2024-04-26T23:11:08.153 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-26T23:11:08.154 DEBUG:teuthology.run:Teuthology command: teuthology --name rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi --archive /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675294 --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/pool-perm} --owner scheduled_rishabh@teuthology --verbose -- /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675294/orig.config.yaml 2024-04-26T23:11:08.199 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T23:11:08.289 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675294 branch: wip-rishabh-testing-20240426.111959 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/pool-perm} email: null first_in_suite: false job_id: '7675294' kernel: client: sha1: distro flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240426.111959 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240426.111959 sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 816 sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 sleep_before_teardown: 0 subset: 166/512 suite: fs suite_branch: wip-rishabh-testing-20240426.111959 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi047.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJuhkAIZrNzUfa5BiRihAr/9yQsVIHqthZJdqNagSQTtDAsb+mEZAsDWgBgGsQQXyEuk8wRTNy5wiSmnIt6JdWA= smithi203.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPST5A8IfAx4FxTfpcq+BZKaAE3MGubSJD5hIpEPXxTlzqJi3F6ZhSulVQafHt2oT3TwmfjhqFJw0vz7VXvLxz4= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_pool_perm teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-26_19:30:57 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-26T23:11:08.289 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa; will attempt to use it 2024-04-26T23:11:08.289 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks 2024-04-26T23:11:08.290 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-26T23:11:08.291 INFO:teuthology.task.internal:Checking packages... 2024-04-26T23:11:08.314 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'bc612ffb9c1d5bc27536bc917a705262a36d4387' 2024-04-26T23:11:08.315 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-26T23:11:08.315 INFO:teuthology.packaging:ref: None 2024-04-26T23:11:08.315 INFO:teuthology.packaging:tag: None 2024-04-26T23:11:08.315 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240426.111959 2024-04-26T23:11:08.315 INFO:teuthology.packaging:sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T23:11:08.315 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240426.111959 2024-04-26T23:11:08.585 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3306.gbc612ffb 2024-04-26T23:11:08.586 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-26T23:11:08.593 INFO:teuthology.task.internal:no buildpackages task found 2024-04-26T23:11:08.593 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-26T23:11:08.628 INFO:teuthology.task.internal:Saving configuration 2024-04-26T23:11:08.644 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-26T23:11:08.661 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-26T23:11:08.685 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi047.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675294', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 23:05:54.118222', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJuhkAIZrNzUfa5BiRihAr/9yQsVIHqthZJdqNagSQTtDAsb+mEZAsDWgBgGsQQXyEuk8wRTNy5wiSmnIt6JdWA='} 2024-04-26T23:11:08.707 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi203.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675294', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 23:05:54.120185', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPST5A8IfAx4FxTfpcq+BZKaAE3MGubSJD5hIpEPXxTlzqJi3F6ZhSulVQafHt2oT3TwmfjhqFJw0vz7VXvLxz4='} 2024-04-26T23:11:08.707 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-26T23:11:08.713 INFO:teuthology.task.internal:roles: ubuntu@smithi047.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-26T23:11:08.713 INFO:teuthology.task.internal:roles: ubuntu@smithi203.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-26T23:11:08.713 INFO:teuthology.run_tasks:Running task console_log... 2024-04-26T23:11:08.807 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f1348cd3160>, signals=[15]) 2024-04-26T23:11:08.808 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-26T23:11:08.814 INFO:teuthology.task.internal:Opening connections... 2024-04-26T23:11:08.814 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi047.front.sepia.ceph.com 2024-04-26T23:11:08.816 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T23:11:08.894 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi203.front.sepia.ceph.com 2024-04-26T23:11:08.895 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T23:11:08.967 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-26T23:11:08.973 DEBUG:teuthology.orchestra.run.smithi047:> uname -m 2024-04-26T23:11:08.991 INFO:teuthology.orchestra.run.smithi047.stdout:x86_64 2024-04-26T23:11:08.992 DEBUG:teuthology.orchestra.run.smithi047:> cat /etc/os-release 2024-04-26T23:11:09.050 INFO:teuthology.orchestra.run.smithi047.stdout:NAME="CentOS Stream" 2024-04-26T23:11:09.050 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION="9" 2024-04-26T23:11:09.050 INFO:teuthology.orchestra.run.smithi047.stdout:ID="centos" 2024-04-26T23:11:09.050 INFO:teuthology.orchestra.run.smithi047.stdout:ID_LIKE="rhel fedora" 2024-04-26T23:11:09.050 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION_ID="9" 2024-04-26T23:11:09.051 INFO:teuthology.orchestra.run.smithi047.stdout:PLATFORM_ID="platform:el9" 2024-04-26T23:11:09.051 INFO:teuthology.orchestra.run.smithi047.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T23:11:09.051 INFO:teuthology.orchestra.run.smithi047.stdout:ANSI_COLOR="0;31" 2024-04-26T23:11:09.051 INFO:teuthology.orchestra.run.smithi047.stdout:LOGO="fedora-logo-icon" 2024-04-26T23:11:09.051 INFO:teuthology.orchestra.run.smithi047.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T23:11:09.051 INFO:teuthology.orchestra.run.smithi047.stdout:HOME_URL="https://centos.org/" 2024-04-26T23:11:09.051 INFO:teuthology.orchestra.run.smithi047.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T23:11:09.052 INFO:teuthology.orchestra.run.smithi047.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T23:11:09.052 INFO:teuthology.orchestra.run.smithi047.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T23:11:09.053 INFO:teuthology.lock.ops:Updating smithi047.front.sepia.ceph.com on lock server 2024-04-26T23:11:09.079 DEBUG:teuthology.orchestra.run.smithi203:> uname -m 2024-04-26T23:11:09.096 INFO:teuthology.orchestra.run.smithi203.stdout:x86_64 2024-04-26T23:11:09.096 DEBUG:teuthology.orchestra.run.smithi203:> cat /etc/os-release 2024-04-26T23:11:09.153 INFO:teuthology.orchestra.run.smithi203.stdout:NAME="CentOS Stream" 2024-04-26T23:11:09.153 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION="9" 2024-04-26T23:11:09.153 INFO:teuthology.orchestra.run.smithi203.stdout:ID="centos" 2024-04-26T23:11:09.153 INFO:teuthology.orchestra.run.smithi203.stdout:ID_LIKE="rhel fedora" 2024-04-26T23:11:09.154 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION_ID="9" 2024-04-26T23:11:09.154 INFO:teuthology.orchestra.run.smithi203.stdout:PLATFORM_ID="platform:el9" 2024-04-26T23:11:09.154 INFO:teuthology.orchestra.run.smithi203.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T23:11:09.154 INFO:teuthology.orchestra.run.smithi203.stdout:ANSI_COLOR="0;31" 2024-04-26T23:11:09.154 INFO:teuthology.orchestra.run.smithi203.stdout:LOGO="fedora-logo-icon" 2024-04-26T23:11:09.154 INFO:teuthology.orchestra.run.smithi203.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T23:11:09.154 INFO:teuthology.orchestra.run.smithi203.stdout:HOME_URL="https://centos.org/" 2024-04-26T23:11:09.154 INFO:teuthology.orchestra.run.smithi203.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T23:11:09.154 INFO:teuthology.orchestra.run.smithi203.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T23:11:09.154 INFO:teuthology.orchestra.run.smithi203.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T23:11:09.154 INFO:teuthology.lock.ops:Updating smithi203.front.sepia.ceph.com on lock server 2024-04-26T23:11:09.316 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-26T23:11:09.326 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-26T23:11:09.353 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-26T23:11:09.353 DEBUG:teuthology.orchestra.run.smithi047:> test '!' -e /home/ubuntu/cephtest 2024-04-26T23:11:09.356 DEBUG:teuthology.orchestra.run.smithi203:> test '!' -e /home/ubuntu/cephtest 2024-04-26T23:11:09.372 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-26T23:11:09.386 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-26T23:11:09.386 DEBUG:teuthology.orchestra.run.smithi047:> test -z $(ls -A /var/lib/ceph) 2024-04-26T23:11:09.413 DEBUG:teuthology.orchestra.run.smithi203:> test -z $(ls -A /var/lib/ceph) 2024-04-26T23:11:09.510 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-26T23:11:09.565 INFO:teuthology.run_tasks:Running task kernel... 2024-04-26T23:11:09.597 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-04-26T23:11:09.597 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2024-04-26T23:11:09.597 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-04-26T23:11:09.598 DEBUG:teuthology.orchestra.run.smithi047:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T23:11:09.598 DEBUG:teuthology.orchestra.run.smithi203:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T23:11:09.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T23:11:09.615 DEBUG:teuthology.orchestra.run.smithi203:> uname -r 2024-04-26T23:11:09.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T23:11:09.616 DEBUG:teuthology.orchestra.run.smithi047:> uname -r 2024-04-26T23:11:09.672 INFO:teuthology.orchestra.run.smithi203.stdout:5.14.0-437.el9.x86_64 2024-04-26T23:11:09.673 INFO:teuthology.task.kernel:Running kernel on smithi203: 5.14.0-437.el9.x86_64 2024-04-26T23:11:09.673 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum install -y kernel 2024-04-26T23:11:09.674 INFO:teuthology.orchestra.run.smithi047.stdout:5.14.0-437.el9.x86_64 2024-04-26T23:11:09.675 INFO:teuthology.task.kernel:Running kernel on smithi047: 5.14.0-437.el9.x86_64 2024-04-26T23:11:09.675 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum install -y kernel 2024-04-26T23:11:12.416 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - BaseOS 7.1 MB/s | 8.0 MB 00:01 2024-04-26T23:11:12.912 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - BaseOS 5.1 MB/s | 8.0 MB 00:01 2024-04-26T23:11:15.302 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - AppStream 14 MB/s | 19 MB 00:01 2024-04-26T23:11:20.131 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - AppStream 2.9 MB/s | 19 MB 00:06 2024-04-26T23:11:20.210 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - CRB 4.8 MB/s | 6.2 MB 00:01 2024-04-26T23:11:21.661 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - Extras packages 52 kB/s | 16 kB 00:00 2024-04-26T23:11:23.213 INFO:teuthology.orchestra.run.smithi203.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-04-26T23:11:24.742 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - CRB 6.6 MB/s | 6.2 MB 00:00 2024-04-26T23:11:26.291 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - Extras packages 47 kB/s | 16 kB 00:00 2024-04-26T23:11:27.450 INFO:teuthology.orchestra.run.smithi047.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-04-26T23:11:28.254 INFO:teuthology.orchestra.run.smithi203.stdout:lab-extras 17 kB/s | 1.7 kB 00:00 2024-04-26T23:11:30.119 INFO:teuthology.orchestra.run.smithi203.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-26T23:11:30.119 INFO:teuthology.orchestra.run.smithi203.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-26T23:11:30.201 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2024-04-26T23:11:30.202 INFO:teuthology.orchestra.run.smithi203.stdout:Nothing to do. 2024-04-26T23:11:30.202 INFO:teuthology.orchestra.run.smithi203.stdout:Complete! 2024-04-26T23:11:30.304 DEBUG:teuthology.orchestra.run.smithi203:> echo no | sudo yum reinstall kernel || true 2024-04-26T23:11:30.803 INFO:teuthology.orchestra.run.smithi203.stdout:Last metadata expiration check: 0:00:02 ago on Fri 26 Apr 2024 11:11:28 PM UTC. 2024-04-26T23:11:30.929 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2024-04-26T23:11:30.940 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-04-26T23:11:30.940 INFO:teuthology.orchestra.run.smithi203.stdout: Package Architecture Version Repository Size 2024-04-26T23:11:30.940 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-04-26T23:11:30.940 INFO:teuthology.orchestra.run.smithi203.stdout:Reinstalling: 2024-04-26T23:11:30.940 INFO:teuthology.orchestra.run.smithi203.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T23:11:30.940 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-26T23:11:30.940 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction Summary 2024-04-26T23:11:30.940 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-04-26T23:11:30.940 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-26T23:11:30.940 INFO:teuthology.orchestra.run.smithi203.stdout:Total download size: 255 k 2024-04-26T23:11:30.940 INFO:teuthology.orchestra.run.smithi203.stdout:Installed size: 0 2024-04-26T23:11:30.940 INFO:teuthology.orchestra.run.smithi203.stderr:Operation aborted. 2024-04-26T23:11:30.999 INFO:teuthology.orchestra.run.smithi203.stdout:Is this ok [y/N]: 2024-04-26T23:11:30.999 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum reinstall -y kernel || true 2024-04-26T23:11:31.493 INFO:teuthology.orchestra.run.smithi203.stdout:Last metadata expiration check: 0:00:03 ago on Fri 26 Apr 2024 11:11:28 PM UTC. 2024-04-26T23:11:31.610 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2024-04-26T23:11:31.611 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-04-26T23:11:31.611 INFO:teuthology.orchestra.run.smithi203.stdout: Package Architecture Version Repository Size 2024-04-26T23:11:31.611 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-04-26T23:11:31.611 INFO:teuthology.orchestra.run.smithi203.stdout:Reinstalling: 2024-04-26T23:11:31.611 INFO:teuthology.orchestra.run.smithi203.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T23:11:31.611 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-26T23:11:31.611 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction Summary 2024-04-26T23:11:31.612 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-04-26T23:11:31.612 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-26T23:11:31.612 INFO:teuthology.orchestra.run.smithi203.stdout:Total download size: 255 k 2024-04-26T23:11:31.612 INFO:teuthology.orchestra.run.smithi203.stdout:Installed size: 0 2024-04-26T23:11:31.612 INFO:teuthology.orchestra.run.smithi203.stdout:Downloading Packages: 2024-04-26T23:11:31.948 INFO:teuthology.orchestra.run.smithi203.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.3 MB/s | 255 kB 00:00 2024-04-26T23:11:31.949 INFO:teuthology.orchestra.run.smithi203.stdout:-------------------------------------------------------------------------------- 2024-04-26T23:11:31.949 INFO:teuthology.orchestra.run.smithi203.stdout:Total 756 kB/s | 255 kB 00:00 2024-04-26T23:11:31.970 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction check 2024-04-26T23:11:32.142 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction check succeeded. 2024-04-26T23:11:32.142 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction test 2024-04-26T23:11:32.148 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction test succeeded. 2024-04-26T23:11:32.148 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction 2024-04-26T23:11:32.269 INFO:teuthology.orchestra.run.smithi203.stdout: Preparing : 1/1 2024-04-26T23:11:32.349 INFO:teuthology.orchestra.run.smithi203.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T23:11:32.349 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T23:11:32.817 INFO:teuthology.orchestra.run.smithi047.stdout:lab-extras 24 kB/s | 1.7 kB 00:00 2024-04-26T23:11:33.102 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T23:11:33.102 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T23:11:33.812 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T23:11:33.812 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-26T23:11:33.813 INFO:teuthology.orchestra.run.smithi203.stdout:Reinstalled: 2024-04-26T23:11:33.813 INFO:teuthology.orchestra.run.smithi203.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-26T23:11:33.813 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-26T23:11:33.813 INFO:teuthology.orchestra.run.smithi203.stdout:Complete! 2024-04-26T23:11:34.002 DEBUG:teuthology.orchestra.run.smithi203:> rpm -q kernel | sort -rV | head -n 1 2024-04-26T23:11:34.049 INFO:teuthology.orchestra.run.smithi203.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-26T23:11:34.049 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-26T23:11:34.049 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-26T23:11:34.049 DEBUG:teuthology.parallel:result is None 2024-04-26T23:11:34.761 INFO:teuthology.orchestra.run.smithi047.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-26T23:11:34.762 INFO:teuthology.orchestra.run.smithi047.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-26T23:11:34.865 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-04-26T23:11:34.866 INFO:teuthology.orchestra.run.smithi047.stdout:Nothing to do. 2024-04-26T23:11:34.866 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-04-26T23:11:34.989 DEBUG:teuthology.orchestra.run.smithi047:> echo no | sudo yum reinstall kernel || true 2024-04-26T23:11:35.508 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:00:03 ago on Fri 26 Apr 2024 11:11:32 PM UTC. 2024-04-26T23:11:35.632 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-04-26T23:11:35.650 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-04-26T23:11:35.650 INFO:teuthology.orchestra.run.smithi047.stdout: Package Architecture Version Repository Size 2024-04-26T23:11:35.650 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-04-26T23:11:35.650 INFO:teuthology.orchestra.run.smithi047.stdout:Reinstalling: 2024-04-26T23:11:35.650 INFO:teuthology.orchestra.run.smithi047.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T23:11:35.651 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:11:35.651 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-04-26T23:11:35.651 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-04-26T23:11:35.651 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:11:35.651 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 255 k 2024-04-26T23:11:35.651 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 0 2024-04-26T23:11:35.651 INFO:teuthology.orchestra.run.smithi047.stderr:Operation aborted. 2024-04-26T23:11:35.706 INFO:teuthology.orchestra.run.smithi047.stdout:Is this ok [y/N]: 2024-04-26T23:11:35.706 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum reinstall -y kernel || true 2024-04-26T23:11:36.224 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:00:04 ago on Fri 26 Apr 2024 11:11:32 PM UTC. 2024-04-26T23:11:36.344 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-04-26T23:11:36.345 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-04-26T23:11:36.345 INFO:teuthology.orchestra.run.smithi047.stdout: Package Architecture Version Repository Size 2024-04-26T23:11:36.345 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-04-26T23:11:36.345 INFO:teuthology.orchestra.run.smithi047.stdout:Reinstalling: 2024-04-26T23:11:36.345 INFO:teuthology.orchestra.run.smithi047.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T23:11:36.345 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:11:36.345 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-04-26T23:11:36.345 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-04-26T23:11:36.345 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:11:36.345 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 255 k 2024-04-26T23:11:36.345 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 0 2024-04-26T23:11:36.345 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-04-26T23:11:37.319 INFO:teuthology.orchestra.run.smithi047.stdout:kernel-5.14.0-437.el9.x86_64.rpm 300 kB/s | 255 kB 00:00 2024-04-26T23:11:37.319 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-04-26T23:11:37.319 INFO:teuthology.orchestra.run.smithi047.stdout:Total 262 kB/s | 255 kB 00:00 2024-04-26T23:11:37.344 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-04-26T23:11:37.529 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-04-26T23:11:37.529 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-04-26T23:11:37.534 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-04-26T23:11:37.534 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-04-26T23:11:37.679 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-04-26T23:11:37.778 INFO:teuthology.orchestra.run.smithi047.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T23:11:37.778 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T23:11:38.427 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T23:11:38.427 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T23:11:39.143 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T23:11:39.143 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:11:39.143 INFO:teuthology.orchestra.run.smithi047.stdout:Reinstalled: 2024-04-26T23:11:39.143 INFO:teuthology.orchestra.run.smithi047.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-26T23:11:39.144 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:11:39.144 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-04-26T23:11:39.341 DEBUG:teuthology.orchestra.run.smithi047:> rpm -q kernel | sort -rV | head -n 1 2024-04-26T23:11:39.388 INFO:teuthology.orchestra.run.smithi047.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-26T23:11:39.389 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-26T23:11:39.389 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-26T23:11:39.389 DEBUG:teuthology.parallel:result is None 2024-04-26T23:11:39.389 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-26T23:11:39.396 INFO:teuthology.task.internal:Creating test directory... 2024-04-26T23:11:39.396 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T23:11:39.399 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T23:11:39.416 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-26T23:11:39.434 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-26T23:11:39.467 INFO:teuthology.task.internal:Creating archive directory... 2024-04-26T23:11:39.467 DEBUG:teuthology.orchestra.run.smithi047:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T23:11:39.470 DEBUG:teuthology.orchestra.run.smithi203:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T23:11:39.503 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-26T23:11:39.509 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-26T23:11:39.510 DEBUG:teuthology.orchestra.run.smithi047:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T23:11:39.532 DEBUG:teuthology.orchestra.run.smithi203:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T23:11:39.572 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T23:11:39.578 INFO:teuthology.orchestra.run.smithi203.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T23:11:39.586 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T23:11:39.598 INFO:teuthology.orchestra.run.smithi203.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T23:11:39.600 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-26T23:11:39.609 INFO:teuthology.task.internal:Configuring sudo... 2024-04-26T23:11:39.609 DEBUG:teuthology.orchestra.run.smithi047:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T23:11:39.630 DEBUG:teuthology.orchestra.run.smithi203:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T23:11:39.666 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-26T23:11:39.675 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-26T23:11:39.675 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T23:11:39.695 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T23:11:39.723 DEBUG:teuthology.orchestra.run.smithi047:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T23:11:39.788 DEBUG:teuthology.orchestra.run.smithi047:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T23:11:39.839 DEBUG:teuthology.orchestra.run.smithi047:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T23:11:39.918 DEBUG:teuthology.orchestra.run.smithi047:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T23:11:39.959 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:11:39.959 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T23:11:40.023 DEBUG:teuthology.orchestra.run.smithi203:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T23:11:40.070 DEBUG:teuthology.orchestra.run.smithi203:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T23:11:40.113 DEBUG:teuthology.orchestra.run.smithi203:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T23:11:40.193 DEBUG:teuthology.orchestra.run.smithi203:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T23:11:40.236 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-26T23:11:40.237 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T23:11:40.301 DEBUG:teuthology.orchestra.run.smithi047:> sudo service rsyslog restart 2024-04-26T23:11:40.304 DEBUG:teuthology.orchestra.run.smithi203:> sudo service rsyslog restart 2024-04-26T23:11:40.335 INFO:teuthology.orchestra.run.smithi047.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T23:11:40.376 INFO:teuthology.orchestra.run.smithi203.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T23:11:40.764 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-26T23:11:40.772 INFO:teuthology.task.internal:Starting timer... 2024-04-26T23:11:40.772 INFO:teuthology.run_tasks:Running task pcp... 2024-04-26T23:11:40.806 INFO:teuthology.run_tasks:Running task selinux... 2024-04-26T23:11:40.853 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-26T23:11:40.853 DEBUG:teuthology.orchestra.run.smithi047:> sudo service auditd rotate 2024-04-26T23:11:40.904 INFO:teuthology.orchestra.run.smithi047.stdout:Rotating logs: 2024-04-26T23:11:40.907 DEBUG:teuthology.orchestra.run.smithi203:> sudo service auditd rotate 2024-04-26T23:11:40.979 INFO:teuthology.orchestra.run.smithi203.stdout:Rotating logs: 2024-04-26T23:11:40.981 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-26T23:11:40.981 DEBUG:teuthology.orchestra.run.smithi047:> /usr/sbin/getenforce 2024-04-26T23:11:41.006 INFO:teuthology.orchestra.run.smithi047.stdout:Permissive 2024-04-26T23:11:41.006 DEBUG:teuthology.orchestra.run.smithi203:> /usr/sbin/getenforce 2024-04-26T23:11:41.027 INFO:teuthology.orchestra.run.smithi203.stdout:Permissive 2024-04-26T23:11:41.027 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi047.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi203.front.sepia.ceph.com': 'permissive'} 2024-04-26T23:11:41.028 DEBUG:teuthology.orchestra.run.smithi047:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T23:11:41.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T23:11:41.073 DEBUG:teuthology.orchestra.run.smithi203:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T23:11:41.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T23:11:41.098 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-26T23:11:41.098 DEBUG:teuthology.orchestra.run.smithi047:> sudo /usr/sbin/setenforce permissive 2024-04-26T23:11:41.139 DEBUG:teuthology.orchestra.run.smithi203:> sudo /usr/sbin/setenforce permissive 2024-04-26T23:11:41.163 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-26T23:11:41.173 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-26T23:11:41.177 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-26T23:11:41.368 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-26T23:11:41.414 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-26T23:11:41.415 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi047.front.sepia.ceph.com,smithi203.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-26T23:15:10.676 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi047.front.sepia.ceph.com'), Remote(name='ubuntu@smithi203.front.sepia.ceph.com')] 2024-04-26T23:15:10.677 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-04-26T23:15:10.679 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T23:15:10.757 DEBUG:teuthology.orchestra.run.smithi047:> true 2024-04-26T23:15:10.833 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-04-26T23:15:10.833 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-04-26T23:15:10.834 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T23:15:10.918 DEBUG:teuthology.orchestra.run.smithi203:> true 2024-04-26T23:15:10.953 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-04-26T23:15:10.953 INFO:teuthology.run_tasks:Running task clock... 2024-04-26T23:15:10.965 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-26T23:15:10.966 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T23:15:10.966 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T23:15:10.968 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T23:15:10.969 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T23:15:10.996 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T23:15:11.010 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T23:15:11.020 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T23:15:11.034 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T23:15:11.034 INFO:teuthology.orchestra.run.smithi047.stderr:sudo: ntpd: command not found 2024-04-26T23:15:11.045 INFO:teuthology.orchestra.run.smithi047.stdout:506 Cannot talk to daemon 2024-04-26T23:15:11.056 INFO:teuthology.orchestra.run.smithi203.stderr:sudo: ntpd: command not found 2024-04-26T23:15:11.058 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T23:15:11.067 INFO:teuthology.orchestra.run.smithi203.stdout:506 Cannot talk to daemon 2024-04-26T23:15:11.072 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T23:15:11.080 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T23:15:11.093 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T23:15:11.133 INFO:teuthology.orchestra.run.smithi047.stderr:bash: line 1: ntpq: command not found 2024-04-26T23:15:11.137 INFO:teuthology.orchestra.run.smithi047.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T23:15:11.137 INFO:teuthology.orchestra.run.smithi047.stdout:=============================================================================== 2024-04-26T23:15:11.137 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T23:15:11.137 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T23:15:11.138 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T23:15:11.138 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T23:15:11.151 INFO:teuthology.orchestra.run.smithi203.stderr:bash: line 1: ntpq: command not found 2024-04-26T23:15:11.154 INFO:teuthology.orchestra.run.smithi203.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T23:15:11.154 INFO:teuthology.orchestra.run.smithi203.stdout:=============================================================================== 2024-04-26T23:15:11.154 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T23:15:11.154 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T23:15:11.155 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T23:15:11.155 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T23:15:11.155 INFO:teuthology.run_tasks:Running task install... 2024-04-26T23:15:11.165 DEBUG:teuthology.task.install:project ceph 2024-04-26T23:15:11.165 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}} 2024-04-26T23:15:11.165 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'} 2024-04-26T23:15:11.165 INFO:teuthology.task.install:Using flavor: default 2024-04-26T23:15:11.174 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-26T23:15:11.174 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-26T23:15:11.176 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-26T23:15:11.177 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T23:15:11.179 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-26T23:15:11.179 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T23:15:11.332 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-26T23:15:11.332 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-26T23:15:11.337 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-26T23:15:11.337 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-26T23:15:11.505 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T23:15:11.506 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-26T23:15:11.506 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T23:15:11.515 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T23:15:11.515 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:15:11.515 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T23:15:11.531 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T23:15:11.531 DEBUG:teuthology.orchestra.run.smithi203:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T23:15:11.540 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T23:15:11.541 DEBUG:teuthology.orchestra.run.smithi047:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T23:15:11.596 DEBUG:teuthology.orchestra.run.smithi203:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T23:15:11.606 DEBUG:teuthology.orchestra.run.smithi047:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T23:15:11.670 DEBUG:teuthology.orchestra.run.smithi203:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T23:15:11.681 DEBUG:teuthology.orchestra.run.smithi047:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T23:15:11.733 INFO:teuthology.orchestra.run.smithi203.stdout:check_obsoletes = 1 2024-04-26T23:15:11.734 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum clean all 2024-04-26T23:15:11.745 INFO:teuthology.orchestra.run.smithi047.stdout:check_obsoletes = 1 2024-04-26T23:15:11.746 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum clean all 2024-04-26T23:15:11.960 INFO:teuthology.orchestra.run.smithi203.stdout:48 files removed 2024-04-26T23:15:11.979 INFO:teuthology.orchestra.run.smithi047.stdout:48 files removed 2024-04-26T23:15:11.987 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T23:15:12.006 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T23:15:12.575 INFO:teuthology.orchestra.run.smithi203.stdout:ceph packages for x86_64 254 kB/s | 84 kB 00:00 2024-04-26T23:15:12.733 INFO:teuthology.orchestra.run.smithi047.stdout:ceph packages for x86_64 184 kB/s | 84 kB 00:00 2024-04-26T23:15:12.968 INFO:teuthology.orchestra.run.smithi203.stdout:ceph noarch packages 33 kB/s | 12 kB 00:00 2024-04-26T23:15:13.121 INFO:teuthology.orchestra.run.smithi047.stdout:ceph noarch packages 34 kB/s | 12 kB 00:00 2024-04-26T23:15:13.269 INFO:teuthology.orchestra.run.smithi203.stdout:ceph source packages 7.1 kB/s | 1.9 kB 00:00 2024-04-26T23:15:13.612 INFO:teuthology.orchestra.run.smithi047.stdout:ceph source packages 4.2 kB/s | 1.9 kB 00:00 2024-04-26T23:15:14.795 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - BaseOS 7.0 MB/s | 8.0 MB 00:01 2024-04-26T23:15:15.277 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - BaseOS 4.1 MB/s | 8.0 MB 00:01 2024-04-26T23:15:16.738 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-04-26T23:15:17.479 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - AppStream 17 MB/s | 19 MB 00:01 2024-04-26T23:15:21.375 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - CRB 7.1 MB/s | 6.2 MB 00:00 2024-04-26T23:15:22.038 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - CRB 6.3 MB/s | 6.2 MB 00:00 2024-04-26T23:15:23.039 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 16 kB 00:00 2024-04-26T23:15:23.890 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - Extras packages 23 kB/s | 16 kB 00:00 2024-04-26T23:15:24.818 INFO:teuthology.orchestra.run.smithi047.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-26T23:15:25.532 INFO:teuthology.orchestra.run.smithi203.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-04-26T23:15:30.049 INFO:teuthology.orchestra.run.smithi047.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-04-26T23:15:30.649 INFO:teuthology.orchestra.run.smithi203.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-04-26T23:15:32.260 INFO:teuthology.orchestra.run.smithi047.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T23:15:32.261 INFO:teuthology.orchestra.run.smithi047.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T23:15:32.272 INFO:teuthology.orchestra.run.smithi047.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T23:15:32.272 INFO:teuthology.orchestra.run.smithi047.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T23:15:32.273 INFO:teuthology.orchestra.run.smithi047.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T23:15:32.274 INFO:teuthology.orchestra.run.smithi047.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T23:15:32.276 INFO:teuthology.orchestra.run.smithi047.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T23:15:32.278 INFO:teuthology.orchestra.run.smithi047.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T23:15:32.279 INFO:teuthology.orchestra.run.smithi047.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T23:15:32.344 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-04-26T23:15:32.354 INFO:teuthology.orchestra.run.smithi047.stdout:======================================================================================= 2024-04-26T23:15:32.354 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repository Size 2024-04-26T23:15:32.354 INFO:teuthology.orchestra.run.smithi047.stdout:======================================================================================= 2024-04-26T23:15:32.354 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-04-26T23:15:32.354 INFO:teuthology.orchestra.run.smithi047.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T23:15:32.354 INFO:teuthology.orchestra.run.smithi047.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-26T23:15:32.354 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T23:15:32.354 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-26T23:15:32.354 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-26T23:15:32.354 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-26T23:15:32.355 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-26T23:15:32.355 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-26T23:15:32.355 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-26T23:15:32.355 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-26T23:15:32.355 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-26T23:15:32.355 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-26T23:15:32.355 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-26T23:15:32.355 INFO:teuthology.orchestra.run.smithi047.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-26T23:15:32.355 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-26T23:15:32.355 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-26T23:15:32.355 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-26T23:15:32.355 INFO:teuthology.orchestra.run.smithi047.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T23:15:32.355 INFO:teuthology.orchestra.run.smithi047.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T23:15:32.356 INFO:teuthology.orchestra.run.smithi047.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T23:15:32.356 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-26T23:15:32.356 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-26T23:15:32.356 INFO:teuthology.orchestra.run.smithi047.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-26T23:15:32.356 INFO:teuthology.orchestra.run.smithi047.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T23:15:32.356 INFO:teuthology.orchestra.run.smithi047.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T23:15:32.356 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T23:15:32.356 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T23:15:32.356 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T23:15:32.356 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T23:15:32.356 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-26T23:15:32.356 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-26T23:15:32.356 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-26T23:15:32.356 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-26T23:15:32.357 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-26T23:15:32.357 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T23:15:32.357 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-26T23:15:32.357 INFO:teuthology.orchestra.run.smithi047.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T23:15:32.357 INFO:teuthology.orchestra.run.smithi047.stdout:Upgrading: 2024-04-26T23:15:32.357 INFO:teuthology.orchestra.run.smithi047.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-26T23:15:32.357 INFO:teuthology.orchestra.run.smithi047.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T23:15:32.357 INFO:teuthology.orchestra.run.smithi047.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T23:15:32.357 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T23:15:32.357 INFO:teuthology.orchestra.run.smithi047.stdout:Installing dependencies: 2024-04-26T23:15:32.357 INFO:teuthology.orchestra.run.smithi047.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T23:15:32.357 INFO:teuthology.orchestra.run.smithi047.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T23:15:32.357 INFO:teuthology.orchestra.run.smithi047.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T23:15:32.358 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-26T23:15:32.358 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-26T23:15:32.358 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-26T23:15:32.358 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-26T23:15:32.358 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-26T23:15:32.358 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-26T23:15:32.358 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-26T23:15:32.358 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-26T23:15:32.358 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T23:15:32.358 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T23:15:32.358 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T23:15:32.358 INFO:teuthology.orchestra.run.smithi047.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T23:15:32.358 INFO:teuthology.orchestra.run.smithi047.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T23:15:32.358 INFO:teuthology.orchestra.run.smithi047.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T23:15:32.359 INFO:teuthology.orchestra.run.smithi047.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T23:15:32.359 INFO:teuthology.orchestra.run.smithi047.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T23:15:32.359 INFO:teuthology.orchestra.run.smithi047.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-26T23:15:32.359 INFO:teuthology.orchestra.run.smithi047.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T23:15:32.359 INFO:teuthology.orchestra.run.smithi047.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T23:15:32.359 INFO:teuthology.orchestra.run.smithi047.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T23:15:32.359 INFO:teuthology.orchestra.run.smithi047.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T23:15:32.359 INFO:teuthology.orchestra.run.smithi047.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T23:15:32.359 INFO:teuthology.orchestra.run.smithi047.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-26T23:15:32.359 INFO:teuthology.orchestra.run.smithi047.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T23:15:32.359 INFO:teuthology.orchestra.run.smithi047.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T23:15:32.359 INFO:teuthology.orchestra.run.smithi047.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T23:15:32.359 INFO:teuthology.orchestra.run.smithi047.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T23:15:32.359 INFO:teuthology.orchestra.run.smithi047.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T23:15:32.359 INFO:teuthology.orchestra.run.smithi047.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T23:15:32.360 INFO:teuthology.orchestra.run.smithi047.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T23:15:32.360 INFO:teuthology.orchestra.run.smithi047.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T23:15:32.360 INFO:teuthology.orchestra.run.smithi047.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T23:15:32.360 INFO:teuthology.orchestra.run.smithi047.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T23:15:32.360 INFO:teuthology.orchestra.run.smithi047.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T23:15:32.360 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T23:15:32.360 INFO:teuthology.orchestra.run.smithi047.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T23:15:32.360 INFO:teuthology.orchestra.run.smithi047.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T23:15:32.360 INFO:teuthology.orchestra.run.smithi047.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T23:15:32.360 INFO:teuthology.orchestra.run.smithi047.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T23:15:32.360 INFO:teuthology.orchestra.run.smithi047.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T23:15:32.360 INFO:teuthology.orchestra.run.smithi047.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T23:15:32.360 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T23:15:32.360 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-26T23:15:32.360 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-26T23:15:32.361 INFO:teuthology.orchestra.run.smithi047.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T23:15:32.361 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T23:15:32.361 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T23:15:32.361 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T23:15:32.361 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T23:15:32.361 INFO:teuthology.orchestra.run.smithi047.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T23:15:32.361 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T23:15:32.361 INFO:teuthology.orchestra.run.smithi047.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T23:15:32.361 INFO:teuthology.orchestra.run.smithi047.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T23:15:32.361 INFO:teuthology.orchestra.run.smithi047.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T23:15:32.361 INFO:teuthology.orchestra.run.smithi047.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T23:15:32.361 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T23:15:32.361 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T23:15:32.361 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T23:15:32.361 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T23:15:32.362 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T23:15:32.362 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T23:15:32.362 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T23:15:32.362 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T23:15:32.362 INFO:teuthology.orchestra.run.smithi047.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T23:15:32.362 INFO:teuthology.orchestra.run.smithi047.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T23:15:32.362 INFO:teuthology.orchestra.run.smithi047.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T23:15:32.362 INFO:teuthology.orchestra.run.smithi047.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T23:15:32.362 INFO:teuthology.orchestra.run.smithi047.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T23:15:32.362 INFO:teuthology.orchestra.run.smithi047.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T23:15:32.362 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T23:15:32.362 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T23:15:32.362 INFO:teuthology.orchestra.run.smithi047.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T23:15:32.362 INFO:teuthology.orchestra.run.smithi047.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T23:15:32.363 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T23:15:32.363 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T23:15:32.363 INFO:teuthology.orchestra.run.smithi047.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T23:15:32.363 INFO:teuthology.orchestra.run.smithi047.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T23:15:32.363 INFO:teuthology.orchestra.run.smithi047.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T23:15:32.363 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T23:15:32.363 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T23:15:32.363 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T23:15:32.363 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T23:15:32.363 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T23:15:32.363 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T23:15:32.363 INFO:teuthology.orchestra.run.smithi047.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T23:15:32.363 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T23:15:32.363 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T23:15:32.363 INFO:teuthology.orchestra.run.smithi047.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T23:15:32.363 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T23:15:32.364 INFO:teuthology.orchestra.run.smithi047.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T23:15:32.364 INFO:teuthology.orchestra.run.smithi047.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T23:15:32.364 INFO:teuthology.orchestra.run.smithi047.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T23:15:32.364 INFO:teuthology.orchestra.run.smithi047.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T23:15:32.364 INFO:teuthology.orchestra.run.smithi047.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T23:15:32.364 INFO:teuthology.orchestra.run.smithi047.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T23:15:32.364 INFO:teuthology.orchestra.run.smithi047.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T23:15:32.364 INFO:teuthology.orchestra.run.smithi047.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T23:15:32.364 INFO:teuthology.orchestra.run.smithi047.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T23:15:32.364 INFO:teuthology.orchestra.run.smithi047.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T23:15:32.364 INFO:teuthology.orchestra.run.smithi047.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T23:15:32.364 INFO:teuthology.orchestra.run.smithi047.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T23:15:32.364 INFO:teuthology.orchestra.run.smithi047.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T23:15:32.364 INFO:teuthology.orchestra.run.smithi047.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T23:15:32.365 INFO:teuthology.orchestra.run.smithi047.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T23:15:32.365 INFO:teuthology.orchestra.run.smithi047.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T23:15:32.365 INFO:teuthology.orchestra.run.smithi047.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T23:15:32.365 INFO:teuthology.orchestra.run.smithi047.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T23:15:32.365 INFO:teuthology.orchestra.run.smithi047.stdout:Installing weak dependencies: 2024-04-26T23:15:32.365 INFO:teuthology.orchestra.run.smithi047.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T23:15:32.365 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:15:32.365 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-04-26T23:15:32.365 INFO:teuthology.orchestra.run.smithi047.stdout:======================================================================================= 2024-04-26T23:15:32.365 INFO:teuthology.orchestra.run.smithi047.stdout:Install 148 Packages 2024-04-26T23:15:32.365 INFO:teuthology.orchestra.run.smithi047.stdout:Upgrade 4 Packages 2024-04-26T23:15:32.365 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:15:32.366 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 222 M 2024-04-26T23:15:32.366 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-04-26T23:15:32.950 INFO:teuthology.orchestra.run.smithi203.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T23:15:32.951 INFO:teuthology.orchestra.run.smithi203.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T23:15:32.962 INFO:teuthology.orchestra.run.smithi203.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T23:15:32.963 INFO:teuthology.orchestra.run.smithi203.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T23:15:32.964 INFO:teuthology.orchestra.run.smithi203.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T23:15:32.965 INFO:teuthology.orchestra.run.smithi203.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T23:15:32.966 INFO:teuthology.orchestra.run.smithi203.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T23:15:32.969 INFO:teuthology.orchestra.run.smithi203.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T23:15:32.969 INFO:teuthology.orchestra.run.smithi203.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T23:15:33.031 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2024-04-26T23:15:33.040 INFO:teuthology.orchestra.run.smithi203.stdout:======================================================================================= 2024-04-26T23:15:33.040 INFO:teuthology.orchestra.run.smithi203.stdout: Package Arch Version Repository Size 2024-04-26T23:15:33.040 INFO:teuthology.orchestra.run.smithi203.stdout:======================================================================================= 2024-04-26T23:15:33.040 INFO:teuthology.orchestra.run.smithi203.stdout:Installing: 2024-04-26T23:15:33.040 INFO:teuthology.orchestra.run.smithi203.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T23:15:33.040 INFO:teuthology.orchestra.run.smithi203.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-26T23:15:33.041 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T23:15:33.041 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-26T23:15:33.041 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-26T23:15:33.041 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-26T23:15:33.041 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-26T23:15:33.041 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-26T23:15:33.041 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-26T23:15:33.041 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-26T23:15:33.041 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-26T23:15:33.041 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-26T23:15:33.041 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-26T23:15:33.041 INFO:teuthology.orchestra.run.smithi203.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-26T23:15:33.041 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-26T23:15:33.041 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-26T23:15:33.041 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-26T23:15:33.042 INFO:teuthology.orchestra.run.smithi203.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T23:15:33.042 INFO:teuthology.orchestra.run.smithi203.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T23:15:33.042 INFO:teuthology.orchestra.run.smithi203.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T23:15:33.042 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-26T23:15:33.042 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-26T23:15:33.042 INFO:teuthology.orchestra.run.smithi203.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-26T23:15:33.042 INFO:teuthology.orchestra.run.smithi203.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T23:15:33.042 INFO:teuthology.orchestra.run.smithi203.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T23:15:33.042 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T23:15:33.042 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T23:15:33.042 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T23:15:33.042 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T23:15:33.042 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-26T23:15:33.042 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-26T23:15:33.042 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-26T23:15:33.043 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-26T23:15:33.043 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-26T23:15:33.043 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T23:15:33.043 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-26T23:15:33.043 INFO:teuthology.orchestra.run.smithi203.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T23:15:33.043 INFO:teuthology.orchestra.run.smithi203.stdout:Upgrading: 2024-04-26T23:15:33.043 INFO:teuthology.orchestra.run.smithi203.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-26T23:15:33.043 INFO:teuthology.orchestra.run.smithi203.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T23:15:33.043 INFO:teuthology.orchestra.run.smithi203.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T23:15:33.043 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T23:15:33.043 INFO:teuthology.orchestra.run.smithi203.stdout:Installing dependencies: 2024-04-26T23:15:33.043 INFO:teuthology.orchestra.run.smithi203.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T23:15:33.043 INFO:teuthology.orchestra.run.smithi203.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T23:15:33.043 INFO:teuthology.orchestra.run.smithi203.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T23:15:33.044 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-26T23:15:33.044 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-26T23:15:33.044 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-26T23:15:33.044 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-26T23:15:33.044 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-26T23:15:33.044 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-26T23:15:33.044 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-26T23:15:33.044 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-26T23:15:33.044 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T23:15:33.044 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T23:15:33.044 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T23:15:33.044 INFO:teuthology.orchestra.run.smithi203.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T23:15:33.044 INFO:teuthology.orchestra.run.smithi203.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T23:15:33.044 INFO:teuthology.orchestra.run.smithi203.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T23:15:33.044 INFO:teuthology.orchestra.run.smithi203.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T23:15:33.045 INFO:teuthology.orchestra.run.smithi203.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T23:15:33.045 INFO:teuthology.orchestra.run.smithi203.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-26T23:15:33.045 INFO:teuthology.orchestra.run.smithi203.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T23:15:33.045 INFO:teuthology.orchestra.run.smithi203.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T23:15:33.045 INFO:teuthology.orchestra.run.smithi203.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T23:15:33.045 INFO:teuthology.orchestra.run.smithi203.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T23:15:33.045 INFO:teuthology.orchestra.run.smithi203.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T23:15:33.045 INFO:teuthology.orchestra.run.smithi203.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-26T23:15:33.045 INFO:teuthology.orchestra.run.smithi203.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T23:15:33.045 INFO:teuthology.orchestra.run.smithi203.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T23:15:33.045 INFO:teuthology.orchestra.run.smithi203.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T23:15:33.045 INFO:teuthology.orchestra.run.smithi203.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T23:15:33.045 INFO:teuthology.orchestra.run.smithi203.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T23:15:33.045 INFO:teuthology.orchestra.run.smithi203.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T23:15:33.045 INFO:teuthology.orchestra.run.smithi203.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T23:15:33.046 INFO:teuthology.orchestra.run.smithi203.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T23:15:33.046 INFO:teuthology.orchestra.run.smithi203.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T23:15:33.046 INFO:teuthology.orchestra.run.smithi203.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T23:15:33.046 INFO:teuthology.orchestra.run.smithi203.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T23:15:33.046 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T23:15:33.046 INFO:teuthology.orchestra.run.smithi203.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T23:15:33.046 INFO:teuthology.orchestra.run.smithi203.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T23:15:33.046 INFO:teuthology.orchestra.run.smithi203.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T23:15:33.046 INFO:teuthology.orchestra.run.smithi203.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T23:15:33.046 INFO:teuthology.orchestra.run.smithi203.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T23:15:33.046 INFO:teuthology.orchestra.run.smithi203.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T23:15:33.046 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T23:15:33.046 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-26T23:15:33.046 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-26T23:15:33.047 INFO:teuthology.orchestra.run.smithi203.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T23:15:33.047 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T23:15:33.047 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T23:15:33.047 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T23:15:33.047 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T23:15:33.047 INFO:teuthology.orchestra.run.smithi203.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T23:15:33.047 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T23:15:33.047 INFO:teuthology.orchestra.run.smithi203.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T23:15:33.047 INFO:teuthology.orchestra.run.smithi203.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T23:15:33.047 INFO:teuthology.orchestra.run.smithi203.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T23:15:33.047 INFO:teuthology.orchestra.run.smithi203.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T23:15:33.047 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T23:15:33.047 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T23:15:33.047 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T23:15:33.047 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T23:15:33.047 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T23:15:33.048 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T23:15:33.048 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T23:15:33.048 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T23:15:33.048 INFO:teuthology.orchestra.run.smithi203.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T23:15:33.048 INFO:teuthology.orchestra.run.smithi203.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T23:15:33.048 INFO:teuthology.orchestra.run.smithi203.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T23:15:33.048 INFO:teuthology.orchestra.run.smithi203.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T23:15:33.048 INFO:teuthology.orchestra.run.smithi203.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T23:15:33.048 INFO:teuthology.orchestra.run.smithi203.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T23:15:33.048 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T23:15:33.048 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T23:15:33.048 INFO:teuthology.orchestra.run.smithi203.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T23:15:33.048 INFO:teuthology.orchestra.run.smithi203.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T23:15:33.048 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T23:15:33.048 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T23:15:33.049 INFO:teuthology.orchestra.run.smithi203.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T23:15:33.049 INFO:teuthology.orchestra.run.smithi203.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T23:15:33.049 INFO:teuthology.orchestra.run.smithi203.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T23:15:33.049 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T23:15:33.049 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T23:15:33.049 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T23:15:33.049 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T23:15:33.049 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T23:15:33.049 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T23:15:33.049 INFO:teuthology.orchestra.run.smithi203.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T23:15:33.049 INFO:teuthology.orchestra.run.smithi203.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T23:15:33.049 INFO:teuthology.orchestra.run.smithi203.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T23:15:33.049 INFO:teuthology.orchestra.run.smithi203.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T23:15:33.049 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T23:15:33.049 INFO:teuthology.orchestra.run.smithi203.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T23:15:33.049 INFO:teuthology.orchestra.run.smithi203.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T23:15:33.050 INFO:teuthology.orchestra.run.smithi203.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T23:15:33.050 INFO:teuthology.orchestra.run.smithi203.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T23:15:33.050 INFO:teuthology.orchestra.run.smithi203.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T23:15:33.050 INFO:teuthology.orchestra.run.smithi203.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T23:15:33.050 INFO:teuthology.orchestra.run.smithi203.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T23:15:33.050 INFO:teuthology.orchestra.run.smithi203.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T23:15:33.050 INFO:teuthology.orchestra.run.smithi203.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T23:15:33.050 INFO:teuthology.orchestra.run.smithi203.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T23:15:33.050 INFO:teuthology.orchestra.run.smithi203.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T23:15:33.050 INFO:teuthology.orchestra.run.smithi203.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T23:15:33.050 INFO:teuthology.orchestra.run.smithi203.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T23:15:33.050 INFO:teuthology.orchestra.run.smithi203.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T23:15:33.050 INFO:teuthology.orchestra.run.smithi203.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T23:15:33.050 INFO:teuthology.orchestra.run.smithi203.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T23:15:33.050 INFO:teuthology.orchestra.run.smithi203.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T23:15:33.051 INFO:teuthology.orchestra.run.smithi203.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T23:15:33.051 INFO:teuthology.orchestra.run.smithi203.stdout:Installing weak dependencies: 2024-04-26T23:15:33.051 INFO:teuthology.orchestra.run.smithi203.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T23:15:33.051 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-26T23:15:33.051 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction Summary 2024-04-26T23:15:33.051 INFO:teuthology.orchestra.run.smithi203.stdout:======================================================================================= 2024-04-26T23:15:33.051 INFO:teuthology.orchestra.run.smithi203.stdout:Install 148 Packages 2024-04-26T23:15:33.051 INFO:teuthology.orchestra.run.smithi203.stdout:Upgrade 4 Packages 2024-04-26T23:15:33.051 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-26T23:15:33.051 INFO:teuthology.orchestra.run.smithi203.stdout:Total download size: 222 M 2024-04-26T23:15:33.052 INFO:teuthology.orchestra.run.smithi203.stdout:Downloading Packages: 2024-04-26T23:15:33.551 INFO:teuthology.orchestra.run.smithi047.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 38 kB/s | 6.6 kB 00:00 2024-04-26T23:15:33.776 INFO:teuthology.orchestra.run.smithi047.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 3.9 MB/s | 889 kB 00:00 2024-04-26T23:15:33.843 INFO:teuthology.orchestra.run.smithi047.stdout:(3/152): ceph-immutable-object-cache-19.0.0-330 2.1 MB/s | 145 kB 00:00 2024-04-26T23:15:34.176 INFO:teuthology.orchestra.run.smithi047.stdout:(4/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 7.1 MB/s | 2.4 MB 00:00 2024-04-26T23:15:34.292 INFO:teuthology.orchestra.run.smithi203.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 12 kB/s | 6.6 kB 00:00 2024-04-26T23:15:34.368 INFO:teuthology.orchestra.run.smithi047.stdout:(5/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 5.4 MB/s | 5.4 MB 00:00 2024-04-26T23:15:34.410 INFO:teuthology.orchestra.run.smithi047.stdout:(6/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 6.5 MB/s | 1.5 MB 00:00 2024-04-26T23:15:34.944 INFO:teuthology.orchestra.run.smithi047.stdout:(7/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 8.2 MB/s | 4.7 MB 00:00 2024-04-26T23:15:35.876 INFO:teuthology.orchestra.run.smithi203.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 562 kB/s | 889 kB 00:01 2024-04-26T23:15:36.069 INFO:teuthology.orchestra.run.smithi047.stdout:(8/152): ceph-common-19.0.0-3306.gbc612ffb.el9. 7.9 MB/s | 21 MB 00:02 2024-04-26T23:15:36.076 INFO:teuthology.orchestra.run.smithi203.stdout:(3/152): ceph-immutable-object-cache-19.0.0-330 726 kB/s | 145 kB 00:00 2024-04-26T23:15:36.253 INFO:teuthology.orchestra.run.smithi047.stdout:(9/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x86 9.1 MB/s | 17 MB 00:01 2024-04-26T23:15:36.259 INFO:teuthology.orchestra.run.smithi203.stdout:(4/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 2.1 MB/s | 5.4 MB 00:02 2024-04-26T23:15:36.278 INFO:teuthology.orchestra.run.smithi047.stdout:(10/152): ceph-selinux-19.0.0-3306.gbc612ffb.el 121 kB/s | 25 kB 00:00 2024-04-26T23:15:36.386 INFO:teuthology.orchestra.run.smithi047.stdout:(11/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el 7.3 MB/s | 11 MB 00:01 2024-04-26T23:15:36.428 INFO:teuthology.orchestra.run.smithi047.stdout:(12/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 1.5 MB/s | 225 kB 00:00 2024-04-26T23:15:36.460 INFO:teuthology.orchestra.run.smithi203.stdout:(5/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 7.6 MB/s | 1.5 MB 00:00 2024-04-26T23:15:36.462 INFO:teuthology.orchestra.run.smithi047.stdout:(13/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 360 kB/s | 27 kB 00:00 2024-04-26T23:15:36.487 INFO:teuthology.orchestra.run.smithi047.stdout:(14/152): libcephfs-devel-19.0.0-3306.gbc612ffb 541 kB/s | 32 kB 00:00 2024-04-26T23:15:36.571 INFO:teuthology.orchestra.run.smithi047.stdout:(15/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 1.9 MB/s | 163 kB 00:00 2024-04-26T23:15:36.638 INFO:teuthology.orchestra.run.smithi047.stdout:(16/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 4.2 MB/s | 747 kB 00:00 2024-04-26T23:15:36.654 INFO:teuthology.orchestra.run.smithi047.stdout:(17/152): librados-devel-19.0.0-3306.gbc612ffb. 1.5 MB/s | 127 kB 00:00 2024-04-26T23:15:36.746 INFO:teuthology.orchestra.run.smithi047.stdout:(18/152): libradosstriper1-19.0.0-3306.gbc612ff 4.5 MB/s | 501 kB 00:00 2024-04-26T23:15:36.838 INFO:teuthology.orchestra.run.smithi047.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 492 kB/s | 45 kB 00:00 2024-04-26T23:15:36.947 INFO:teuthology.orchestra.run.smithi047.stdout:(20/152): python3-ceph-common-19.0.0-3306.gbc61 1.2 MB/s | 135 kB 00:00 2024-04-26T23:15:37.056 INFO:teuthology.orchestra.run.smithi047.stdout:(21/152): python3-cephfs-19.0.0-3306.gbc612ffb. 1.5 MB/s | 161 kB 00:00 2024-04-26T23:15:37.156 INFO:teuthology.orchestra.run.smithi047.stdout:(22/152): python3-rados-19.0.0-3306.gbc612ffb.e 3.1 MB/s | 321 kB 00:00 2024-04-26T23:15:37.177 INFO:teuthology.orchestra.run.smithi203.stdout:(6/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 6.6 MB/s | 4.7 MB 00:00 2024-04-26T23:15:37.257 INFO:teuthology.orchestra.run.smithi047.stdout:(23/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 2.9 MB/s | 298 kB 00:00 2024-04-26T23:15:37.374 INFO:teuthology.orchestra.run.smithi047.stdout:(24/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 852 kB/s | 99 kB 00:00 2024-04-26T23:15:37.524 INFO:teuthology.orchestra.run.smithi047.stdout:(25/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 6.2 MB/s | 5.4 MB 00:00 2024-04-26T23:15:37.549 INFO:teuthology.orchestra.run.smithi047.stdout:(26/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 486 kB/s | 85 kB 00:00 2024-04-26T23:15:37.667 INFO:teuthology.orchestra.run.smithi047.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 1.4 MB/s | 171 kB 00:00 2024-04-26T23:15:37.775 INFO:teuthology.orchestra.run.smithi047.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3306.g 316 kB/s | 34 kB 00:00 2024-04-26T23:15:37.875 INFO:teuthology.orchestra.run.smithi047.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 1.4 MB/s | 142 kB 00:00 2024-04-26T23:15:37.992 INFO:teuthology.orchestra.run.smithi047.stdout:(30/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 6.6 MB/s | 3.1 MB 00:00 2024-04-26T23:15:38.334 INFO:teuthology.orchestra.run.smithi047.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 7.7 MB/s | 3.5 MB 00:00 2024-04-26T23:15:38.426 INFO:teuthology.orchestra.run.smithi047.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3306.gbc 2.7 MB/s | 249 kB 00:00 2024-04-26T23:15:38.510 INFO:teuthology.orchestra.run.smithi047.stdout:(33/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 593 kB/s | 49 kB 00:00 2024-04-26T23:15:38.593 INFO:teuthology.orchestra.run.smithi047.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3306.gb 197 kB/s | 16 kB 00:00 2024-04-26T23:15:38.685 INFO:teuthology.orchestra.run.smithi047.stdout:(35/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 2.9 MB/s | 273 kB 00:00 2024-04-26T23:15:38.802 INFO:teuthology.orchestra.run.smithi047.stdout:(36/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 6.4 MB/s | 762 kB 00:00 2024-04-26T23:15:38.886 INFO:teuthology.orchestra.run.smithi047.stdout:(37/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 219 kB/s | 18 kB 00:00 2024-04-26T23:15:38.944 INFO:teuthology.orchestra.run.smithi203.stdout:(7/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 843 kB/s | 2.4 MB 00:02 2024-04-26T23:15:39.053 INFO:teuthology.orchestra.run.smithi047.stdout:(38/152): ceph-mgr-diskprediction-local-19.0.0- 7.0 MB/s | 7.4 MB 00:01 2024-04-26T23:15:39.344 INFO:teuthology.orchestra.run.smithi203.stdout:(8/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x86 7.8 MB/s | 17 MB 00:02 2024-04-26T23:15:39.394 INFO:teuthology.orchestra.run.smithi203.stdout:(9/152): ceph-selinux-19.0.0-3306.gbc612ffb.el9 508 kB/s | 25 kB 00:00 2024-04-26T23:15:39.528 INFO:teuthology.orchestra.run.smithi047.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 125 kB/s | 80 kB 00:00 2024-04-26T23:15:40.287 INFO:teuthology.orchestra.run.smithi047.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 258 kB/s | 195 kB 00:00 2024-04-26T23:15:40.737 INFO:teuthology.orchestra.run.smithi047.stdout:(41/152): ceph-test-19.0.0-3306.gbc612ffb.el9.x 11 MB/s | 49 MB 00:04 2024-04-26T23:15:40.779 INFO:teuthology.orchestra.run.smithi047.stdout:(42/152): libgfortran-11.4.1-3.el9.x86_64.rpm 465 kB/s | 803 kB 00:01 2024-04-26T23:15:41.454 INFO:teuthology.orchestra.run.smithi047.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 158 kB/s | 106 kB 00:00 2024-04-26T23:15:41.554 INFO:teuthology.orchestra.run.smithi047.stdout:(44/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 200 kB/s | 253 kB 00:01 2024-04-26T23:15:42.188 INFO:teuthology.orchestra.run.smithi047.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 184 kB/s | 135 kB 00:00 2024-04-26T23:15:42.246 INFO:teuthology.orchestra.run.smithi047.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 180 kB/s | 125 kB 00:00 2024-04-26T23:15:42.530 INFO:teuthology.orchestra.run.smithi047.stdout:(47/152): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 689 kB 00:00 2024-04-26T23:15:42.605 INFO:teuthology.orchestra.run.smithi047.stdout:(48/152): python3-cryptography-36.0.1-4.el9.x86 672 kB/s | 1.2 MB 00:01 2024-04-26T23:15:42.780 INFO:teuthology.orchestra.run.smithi047.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 612 kB/s | 107 kB 00:00 2024-04-26T23:15:42.822 INFO:teuthology.orchestra.run.smithi047.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 518 kB/s | 21 kB 00:00 2024-04-26T23:15:42.872 INFO:teuthology.orchestra.run.smithi047.stdout:(51/152): bison-3.7.4-5.el9.x86_64.rpm 2.7 MB/s | 943 kB 00:00 2024-04-26T23:15:42.917 INFO:teuthology.orchestra.run.smithi047.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 784 kB/s | 33 kB 00:00 2024-04-26T23:15:42.940 INFO:teuthology.orchestra.run.smithi047.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-04-26T23:15:42.977 INFO:teuthology.orchestra.run.smithi203.stdout:(10/152): ceph-test-19.0.0-3306.gbc612ffb.el9.x 14 MB/s | 49 MB 00:03 2024-04-26T23:15:42.982 INFO:teuthology.orchestra.run.smithi047.stdout:(54/152): python3-urllib3-1.26.5-5.el9.noarch.r 270 kB/s | 215 kB 00:00 2024-04-26T23:15:43.007 INFO:teuthology.orchestra.run.smithi047.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 270 kB/s | 18 kB 00:00 2024-04-26T23:15:43.044 INFO:teuthology.orchestra.run.smithi203.stdout:(11/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 3.3 MB/s | 225 kB 00:00 2024-04-26T23:15:43.066 INFO:teuthology.orchestra.run.smithi047.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 133 kB/s | 11 kB 00:00 2024-04-26T23:15:43.086 INFO:teuthology.orchestra.run.smithi203.stdout:(12/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 651 kB/s | 27 kB 00:00 2024-04-26T23:15:43.091 INFO:teuthology.orchestra.run.smithi047.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-04-26T23:15:43.116 INFO:teuthology.orchestra.run.smithi047.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 903 kB/s | 45 kB 00:00 2024-04-26T23:15:43.128 INFO:teuthology.orchestra.run.smithi203.stdout:(13/152): libcephfs-devel-19.0.0-3306.gbc612ffb 763 kB/s | 32 kB 00:00 2024-04-26T23:15:43.220 INFO:teuthology.orchestra.run.smithi203.stdout:(14/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 8.0 MB/s | 747 kB 00:00 2024-04-26T23:15:43.258 INFO:teuthology.orchestra.run.smithi047.stdout:(59/152): libtool-2.4.6-45.el9.x86_64.rpm 4.0 MB/s | 582 kB 00:00 2024-04-26T23:15:43.270 INFO:teuthology.orchestra.run.smithi203.stdout:(15/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 3.2 MB/s | 163 kB 00:00 2024-04-26T23:15:43.321 INFO:teuthology.orchestra.run.smithi203.stdout:(16/152): librados-devel-19.0.0-3306.gbc612ffb. 2.5 MB/s | 127 kB 00:00 2024-04-26T23:15:43.341 INFO:teuthology.orchestra.run.smithi047.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 2.6 MB/s | 662 kB 00:00 2024-04-26T23:15:43.388 INFO:teuthology.orchestra.run.smithi203.stdout:(17/152): libradosstriper1-19.0.0-3306.gbc612ff 7.3 MB/s | 501 kB 00:00 2024-04-26T23:15:43.392 INFO:teuthology.orchestra.run.smithi047.stdout:(61/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 950 kB/s | 47 kB 00:00 2024-04-26T23:15:43.417 INFO:teuthology.orchestra.run.smithi047.stdout:(62/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 160 kB/s | 25 kB 00:00 2024-04-26T23:15:43.501 INFO:teuthology.orchestra.run.smithi047.stdout:(63/152): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-04-26T23:15:43.534 INFO:teuthology.orchestra.run.smithi047.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.0 MB/s | 292 kB 00:00 2024-04-26T23:15:43.560 INFO:teuthology.orchestra.run.smithi047.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 601 kB/s | 35 kB 00:00 2024-04-26T23:15:43.730 INFO:teuthology.orchestra.run.smithi203.stdout:(18/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 16 MB/s | 5.4 MB 00:00 2024-04-26T23:15:43.772 INFO:teuthology.orchestra.run.smithi203.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 1.1 MB/s | 45 kB 00:00 2024-04-26T23:15:43.822 INFO:teuthology.orchestra.run.smithi203.stdout:(20/152): python3-ceph-common-19.0.0-3306.gbc61 2.6 MB/s | 135 kB 00:00 2024-04-26T23:15:43.852 INFO:teuthology.orchestra.run.smithi047.stdout:(66/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.2 MB/s | 3.0 MB 00:00 2024-04-26T23:15:43.872 INFO:teuthology.orchestra.run.smithi203.stdout:(21/152): python3-cephfs-19.0.0-3306.gbc612ffb. 3.2 MB/s | 161 kB 00:00 2024-04-26T23:15:43.931 INFO:teuthology.orchestra.run.smithi203.stdout:(22/152): python3-rados-19.0.0-3306.gbc612ffb.e 5.4 MB/s | 321 kB 00:00 2024-04-26T23:15:44.048 INFO:teuthology.orchestra.run.smithi203.stdout:(23/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el 2.1 MB/s | 11 MB 00:05 2024-04-26T23:15:44.081 INFO:teuthology.orchestra.run.smithi203.stdout:(24/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 1.9 MB/s | 298 kB 00:00 2024-04-26T23:15:44.115 INFO:teuthology.orchestra.run.smithi203.stdout:(25/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 1.5 MB/s | 99 kB 00:00 2024-04-26T23:15:44.141 INFO:teuthology.orchestra.run.smithi203.stdout:(26/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 1.4 MB/s | 85 kB 00:00 2024-04-26T23:15:44.191 INFO:teuthology.orchestra.run.smithi203.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 3.3 MB/s | 171 kB 00:00 2024-04-26T23:15:44.233 INFO:teuthology.orchestra.run.smithi203.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3306.g 822 kB/s | 34 kB 00:00 2024-04-26T23:15:44.260 INFO:teuthology.orchestra.run.smithi047.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 3.9 MB/s | 1.6 MB 00:00 2024-04-26T23:15:44.283 INFO:teuthology.orchestra.run.smithi203.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 2.8 MB/s | 142 kB 00:00 2024-04-26T23:15:44.477 INFO:teuthology.orchestra.run.smithi047.stdout:(68/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 4.8 MB/s | 4.4 MB 00:00 2024-04-26T23:15:44.536 INFO:teuthology.orchestra.run.smithi047.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 3.2 MB/s | 911 kB 00:00 2024-04-26T23:15:44.542 INFO:teuthology.orchestra.run.smithi203.stdout:(30/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 14 MB/s | 3.5 MB 00:00 2024-04-26T23:15:44.561 INFO:teuthology.orchestra.run.smithi047.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.6 MB/s | 133 kB 00:00 2024-04-26T23:15:44.769 INFO:teuthology.orchestra.run.smithi047.stdout:(71/152): openblas-openmp-0.3.21-2.el9.x86_64.r 3.8 MB/s | 4.7 MB 00:01 2024-04-26T23:15:44.803 INFO:teuthology.orchestra.run.smithi047.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 4.2 MB/s | 1.0 MB 00:00 2024-04-26T23:15:44.828 INFO:teuthology.orchestra.run.smithi047.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-26T23:15:44.912 INFO:teuthology.orchestra.run.smithi047.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 2.9 MB/s | 244 kB 00:00 2024-04-26T23:15:44.995 INFO:teuthology.orchestra.run.smithi047.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-04-26T23:15:45.026 INFO:teuthology.orchestra.run.smithi203.stdout:(31/152): ceph-mgr-diskprediction-local-19.0.0- 15 MB/s | 7.4 MB 00:00 2024-04-26T23:15:45.046 INFO:teuthology.orchestra.run.smithi047.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 513 kB/s | 26 kB 00:00 2024-04-26T23:15:45.076 INFO:teuthology.orchestra.run.smithi203.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3306.gbc 4.9 MB/s | 249 kB 00:00 2024-04-26T23:15:45.121 INFO:teuthology.orchestra.run.smithi047.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 255 kB/s | 19 kB 00:00 2024-04-26T23:15:45.135 INFO:teuthology.orchestra.run.smithi203.stdout:(33/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 3.0 MB/s | 3.1 MB 00:01 2024-04-26T23:15:45.161 INFO:teuthology.orchestra.run.smithi203.stdout:(34/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 589 kB/s | 49 kB 00:00 2024-04-26T23:15:45.194 INFO:teuthology.orchestra.run.smithi203.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3306.gb 281 kB/s | 16 kB 00:00 2024-04-26T23:15:45.196 INFO:teuthology.orchestra.run.smithi047.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-04-26T23:15:45.219 INFO:teuthology.orchestra.run.smithi203.stdout:(36/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 4.6 MB/s | 273 kB 00:00 2024-04-26T23:15:45.246 INFO:teuthology.orchestra.run.smithi047.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 698 kB/s | 35 kB 00:00 2024-04-26T23:15:45.261 INFO:teuthology.orchestra.run.smithi203.stdout:(37/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 437 kB/s | 18 kB 00:00 2024-04-26T23:15:45.512 INFO:teuthology.orchestra.run.smithi203.stdout:(38/152): ceph-common-19.0.0-3306.gbc612ffb.el9 1.8 MB/s | 21 MB 00:11 2024-04-26T23:15:45.537 INFO:teuthology.orchestra.run.smithi203.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 292 kB/s | 80 kB 00:00 2024-04-26T23:15:45.562 INFO:teuthology.orchestra.run.smithi203.stdout:(40/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 2.0 MB/s | 762 kB 00:00 2024-04-26T23:15:45.679 INFO:teuthology.orchestra.run.smithi203.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 195 kB 00:00 2024-04-26T23:15:45.729 INFO:teuthology.orchestra.run.smithi203.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.5 MB/s | 253 kB 00:00 2024-04-26T23:15:45.755 INFO:teuthology.orchestra.run.smithi047.stdout:(80/152): python3-babel-2.9.1-2.el9.noarch.rpm 6.3 MB/s | 6.0 MB 00:00 2024-04-26T23:15:45.788 INFO:teuthology.orchestra.run.smithi203.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-26T23:15:45.822 INFO:teuthology.orchestra.run.smithi203.stdout:(44/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.5 MB/s | 803 kB 00:00 2024-04-26T23:15:45.838 INFO:teuthology.orchestra.run.smithi047.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.9 MB/s | 250 kB 00:00 2024-04-26T23:15:45.872 INFO:teuthology.orchestra.run.smithi203.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-04-26T23:15:45.897 INFO:teuthology.orchestra.run.smithi203.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-04-26T23:15:45.956 INFO:teuthology.orchestra.run.smithi203.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-04-26T23:15:45.998 INFO:teuthology.orchestra.run.smithi203.stdout:(48/152): python3-cryptography-36.0.1-4.el9.x86 3.9 MB/s | 1.2 MB 00:00 2024-04-26T23:15:46.039 INFO:teuthology.orchestra.run.smithi047.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 1.1 MB/s | 222 kB 00:00 2024-04-26T23:15:46.089 INFO:teuthology.orchestra.run.smithi047.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-26T23:15:46.148 INFO:teuthology.orchestra.run.smithi047.stdout:(84/152): python3-prettytable-0.7.2-27.el9.noar 726 kB/s | 42 kB 00:00 2024-04-26T23:15:46.232 INFO:teuthology.orchestra.run.smithi047.stdout:(85/152): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 269 kB 00:00 2024-04-26T23:15:46.290 INFO:teuthology.orchestra.run.smithi047.stdout:(86/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-04-26T23:15:46.391 INFO:teuthology.orchestra.run.smithi047.stdout:(87/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.5 MB/s | 5.1 MB 00:01 2024-04-26T23:15:46.458 INFO:teuthology.orchestra.run.smithi047.stdout:(88/152): postgresql-server-13.14-1.el9.x86_64. 3.0 MB/s | 5.8 MB 00:01 2024-04-26T23:15:46.483 INFO:teuthology.orchestra.run.smithi047.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 555 kB/s | 51 kB 00:00 2024-04-26T23:15:46.508 INFO:teuthology.orchestra.run.smithi047.stdout:(90/152): python3-pyasn1-modules-0.4.8-6.el9.no 1.3 MB/s | 279 kB 00:00 2024-04-26T23:15:46.533 INFO:teuthology.orchestra.run.smithi047.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 710 kB/s | 54 kB 00:00 2024-04-26T23:15:46.558 INFO:teuthology.orchestra.run.smithi047.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 833 kB/s | 42 kB 00:00 2024-04-26T23:15:46.584 INFO:teuthology.orchestra.run.smithi047.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 882 kB/s | 44 kB 00:00 2024-04-26T23:15:46.618 INFO:teuthology.orchestra.run.smithi047.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-04-26T23:15:46.657 INFO:teuthology.orchestra.run.smithi203.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 162 kB/s | 107 kB 00:00 2024-04-26T23:15:46.703 INFO:teuthology.orchestra.run.smithi047.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 303 kB/s | 36 kB 00:00 2024-04-26T23:15:46.726 INFO:teuthology.orchestra.run.smithi047.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 436 kB/s | 47 kB 00:00 2024-04-26T23:15:46.777 INFO:teuthology.orchestra.run.smithi047.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-26T23:15:46.827 INFO:teuthology.orchestra.run.smithi047.stdout:(98/152): socat-1.7.4.1-5.el9.x86_64.rpm 2.4 MB/s | 305 kB 00:00 2024-04-26T23:15:46.853 INFO:teuthology.orchestra.run.smithi047.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.2 MB/s | 94 kB 00:00 2024-04-26T23:15:47.003 INFO:teuthology.orchestra.run.smithi047.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 364 kB/s | 64 kB 00:00 2024-04-26T23:15:47.019 INFO:teuthology.orchestra.run.smithi047.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 268 kB/s | 45 kB 00:00 2024-04-26T23:15:47.128 INFO:teuthology.orchestra.run.smithi047.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 178 kB/s | 22 kB 00:00 2024-04-26T23:15:47.165 INFO:teuthology.orchestra.run.smithi203.stdout:(50/152): automake-1.16.2-8.el9.noarch.rpm 544 kB/s | 689 kB 00:01 2024-04-26T23:15:47.199 INFO:teuthology.orchestra.run.smithi203.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 40 kB/s | 21 kB 00:00 2024-04-26T23:15:47.287 INFO:teuthology.orchestra.run.smithi047.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 3.2 MB/s | 863 kB 00:00 2024-04-26T23:15:47.466 INFO:teuthology.orchestra.run.smithi203.stdout:(52/152): bison-3.7.4-5.el9.x86_64.rpm 625 kB/s | 943 kB 00:01 2024-04-26T23:15:47.716 INFO:teuthology.orchestra.run.smithi203.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 63 kB/s | 33 kB 00:00 2024-04-26T23:15:48.058 INFO:teuthology.orchestra.run.smithi203.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 352 kB/s | 313 kB 00:00 2024-04-26T23:15:48.138 INFO:teuthology.orchestra.run.smithi047.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 363 kB/s | 309 kB 00:00 2024-04-26T23:15:48.175 INFO:teuthology.orchestra.run.smithi203.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 40 kB/s | 18 kB 00:00 2024-04-26T23:15:48.246 INFO:teuthology.orchestra.run.smithi047.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 180 kB/s | 19 kB 00:00 2024-04-26T23:15:48.321 INFO:teuthology.orchestra.run.smithi047.stdout:(106/152): abseil-cpp-20211102.0-3.el9.x86_64.r 460 kB/s | 548 kB 00:01 2024-04-26T23:15:48.421 INFO:teuthology.orchestra.run.smithi047.stdout:(107/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 260 kB/s | 26 kB 00:00 2024-04-26T23:15:48.509 INFO:teuthology.orchestra.run.smithi203.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 25 kB/s | 11 kB 00:00 2024-04-26T23:15:48.521 INFO:teuthology.orchestra.run.smithi047.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 486 kB/s | 49 kB 00:00 2024-04-26T23:15:48.705 INFO:teuthology.orchestra.run.smithi047.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 367 kB/s | 67 kB 00:00 2024-04-26T23:15:48.875 INFO:teuthology.orchestra.run.smithi203.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 229 kB/s | 160 kB 00:00 2024-04-26T23:15:48.963 INFO:teuthology.orchestra.run.smithi047.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 585 kB/s | 151 kB 00:00 2024-04-26T23:15:49.076 INFO:teuthology.orchestra.run.smithi203.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 80 kB/s | 45 kB 00:00 2024-04-26T23:15:49.989 INFO:teuthology.orchestra.run.smithi047.stdout:(111/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 4.6 MB/s | 16 MB 00:03 2024-04-26T23:15:50.097 INFO:teuthology.orchestra.run.smithi047.stdout:(112/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 740 kB/s | 839 kB 00:01 2024-04-26T23:15:50.134 INFO:teuthology.orchestra.run.smithi203.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 526 kB/s | 662 kB 00:01 2024-04-26T23:15:50.231 INFO:teuthology.orchestra.run.smithi047.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 415 kB/s | 43 kB 00:00 2024-04-26T23:15:50.251 INFO:teuthology.orchestra.run.smithi203.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 507 kB/s | 582 kB 00:01 2024-04-26T23:15:50.331 INFO:teuthology.orchestra.run.smithi047.stdout:(114/152): python3-cachetools-4.2.4-1.el9.noarc 326 kB/s | 32 kB 00:00 2024-04-26T23:15:50.423 INFO:teuthology.orchestra.run.smithi047.stdout:(115/152): python3-certifi-2021.10.8-2.el9.noar 163 kB/s | 15 kB 00:00 2024-04-26T23:15:50.652 INFO:teuthology.orchestra.run.smithi203.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 49 kB/s | 25 kB 00:00 2024-04-26T23:15:50.692 INFO:teuthology.orchestra.run.smithi047.stdout:(116/152): python3-cheroot-8.6.0-4.el9.noarch.r 644 kB/s | 172 kB 00:00 2024-04-26T23:15:50.833 INFO:teuthology.orchestra.run.smithi203.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 82 kB/s | 47 kB 00:00 2024-04-26T23:15:51.191 INFO:teuthology.orchestra.run.smithi047.stdout:(117/152): python3-cherrypy-18.6.1-2.el9.noarch 717 kB/s | 358 kB 00:00 2024-04-26T23:15:51.216 INFO:teuthology.orchestra.run.smithi047.stdout:(118/152): python3-asyncssh-2.13.2-5.el9.noarch 447 kB/s | 548 kB 00:01 2024-04-26T23:15:51.252 INFO:teuthology.orchestra.run.smithi203.stdout:(63/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 809 kB/s | 3.0 MB 00:03 2024-04-26T23:15:51.316 INFO:teuthology.orchestra.run.smithi047.stdout:(119/152): python3-colorama-0.4.4-12.el9.noarch 354 kB/s | 35 kB 00:00 2024-04-26T23:15:51.502 INFO:teuthology.orchestra.run.smithi203.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 344 kB/s | 292 kB 00:00 2024-04-26T23:15:51.616 INFO:teuthology.orchestra.run.smithi047.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 712 kB/s | 303 kB 00:00 2024-04-26T23:15:51.619 INFO:teuthology.orchestra.run.smithi203.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 238 kB/s | 188 kB 00:00 2024-04-26T23:15:51.658 INFO:teuthology.orchestra.run.smithi047.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 601 kB/s | 206 kB 00:00 2024-04-26T23:15:51.811 INFO:teuthology.orchestra.run.smithi203.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 63 kB/s | 35 kB 00:00 2024-04-26T23:15:51.967 INFO:teuthology.orchestra.run.smithi047.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 469 kB/s | 144 kB 00:00 2024-04-26T23:15:52.084 INFO:teuthology.orchestra.run.smithi047.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 92 kB/s | 11 kB 00:00 2024-04-26T23:15:52.201 INFO:teuthology.orchestra.run.smithi047.stdout:(124/152): python3-jaraco-classes-3.2.1-5.el9.n 151 kB/s | 18 kB 00:00 2024-04-26T23:15:52.309 INFO:teuthology.orchestra.run.smithi047.stdout:(125/152): python3-jaraco-collections-3.0.0-8.e 214 kB/s | 23 kB 00:00 2024-04-26T23:15:52.418 INFO:teuthology.orchestra.run.smithi047.stdout:(126/152): python3-jaraco-functools-3.5.0-2.el9 180 kB/s | 19 kB 00:00 2024-04-26T23:15:52.518 INFO:teuthology.orchestra.run.smithi047.stdout:(127/152): python3-jaraco-text-3.2.0-6.el9.noar 197 kB/s | 20 kB 00:00 2024-04-26T23:15:53.660 INFO:teuthology.orchestra.run.smithi047.stdout:(128/152): libarrow-9.0.0-11.el9.x86_64.rpm 835 kB/s | 4.4 MB 00:05 2024-04-26T23:15:53.769 INFO:teuthology.orchestra.run.smithi047.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 428 kB/s | 46 kB 00:00 2024-04-26T23:15:53.852 INFO:teuthology.orchestra.run.smithi047.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 784 kB/s | 1.0 MB 00:01 2024-04-26T23:15:53.962 INFO:teuthology.orchestra.run.smithi047.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 411 kB/s | 79 kB 00:00 2024-04-26T23:15:54.036 INFO:teuthology.orchestra.run.smithi047.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 315 kB/s | 58 kB 00:00 2024-04-26T23:15:54.120 INFO:teuthology.orchestra.run.smithi203.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 705 kB/s | 1.6 MB 00:02 2024-04-26T23:15:54.136 INFO:teuthology.orchestra.run.smithi047.stdout:(133/152): python3-portend-3.1.0-2.el9.noarch.r 164 kB/s | 16 kB 00:00 2024-04-26T23:15:54.262 INFO:teuthology.orchestra.run.smithi047.stdout:(134/152): python3-grpcio-1.46.7-10.el9.x86_64. 789 kB/s | 2.0 MB 00:02 2024-04-26T23:15:54.320 INFO:teuthology.orchestra.run.smithi047.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 490 kB/s | 90 kB 00:00 2024-04-26T23:15:54.362 INFO:teuthology.orchestra.run.smithi047.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 282 kB/s | 28 kB 00:00 2024-04-26T23:15:54.405 INFO:teuthology.orchestra.run.smithi047.stdout:(137/152): python3-pecan-1.4.2-3.el9.noarch.rpm 615 kB/s | 272 kB 00:00 2024-04-26T23:15:54.429 INFO:teuthology.orchestra.run.smithi047.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 283 kB/s | 31 kB 00:00 2024-04-26T23:15:54.538 INFO:teuthology.orchestra.run.smithi047.stdout:(139/152): python3-tempora-5.0.0-2.el9.noarch.r 330 kB/s | 36 kB 00:00 2024-04-26T23:15:54.588 INFO:teuthology.orchestra.run.smithi047.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 322 kB/s | 59 kB 00:00 2024-04-26T23:15:54.647 INFO:teuthology.orchestra.run.smithi047.stdout:(141/152): python3-routes-2.5.1-5.el9.noarch.rp 663 kB/s | 188 kB 00:00 2024-04-26T23:15:54.672 INFO:teuthology.orchestra.run.smithi047.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 381 kB/s | 51 kB 00:00 2024-04-26T23:15:54.881 INFO:teuthology.orchestra.run.smithi047.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 384 kB/s | 90 kB 00:00 2024-04-26T23:15:54.998 INFO:teuthology.orchestra.run.smithi047.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 561 kB/s | 230 kB 00:00 2024-04-26T23:15:55.023 INFO:teuthology.orchestra.run.smithi047.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 140 kB/s | 20 kB 00:00 2024-04-26T23:15:55.298 INFO:teuthology.orchestra.run.smithi047.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 639 kB/s | 192 kB 00:00 2024-04-26T23:15:55.382 INFO:teuthology.orchestra.run.smithi047.stdout:(147/152): python3-werkzeug-2.0.3-3.el9.1.noarc 602 kB/s | 427 kB 00:00 2024-04-26T23:15:55.599 INFO:teuthology.orchestra.run.smithi047.stdout:(148/152): librados2-19.0.0-3306.gbc612ffb.el9. 11 MB/s | 3.4 MB 00:00 2024-04-26T23:15:55.854 INFO:teuthology.orchestra.run.smithi203.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 526 kB/s | 911 kB 00:01 2024-04-26T23:15:56.066 INFO:teuthology.orchestra.run.smithi047.stdout:(149/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 4.6 MB/s | 3.1 MB 00:00 2024-04-26T23:15:56.770 INFO:teuthology.orchestra.run.smithi203.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 145 kB/s | 133 kB 00:00 2024-04-26T23:15:57.533 INFO:teuthology.orchestra.run.smithi047.stdout:(150/152): thrift-0.15.0-2.el9.x86_64.rpm 645 kB/s | 1.6 MB 00:02 2024-04-26T23:15:57.778 INFO:teuthology.orchestra.run.smithi203.stdout:(70/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 736 kB/s | 4.4 MB 00:06 2024-04-26T23:15:58.229 INFO:teuthology.orchestra.run.smithi203.stdout:(71/152): openblas-openmp-0.3.21-2.el9.x86_64.r 721 kB/s | 4.7 MB 00:06 2024-04-26T23:15:58.324 INFO:teuthology.orchestra.run.smithi047.stdout:(151/152): openssl-3.2.1-1.el9.x86_64.rpm 507 kB/s | 1.3 MB 00:02 2024-04-26T23:15:59.079 INFO:teuthology.orchestra.run.smithi203.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 103 kB/s | 87 kB 00:00 2024-04-26T23:16:00.029 INFO:teuthology.orchestra.run.smithi203.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 459 kB/s | 1.0 MB 00:02 2024-04-26T23:16:00.449 INFO:teuthology.orchestra.run.smithi047.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 559 kB/s | 2.4 MB 00:04 2024-04-26T23:16:00.456 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-04-26T23:16:00.456 INFO:teuthology.orchestra.run.smithi047.stdout:Total 7.9 MB/s | 222 MB 00:28 2024-04-26T23:16:01.212 INFO:teuthology.orchestra.run.smithi203.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 206 kB/s | 244 kB 00:01 2024-04-26T23:16:01.588 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-04-26T23:16:01.749 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-04-26T23:16:01.749 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-04-26T23:16:02.029 INFO:teuthology.orchestra.run.smithi203.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 304 kB/s | 248 kB 00:00 2024-04-26T23:16:02.538 INFO:teuthology.orchestra.run.smithi203.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 50 kB/s | 26 kB 00:00 2024-04-26T23:16:03.180 INFO:teuthology.orchestra.run.smithi203.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 30 kB/s | 19 kB 00:00 2024-04-26T23:16:03.625 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-04-26T23:16:03.625 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-04-26T23:16:03.964 INFO:teuthology.orchestra.run.smithi203.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 220 kB/s | 172 kB 00:00 2024-04-26T23:16:04.497 INFO:teuthology.orchestra.run.smithi203.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 65 kB/s | 35 kB 00:00 2024-04-26T23:16:05.197 INFO:teuthology.orchestra.run.smithi203.stdout:(80/152): postgresql-server-13.14-1.el9.x86_64. 702 kB/s | 5.8 MB 00:08 2024-04-26T23:16:06.039 INFO:teuthology.orchestra.run.smithi203.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 298 kB/s | 250 kB 00:00 2024-04-26T23:16:06.806 INFO:teuthology.orchestra.run.smithi203.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 290 kB/s | 222 kB 00:00 2024-04-26T23:16:07.265 INFO:teuthology.orchestra.run.smithi203.stdout:(83/152): python3-babel-2.9.1-2.el9.noarch.rpm 746 kB/s | 6.0 MB 00:08 2024-04-26T23:16:07.450 INFO:teuthology.orchestra.run.smithi203.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 120 kB/s | 77 kB 00:00 2024-04-26T23:16:07.815 INFO:teuthology.orchestra.run.smithi203.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 77 kB/s | 42 kB 00:00 2024-04-26T23:16:08.299 INFO:teuthology.orchestra.run.smithi203.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 317 kB/s | 269 kB 00:00 2024-04-26T23:16:08.507 INFO:teuthology.orchestra.run.smithi203.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 230 kB/s | 159 kB 00:00 2024-04-26T23:16:09.116 INFO:teuthology.orchestra.run.smithi203.stdout:(88/152): python3-pytz-2021.1-5.el9.noarch.rpm 88 kB/s | 51 kB 00:00 2024-04-26T23:16:09.150 INFO:teuthology.orchestra.run.smithi203.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 328 kB/s | 279 kB 00:00 2024-04-26T23:16:09.708 INFO:teuthology.orchestra.run.smithi203.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 91 kB/s | 54 kB 00:00 2024-04-26T23:16:09.864 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-04-26T23:16:09.970 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T23:16:10.009 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T23:16:10.062 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T23:16:10.096 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T23:16:10.238 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T23:16:10.282 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T23:16:10.300 INFO:teuthology.orchestra.run.smithi203.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 70 kB/s | 42 kB 00:00 2024-04-26T23:16:10.628 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T23:16:10.692 INFO:teuthology.orchestra.run.smithi203.stdout:(92/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 843 kB/s | 5.1 MB 00:06 2024-04-26T23:16:10.739 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T23:16:10.863 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T23:16:10.889 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T23:16:10.892 INFO:teuthology.orchestra.run.smithi203.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 75 kB/s | 44 kB 00:00 2024-04-26T23:16:11.091 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T23:16:11.136 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T23:16:11.294 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T23:16:11.341 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T23:16:11.369 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T23:16:11.451 INFO:teuthology.orchestra.run.smithi203.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 67 kB/s | 36 kB 00:00 2024-04-26T23:16:11.474 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T23:16:11.476 INFO:teuthology.orchestra.run.smithi203.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 282 kB/s | 221 kB 00:00 2024-04-26T23:16:11.503 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T23:16:11.566 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T23:16:11.587 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T23:16:11.613 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T23:16:11.806 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T23:16:11.837 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T23:16:11.950 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T23:16:12.027 INFO:teuthology.orchestra.run.smithi203.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 82 kB/s | 47 kB 00:00 2024-04-26T23:16:12.044 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T23:16:12.203 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T23:16:12.238 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T23:16:12.315 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T23:16:12.352 INFO:teuthology.orchestra.run.smithi203.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 348 kB/s | 305 kB 00:00 2024-04-26T23:16:12.354 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T23:16:12.398 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T23:16:12.429 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T23:16:12.457 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T23:16:12.529 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T23:16:12.582 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T23:16:12.618 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T23:16:12.619 INFO:teuthology.orchestra.run.smithi203.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 98 kB/s | 58 kB 00:00 2024-04-26T23:16:12.654 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T23:16:12.734 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T23:16:12.773 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-26T23:16:12.817 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-26T23:16:12.845 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-26T23:16:12.918 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T23:16:12.969 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T23:16:12.986 INFO:teuthology.orchestra.run.smithi203.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 148 kB/s | 94 kB 00:00 2024-04-26T23:16:13.047 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T23:16:13.203 INFO:teuthology.orchestra.run.smithi203.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 109 kB/s | 64 kB 00:00 2024-04-26T23:16:13.250 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T23:16:13.305 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T23:16:13.562 INFO:teuthology.orchestra.run.smithi203.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 78 kB/s | 45 kB 00:00 2024-04-26T23:16:13.737 INFO:teuthology.orchestra.run.smithi203.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 44 kB/s | 22 kB 00:00 2024-04-26T23:16:13.864 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T23:16:13.952 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T23:16:13.998 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T23:16:14.038 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T23:16:14.046 INFO:teuthology.orchestra.run.smithi203.stdout:(103/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.7 MB/s | 548 kB 00:00 2024-04-26T23:16:14.076 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-26T23:16:14.119 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T23:16:14.129 INFO:teuthology.orchestra.run.smithi203.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 3.6 MB/s | 309 kB 00:00 2024-04-26T23:16:14.197 INFO:teuthology.orchestra.run.smithi203.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 291 kB/s | 19 kB 00:00 2024-04-26T23:16:14.343 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T23:16:14.422 INFO:teuthology.orchestra.run.smithi203.stdout:(106/152): libarrow-9.0.0-11.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-04-26T23:16:14.480 INFO:teuthology.orchestra.run.smithi203.stdout:(107/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 445 kB/s | 26 kB 00:00 2024-04-26T23:16:14.539 INFO:teuthology.orchestra.run.smithi203.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 834 kB/s | 49 kB 00:00 2024-04-26T23:16:14.579 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T23:16:14.598 INFO:teuthology.orchestra.run.smithi203.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-04-26T23:16:14.636 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T23:16:14.656 INFO:teuthology.orchestra.run.smithi203.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-04-26T23:16:14.709 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T23:16:14.732 INFO:teuthology.orchestra.run.smithi203.stdout:(111/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-04-26T23:16:14.746 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T23:16:14.783 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T23:16:14.815 INFO:teuthology.orchestra.run.smithi203.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 6.5 MB/s | 548 kB 00:00 2024-04-26T23:16:14.831 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T23:16:14.866 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T23:16:14.866 INFO:teuthology.orchestra.run.smithi203.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 867 kB/s | 43 kB 00:00 2024-04-26T23:16:14.909 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T23:16:14.916 INFO:teuthology.orchestra.run.smithi203.stdout:(114/152): python3-cachetools-4.2.4-1.el9.noarc 644 kB/s | 32 kB 00:00 2024-04-26T23:16:14.951 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T23:16:14.983 INFO:teuthology.orchestra.run.smithi203.stdout:(115/152): python3-certifi-2021.10.8-2.el9.noar 224 kB/s | 15 kB 00:00 2024-04-26T23:16:15.033 INFO:teuthology.orchestra.run.smithi203.stdout:(116/152): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-04-26T23:16:15.092 INFO:teuthology.orchestra.run.smithi203.stdout:(117/152): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-04-26T23:16:15.097 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T23:16:15.134 INFO:teuthology.orchestra.run.smithi203.stdout:(118/152): protobuf-compiler-3.14.0-13.el9.x86_ 549 kB/s | 863 kB 00:01 2024-04-26T23:16:15.140 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T23:16:15.159 INFO:teuthology.orchestra.run.smithi203.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.5 MB/s | 303 kB 00:00 2024-04-26T23:16:15.170 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T23:16:15.212 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T23:16:15.226 INFO:teuthology.orchestra.run.smithi203.stdout:(120/152): python3-google-auth-2.29.0-1.el9.noa 3.0 MB/s | 206 kB 00:00 2024-04-26T23:16:15.260 INFO:teuthology.orchestra.run.smithi203.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 281 kB/s | 35 kB 00:00 2024-04-26T23:16:15.311 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T23:16:15.327 INFO:teuthology.orchestra.run.smithi203.stdout:(122/152): python3-grpcio-1.46.7-10.el9.x86_64. 20 MB/s | 2.0 MB 00:00 2024-04-26T23:16:15.345 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T23:16:15.377 INFO:teuthology.orchestra.run.smithi203.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 214 kB/s | 11 kB 00:00 2024-04-26T23:16:15.385 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T23:16:15.418 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T23:16:15.419 INFO:teuthology.orchestra.run.smithi203.stdout:(124/152): python3-grpcio-tools-1.46.7-10.el9.x 908 kB/s | 144 kB 00:00 2024-04-26T23:16:15.443 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T23:16:15.444 INFO:teuthology.orchestra.run.smithi203.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 263 kB/s | 18 kB 00:00 2024-04-26T23:16:15.478 INFO:teuthology.orchestra.run.smithi203.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 394 kB/s | 23 kB 00:00 2024-04-26T23:16:15.504 INFO:teuthology.orchestra.run.smithi203.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 330 kB/s | 19 kB 00:00 2024-04-26T23:16:15.522 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T23:16:15.537 INFO:teuthology.orchestra.run.smithi203.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 336 kB/s | 20 kB 00:00 2024-04-26T23:16:15.596 INFO:teuthology.orchestra.run.smithi203.stdout:(129/152): python3-kubernetes-26.1.0-2.el9.noar 11 MB/s | 1.0 MB 00:00 2024-04-26T23:16:15.621 INFO:teuthology.orchestra.run.smithi203.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 553 kB/s | 46 kB 00:00 2024-04-26T23:16:15.647 INFO:teuthology.orchestra.run.smithi203.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-04-26T23:16:15.659 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T23:16:15.689 INFO:teuthology.orchestra.run.smithi203.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 861 kB/s | 58 kB 00:00 2024-04-26T23:16:15.700 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T23:16:15.714 INFO:teuthology.orchestra.run.smithi203.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-04-26T23:16:15.728 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T23:16:15.739 INFO:teuthology.orchestra.run.smithi203.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-04-26T23:16:15.762 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T23:16:15.773 INFO:teuthology.orchestra.run.smithi203.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-04-26T23:16:15.790 INFO:teuthology.orchestra.run.smithi203.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-04-26T23:16:15.802 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T23:16:15.832 INFO:teuthology.orchestra.run.smithi203.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 525 kB/s | 31 kB 00:00 2024-04-26T23:16:15.852 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T23:16:15.888 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T23:16:15.890 INFO:teuthology.orchestra.run.smithi203.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-04-26T23:16:15.916 INFO:teuthology.orchestra.run.smithi203.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 708 kB/s | 59 kB 00:00 2024-04-26T23:16:15.923 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T23:16:15.950 INFO:teuthology.orchestra.run.smithi203.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 610 kB/s | 36 kB 00:00 2024-04-26T23:16:15.956 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T23:16:15.983 INFO:teuthology.orchestra.run.smithi203.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 765 kB/s | 51 kB 00:00 2024-04-26T23:16:15.998 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T23:16:16.033 INFO:teuthology.orchestra.run.smithi203.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-04-26T23:16:16.059 INFO:teuthology.orchestra.run.smithi203.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-26T23:16:16.082 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T23:16:16.117 INFO:teuthology.orchestra.run.smithi203.stdout:(144/152): python3-zc-lockfile-2.0-10.el9.noarc 343 kB/s | 20 kB 00:00 2024-04-26T23:16:16.143 INFO:teuthology.orchestra.run.smithi203.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-04-26T23:16:16.164 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T23:16:16.176 INFO:teuthology.orchestra.run.smithi203.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 3.2 MB/s | 192 kB 00:00 2024-04-26T23:16:16.213 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T23:16:16.247 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T23:16:16.292 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T23:16:16.312 INFO:teuthology.orchestra.run.smithi203.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-26T23:16:16.403 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T23:16:16.455 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T23:16:16.594 INFO:teuthology.orchestra.run.smithi203.stdout:(148/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 10 MB/s | 3.1 MB 00:00 2024-04-26T23:16:17.035 INFO:teuthology.orchestra.run.smithi203.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 3.1 MB/s | 1.3 MB 00:00 2024-04-26T23:16:17.234 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T23:16:17.269 INFO:teuthology.orchestra.run.smithi203.stdout:(150/152): librados2-19.0.0-3306.gbc612ffb.el9. 3.1 MB/s | 3.4 MB 00:01 2024-04-26T23:16:17.283 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T23:16:17.317 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T23:16:17.421 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T23:16:17.476 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T23:16:17.566 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T23:16:17.611 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T23:16:17.744 INFO:teuthology.orchestra.run.smithi203.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 3.4 MB/s | 2.4 MB 00:00 2024-04-26T23:16:17.795 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T23:16:17.833 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T23:16:17.913 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T23:16:18.605 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T23:16:18.727 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T23:16:20.034 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T23:16:20.122 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T23:16:20.149 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T23:16:20.181 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T23:16:20.510 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T23:16:20.552 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T23:16:20.643 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T23:16:20.673 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-26T23:16:20.710 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T23:16:21.219 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T23:16:21.246 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T23:16:21.318 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T23:16:21.348 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-26T23:16:23.724 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T23:16:23.761 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T23:16:24.330 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T23:16:24.372 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T23:16:24.473 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T23:16:24.473 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T23:16:24.473 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T23:16:24.473 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:16:24.507 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T23:16:27.635 INFO:teuthology.orchestra.run.smithi203.stdout:(152/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 900 kB/s | 16 MB 00:18 2024-04-26T23:16:27.640 INFO:teuthology.orchestra.run.smithi203.stdout:-------------------------------------------------------------------------------- 2024-04-26T23:16:27.640 INFO:teuthology.orchestra.run.smithi203.stdout:Total 4.1 MB/s | 222 MB 00:54 2024-04-26T23:16:28.734 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction check 2024-04-26T23:16:28.878 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction check succeeded. 2024-04-26T23:16:28.878 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction test 2024-04-26T23:16:30.738 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction test succeeded. 2024-04-26T23:16:30.738 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction 2024-04-26T23:16:36.479 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T23:16:36.479 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /sys 2024-04-26T23:16:36.479 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /proc 2024-04-26T23:16:36.479 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /mnt 2024-04-26T23:16:36.480 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /var/tmp 2024-04-26T23:16:36.480 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /home 2024-04-26T23:16:36.480 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /root 2024-04-26T23:16:36.480 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /tmp 2024-04-26T23:16:36.480 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:16:36.652 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T23:16:36.668 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T23:16:36.669 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:16:36.669 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T23:16:36.669 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:16:36.715 INFO:teuthology.orchestra.run.smithi203.stdout: Preparing : 1/1 2024-04-26T23:16:36.805 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T23:16:36.844 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T23:16:36.906 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T23:16:36.940 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T23:16:36.965 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T23:16:37.008 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T23:16:37.339 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T23:16:37.458 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T23:16:37.579 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T23:16:37.601 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T23:16:37.692 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T23:16:37.722 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T23:16:37.877 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T23:16:37.928 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T23:16:37.955 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T23:16:38.059 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T23:16:38.082 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T23:16:38.144 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T23:16:38.166 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T23:16:38.187 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T23:16:38.206 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T23:16:38.232 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T23:16:38.232 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:16:38.232 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T23:16:38.233 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T23:16:38.233 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T23:16:38.233 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:16:38.383 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T23:16:38.416 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T23:16:38.464 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T23:16:38.488 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T23:16:38.488 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:16:38.488 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T23:16:38.488 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T23:16:38.489 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T23:16:38.489 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:16:38.524 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T23:16:38.621 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T23:16:38.657 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T23:16:38.692 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T23:16:38.775 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T23:16:38.816 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T23:16:38.859 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T23:16:38.892 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T23:16:38.920 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T23:16:38.957 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T23:16:38.981 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T23:16:38.981 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:16:38.981 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T23:16:38.981 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T23:16:38.981 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T23:16:38.981 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:16:38.990 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T23:16:39.051 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T23:16:39.081 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T23:16:39.096 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T23:16:39.116 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T23:16:39.154 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T23:16:39.194 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T23:16:39.218 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T23:16:39.227 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-26T23:16:39.271 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-26T23:16:39.300 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-26T23:16:39.380 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T23:16:39.421 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T23:16:39.423 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T23:16:39.466 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T23:16:39.499 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T23:16:39.546 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T23:16:39.636 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T23:16:39.643 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T23:16:39.702 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T23:16:39.788 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T23:16:39.833 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T23:16:39.865 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T23:16:39.892 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T23:16:40.239 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T23:16:40.321 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T23:16:40.322 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T23:16:40.349 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T23:16:40.350 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T23:16:40.381 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-26T23:16:40.393 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T23:16:40.431 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-26T23:16:40.454 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-26T23:16:40.473 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T23:16:40.480 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T23:16:40.628 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T23:16:40.685 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T23:16:40.741 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T23:16:40.814 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T23:16:40.844 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T23:16:40.872 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T23:16:40.919 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T23:16:40.955 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T23:16:40.999 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T23:16:41.041 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T23:16:41.178 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T23:16:41.222 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T23:16:41.252 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T23:16:41.294 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T23:16:41.389 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T23:16:41.427 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T23:16:41.467 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T23:16:41.492 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T23:16:41.517 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T23:16:41.579 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T23:16:41.593 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T23:16:41.616 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T23:16:41.720 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T23:16:41.759 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T23:16:41.803 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T23:16:41.837 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T23:16:41.877 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T23:16:41.917 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T23:16:41.946 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T23:16:41.981 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T23:16:42.014 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T23:16:42.056 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T23:16:42.091 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T23:16:42.178 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T23:16:42.230 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T23:16:42.261 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T23:16:42.325 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T23:16:42.425 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T23:16:42.480 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T23:16:42.630 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T23:16:42.657 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T23:16:42.767 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T23:16:42.934 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-26T23:16:42.987 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T23:16:43.011 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T23:16:43.011 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:16:43.012 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T23:16:43.012 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T23:16:43.012 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T23:16:43.012 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:16:43.060 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T23:16:43.071 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T23:16:43.172 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-26T23:16:43.219 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T23:16:43.226 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-26T23:16:43.260 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T23:16:43.294 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T23:16:43.397 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T23:16:43.453 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T23:16:43.541 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T23:16:43.579 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T23:16:43.757 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T23:16:43.785 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T23:16:43.863 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T23:16:44.201 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T23:16:44.229 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T23:16:44.253 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T23:16:44.253 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:16:44.253 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T23:16:44.253 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T23:16:44.253 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T23:16:44.253 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:16:44.296 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T23:16:44.319 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T23:16:44.319 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:16:44.319 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T23:16:44.319 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:16:44.372 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T23:16:44.395 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T23:16:44.395 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:16:44.396 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T23:16:44.396 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T23:16:44.396 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T23:16:44.396 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:16:44.527 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T23:16:44.653 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T23:16:44.714 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T23:16:44.738 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T23:16:44.738 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:16:44.738 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T23:16:44.738 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T23:16:44.738 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T23:16:44.738 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:16:45.910 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T23:16:46.084 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T23:16:46.111 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T23:16:46.144 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T23:16:46.459 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T23:16:46.516 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T23:16:46.606 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T23:16:46.637 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-26T23:16:46.673 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T23:16:47.153 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T23:16:47.186 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T23:16:47.257 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T23:16:47.279 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-26T23:16:49.601 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-26T23:16:49.643 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-26T23:16:49.684 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-26T23:16:49.716 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T23:16:49.743 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T23:16:49.844 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T23:16:49.884 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T23:16:49.937 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-26T23:16:49.981 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T23:16:50.010 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-26T23:16:50.431 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T23:16:50.472 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T23:16:50.568 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T23:16:50.569 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T23:16:50.569 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T23:16:50.569 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-26T23:16:50.598 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T23:16:50.648 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T23:16:50.691 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T23:16:50.711 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T23:16:51.626 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-26T23:16:51.720 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T23:16:51.762 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T23:16:51.846 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T23:16:51.885 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T23:16:51.885 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T23:16:51.969 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T23:16:52.002 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T23:16:52.002 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T23:16:52.094 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T23:16:52.162 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T23:16:57.807 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T23:16:57.807 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-26T23:16:57.807 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-26T23:16:57.808 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-26T23:16:57.808 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-26T23:16:57.808 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-26T23:16:57.808 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-26T23:16:57.808 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-26T23:16:57.808 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T23:16:57.808 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T23:16:57.808 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T23:16:57.808 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-26T23:16:57.808 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-26T23:16:57.808 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T23:16:57.809 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-26T23:16:57.809 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-26T23:16:57.809 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-26T23:16:57.809 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-26T23:16:57.809 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-26T23:16:57.809 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-26T23:16:57.809 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-26T23:16:57.809 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-26T23:16:57.809 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-26T23:16:57.809 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-26T23:16:57.809 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-26T23:16:57.809 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-26T23:16:57.810 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-26T23:16:57.810 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-26T23:16:57.810 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-26T23:16:57.810 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-26T23:16:57.810 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-26T23:16:57.810 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-26T23:16:57.810 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-26T23:16:57.810 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-26T23:16:57.810 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-26T23:16:57.810 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-26T23:16:57.810 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-26T23:16:57.810 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-26T23:16:57.811 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-26T23:16:57.811 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-26T23:16:57.811 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T23:16:57.811 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T23:16:57.811 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T23:16:57.811 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T23:16:57.811 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T23:16:57.811 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T23:16:57.811 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T23:16:57.811 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T23:16:57.811 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T23:16:57.811 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T23:16:57.812 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T23:16:57.812 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T23:16:57.812 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T23:16:57.812 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T23:16:57.812 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T23:16:57.812 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T23:16:57.812 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T23:16:57.812 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T23:16:57.812 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T23:16:57.812 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T23:16:57.812 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T23:16:57.812 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T23:16:57.813 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T23:16:57.813 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T23:16:57.813 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T23:16:57.813 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T23:16:57.813 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T23:16:57.813 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T23:16:57.813 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T23:16:57.813 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T23:16:57.813 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T23:16:57.813 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T23:16:57.813 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T23:16:57.813 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T23:16:57.813 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T23:16:57.814 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T23:16:57.814 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T23:16:57.814 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T23:16:57.814 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T23:16:57.814 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T23:16:57.814 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T23:16:57.814 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T23:16:57.814 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T23:16:57.814 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T23:16:57.814 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T23:16:57.814 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T23:16:57.814 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T23:16:57.814 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T23:16:57.815 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T23:16:57.815 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T23:16:57.815 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T23:16:57.815 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T23:16:57.815 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T23:16:57.815 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T23:16:57.815 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T23:16:57.815 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T23:16:57.815 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T23:16:57.815 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T23:16:57.815 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T23:16:57.816 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T23:16:57.816 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T23:16:57.816 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T23:16:57.816 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T23:16:57.816 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T23:16:57.816 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T23:16:57.816 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T23:16:57.816 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T23:16:57.816 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T23:16:57.816 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T23:16:57.816 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T23:16:57.816 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T23:16:57.817 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T23:16:57.817 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T23:16:57.817 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T23:16:57.817 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T23:16:57.817 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T23:16:57.817 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T23:16:57.817 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T23:16:57.817 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T23:16:57.817 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T23:16:57.817 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T23:16:57.817 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T23:16:57.817 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T23:16:57.817 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T23:16:57.818 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T23:16:57.818 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T23:16:57.818 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T23:16:57.818 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T23:16:57.818 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T23:16:57.818 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T23:16:57.818 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T23:16:57.818 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T23:16:57.818 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T23:16:57.818 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T23:16:57.818 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T23:16:57.818 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T23:16:57.819 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T23:16:57.819 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T23:16:57.819 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T23:16:57.819 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T23:16:57.819 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T23:16:57.819 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T23:16:57.819 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T23:16:57.819 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T23:16:57.819 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T23:16:57.819 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T23:16:57.819 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T23:16:57.819 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T23:16:57.820 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T23:16:57.820 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-26T23:16:57.820 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T23:16:57.820 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-26T23:16:57.820 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T23:16:57.820 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T23:16:57.820 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T23:16:57.820 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T23:16:59.898 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T23:16:59.898 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:16:59.898 INFO:teuthology.orchestra.run.smithi047.stdout:Upgraded: 2024-04-26T23:16:59.898 INFO:teuthology.orchestra.run.smithi047.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:16:59.898 INFO:teuthology.orchestra.run.smithi047.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:16:59.898 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T23:16:59.898 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T23:16:59.898 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-04-26T23:16:59.899 INFO:teuthology.orchestra.run.smithi047.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T23:16:59.899 INFO:teuthology.orchestra.run.smithi047.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T23:16:59.899 INFO:teuthology.orchestra.run.smithi047.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T23:16:59.899 INFO:teuthology.orchestra.run.smithi047.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T23:16:59.899 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:16:59.899 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:16:59.899 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:16:59.899 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:16:59.899 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:16:59.899 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:16:59.899 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:16:59.899 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:16:59.899 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:16:59.899 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:16:59.899 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:16:59.900 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:16:59.900 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:16:59.900 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:16:59.900 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:16:59.900 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:16:59.900 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:16:59.900 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:16:59.900 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:16:59.900 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:16:59.900 INFO:teuthology.orchestra.run.smithi047.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:16:59.900 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:16:59.900 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:16:59.900 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:16:59.900 INFO:teuthology.orchestra.run.smithi047.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T23:16:59.900 INFO:teuthology.orchestra.run.smithi047.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T23:16:59.901 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T23:16:59.901 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T23:16:59.901 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T23:16:59.901 INFO:teuthology.orchestra.run.smithi047.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T23:16:59.901 INFO:teuthology.orchestra.run.smithi047.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T23:16:59.901 INFO:teuthology.orchestra.run.smithi047.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T23:16:59.901 INFO:teuthology.orchestra.run.smithi047.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T23:16:59.901 INFO:teuthology.orchestra.run.smithi047.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T23:16:59.901 INFO:teuthology.orchestra.run.smithi047.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T23:16:59.901 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:16:59.901 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:16:59.901 INFO:teuthology.orchestra.run.smithi047.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:16:59.901 INFO:teuthology.orchestra.run.smithi047.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T23:16:59.901 INFO:teuthology.orchestra.run.smithi047.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T23:16:59.902 INFO:teuthology.orchestra.run.smithi047.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T23:16:59.902 INFO:teuthology.orchestra.run.smithi047.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T23:16:59.902 INFO:teuthology.orchestra.run.smithi047.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T23:16:59.902 INFO:teuthology.orchestra.run.smithi047.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:16:59.902 INFO:teuthology.orchestra.run.smithi047.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:16:59.902 INFO:teuthology.orchestra.run.smithi047.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T23:16:59.902 INFO:teuthology.orchestra.run.smithi047.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:16:59.902 INFO:teuthology.orchestra.run.smithi047.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T23:16:59.902 INFO:teuthology.orchestra.run.smithi047.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T23:16:59.902 INFO:teuthology.orchestra.run.smithi047.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T23:16:59.902 INFO:teuthology.orchestra.run.smithi047.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T23:16:59.902 INFO:teuthology.orchestra.run.smithi047.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T23:16:59.902 INFO:teuthology.orchestra.run.smithi047.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T23:16:59.902 INFO:teuthology.orchestra.run.smithi047.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T23:16:59.903 INFO:teuthology.orchestra.run.smithi047.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T23:16:59.903 INFO:teuthology.orchestra.run.smithi047.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T23:16:59.903 INFO:teuthology.orchestra.run.smithi047.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T23:16:59.903 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T23:16:59.903 INFO:teuthology.orchestra.run.smithi047.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T23:16:59.903 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T23:16:59.903 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T23:16:59.903 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T23:16:59.903 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T23:16:59.903 INFO:teuthology.orchestra.run.smithi047.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T23:16:59.903 INFO:teuthology.orchestra.run.smithi047.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T23:16:59.903 INFO:teuthology.orchestra.run.smithi047.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T23:16:59.903 INFO:teuthology.orchestra.run.smithi047.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T23:16:59.903 INFO:teuthology.orchestra.run.smithi047.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T23:16:59.903 INFO:teuthology.orchestra.run.smithi047.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T23:16:59.903 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T23:16:59.904 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:16:59.904 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:16:59.904 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:16:59.904 INFO:teuthology.orchestra.run.smithi047.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T23:16:59.904 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T23:16:59.904 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T23:16:59.904 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T23:16:59.904 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T23:16:59.904 INFO:teuthology.orchestra.run.smithi047.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T23:16:59.904 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T23:16:59.904 INFO:teuthology.orchestra.run.smithi047.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T23:16:59.904 INFO:teuthology.orchestra.run.smithi047.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T23:16:59.904 INFO:teuthology.orchestra.run.smithi047.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T23:16:59.904 INFO:teuthology.orchestra.run.smithi047.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T23:16:59.904 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T23:16:59.905 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T23:16:59.905 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T23:16:59.905 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T23:16:59.905 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T23:16:59.905 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T23:16:59.905 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T23:16:59.905 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T23:16:59.905 INFO:teuthology.orchestra.run.smithi047.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T23:16:59.905 INFO:teuthology.orchestra.run.smithi047.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T23:16:59.905 INFO:teuthology.orchestra.run.smithi047.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T23:16:59.905 INFO:teuthology.orchestra.run.smithi047.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T23:16:59.905 INFO:teuthology.orchestra.run.smithi047.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T23:16:59.905 INFO:teuthology.orchestra.run.smithi047.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T23:16:59.905 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T23:16:59.905 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T23:16:59.906 INFO:teuthology.orchestra.run.smithi047.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T23:16:59.906 INFO:teuthology.orchestra.run.smithi047.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T23:16:59.906 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T23:16:59.906 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T23:16:59.906 INFO:teuthology.orchestra.run.smithi047.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T23:16:59.906 INFO:teuthology.orchestra.run.smithi047.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T23:16:59.906 INFO:teuthology.orchestra.run.smithi047.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T23:16:59.906 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T23:16:59.906 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T23:16:59.906 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T23:16:59.906 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T23:16:59.906 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T23:16:59.906 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T23:16:59.906 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:16:59.906 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:16:59.907 INFO:teuthology.orchestra.run.smithi047.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T23:16:59.907 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T23:16:59.907 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T23:16:59.907 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:16:59.907 INFO:teuthology.orchestra.run.smithi047.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T23:16:59.907 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T23:16:59.907 INFO:teuthology.orchestra.run.smithi047.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T23:16:59.907 INFO:teuthology.orchestra.run.smithi047.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T23:16:59.907 INFO:teuthology.orchestra.run.smithi047.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T23:16:59.907 INFO:teuthology.orchestra.run.smithi047.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T23:16:59.907 INFO:teuthology.orchestra.run.smithi047.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T23:16:59.907 INFO:teuthology.orchestra.run.smithi047.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T23:16:59.907 INFO:teuthology.orchestra.run.smithi047.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T23:16:59.907 INFO:teuthology.orchestra.run.smithi047.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T23:16:59.907 INFO:teuthology.orchestra.run.smithi047.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T23:16:59.908 INFO:teuthology.orchestra.run.smithi047.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T23:16:59.908 INFO:teuthology.orchestra.run.smithi047.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T23:16:59.908 INFO:teuthology.orchestra.run.smithi047.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T23:16:59.908 INFO:teuthology.orchestra.run.smithi047.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T23:16:59.908 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:16:59.908 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:16:59.908 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:16:59.908 INFO:teuthology.orchestra.run.smithi047.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T23:16:59.908 INFO:teuthology.orchestra.run.smithi047.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T23:16:59.908 INFO:teuthology.orchestra.run.smithi047.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T23:16:59.908 INFO:teuthology.orchestra.run.smithi047.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T23:16:59.908 INFO:teuthology.orchestra.run.smithi047.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T23:16:59.908 INFO:teuthology.orchestra.run.smithi047.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T23:16:59.909 INFO:teuthology.orchestra.run.smithi047.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T23:16:59.909 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:16:59.909 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-04-26T23:17:00.283 DEBUG:teuthology.parallel:result is None 2024-04-26T23:17:02.264 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T23:17:02.264 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /sys 2024-04-26T23:17:02.264 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /proc 2024-04-26T23:17:02.264 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /mnt 2024-04-26T23:17:02.264 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /var/tmp 2024-04-26T23:17:02.264 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /home 2024-04-26T23:17:02.264 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /root 2024-04-26T23:17:02.264 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /tmp 2024-04-26T23:17:02.264 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-26T23:17:02.440 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T23:17:02.456 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T23:17:02.456 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:17:02.456 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T23:17:02.456 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-26T23:17:03.927 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T23:17:03.952 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T23:17:03.953 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:17:03.953 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T23:17:03.953 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T23:17:03.953 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T23:17:03.953 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-26T23:17:04.178 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T23:17:04.200 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T23:17:04.200 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:17:04.200 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T23:17:04.200 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T23:17:04.201 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T23:17:04.201 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-26T23:17:04.645 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T23:17:04.668 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T23:17:04.668 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:17:04.668 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T23:17:04.668 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T23:17:04.668 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T23:17:04.668 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-26T23:17:04.776 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T23:17:04.833 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T23:17:04.889 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T23:17:05.077 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T23:17:05.122 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T23:17:05.201 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T23:17:05.290 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T23:17:05.429 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T23:17:05.472 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T23:17:05.504 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T23:17:05.532 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T23:17:05.917 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T23:17:05.948 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T23:17:05.980 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-26T23:17:06.050 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-26T23:17:06.088 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T23:17:07.134 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T23:17:07.173 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T23:17:08.144 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T23:17:08.173 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T23:17:08.275 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T23:17:08.446 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-26T23:17:08.475 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T23:17:08.498 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T23:17:08.499 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:17:08.499 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T23:17:08.499 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T23:17:08.499 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T23:17:08.499 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-26T23:17:08.543 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T23:17:08.555 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T23:17:08.652 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-26T23:17:08.701 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-26T23:17:09.628 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T23:17:09.654 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T23:17:09.677 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T23:17:09.677 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:17:09.677 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T23:17:09.677 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T23:17:09.678 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T23:17:09.678 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-26T23:17:09.788 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T23:17:09.810 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T23:17:09.811 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:17:09.811 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T23:17:09.811 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-26T23:17:09.856 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T23:17:09.878 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T23:17:09.878 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:17:09.878 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T23:17:09.878 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T23:17:09.879 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T23:17:09.879 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-26T23:17:10.190 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T23:17:10.212 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T23:17:10.212 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:17:10.212 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T23:17:10.213 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T23:17:10.213 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T23:17:10.213 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-26T23:17:14.811 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-26T23:17:14.852 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-26T23:17:14.893 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-26T23:17:14.925 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T23:17:14.953 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T23:17:15.145 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-26T23:17:15.210 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T23:17:15.240 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-26T23:17:15.857 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T23:17:15.895 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T23:17:15.915 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T23:17:16.766 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-26T23:17:16.850 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T23:17:16.891 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T23:17:16.974 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T23:17:17.008 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T23:17:17.008 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T23:17:17.089 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T23:17:17.121 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T23:17:17.122 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T23:17:17.244 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T23:17:17.312 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T23:17:23.338 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T23:17:23.338 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-26T23:17:23.338 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-26T23:17:23.338 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-26T23:17:23.338 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-26T23:17:23.338 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-26T23:17:23.338 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-26T23:17:23.338 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-26T23:17:23.338 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T23:17:23.339 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T23:17:23.339 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T23:17:23.339 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-26T23:17:23.339 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-26T23:17:23.339 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T23:17:23.339 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-26T23:17:23.339 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-26T23:17:23.339 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-26T23:17:23.339 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-26T23:17:23.339 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-26T23:17:23.339 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-26T23:17:23.339 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-26T23:17:23.339 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-26T23:17:23.339 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-26T23:17:23.340 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-26T23:17:23.340 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-26T23:17:23.340 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-26T23:17:23.340 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-26T23:17:23.340 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-26T23:17:23.340 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-26T23:17:23.340 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-26T23:17:23.341 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-26T23:17:23.341 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-26T23:17:23.341 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-26T23:17:23.341 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-26T23:17:23.341 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-26T23:17:23.341 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-26T23:17:23.342 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-26T23:17:23.342 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-26T23:17:23.342 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-26T23:17:23.342 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-26T23:17:23.342 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T23:17:23.343 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T23:17:23.343 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T23:17:23.343 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T23:17:23.343 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T23:17:23.343 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T23:17:23.343 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T23:17:23.345 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T23:17:23.346 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T23:17:23.346 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T23:17:23.346 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T23:17:23.346 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T23:17:23.346 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T23:17:23.347 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T23:17:23.347 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T23:17:23.347 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T23:17:23.347 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T23:17:23.347 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T23:17:23.348 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T23:17:23.348 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T23:17:23.348 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T23:17:23.348 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T23:17:23.348 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T23:17:23.348 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T23:17:23.349 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T23:17:23.349 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T23:17:23.349 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T23:17:23.349 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T23:17:23.349 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T23:17:23.349 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T23:17:23.349 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T23:17:23.350 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T23:17:23.350 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T23:17:23.350 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T23:17:23.350 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T23:17:23.350 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T23:17:23.350 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T23:17:23.351 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T23:17:23.351 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T23:17:23.351 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T23:17:23.351 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T23:17:23.351 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T23:17:23.351 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T23:17:23.351 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T23:17:23.352 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T23:17:23.352 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T23:17:23.352 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T23:17:23.352 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T23:17:23.352 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T23:17:23.352 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T23:17:23.352 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T23:17:23.353 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T23:17:23.353 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T23:17:23.353 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T23:17:23.353 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T23:17:23.353 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T23:17:23.353 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T23:17:23.353 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T23:17:23.354 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T23:17:23.354 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T23:17:23.354 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T23:17:23.354 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T23:17:23.354 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T23:17:23.354 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T23:17:23.354 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T23:17:23.355 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T23:17:23.355 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T23:17:23.355 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T23:17:23.355 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T23:17:23.355 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T23:17:23.355 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T23:17:23.356 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T23:17:23.356 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T23:17:23.356 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T23:17:23.356 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T23:17:23.356 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T23:17:23.356 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T23:17:23.357 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T23:17:23.357 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T23:17:23.357 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T23:17:23.357 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T23:17:23.357 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T23:17:23.357 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T23:17:23.358 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T23:17:23.358 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T23:17:23.358 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T23:17:23.358 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T23:17:23.358 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T23:17:23.359 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T23:17:23.359 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T23:17:23.359 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T23:17:23.359 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T23:17:23.359 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T23:17:23.359 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T23:17:23.359 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T23:17:23.359 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T23:17:23.360 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T23:17:23.360 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T23:17:23.360 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T23:17:23.360 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T23:17:23.360 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T23:17:23.360 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T23:17:23.360 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T23:17:23.360 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T23:17:23.360 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T23:17:23.360 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T23:17:23.360 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T23:17:23.361 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T23:17:23.361 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T23:17:23.361 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-26T23:17:23.361 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T23:17:23.361 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-26T23:17:23.361 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T23:17:23.361 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T23:17:23.361 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T23:17:23.361 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T23:17:25.485 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T23:17:25.485 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-26T23:17:25.485 INFO:teuthology.orchestra.run.smithi203.stdout:Upgraded: 2024-04-26T23:17:25.485 INFO:teuthology.orchestra.run.smithi203.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:17:25.485 INFO:teuthology.orchestra.run.smithi203.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:17:25.485 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T23:17:25.485 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T23:17:25.485 INFO:teuthology.orchestra.run.smithi203.stdout:Installed: 2024-04-26T23:17:25.485 INFO:teuthology.orchestra.run.smithi203.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T23:17:25.486 INFO:teuthology.orchestra.run.smithi203.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T23:17:25.486 INFO:teuthology.orchestra.run.smithi203.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T23:17:25.486 INFO:teuthology.orchestra.run.smithi203.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T23:17:25.486 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:17:25.486 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:17:25.486 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:17:25.486 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:17:25.486 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:17:25.486 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:17:25.486 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:17:25.486 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:17:25.486 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:17:25.486 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:17:25.486 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:17:25.487 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:17:25.487 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:17:25.487 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:17:25.487 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:17:25.487 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:17:25.487 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:17:25.487 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:17:25.487 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:17:25.487 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:17:25.487 INFO:teuthology.orchestra.run.smithi203.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:17:25.487 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:17:25.487 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:17:25.487 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:17:25.488 INFO:teuthology.orchestra.run.smithi203.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T23:17:25.488 INFO:teuthology.orchestra.run.smithi203.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T23:17:25.488 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T23:17:25.488 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T23:17:25.488 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T23:17:25.488 INFO:teuthology.orchestra.run.smithi203.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T23:17:25.488 INFO:teuthology.orchestra.run.smithi203.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T23:17:25.488 INFO:teuthology.orchestra.run.smithi203.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T23:17:25.488 INFO:teuthology.orchestra.run.smithi203.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T23:17:25.488 INFO:teuthology.orchestra.run.smithi203.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T23:17:25.488 INFO:teuthology.orchestra.run.smithi203.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T23:17:25.488 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:17:25.488 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:17:25.488 INFO:teuthology.orchestra.run.smithi203.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:17:25.488 INFO:teuthology.orchestra.run.smithi203.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T23:17:25.489 INFO:teuthology.orchestra.run.smithi203.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T23:17:25.489 INFO:teuthology.orchestra.run.smithi203.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T23:17:25.489 INFO:teuthology.orchestra.run.smithi203.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T23:17:25.489 INFO:teuthology.orchestra.run.smithi203.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T23:17:25.489 INFO:teuthology.orchestra.run.smithi203.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:17:25.489 INFO:teuthology.orchestra.run.smithi203.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:17:25.489 INFO:teuthology.orchestra.run.smithi203.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T23:17:25.489 INFO:teuthology.orchestra.run.smithi203.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:17:25.489 INFO:teuthology.orchestra.run.smithi203.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T23:17:25.489 INFO:teuthology.orchestra.run.smithi203.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T23:17:25.489 INFO:teuthology.orchestra.run.smithi203.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T23:17:25.489 INFO:teuthology.orchestra.run.smithi203.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T23:17:25.489 INFO:teuthology.orchestra.run.smithi203.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T23:17:25.489 INFO:teuthology.orchestra.run.smithi203.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T23:17:25.490 INFO:teuthology.orchestra.run.smithi203.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T23:17:25.490 INFO:teuthology.orchestra.run.smithi203.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T23:17:25.490 INFO:teuthology.orchestra.run.smithi203.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T23:17:25.490 INFO:teuthology.orchestra.run.smithi203.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T23:17:25.490 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T23:17:25.490 INFO:teuthology.orchestra.run.smithi203.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T23:17:25.490 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T23:17:25.490 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T23:17:25.490 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T23:17:25.490 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T23:17:25.490 INFO:teuthology.orchestra.run.smithi203.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T23:17:25.490 INFO:teuthology.orchestra.run.smithi203.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T23:17:25.490 INFO:teuthology.orchestra.run.smithi203.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T23:17:25.490 INFO:teuthology.orchestra.run.smithi203.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T23:17:25.490 INFO:teuthology.orchestra.run.smithi203.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T23:17:25.491 INFO:teuthology.orchestra.run.smithi203.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T23:17:25.491 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T23:17:25.491 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:17:25.491 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:17:25.491 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:17:25.491 INFO:teuthology.orchestra.run.smithi203.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T23:17:25.491 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T23:17:25.491 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T23:17:25.491 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T23:17:25.491 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T23:17:25.491 INFO:teuthology.orchestra.run.smithi203.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T23:17:25.491 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T23:17:25.491 INFO:teuthology.orchestra.run.smithi203.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T23:17:25.491 INFO:teuthology.orchestra.run.smithi203.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T23:17:25.492 INFO:teuthology.orchestra.run.smithi203.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T23:17:25.492 INFO:teuthology.orchestra.run.smithi203.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T23:17:25.492 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T23:17:25.492 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T23:17:25.492 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T23:17:25.492 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T23:17:25.492 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T23:17:25.492 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T23:17:25.492 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T23:17:25.492 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T23:17:25.492 INFO:teuthology.orchestra.run.smithi203.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T23:17:25.493 INFO:teuthology.orchestra.run.smithi203.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T23:17:25.493 INFO:teuthology.orchestra.run.smithi203.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T23:17:25.493 INFO:teuthology.orchestra.run.smithi203.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T23:17:25.493 INFO:teuthology.orchestra.run.smithi203.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T23:17:25.493 INFO:teuthology.orchestra.run.smithi203.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T23:17:25.493 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T23:17:25.493 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T23:17:25.493 INFO:teuthology.orchestra.run.smithi203.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T23:17:25.493 INFO:teuthology.orchestra.run.smithi203.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T23:17:25.493 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T23:17:25.493 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T23:17:25.493 INFO:teuthology.orchestra.run.smithi203.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T23:17:25.493 INFO:teuthology.orchestra.run.smithi203.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T23:17:25.493 INFO:teuthology.orchestra.run.smithi203.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T23:17:25.493 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T23:17:25.494 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T23:17:25.494 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T23:17:25.494 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T23:17:25.494 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T23:17:25.494 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T23:17:25.494 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:17:25.494 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:17:25.494 INFO:teuthology.orchestra.run.smithi203.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T23:17:25.494 INFO:teuthology.orchestra.run.smithi203.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T23:17:25.494 INFO:teuthology.orchestra.run.smithi203.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T23:17:25.494 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:17:25.495 INFO:teuthology.orchestra.run.smithi203.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T23:17:25.495 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T23:17:25.495 INFO:teuthology.orchestra.run.smithi203.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T23:17:25.495 INFO:teuthology.orchestra.run.smithi203.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T23:17:25.495 INFO:teuthology.orchestra.run.smithi203.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T23:17:25.495 INFO:teuthology.orchestra.run.smithi203.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T23:17:25.495 INFO:teuthology.orchestra.run.smithi203.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T23:17:25.495 INFO:teuthology.orchestra.run.smithi203.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T23:17:25.495 INFO:teuthology.orchestra.run.smithi203.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T23:17:25.495 INFO:teuthology.orchestra.run.smithi203.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T23:17:25.495 INFO:teuthology.orchestra.run.smithi203.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T23:17:25.495 INFO:teuthology.orchestra.run.smithi203.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T23:17:25.495 INFO:teuthology.orchestra.run.smithi203.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T23:17:25.495 INFO:teuthology.orchestra.run.smithi203.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T23:17:25.495 INFO:teuthology.orchestra.run.smithi203.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T23:17:25.496 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:17:25.496 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:17:25.496 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:17:25.496 INFO:teuthology.orchestra.run.smithi203.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T23:17:25.496 INFO:teuthology.orchestra.run.smithi203.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T23:17:25.496 INFO:teuthology.orchestra.run.smithi203.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T23:17:25.496 INFO:teuthology.orchestra.run.smithi203.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T23:17:25.496 INFO:teuthology.orchestra.run.smithi203.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T23:17:25.496 INFO:teuthology.orchestra.run.smithi203.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T23:17:25.496 INFO:teuthology.orchestra.run.smithi203.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T23:17:25.497 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-26T23:17:25.497 INFO:teuthology.orchestra.run.smithi203.stdout:Complete! 2024-04-26T23:17:25.896 DEBUG:teuthology.parallel:result is None 2024-04-26T23:17:25.897 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T23:17:26.127 DEBUG:teuthology.orchestra.run.smithi047:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T23:17:26.153 INFO:teuthology.orchestra.run.smithi047.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-26T23:17:26.154 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-26T23:17:26.154 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-26T23:17:26.156 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T23:17:26.292 DEBUG:teuthology.orchestra.run.smithi203:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T23:17:26.313 INFO:teuthology.orchestra.run.smithi203.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-26T23:17:26.313 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-26T23:17:26.313 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-26T23:17:26.315 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-26T23:17:26.315 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:17:26.315 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T23:17:26.347 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-26T23:17:26.347 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T23:17:26.380 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-26T23:17:26.380 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:17:26.380 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T23:17:26.417 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T23:17:26.486 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-26T23:17:26.486 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T23:17:26.510 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T23:17:26.575 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-26T23:17:26.575 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:17:26.575 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T23:17:26.605 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T23:17:26.673 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-26T23:17:26.673 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T23:17:26.697 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T23:17:26.761 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-26T23:17:26.761 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:17:26.762 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T23:17:26.786 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T23:17:26.851 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-26T23:17:26.852 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T23:17:26.875 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T23:17:26.939 INFO:teuthology.run_tasks:Running task ceph... 2024-04-26T23:17:27.033 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-26T23:17:27.033 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 777 /var/log/ceph 2024-04-26T23:17:27.036 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 777 /var/log/ceph 2024-04-26T23:17:27.059 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-26T23:17:27.059 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-26T23:17:27.101 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-26T23:17:27.125 INFO:tasks.ceph:Creating extra log directories... 2024-04-26T23:17:27.125 DEBUG:teuthology.orchestra.run.smithi047:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-26T23:17:27.166 DEBUG:teuthology.orchestra.run.smithi203:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-26T23:17:27.196 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-26T23:17:27.196 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-26T23:17:27.196 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-26T23:17:27.196 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:17:27.196 DEBUG:teuthology.orchestra.run.smithi047:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-26T23:17:27.244 DEBUG:teuthology.orchestra.run.smithi047:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-26T23:17:27.354 DEBUG:teuthology.orchestra.run.smithi047:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-26T23:17:27.378 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-26T23:17:27.378 DEBUG:teuthology.orchestra.run.smithi203:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-26T23:17:27.393 DEBUG:teuthology.orchestra.run.smithi203:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-26T23:17:27.501 DEBUG:teuthology.orchestra.run.smithi203:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-26T23:17:27.547 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-26T23:17:27.548 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-26T23:17:27.548 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675294', 'branch': 'wip-rishabh-testing-20240426.111959', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/pool-perm}', 'email': None, 'first_in_suite': False, 'job_id': '7675294', 'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-testing-20240426.111959'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-testing-20240426.111959', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 816, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'sleep_before_teardown': 0, 'subset': '166/512', 'suite': 'fs', 'suite_branch': 'wip-rishabh-testing-20240426.111959', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi047.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJuhkAIZrNzUfa5BiRihAr/9yQsVIHqthZJdqNagSQTtDAsb+mEZAsDWgBgGsQQXyEuk8wRTNy5wiSmnIt6JdWA=', 'smithi203.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPST5A8IfAx4FxTfpcq+BZKaAE3MGubSJD5hIpEPXxTlzqJi3F6ZhSulVQafHt2oT3TwmfjhqFJw0vz7VXvLxz4='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_pool_perm']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-26_19:30:57', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-26T23:17:27.548 DEBUG:teuthology.orchestra.run.smithi047:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-26T23:17:27.550 DEBUG:teuthology.orchestra.run.smithi203:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-26T23:17:27.596 DEBUG:teuthology.orchestra.run.smithi047:> sudo install -d -m0777 -- /var/run/ceph 2024-04-26T23:17:27.606 DEBUG:teuthology.orchestra.run.smithi203:> sudo install -d -m0777 -- /var/run/ceph 2024-04-26T23:17:27.660 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:17:27.660 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T23:17:27.684 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T23:17:27.684 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_1 2024-04-26T23:17:27.739 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T23:17:27.739 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T23:17:27.739 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-26T23:17:27.739 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T23:17:27.739 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T23:17:27.739 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-04-26 23:16:52.523602049 +0000 2024-04-26T23:17:27.739 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-04-26 23:14:52.291074929 +0000 2024-04-26T23:17:27.739 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-04-26 23:14:52.291074929 +0000 2024-04-26T23:17:27.740 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-04-26 23:14:52.291074929 +0000 2024-04-26T23:17:27.740 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T23:17:27.802 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-04-26T23:17:27.802 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-04-26T23:17:27.802 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.00010342 s, 5.0 MB/s 2024-04-26T23:17:27.803 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T23:17:27.859 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_2 2024-04-26T23:17:27.915 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T23:17:27.915 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T23:17:27.915 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-26T23:17:27.915 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T23:17:27.915 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T23:17:27.916 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-04-26 23:16:52.523602049 +0000 2024-04-26T23:17:27.916 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-04-26 23:14:52.628067384 +0000 2024-04-26T23:17:27.916 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-04-26 23:14:52.628067384 +0000 2024-04-26T23:17:27.916 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-04-26 23:14:52.628067384 +0000 2024-04-26T23:17:27.916 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T23:17:27.978 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-04-26T23:17:27.978 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-04-26T23:17:27.978 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 9.6584e-05 s, 5.3 MB/s 2024-04-26T23:17:27.979 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T23:17:28.035 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_3 2024-04-26T23:17:28.090 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T23:17:28.090 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T23:17:28.090 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-26T23:17:28.091 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T23:17:28.091 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T23:17:28.091 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-04-26 23:16:52.524602028 +0000 2024-04-26T23:17:28.091 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-04-26 23:14:52.956060041 +0000 2024-04-26T23:17:28.091 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-04-26 23:14:52.956060041 +0000 2024-04-26T23:17:28.091 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-04-26 23:14:52.956060041 +0000 2024-04-26T23:17:28.091 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T23:17:28.154 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-04-26T23:17:28.154 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-04-26T23:17:28.154 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 8.6664e-05 s, 5.9 MB/s 2024-04-26T23:17:28.155 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T23:17:28.211 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_4 2024-04-26T23:17:28.266 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T23:17:28.267 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T23:17:28.267 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-26T23:17:28.267 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T23:17:28.267 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T23:17:28.267 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-04-26 23:16:52.524602028 +0000 2024-04-26T23:17:28.267 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-04-26 23:14:53.299052362 +0000 2024-04-26T23:17:28.267 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-04-26 23:14:53.299052362 +0000 2024-04-26T23:17:28.267 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-04-26 23:14:53.299052362 +0000 2024-04-26T23:17:28.267 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T23:17:28.330 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-04-26T23:17:28.330 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-04-26T23:17:28.330 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000102206 s, 5.0 MB/s 2024-04-26T23:17:28.331 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T23:17:28.387 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T23:17:28.387 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-26T23:17:28.387 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T23:17:28.404 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T23:17:28.404 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_1 2024-04-26T23:17:28.460 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T23:17:28.460 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T23:17:28.460 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-26T23:17:28.460 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T23:17:28.460 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T23:17:28.460 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-04-26 23:17:17.676465128 +0000 2024-04-26T23:17:28.461 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-04-26 23:14:47.308620594 +0000 2024-04-26T23:17:28.461 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-04-26 23:14:47.308620594 +0000 2024-04-26T23:17:28.461 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2024-04-26 23:14:47.308620594 +0000 2024-04-26T23:17:28.461 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T23:17:28.525 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-04-26T23:17:28.525 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-04-26T23:17:28.525 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000197545 s, 2.6 MB/s 2024-04-26T23:17:28.527 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T23:17:28.584 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_2 2024-04-26T23:17:28.641 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T23:17:28.641 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T23:17:28.641 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-26T23:17:28.641 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T23:17:28.641 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T23:17:28.641 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-04-26 23:17:17.677465105 +0000 2024-04-26T23:17:28.641 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-04-26 23:14:47.648611625 +0000 2024-04-26T23:17:28.642 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-04-26 23:14:47.648611625 +0000 2024-04-26T23:17:28.642 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2024-04-26 23:14:47.648611625 +0000 2024-04-26T23:17:28.642 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T23:17:28.705 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-04-26T23:17:28.705 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-04-26T23:17:28.705 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000191436 s, 2.7 MB/s 2024-04-26T23:17:28.706 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T23:17:28.763 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_3 2024-04-26T23:17:28.819 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T23:17:28.819 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T23:17:28.819 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-26T23:17:28.819 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T23:17:28.819 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T23:17:28.819 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-04-26 23:17:17.677465105 +0000 2024-04-26T23:17:28.819 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-04-26 23:14:47.996602446 +0000 2024-04-26T23:17:28.819 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-04-26 23:14:47.996602446 +0000 2024-04-26T23:17:28.819 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2024-04-26 23:14:47.996602446 +0000 2024-04-26T23:17:28.820 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T23:17:28.882 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-04-26T23:17:28.882 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-04-26T23:17:28.883 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000171176 s, 3.0 MB/s 2024-04-26T23:17:28.884 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T23:17:28.939 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_4 2024-04-26T23:17:28.994 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T23:17:28.994 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T23:17:28.994 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-26T23:17:28.994 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T23:17:28.994 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T23:17:28.994 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-04-26 23:17:17.677465105 +0000 2024-04-26T23:17:28.994 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-04-26 23:14:48.334593529 +0000 2024-04-26T23:17:28.994 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-04-26 23:14:48.334593529 +0000 2024-04-26T23:17:28.995 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2024-04-26 23:14:48.334593529 +0000 2024-04-26T23:17:28.995 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T23:17:29.058 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-04-26T23:17:29.059 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-04-26T23:17:29.059 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000205986 s, 2.5 MB/s 2024-04-26T23:17:29.060 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T23:17:29.116 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T23:17:29.116 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi047.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi203.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-26T23:17:29.116 INFO:tasks.ceph:Generating config... 2024-04-26T23:17:29.117 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-26T23:17:29.117 INFO:tasks.ceph:[client] debug client = 20 2024-04-26T23:17:29.117 INFO:tasks.ceph:[client] debug ms = 1 2024-04-26T23:17:29.117 INFO:tasks.ceph:[client] pid file = 2024-04-26T23:17:29.118 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-26T23:17:29.118 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-26T23:17:29.118 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-04-26T23:17:29.118 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-26T23:17:29.118 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-26T23:17:29.118 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-26T23:17:29.118 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-26T23:17:29.118 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-26T23:17:29.118 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-26T23:17:29.118 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-26T23:17:29.118 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-26T23:17:29.118 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-26T23:17:29.118 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-26T23:17:29.118 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-26T23:17:29.118 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-26T23:17:29.119 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-26T23:17:29.119 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-26T23:17:29.119 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-26T23:17:29.119 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-26T23:17:29.119 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-26T23:17:29.119 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-26T23:17:29.119 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-26T23:17:29.119 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-26T23:17:29.119 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-04-26T23:17:29.119 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-04-26T23:17:29.120 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-26T23:17:29.120 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-26T23:17:29.120 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-04-26T23:17:29.120 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-26T23:17:29.120 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-26T23:17:29.120 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-26T23:17:29.120 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-26T23:17:29.120 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-26T23:17:29.120 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-26T23:17:29.120 INFO:tasks.ceph:Setting up mon.a... 2024-04-26T23:17:29.120 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-26T23:17:29.173 INFO:teuthology.orchestra.run.smithi047.stdout:creating /etc/ceph/ceph.keyring 2024-04-26T23:17:29.176 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-26T23:17:29.223 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-26T23:17:29.248 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.47'), ('mon.b', '172.21.15.203'), ('mon.c', '[v2:172.21.15.203:3301,v1:172.21.15.203:6790]')] 2024-04-26T23:17:29.249 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.47,172.21.15.203,[v2:172.21.15.203:3301,v1:172.21.15.203:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-26T23:17:29.250 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:17:29.250 DEBUG:teuthology.orchestra.run.smithi047:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-26T23:17:29.309 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.47 --add b 172.21.15.203 --addv c '[v2:172.21.15.203:3301,v1:172.21.15.203:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-26T23:17:29.417 INFO:teuthology.orchestra.run.smithi047.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-26T23:17:29.418 INFO:teuthology.orchestra.run.smithi047.stdout:monmaptool: generated fsid 399006d3-4db1-43e6-ba81-a79a276eabfd 2024-04-26T23:17:29.418 INFO:teuthology.orchestra.run.smithi047.stdout:setting min_mon_release = quincy 2024-04-26T23:17:29.418 INFO:teuthology.orchestra.run.smithi047.stdout:epoch 0 2024-04-26T23:17:29.418 INFO:teuthology.orchestra.run.smithi047.stdout:fsid 399006d3-4db1-43e6-ba81-a79a276eabfd 2024-04-26T23:17:29.418 INFO:teuthology.orchestra.run.smithi047.stdout:last_changed 2024-04-26T23:17:29.417576+0000 2024-04-26T23:17:29.418 INFO:teuthology.orchestra.run.smithi047.stdout:created 2024-04-26T23:17:29.417576+0000 2024-04-26T23:17:29.418 INFO:teuthology.orchestra.run.smithi047.stdout:min_mon_release 17 (quincy) 2024-04-26T23:17:29.418 INFO:teuthology.orchestra.run.smithi047.stdout:election_strategy: 1 2024-04-26T23:17:29.418 INFO:teuthology.orchestra.run.smithi047.stdout:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-26T23:17:29.418 INFO:teuthology.orchestra.run.smithi047.stdout:1: [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] mon.b 2024-04-26T23:17:29.419 INFO:teuthology.orchestra.run.smithi047.stdout:2: [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] mon.c 2024-04-26T23:17:29.419 INFO:teuthology.orchestra.run.smithi047.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-26T23:17:29.420 DEBUG:teuthology.orchestra.run.smithi047:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-26T23:17:29.440 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 399006d3-4db1-43e6-ba81-a79a276eabfd... 2024-04-26T23:17:29.441 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-26T23:17:29.488 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-26T23:17:29.528 INFO:teuthology.orchestra.run.smithi047.stdout:[global] 2024-04-26T23:17:29.528 INFO:teuthology.orchestra.run.smithi047.stdout: chdir = "" 2024-04-26T23:17:29.528 INFO:teuthology.orchestra.run.smithi047.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-26T23:17:29.528 INFO:teuthology.orchestra.run.smithi047.stdout: auth supported = cephx 2024-04-26T23:17:29.529 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:17:29.529 INFO:teuthology.orchestra.run.smithi047.stdout: filestore xattr use omap = true 2024-04-26T23:17:29.529 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:17:29.529 INFO:teuthology.orchestra.run.smithi047.stdout: mon clock drift allowed = 1.000 2024-04-26T23:17:29.529 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:17:29.529 INFO:teuthology.orchestra.run.smithi047.stdout: osd crush chooseleaf type = 0 2024-04-26T23:17:29.529 INFO:teuthology.orchestra.run.smithi047.stdout: auth debug = true 2024-04-26T23:17:29.529 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:17:29.529 INFO:teuthology.orchestra.run.smithi047.stdout: ms die on old message = true 2024-04-26T23:17:29.529 INFO:teuthology.orchestra.run.smithi047.stdout: ms die on bug = true 2024-04-26T23:17:29.529 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:17:29.529 INFO:teuthology.orchestra.run.smithi047.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-26T23:17:29.529 INFO:teuthology.orchestra.run.smithi047.stdout: mon pg warn max object skew = 0 2024-04-26T23:17:29.529 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:17:29.529 INFO:teuthology.orchestra.run.smithi047.stdout: # disable pg_autoscaler by default for new pools 2024-04-26T23:17:29.530 INFO:teuthology.orchestra.run.smithi047.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-26T23:17:29.530 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:17:29.530 INFO:teuthology.orchestra.run.smithi047.stdout: osd pool default size = 2 2024-04-26T23:17:29.530 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:17:29.530 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd allow primary affinity = true 2024-04-26T23:17:29.530 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd allow pg remap = true 2024-04-26T23:17:29.530 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on legacy crush tunables = false 2024-04-26T23:17:29.530 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on crush straw calc version zero = false 2024-04-26T23:17:29.530 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on no sortbitwise = false 2024-04-26T23:17:29.530 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on osd down out interval zero = false 2024-04-26T23:17:29.530 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on too few osds = false 2024-04-26T23:17:29.530 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-26T23:17:29.531 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-26T23:17:29.531 INFO:teuthology.orchestra.run.smithi047.stdout: mon_allow_pool_size_one = true 2024-04-26T23:17:29.531 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:17:29.531 INFO:teuthology.orchestra.run.smithi047.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-26T23:17:29.531 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:17:29.531 INFO:teuthology.orchestra.run.smithi047.stdout: osd default data pool replay window = 5 2024-04-26T23:17:29.531 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:17:29.531 INFO:teuthology.orchestra.run.smithi047.stdout: mon allow pool delete = true 2024-04-26T23:17:29.531 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:17:29.531 INFO:teuthology.orchestra.run.smithi047.stdout: mon cluster log file level = debug 2024-04-26T23:17:29.532 INFO:teuthology.orchestra.run.smithi047.stdout: debug asserts on shutdown = true 2024-04-26T23:17:29.532 INFO:teuthology.orchestra.run.smithi047.stdout: mon health detail to clog = false 2024-04-26T23:17:29.532 INFO:teuthology.orchestra.run.smithi047.stdout: mon host = "172.21.15.47,172.21.15.203,[v2:172.21.15.203:3301,v1:172.21.15.203:6790]" 2024-04-26T23:17:29.532 INFO:teuthology.orchestra.run.smithi047.stdout: ms die on skipped message = False 2024-04-26T23:17:29.532 INFO:teuthology.orchestra.run.smithi047.stdout: fsid = 399006d3-4db1-43e6-ba81-a79a276eabfd 2024-04-26T23:17:29.532 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:17:29.532 INFO:teuthology.orchestra.run.smithi047.stdout:[osd] 2024-04-26T23:17:29.532 INFO:teuthology.orchestra.run.smithi047.stdout: osd journal size = 100 2024-04-26T23:17:29.532 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:17:29.532 INFO:teuthology.orchestra.run.smithi047.stdout: osd scrub load threshold = 5.0 2024-04-26T23:17:29.532 INFO:teuthology.orchestra.run.smithi047.stdout: osd scrub max interval = 600 2024-04-26T23:17:29.532 INFO:teuthology.orchestra.run.smithi047.stdout: osd mclock profile = high_recovery_ops 2024-04-26T23:17:29.532 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:17:29.532 INFO:teuthology.orchestra.run.smithi047.stdout: osd recover clone overlap = true 2024-04-26T23:17:29.532 INFO:teuthology.orchestra.run.smithi047.stdout: osd recovery max chunk = 1048576 2024-04-26T23:17:29.533 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:17:29.533 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug shutdown = true 2024-04-26T23:17:29.533 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug op order = true 2024-04-26T23:17:29.533 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug verify stray on activate = true 2024-04-26T23:17:29.533 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug trim objects = true 2024-04-26T23:17:29.533 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:17:29.533 INFO:teuthology.orchestra.run.smithi047.stdout: osd open classes on start = true 2024-04-26T23:17:29.533 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug pg log writeout = true 2024-04-26T23:17:29.533 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:17:29.533 INFO:teuthology.orchestra.run.smithi047.stdout: osd deep scrub update digest min age = 30 2024-04-26T23:17:29.533 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:17:29.533 INFO:teuthology.orchestra.run.smithi047.stdout: osd map max advance = 10 2024-04-26T23:17:29.533 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:17:29.533 INFO:teuthology.orchestra.run.smithi047.stdout: journal zero on create = true 2024-04-26T23:17:29.533 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:17:29.534 INFO:teuthology.orchestra.run.smithi047.stdout: filestore ondisk finisher threads = 3 2024-04-26T23:17:29.534 INFO:teuthology.orchestra.run.smithi047.stdout: filestore apply finisher threads = 3 2024-04-26T23:17:29.534 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:17:29.534 INFO:teuthology.orchestra.run.smithi047.stdout: bdev debug aio = true 2024-04-26T23:17:29.534 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug misdirected ops = true 2024-04-26T23:17:29.534 INFO:teuthology.orchestra.run.smithi047.stdout: bluestore block size = 96636764160 2024-04-26T23:17:29.534 INFO:teuthology.orchestra.run.smithi047.stdout: bluestore fsck on mount = True 2024-04-26T23:17:29.534 INFO:teuthology.orchestra.run.smithi047.stdout: debug bluefs = 20 2024-04-26T23:17:29.534 INFO:teuthology.orchestra.run.smithi047.stdout: debug bluestore = 20 2024-04-26T23:17:29.534 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-04-26T23:17:29.534 INFO:teuthology.orchestra.run.smithi047.stdout: debug osd = 20 2024-04-26T23:17:29.534 INFO:teuthology.orchestra.run.smithi047.stdout: debug rocksdb = 10 2024-04-26T23:17:29.534 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd backfillfull_ratio = 0.85 2024-04-26T23:17:29.534 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd full ratio = 0.9 2024-04-26T23:17:29.534 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd nearfull ratio = 0.8 2024-04-26T23:17:29.535 INFO:teuthology.orchestra.run.smithi047.stdout: osd failsafe full ratio = 0.95 2024-04-26T23:17:29.535 INFO:teuthology.orchestra.run.smithi047.stdout: osd objectstore = bluestore 2024-04-26T23:17:29.535 INFO:teuthology.orchestra.run.smithi047.stdout: osd op complaint time = 180 2024-04-26T23:17:29.535 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:17:29.535 INFO:teuthology.orchestra.run.smithi047.stdout:[mgr] 2024-04-26T23:17:29.535 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-04-26T23:17:29.535 INFO:teuthology.orchestra.run.smithi047.stdout: debug mgr = 20 2024-04-26T23:17:29.535 INFO:teuthology.orchestra.run.smithi047.stdout: debug mon = 20 2024-04-26T23:17:29.535 INFO:teuthology.orchestra.run.smithi047.stdout: debug auth = 20 2024-04-26T23:17:29.535 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min pgs per osd = 4 2024-04-26T23:17:29.535 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min bytes per osd = 10 2024-04-26T23:17:29.535 INFO:teuthology.orchestra.run.smithi047.stdout: mgr/telemetry/nag = false 2024-04-26T23:17:29.535 INFO:teuthology.orchestra.run.smithi047.stdout: debug client = 20 2024-04-26T23:17:29.535 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:17:29.535 INFO:teuthology.orchestra.run.smithi047.stdout:[mon] 2024-04-26T23:17:29.535 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-04-26T23:17:29.536 INFO:teuthology.orchestra.run.smithi047.stdout: debug mon = 20 2024-04-26T23:17:29.536 INFO:teuthology.orchestra.run.smithi047.stdout: debug paxos = 20 2024-04-26T23:17:29.536 INFO:teuthology.orchestra.run.smithi047.stdout: debug auth = 20 2024-04-26T23:17:29.536 INFO:teuthology.orchestra.run.smithi047.stdout: mon data avail warn = 5 2024-04-26T23:17:29.536 INFO:teuthology.orchestra.run.smithi047.stdout: mon mgr mkfs grace = 240 2024-04-26T23:17:29.536 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min pgs per osd = 4 2024-04-26T23:17:29.536 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd reporter subtree level = osd 2024-04-26T23:17:29.536 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd prime pg temp = true 2024-04-26T23:17:29.536 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min bytes per osd = 10 2024-04-26T23:17:29.536 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:17:29.536 INFO:teuthology.orchestra.run.smithi047.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-26T23:17:29.536 INFO:teuthology.orchestra.run.smithi047.stdout: auth mon ticket ttl = 660 # 11m 2024-04-26T23:17:29.536 INFO:teuthology.orchestra.run.smithi047.stdout: auth service ticket ttl = 240 # 4m 2024-04-26T23:17:29.536 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:17:29.536 INFO:teuthology.orchestra.run.smithi047.stdout: # don't complain about insecure global_id in the test suite 2024-04-26T23:17:29.537 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-26T23:17:29.537 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-26T23:17:29.537 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:17:29.537 INFO:teuthology.orchestra.run.smithi047.stdout: # 1m isn't quite enough 2024-04-26T23:17:29.537 INFO:teuthology.orchestra.run.smithi047.stdout: mon_down_mkfs_grace = 2m 2024-04-26T23:17:29.537 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:17:29.537 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_filestore_osds = false 2024-04-26T23:17:29.537 INFO:teuthology.orchestra.run.smithi047.stdout: mon down mkfs grace = 300 2024-04-26T23:17:29.537 INFO:teuthology.orchestra.run.smithi047.stdout: mon op complaint time = 120 2024-04-26T23:17:29.537 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:17:29.537 INFO:teuthology.orchestra.run.smithi047.stdout:[client] 2024-04-26T23:17:29.537 INFO:teuthology.orchestra.run.smithi047.stdout: rgw cache enabled = true 2024-04-26T23:17:29.537 INFO:teuthology.orchestra.run.smithi047.stdout: rgw enable ops log = true 2024-04-26T23:17:29.537 INFO:teuthology.orchestra.run.smithi047.stdout: rgw enable usage log = true 2024-04-26T23:17:29.537 INFO:teuthology.orchestra.run.smithi047.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-26T23:17:29.537 INFO:teuthology.orchestra.run.smithi047.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-26T23:17:29.538 INFO:teuthology.orchestra.run.smithi047.stdout: client mount timeout = 600 2024-04-26T23:17:29.538 INFO:teuthology.orchestra.run.smithi047.stdout: debug client = 20 2024-04-26T23:17:29.538 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-04-26T23:17:29.538 INFO:teuthology.orchestra.run.smithi047.stdout: pid file = "" 2024-04-26T23:17:29.538 INFO:teuthology.orchestra.run.smithi047.stdout: rados mon op timeout = 900 2024-04-26T23:17:29.538 INFO:teuthology.orchestra.run.smithi047.stdout: rados osd op timeout = 900 2024-04-26T23:17:29.538 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.a] 2024-04-26T23:17:29.538 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.b] 2024-04-26T23:17:29.538 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.c] 2024-04-26T23:17:29.538 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.a] 2024-04-26T23:17:29.538 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.c] 2024-04-26T23:17:29.538 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.b] 2024-04-26T23:17:29.538 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.d] 2024-04-26T23:17:29.538 INFO:teuthology.orchestra.run.smithi047.stdout:[mds] 2024-04-26T23:17:29.538 INFO:teuthology.orchestra.run.smithi047.stdout: debug mds = 20 2024-04-26T23:17:29.539 INFO:teuthology.orchestra.run.smithi047.stdout: debug mds balancer = 20 2024-04-26T23:17:29.539 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-04-26T23:17:29.539 INFO:teuthology.orchestra.run.smithi047.stdout: mds debug frag = True 2024-04-26T23:17:29.539 INFO:teuthology.orchestra.run.smithi047.stdout: mds debug scatterstat = True 2024-04-26T23:17:29.539 INFO:teuthology.orchestra.run.smithi047.stdout: mds op complaint time = 180 2024-04-26T23:17:29.539 INFO:teuthology.orchestra.run.smithi047.stdout: mds verify scatter = True 2024-04-26T23:17:29.539 INFO:teuthology.orchestra.run.smithi047.stdout: osd op complaint time = 180 2024-04-26T23:17:29.539 INFO:teuthology.orchestra.run.smithi047.stdout: rados mon op timeout = 900 2024-04-26T23:17:29.539 INFO:teuthology.orchestra.run.smithi047.stdout: rados osd op timeout = 900 2024-04-26T23:17:29.540 INFO:teuthology.orchestra.run.smithi203.stdout:[global] 2024-04-26T23:17:29.540 INFO:teuthology.orchestra.run.smithi203.stdout: chdir = "" 2024-04-26T23:17:29.540 INFO:teuthology.orchestra.run.smithi203.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-26T23:17:29.540 INFO:teuthology.orchestra.run.smithi203.stdout: auth supported = cephx 2024-04-26T23:17:29.540 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-26T23:17:29.540 INFO:teuthology.orchestra.run.smithi203.stdout: filestore xattr use omap = true 2024-04-26T23:17:29.540 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-26T23:17:29.540 INFO:teuthology.orchestra.run.smithi203.stdout: mon clock drift allowed = 1.000 2024-04-26T23:17:29.540 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-26T23:17:29.540 INFO:teuthology.orchestra.run.smithi203.stdout: osd crush chooseleaf type = 0 2024-04-26T23:17:29.540 INFO:teuthology.orchestra.run.smithi203.stdout: auth debug = true 2024-04-26T23:17:29.540 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-26T23:17:29.540 INFO:teuthology.orchestra.run.smithi203.stdout: ms die on old message = true 2024-04-26T23:17:29.541 INFO:teuthology.orchestra.run.smithi203.stdout: ms die on bug = true 2024-04-26T23:17:29.541 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-26T23:17:29.541 INFO:teuthology.orchestra.run.smithi203.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-26T23:17:29.541 INFO:teuthology.orchestra.run.smithi203.stdout: mon pg warn max object skew = 0 2024-04-26T23:17:29.541 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-26T23:17:29.541 INFO:teuthology.orchestra.run.smithi203.stdout: # disable pg_autoscaler by default for new pools 2024-04-26T23:17:29.541 INFO:teuthology.orchestra.run.smithi203.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-26T23:17:29.541 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-26T23:17:29.541 INFO:teuthology.orchestra.run.smithi203.stdout: osd pool default size = 2 2024-04-26T23:17:29.541 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-26T23:17:29.541 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd allow primary affinity = true 2024-04-26T23:17:29.541 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd allow pg remap = true 2024-04-26T23:17:29.541 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on legacy crush tunables = false 2024-04-26T23:17:29.541 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on crush straw calc version zero = false 2024-04-26T23:17:29.542 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on no sortbitwise = false 2024-04-26T23:17:29.542 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on osd down out interval zero = false 2024-04-26T23:17:29.542 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on too few osds = false 2024-04-26T23:17:29.542 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-26T23:17:29.542 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-26T23:17:29.542 INFO:teuthology.orchestra.run.smithi203.stdout: mon_allow_pool_size_one = true 2024-04-26T23:17:29.542 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-26T23:17:29.542 INFO:teuthology.orchestra.run.smithi203.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-26T23:17:29.542 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-26T23:17:29.542 INFO:teuthology.orchestra.run.smithi203.stdout: osd default data pool replay window = 5 2024-04-26T23:17:29.542 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-26T23:17:29.542 INFO:teuthology.orchestra.run.smithi203.stdout: mon allow pool delete = true 2024-04-26T23:17:29.542 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-26T23:17:29.542 INFO:teuthology.orchestra.run.smithi203.stdout: mon cluster log file level = debug 2024-04-26T23:17:29.542 INFO:teuthology.orchestra.run.smithi203.stdout: debug asserts on shutdown = true 2024-04-26T23:17:29.543 INFO:teuthology.orchestra.run.smithi203.stdout: mon health detail to clog = false 2024-04-26T23:17:29.543 INFO:teuthology.orchestra.run.smithi203.stdout: mon host = "172.21.15.47,172.21.15.203,[v2:172.21.15.203:3301,v1:172.21.15.203:6790]" 2024-04-26T23:17:29.543 INFO:teuthology.orchestra.run.smithi203.stdout: ms die on skipped message = False 2024-04-26T23:17:29.543 INFO:teuthology.orchestra.run.smithi203.stdout: fsid = 399006d3-4db1-43e6-ba81-a79a276eabfd 2024-04-26T23:17:29.543 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-26T23:17:29.543 INFO:teuthology.orchestra.run.smithi203.stdout:[osd] 2024-04-26T23:17:29.543 INFO:teuthology.orchestra.run.smithi203.stdout: osd journal size = 100 2024-04-26T23:17:29.543 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-26T23:17:29.543 INFO:teuthology.orchestra.run.smithi203.stdout: osd scrub load threshold = 5.0 2024-04-26T23:17:29.543 INFO:teuthology.orchestra.run.smithi203.stdout: osd scrub max interval = 600 2024-04-26T23:17:29.543 INFO:teuthology.orchestra.run.smithi203.stdout: osd mclock profile = high_recovery_ops 2024-04-26T23:17:29.543 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-26T23:17:29.543 INFO:teuthology.orchestra.run.smithi203.stdout: osd recover clone overlap = true 2024-04-26T23:17:29.543 INFO:teuthology.orchestra.run.smithi203.stdout: osd recovery max chunk = 1048576 2024-04-26T23:17:29.543 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-26T23:17:29.543 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug shutdown = true 2024-04-26T23:17:29.544 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug op order = true 2024-04-26T23:17:29.544 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug verify stray on activate = true 2024-04-26T23:17:29.544 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug trim objects = true 2024-04-26T23:17:29.544 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-26T23:17:29.544 INFO:teuthology.orchestra.run.smithi203.stdout: osd open classes on start = true 2024-04-26T23:17:29.544 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug pg log writeout = true 2024-04-26T23:17:29.544 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-26T23:17:29.544 INFO:teuthology.orchestra.run.smithi203.stdout: osd deep scrub update digest min age = 30 2024-04-26T23:17:29.544 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-26T23:17:29.544 INFO:teuthology.orchestra.run.smithi203.stdout: osd map max advance = 10 2024-04-26T23:17:29.544 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-26T23:17:29.544 INFO:teuthology.orchestra.run.smithi203.stdout: journal zero on create = true 2024-04-26T23:17:29.544 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-26T23:17:29.544 INFO:teuthology.orchestra.run.smithi203.stdout: filestore ondisk finisher threads = 3 2024-04-26T23:17:29.544 INFO:teuthology.orchestra.run.smithi203.stdout: filestore apply finisher threads = 3 2024-04-26T23:17:29.544 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-26T23:17:29.545 INFO:teuthology.orchestra.run.smithi203.stdout: bdev debug aio = true 2024-04-26T23:17:29.545 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug misdirected ops = true 2024-04-26T23:17:29.545 INFO:teuthology.orchestra.run.smithi203.stdout: bluestore block size = 96636764160 2024-04-26T23:17:29.545 INFO:teuthology.orchestra.run.smithi203.stdout: bluestore fsck on mount = True 2024-04-26T23:17:29.545 INFO:teuthology.orchestra.run.smithi203.stdout: debug bluefs = 20 2024-04-26T23:17:29.545 INFO:teuthology.orchestra.run.smithi203.stdout: debug bluestore = 20 2024-04-26T23:17:29.545 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2024-04-26T23:17:29.545 INFO:teuthology.orchestra.run.smithi203.stdout: debug osd = 20 2024-04-26T23:17:29.545 INFO:teuthology.orchestra.run.smithi203.stdout: debug rocksdb = 10 2024-04-26T23:17:29.545 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd backfillfull_ratio = 0.85 2024-04-26T23:17:29.545 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd full ratio = 0.9 2024-04-26T23:17:29.545 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd nearfull ratio = 0.8 2024-04-26T23:17:29.545 INFO:teuthology.orchestra.run.smithi203.stdout: osd failsafe full ratio = 0.95 2024-04-26T23:17:29.545 INFO:teuthology.orchestra.run.smithi203.stdout: osd objectstore = bluestore 2024-04-26T23:17:29.545 INFO:teuthology.orchestra.run.smithi203.stdout: osd op complaint time = 180 2024-04-26T23:17:29.545 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-26T23:17:29.546 INFO:teuthology.orchestra.run.smithi203.stdout:[mgr] 2024-04-26T23:17:29.546 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2024-04-26T23:17:29.546 INFO:teuthology.orchestra.run.smithi203.stdout: debug mgr = 20 2024-04-26T23:17:29.546 INFO:teuthology.orchestra.run.smithi203.stdout: debug mon = 20 2024-04-26T23:17:29.546 INFO:teuthology.orchestra.run.smithi203.stdout: debug auth = 20 2024-04-26T23:17:29.546 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min pgs per osd = 4 2024-04-26T23:17:29.546 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min bytes per osd = 10 2024-04-26T23:17:29.546 INFO:teuthology.orchestra.run.smithi203.stdout: mgr/telemetry/nag = false 2024-04-26T23:17:29.546 INFO:teuthology.orchestra.run.smithi203.stdout: debug client = 20 2024-04-26T23:17:29.546 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-26T23:17:29.546 INFO:teuthology.orchestra.run.smithi203.stdout:[mon] 2024-04-26T23:17:29.546 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2024-04-26T23:17:29.547 INFO:teuthology.orchestra.run.smithi203.stdout: debug mon = 20 2024-04-26T23:17:29.547 INFO:teuthology.orchestra.run.smithi203.stdout: debug paxos = 20 2024-04-26T23:17:29.547 INFO:teuthology.orchestra.run.smithi203.stdout: debug auth = 20 2024-04-26T23:17:29.547 INFO:teuthology.orchestra.run.smithi203.stdout: mon data avail warn = 5 2024-04-26T23:17:29.547 INFO:teuthology.orchestra.run.smithi203.stdout: mon mgr mkfs grace = 240 2024-04-26T23:17:29.547 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min pgs per osd = 4 2024-04-26T23:17:29.547 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd reporter subtree level = osd 2024-04-26T23:17:29.547 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd prime pg temp = true 2024-04-26T23:17:29.547 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min bytes per osd = 10 2024-04-26T23:17:29.547 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-26T23:17:29.547 INFO:teuthology.orchestra.run.smithi203.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-26T23:17:29.547 INFO:teuthology.orchestra.run.smithi203.stdout: auth mon ticket ttl = 660 # 11m 2024-04-26T23:17:29.547 INFO:teuthology.orchestra.run.smithi203.stdout: auth service ticket ttl = 240 # 4m 2024-04-26T23:17:29.547 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-26T23:17:29.547 INFO:teuthology.orchestra.run.smithi203.stdout: # don't complain about insecure global_id in the test suite 2024-04-26T23:17:29.548 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-26T23:17:29.548 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-26T23:17:29.548 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-26T23:17:29.548 INFO:teuthology.orchestra.run.smithi203.stdout: # 1m isn't quite enough 2024-04-26T23:17:29.548 INFO:teuthology.orchestra.run.smithi203.stdout: mon_down_mkfs_grace = 2m 2024-04-26T23:17:29.548 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-26T23:17:29.548 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_filestore_osds = false 2024-04-26T23:17:29.548 INFO:teuthology.orchestra.run.smithi203.stdout: mon down mkfs grace = 300 2024-04-26T23:17:29.548 INFO:teuthology.orchestra.run.smithi203.stdout: mon op complaint time = 120 2024-04-26T23:17:29.548 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-26T23:17:29.548 INFO:teuthology.orchestra.run.smithi203.stdout:[client] 2024-04-26T23:17:29.548 INFO:teuthology.orchestra.run.smithi203.stdout: rgw cache enabled = true 2024-04-26T23:17:29.548 INFO:teuthology.orchestra.run.smithi203.stdout: rgw enable ops log = true 2024-04-26T23:17:29.548 INFO:teuthology.orchestra.run.smithi203.stdout: rgw enable usage log = true 2024-04-26T23:17:29.548 INFO:teuthology.orchestra.run.smithi203.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-26T23:17:29.548 INFO:teuthology.orchestra.run.smithi203.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-26T23:17:29.549 INFO:teuthology.orchestra.run.smithi203.stdout: client mount timeout = 600 2024-04-26T23:17:29.549 INFO:teuthology.orchestra.run.smithi203.stdout: debug client = 20 2024-04-26T23:17:29.549 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2024-04-26T23:17:29.549 INFO:teuthology.orchestra.run.smithi203.stdout: pid file = "" 2024-04-26T23:17:29.549 INFO:teuthology.orchestra.run.smithi203.stdout: rados mon op timeout = 900 2024-04-26T23:17:29.549 INFO:teuthology.orchestra.run.smithi203.stdout: rados osd op timeout = 900 2024-04-26T23:17:29.549 INFO:teuthology.orchestra.run.smithi203.stdout:[mon.a] 2024-04-26T23:17:29.549 INFO:teuthology.orchestra.run.smithi203.stdout:[mon.b] 2024-04-26T23:17:29.549 INFO:teuthology.orchestra.run.smithi203.stdout:[mon.c] 2024-04-26T23:17:29.549 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.a] 2024-04-26T23:17:29.549 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.c] 2024-04-26T23:17:29.549 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.b] 2024-04-26T23:17:29.549 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.d] 2024-04-26T23:17:29.549 INFO:teuthology.orchestra.run.smithi203.stdout:[mds] 2024-04-26T23:17:29.549 INFO:teuthology.orchestra.run.smithi203.stdout: debug mds = 20 2024-04-26T23:17:29.549 INFO:teuthology.orchestra.run.smithi203.stdout: debug mds balancer = 20 2024-04-26T23:17:29.550 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2024-04-26T23:17:29.550 INFO:teuthology.orchestra.run.smithi203.stdout: mds debug frag = True 2024-04-26T23:17:29.550 INFO:teuthology.orchestra.run.smithi203.stdout: mds debug scatterstat = True 2024-04-26T23:17:29.550 INFO:teuthology.orchestra.run.smithi203.stdout: mds op complaint time = 180 2024-04-26T23:17:29.550 INFO:teuthology.orchestra.run.smithi203.stdout: mds verify scatter = True 2024-04-26T23:17:29.550 INFO:teuthology.orchestra.run.smithi203.stdout: osd op complaint time = 180 2024-04-26T23:17:29.550 INFO:teuthology.orchestra.run.smithi203.stdout: rados mon op timeout = 900 2024-04-26T23:17:29.550 INFO:teuthology.orchestra.run.smithi203.stdout: rados osd op timeout = 900 2024-04-26T23:17:29.552 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-26T23:17:29.552 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-26T23:17:29.646 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-26T23:17:29.646 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:17:29.646 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-26T23:17:29.661 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:17:29.661 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-26T23:17:29.716 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi047.front.sepia.ceph.com 2024-04-26T23:17:29.717 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:17:29.717 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-26T23:17:29.717 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-26T23:17:29.790 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:17:29.790 DEBUG:teuthology.orchestra.run.smithi047:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-26T23:17:29.846 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi203.front.sepia.ceph.com 2024-04-26T23:17:29.846 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-26T23:17:29.846 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-26T23:17:29.846 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-26T23:17:29.880 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-26T23:17:29.880 DEBUG:teuthology.orchestra.run.smithi203:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-26T23:17:29.935 INFO:tasks.ceph:Setting up mon nodes... 2024-04-26T23:17:29.936 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-26T23:17:29.936 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-26T23:17:29.986 INFO:teuthology.orchestra.run.smithi047.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-26T23:17:29.988 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-26T23:17:30.060 INFO:teuthology.orchestra.run.smithi203.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-26T23:17:30.062 INFO:tasks.ceph:Setting up mds nodes... 2024-04-26T23:17:30.062 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-26T23:17:30.112 INFO:teuthology.orchestra.run.smithi047.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-26T23:17:30.115 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-26T23:17:30.139 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-26T23:17:30.228 INFO:teuthology.orchestra.run.smithi047.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-26T23:17:30.231 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-26T23:17:30.255 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-26T23:17:30.306 INFO:teuthology.orchestra.run.smithi203.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-26T23:17:30.308 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-26T23:17:30.331 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-26T23:17:30.419 INFO:teuthology.orchestra.run.smithi203.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-26T23:17:30.422 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-26T23:17:30.445 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-26T23:17:30.445 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-26T23:17:30.486 INFO:teuthology.orchestra.run.smithi047.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-26T23:17:30.498 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-26T23:17:30.579 INFO:teuthology.orchestra.run.smithi047.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-26T23:17:30.591 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-26T23:17:30.632 INFO:teuthology.orchestra.run.smithi203.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-26T23:17:30.645 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-26T23:17:30.723 INFO:teuthology.orchestra.run.smithi203.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-26T23:17:30.735 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-26T23:17:30.735 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi047.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi203.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-26T23:17:30.735 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-26T23:17:30.759 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T23:17:30.760 INFO:tasks.ceph:role: osd.0 2024-04-26T23:17:30.760 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi047.front.sepia.ceph.com 2024-04-26T23:17:30.760 DEBUG:teuthology.orchestra.run.smithi047:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-26T23:17:30.824 INFO:teuthology.orchestra.run.smithi047.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T23:17:30.824 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T23:17:30.824 INFO:teuthology.orchestra.run.smithi047.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T23:17:30.824 INFO:teuthology.orchestra.run.smithi047.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T23:17:30.824 INFO:teuthology.orchestra.run.smithi047.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T23:17:30.824 INFO:teuthology.orchestra.run.smithi047.stdout: = sunit=0 swidth=0 blks 2024-04-26T23:17:30.824 INFO:teuthology.orchestra.run.smithi047.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T23:17:30.824 INFO:teuthology.orchestra.run.smithi047.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T23:17:30.824 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T23:17:30.824 INFO:teuthology.orchestra.run.smithi047.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T23:17:30.856 INFO:teuthology.orchestra.run.smithi047.stdout:Discarding blocks...Done. 2024-04-26T23:17:30.858 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi047.front.sepia.ceph.com -o noatime 2024-04-26T23:17:30.858 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-26T23:17:30.927 DEBUG:teuthology.orchestra.run.smithi047:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-26T23:17:30.996 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-26T23:17:31.062 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T23:17:31.062 INFO:tasks.ceph:role: osd.1 2024-04-26T23:17:31.062 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi047.front.sepia.ceph.com 2024-04-26T23:17:31.062 DEBUG:teuthology.orchestra.run.smithi047:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-26T23:17:31.126 INFO:teuthology.orchestra.run.smithi047.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T23:17:31.126 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T23:17:31.126 INFO:teuthology.orchestra.run.smithi047.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T23:17:31.126 INFO:teuthology.orchestra.run.smithi047.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T23:17:31.126 INFO:teuthology.orchestra.run.smithi047.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T23:17:31.126 INFO:teuthology.orchestra.run.smithi047.stdout: = sunit=0 swidth=0 blks 2024-04-26T23:17:31.126 INFO:teuthology.orchestra.run.smithi047.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T23:17:31.126 INFO:teuthology.orchestra.run.smithi047.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T23:17:31.126 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T23:17:31.126 INFO:teuthology.orchestra.run.smithi047.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T23:17:31.159 INFO:teuthology.orchestra.run.smithi047.stdout:Discarding blocks...Done. 2024-04-26T23:17:31.160 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi047.front.sepia.ceph.com -o noatime 2024-04-26T23:17:31.160 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-26T23:17:31.235 DEBUG:teuthology.orchestra.run.smithi047:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-26T23:17:31.302 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-26T23:17:31.367 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T23:17:31.367 INFO:tasks.ceph:role: osd.2 2024-04-26T23:17:31.367 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi047.front.sepia.ceph.com 2024-04-26T23:17:31.367 DEBUG:teuthology.orchestra.run.smithi047:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-26T23:17:31.431 INFO:teuthology.orchestra.run.smithi047.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T23:17:31.431 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T23:17:31.431 INFO:teuthology.orchestra.run.smithi047.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T23:17:31.431 INFO:teuthology.orchestra.run.smithi047.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T23:17:31.431 INFO:teuthology.orchestra.run.smithi047.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T23:17:31.431 INFO:teuthology.orchestra.run.smithi047.stdout: = sunit=0 swidth=0 blks 2024-04-26T23:17:31.431 INFO:teuthology.orchestra.run.smithi047.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T23:17:31.431 INFO:teuthology.orchestra.run.smithi047.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T23:17:31.431 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T23:17:31.431 INFO:teuthology.orchestra.run.smithi047.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T23:17:31.463 INFO:teuthology.orchestra.run.smithi047.stdout:Discarding blocks...Done. 2024-04-26T23:17:31.465 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi047.front.sepia.ceph.com -o noatime 2024-04-26T23:17:31.465 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-26T23:17:31.533 DEBUG:teuthology.orchestra.run.smithi047:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-26T23:17:31.602 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-26T23:17:31.666 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T23:17:31.666 INFO:tasks.ceph:role: osd.3 2024-04-26T23:17:31.666 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi047.front.sepia.ceph.com 2024-04-26T23:17:31.666 DEBUG:teuthology.orchestra.run.smithi047:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-26T23:17:31.729 INFO:teuthology.orchestra.run.smithi047.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T23:17:31.730 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T23:17:31.730 INFO:teuthology.orchestra.run.smithi047.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T23:17:31.730 INFO:teuthology.orchestra.run.smithi047.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T23:17:31.730 INFO:teuthology.orchestra.run.smithi047.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T23:17:31.730 INFO:teuthology.orchestra.run.smithi047.stdout: = sunit=0 swidth=0 blks 2024-04-26T23:17:31.730 INFO:teuthology.orchestra.run.smithi047.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T23:17:31.730 INFO:teuthology.orchestra.run.smithi047.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T23:17:31.730 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T23:17:31.730 INFO:teuthology.orchestra.run.smithi047.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T23:17:31.763 INFO:teuthology.orchestra.run.smithi047.stdout:Discarding blocks...Done. 2024-04-26T23:17:31.764 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi047.front.sepia.ceph.com -o noatime 2024-04-26T23:17:31.764 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-26T23:17:31.833 DEBUG:teuthology.orchestra.run.smithi047:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-26T23:17:31.901 DEBUG:teuthology.orchestra.run.smithi047:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T23:17:31.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:17:31.993+0000 7fe07760e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-26T23:17:31.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:17:31.993+0000 7fe07760e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-26T23:17:31.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:17:31.993+0000 7fe07760e640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-26T23:17:31.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:17:31.993+0000 7fe07760e640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-26T23:17:31.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:17:31.993+0000 7fe07760e640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-26T23:17:31.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:17:31.994+0000 7fe07760e640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-26T23:17:34.843 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-26T23:17:34.872 DEBUG:teuthology.orchestra.run.smithi047:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T23:17:34.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:17:34.961+0000 7f188781b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-26T23:17:34.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:17:34.961+0000 7f188781b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-26T23:17:34.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:17:34.961+0000 7f188781b640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-26T23:17:34.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:17:34.961+0000 7f188781b640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-26T23:17:34.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:17:34.961+0000 7f188781b640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-26T23:17:34.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:17:34.962+0000 7f188781b640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-26T23:17:37.813 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-26T23:17:37.842 DEBUG:teuthology.orchestra.run.smithi047:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T23:17:37.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:17:37.932+0000 7f8504e63640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-26T23:17:37.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:17:37.932+0000 7f8504e63640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-26T23:17:37.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:17:37.932+0000 7f8504e63640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-26T23:17:37.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:17:37.932+0000 7f8504e63640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-26T23:17:37.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:17:37.932+0000 7f8504e63640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-26T23:17:37.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:17:37.932+0000 7f8504e63640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-26T23:17:40.793 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-26T23:17:40.822 DEBUG:teuthology.orchestra.run.smithi047:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T23:17:40.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:17:40.910+0000 7f6faa412640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-26T23:17:40.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:17:40.911+0000 7f6faa412640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-26T23:17:40.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:17:40.911+0000 7f6faa412640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-26T23:17:40.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:17:40.911+0000 7f6faa412640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-26T23:17:40.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:17:40.911+0000 7f6faa412640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-26T23:17:40.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:17:40.911+0000 7f6faa412640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-26T23:17:43.759 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-26T23:17:43.789 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-26T23:17:43.818 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T23:17:43.818 INFO:tasks.ceph:role: osd.4 2024-04-26T23:17:43.818 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi203.front.sepia.ceph.com 2024-04-26T23:17:43.818 DEBUG:teuthology.orchestra.run.smithi203:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-26T23:17:43.893 INFO:teuthology.orchestra.run.smithi203.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T23:17:43.893 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T23:17:43.893 INFO:teuthology.orchestra.run.smithi203.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T23:17:43.893 INFO:teuthology.orchestra.run.smithi203.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T23:17:43.893 INFO:teuthology.orchestra.run.smithi203.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T23:17:43.894 INFO:teuthology.orchestra.run.smithi203.stdout: = sunit=0 swidth=0 blks 2024-04-26T23:17:43.894 INFO:teuthology.orchestra.run.smithi203.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T23:17:43.894 INFO:teuthology.orchestra.run.smithi203.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T23:17:43.894 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T23:17:43.894 INFO:teuthology.orchestra.run.smithi203.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T23:17:44.406 INFO:teuthology.orchestra.run.smithi203.stdout:Discarding blocks...Done. 2024-04-26T23:17:44.408 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi203.front.sepia.ceph.com -o noatime 2024-04-26T23:17:44.408 DEBUG:teuthology.orchestra.run.smithi203:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-26T23:17:44.530 DEBUG:teuthology.orchestra.run.smithi203:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-26T23:17:44.563 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-26T23:17:44.629 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T23:17:44.629 INFO:tasks.ceph:role: osd.5 2024-04-26T23:17:44.629 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi203.front.sepia.ceph.com 2024-04-26T23:17:44.630 DEBUG:teuthology.orchestra.run.smithi203:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-26T23:17:44.703 INFO:teuthology.orchestra.run.smithi203.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T23:17:44.703 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T23:17:44.703 INFO:teuthology.orchestra.run.smithi203.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T23:17:44.703 INFO:teuthology.orchestra.run.smithi203.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T23:17:44.704 INFO:teuthology.orchestra.run.smithi203.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T23:17:44.704 INFO:teuthology.orchestra.run.smithi203.stdout: = sunit=0 swidth=0 blks 2024-04-26T23:17:44.704 INFO:teuthology.orchestra.run.smithi203.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T23:17:44.704 INFO:teuthology.orchestra.run.smithi203.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T23:17:44.704 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T23:17:44.704 INFO:teuthology.orchestra.run.smithi203.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T23:17:45.162 INFO:teuthology.orchestra.run.smithi203.stdout:Discarding blocks...Done. 2024-04-26T23:17:45.165 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi203.front.sepia.ceph.com -o noatime 2024-04-26T23:17:45.165 DEBUG:teuthology.orchestra.run.smithi203:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-26T23:17:45.250 DEBUG:teuthology.orchestra.run.smithi203:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-26T23:17:45.278 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-26T23:17:45.342 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T23:17:45.342 INFO:tasks.ceph:role: osd.6 2024-04-26T23:17:45.342 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi203.front.sepia.ceph.com 2024-04-26T23:17:45.343 DEBUG:teuthology.orchestra.run.smithi203:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-26T23:17:45.413 INFO:teuthology.orchestra.run.smithi203.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T23:17:45.413 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T23:17:45.414 INFO:teuthology.orchestra.run.smithi203.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T23:17:45.414 INFO:teuthology.orchestra.run.smithi203.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T23:17:45.414 INFO:teuthology.orchestra.run.smithi203.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T23:17:45.414 INFO:teuthology.orchestra.run.smithi203.stdout: = sunit=0 swidth=0 blks 2024-04-26T23:17:45.414 INFO:teuthology.orchestra.run.smithi203.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T23:17:45.414 INFO:teuthology.orchestra.run.smithi203.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T23:17:45.414 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T23:17:45.414 INFO:teuthology.orchestra.run.smithi203.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T23:17:45.901 INFO:teuthology.orchestra.run.smithi203.stdout:Discarding blocks...Done. 2024-04-26T23:17:45.903 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi203.front.sepia.ceph.com -o noatime 2024-04-26T23:17:45.903 DEBUG:teuthology.orchestra.run.smithi203:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-26T23:17:46.041 DEBUG:teuthology.orchestra.run.smithi203:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-26T23:17:46.070 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-26T23:17:46.133 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T23:17:46.133 INFO:tasks.ceph:role: osd.7 2024-04-26T23:17:46.134 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi203.front.sepia.ceph.com 2024-04-26T23:17:46.134 DEBUG:teuthology.orchestra.run.smithi203:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-26T23:17:46.205 INFO:teuthology.orchestra.run.smithi203.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T23:17:46.205 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T23:17:46.205 INFO:teuthology.orchestra.run.smithi203.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T23:17:46.205 INFO:teuthology.orchestra.run.smithi203.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T23:17:46.205 INFO:teuthology.orchestra.run.smithi203.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T23:17:46.205 INFO:teuthology.orchestra.run.smithi203.stdout: = sunit=0 swidth=0 blks 2024-04-26T23:17:46.205 INFO:teuthology.orchestra.run.smithi203.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T23:17:46.205 INFO:teuthology.orchestra.run.smithi203.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T23:17:46.205 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T23:17:46.205 INFO:teuthology.orchestra.run.smithi203.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T23:17:46.708 INFO:teuthology.orchestra.run.smithi203.stdout:Discarding blocks...Done. 2024-04-26T23:17:46.710 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi203.front.sepia.ceph.com -o noatime 2024-04-26T23:17:46.711 DEBUG:teuthology.orchestra.run.smithi203:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-26T23:17:46.847 DEBUG:teuthology.orchestra.run.smithi203:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-26T23:17:46.880 DEBUG:teuthology.orchestra.run.smithi203:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T23:17:46.974 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:17:46.973+0000 7fca32c2d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-26T23:17:46.974 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:17:46.973+0000 7fca32c2d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-26T23:17:46.974 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:17:46.973+0000 7fca32c2d640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-26T23:17:46.975 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:17:46.973+0000 7fca32c2d640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-26T23:17:46.975 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:17:46.973+0000 7fca32c2d640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-26T23:17:46.975 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:17:46.974+0000 7fca32c2d640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-26T23:17:49.832 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-26T23:17:49.862 DEBUG:teuthology.orchestra.run.smithi203:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T23:17:49.957 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:17:49.956+0000 7feacd41e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-26T23:17:49.957 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:17:49.956+0000 7feacd41e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-26T23:17:49.957 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:17:49.956+0000 7feacd41e640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-26T23:17:49.957 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:17:49.956+0000 7feacd41e640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-26T23:17:49.957 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:17:49.956+0000 7feacd41e640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-26T23:17:49.958 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:17:49.957+0000 7feacd41e640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-26T23:17:52.811 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-26T23:17:52.839 DEBUG:teuthology.orchestra.run.smithi203:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T23:17:52.927 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:17:52.926+0000 7f64b9733640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-26T23:17:52.928 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:17:52.927+0000 7f64b9733640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-26T23:17:52.928 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:17:52.927+0000 7f64b9733640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-26T23:17:52.928 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:17:52.927+0000 7f64b9733640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-26T23:17:52.928 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:17:52.927+0000 7f64b9733640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-26T23:17:52.928 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:17:52.927+0000 7f64b9733640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-26T23:17:55.787 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-26T23:17:55.814 DEBUG:teuthology.orchestra.run.smithi203:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T23:17:55.900 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:17:55.899+0000 7fdcf2843640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-26T23:17:55.900 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:17:55.899+0000 7fdcf2843640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-26T23:17:55.900 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:17:55.899+0000 7fdcf2843640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-26T23:17:55.900 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:17:55.899+0000 7fdcf2843640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-26T23:17:55.900 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:17:55.899+0000 7fdcf2843640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-26T23:17:55.900 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:17:55.899+0000 7fdcf2843640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-26T23:17:57.549 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:17:57.556 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:17:58.747 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-26T23:17:58.772 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-26T23:17:58.772 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:17:58.772 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-26T23:17:58.802 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:17:58.802 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-26T23:17:58.870 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:17:58.870 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-26T23:17:58.937 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:17:58.937 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-26T23:17:59.004 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:17:59.004 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-26T23:17:59.070 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:17:59.070 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-26T23:17:59.136 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:17:59.136 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-26T23:17:59.203 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-26T23:17:59.203 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-26T23:17:59.230 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-26T23:17:59.231 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-26T23:17:59.297 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-26T23:17:59.297 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-26T23:17:59.363 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-26T23:17:59.363 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-26T23:17:59.428 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-26T23:17:59.429 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-26T23:17:59.494 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-26T23:17:59.494 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-26T23:17:59.557 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-26T23:17:59.557 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-26T23:17:59.621 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:17:59.622 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-26T23:17:59.639 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:17:59.639 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-26T23:17:59.697 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-26T23:17:59.697 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-26T23:17:59.712 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-26T23:17:59.712 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-26T23:17:59.767 INFO:tasks.ceph:Adding keys to all mons... 2024-04-26T23:17:59.767 DEBUG:teuthology.orchestra.run.smithi047:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-26T23:17:59.769 DEBUG:teuthology.orchestra.run.smithi203:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-26T23:17:59.811 INFO:teuthology.orchestra.run.smithi047.stdout:[mgr.y] 2024-04-26T23:17:59.811 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQAJNixmQ2HSOhAACgAm3LAXwI4PB/P1+oRdyw== 2024-04-26T23:17:59.811 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.a] 2024-04-26T23:17:59.811 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQAKNixmrb+7BhAA9Tl5qHRAAITpDe0o1RpB8A== 2024-04-26T23:17:59.811 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.c] 2024-04-26T23:17:59.811 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQAKNixmy+yoDRAAeSQ/dzhhLyysouyco6YrRw== 2024-04-26T23:17:59.811 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.0] 2024-04-26T23:17:59.811 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQALNixm3dBSOxAA0heGY4osjvaNAB25znAAZw== 2024-04-26T23:17:59.811 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.1] 2024-04-26T23:17:59.811 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQAONixmH0ZoORAAaNuvYE66+ypC0bIeWxVaPw== 2024-04-26T23:17:59.811 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.2] 2024-04-26T23:17:59.812 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQARNixmQCOoNxAAl7vmPd2PBbBHjl7VJobIeg== 2024-04-26T23:17:59.812 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.3] 2024-04-26T23:17:59.812 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQAUNixmMtdiNhAA+xeZUbroEshbAQO7BQm9kg== 2024-04-26T23:17:59.812 INFO:teuthology.orchestra.run.smithi047.stdout:[mgr.x] 2024-04-26T23:17:59.812 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQAKNixmf2OaAxAAv3TUVplM33vJKhqsWyV84w== 2024-04-26T23:17:59.812 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.b] 2024-04-26T23:17:59.812 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQAKNixmZYgrEhAA4XwxVuQT3PVAfsalOSSpRg== 2024-04-26T23:17:59.812 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.d] 2024-04-26T23:17:59.812 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQAKNixmr1gJGRAA2M7//N7u9lk7edXxKxKdJQ== 2024-04-26T23:17:59.812 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.4] 2024-04-26T23:17:59.812 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQAaNixmiYETOhAAkz/BfU0a8SnawltF0pBU/A== 2024-04-26T23:17:59.812 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.5] 2024-04-26T23:17:59.812 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQAdNixmKCoPORAAp3FfD88sxG9IlXK1HqpiPA== 2024-04-26T23:17:59.812 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.6] 2024-04-26T23:17:59.813 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQAgNixm/ipONxAAzgJ2wJAHFbHlv09nqA3soQ== 2024-04-26T23:17:59.813 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.7] 2024-04-26T23:17:59.813 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQAjNixmXDWkNRAAJYh2B7nH7ViaGPii7Yk69A== 2024-04-26T23:17:59.813 INFO:teuthology.orchestra.run.smithi047.stdout:[client.0] 2024-04-26T23:17:59.813 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQAKNixmA5QFHRAAT89WPyzDs0acPluyvU4svw== 2024-04-26T23:17:59.813 INFO:teuthology.orchestra.run.smithi047.stdout:[client.2] 2024-04-26T23:17:59.813 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQAKNixmTb2VIhAAhoBSsrzlPIsljZt1jZVscQ== 2024-04-26T23:17:59.813 INFO:teuthology.orchestra.run.smithi047.stdout:[client.1] 2024-04-26T23:17:59.813 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQAKNixmtRW8JRAAqh+Q2ZhqvdhTxty04GtU2g== 2024-04-26T23:17:59.813 INFO:teuthology.orchestra.run.smithi047.stdout:[client.3] 2024-04-26T23:17:59.813 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQAKNixmDE8nKxAA9zbeR9CqdL0p8gcuy3k/kQ== 2024-04-26T23:17:59.829 INFO:teuthology.orchestra.run.smithi203.stdout:[mgr.y] 2024-04-26T23:17:59.829 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQAJNixmQ2HSOhAACgAm3LAXwI4PB/P1+oRdyw== 2024-04-26T23:17:59.829 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.a] 2024-04-26T23:17:59.829 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQAKNixmrb+7BhAA9Tl5qHRAAITpDe0o1RpB8A== 2024-04-26T23:17:59.829 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.c] 2024-04-26T23:17:59.829 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQAKNixmy+yoDRAAeSQ/dzhhLyysouyco6YrRw== 2024-04-26T23:17:59.829 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.0] 2024-04-26T23:17:59.830 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQALNixm3dBSOxAA0heGY4osjvaNAB25znAAZw== 2024-04-26T23:17:59.830 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.1] 2024-04-26T23:17:59.830 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQAONixmH0ZoORAAaNuvYE66+ypC0bIeWxVaPw== 2024-04-26T23:17:59.830 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.2] 2024-04-26T23:17:59.830 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQARNixmQCOoNxAAl7vmPd2PBbBHjl7VJobIeg== 2024-04-26T23:17:59.830 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.3] 2024-04-26T23:17:59.830 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQAUNixmMtdiNhAA+xeZUbroEshbAQO7BQm9kg== 2024-04-26T23:17:59.830 INFO:teuthology.orchestra.run.smithi203.stdout:[mgr.x] 2024-04-26T23:17:59.830 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQAKNixmf2OaAxAAv3TUVplM33vJKhqsWyV84w== 2024-04-26T23:17:59.830 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.b] 2024-04-26T23:17:59.830 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQAKNixmZYgrEhAA4XwxVuQT3PVAfsalOSSpRg== 2024-04-26T23:17:59.830 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.d] 2024-04-26T23:17:59.830 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQAKNixmr1gJGRAA2M7//N7u9lk7edXxKxKdJQ== 2024-04-26T23:17:59.830 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.4] 2024-04-26T23:17:59.830 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQAaNixmiYETOhAAkz/BfU0a8SnawltF0pBU/A== 2024-04-26T23:17:59.831 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.5] 2024-04-26T23:17:59.831 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQAdNixmKCoPORAAp3FfD88sxG9IlXK1HqpiPA== 2024-04-26T23:17:59.831 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.6] 2024-04-26T23:17:59.831 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQAgNixm/ipONxAAzgJ2wJAHFbHlv09nqA3soQ== 2024-04-26T23:17:59.831 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.7] 2024-04-26T23:17:59.831 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQAjNixmXDWkNRAAJYh2B7nH7ViaGPii7Yk69A== 2024-04-26T23:17:59.831 INFO:teuthology.orchestra.run.smithi203.stdout:[client.0] 2024-04-26T23:17:59.831 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQAKNixmA5QFHRAAT89WPyzDs0acPluyvU4svw== 2024-04-26T23:17:59.831 INFO:teuthology.orchestra.run.smithi203.stdout:[client.2] 2024-04-26T23:17:59.831 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQAKNixmTb2VIhAAhoBSsrzlPIsljZt1jZVscQ== 2024-04-26T23:17:59.831 INFO:teuthology.orchestra.run.smithi203.stdout:[client.1] 2024-04-26T23:17:59.831 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQAKNixmtRW8JRAAqh+Q2ZhqvdhTxty04GtU2g== 2024-04-26T23:17:59.831 INFO:teuthology.orchestra.run.smithi203.stdout:[client.3] 2024-04-26T23:17:59.831 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQAKNixmDE8nKxAA9zbeR9CqdL0p8gcuy3k/kQ== 2024-04-26T23:17:59.832 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T23:17:59.857 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T23:17:59.921 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T23:17:59.924 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T23:17:59.977 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T23:17:59.980 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T23:18:00.032 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T23:18:00.034 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T23:18:00.083 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T23:18:00.085 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T23:18:00.134 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T23:18:00.136 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T23:18:00.185 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T23:18:00.188 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T23:18:00.237 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T23:18:00.239 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T23:18:00.288 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T23:18:00.293 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T23:18:00.347 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T23:18:00.352 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T23:18:00.409 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T23:18:00.419 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T23:18:00.470 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T23:18:00.476 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T23:18:00.568 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T23:18:00.573 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T23:18:00.624 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T23:18:00.632 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T23:18:00.683 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T23:18:00.686 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T23:18:00.738 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T23:18:00.744 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T23:18:00.797 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T23:18:00.800 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T23:18:00.853 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T23:18:00.856 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T23:18:00.907 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-26T23:18:00.907 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-26T23:18:00.932 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-26T23:18:01.027 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-26T23:18:01.052 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-26T23:18:01.076 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-26T23:18:01.170 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-26T23:18:01.194 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-26T23:18:01.258 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-26T23:18:01.352 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-26T23:18:01.375 DEBUG:teuthology.orchestra.run.smithi047:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-26T23:18:01.377 DEBUG:teuthology.orchestra.run.smithi203:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-26T23:18:01.430 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-26T23:18:01.430 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-26T23:18:01.431 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-26T23:18:01.433 INFO:tasks.ceph.mon.a:Started 2024-04-26T23:18:01.433 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-26T23:18:01.434 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-26T23:18:01.473 INFO:tasks.ceph.mon.b:Started 2024-04-26T23:18:01.473 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-26T23:18:01.473 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-26T23:18:01.475 INFO:tasks.ceph.mon.c:Started 2024-04-26T23:18:01.475 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-26T23:18:01.475 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-26T23:18:01.476 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-26T23:18:01.478 INFO:tasks.ceph.mgr.y:Started 2024-04-26T23:18:01.478 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-26T23:18:01.478 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-26T23:18:01.480 INFO:tasks.ceph.mgr.x:Started 2024-04-26T23:18:01.480 INFO:tasks.ceph:Setting crush tunables to default 2024-04-26T23:18:01.480 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --cluster ceph osd crush tunables default 2024-04-26T23:18:01.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:01.581+0000 7ff50b059640 1 Processor -- start 2024-04-26T23:18:01.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:01.581+0000 7ff50b059640 1 -- start start 2024-04-26T23:18:01.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:01.582+0000 7ff50b059640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff50415f5a0 0x7ff5040d5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:01.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:01.582+0000 7ff50b059640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff504163610 0x7ff5040cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:01.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:01.582+0000 7ff50b059640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5040d1a70 con 0x7ff5040d2830 2024-04-26T23:18:01.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:01.582+0000 7ff50b059640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7ff504163d30 con 0x7ff504163610 2024-04-26T23:18:01.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:01.582+0000 7ff50b059640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7ff5040d1d50 con 0x7ff50415f5a0 2024-04-26T23:18:01.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:01.582+0000 7ff5095f6640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff504163610 0x7ff5040cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:01.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:01.582+0000 7ff5095f6640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff504163610 0x7ff5040cfda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44516/0 (socket says 172.21.15.47:44516) 2024-04-26T23:18:01.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:01.582+0000 7ff5095f6640 1 -- 172.21.15.47:0/4084914320 learned_addr learned my addr 172.21.15.47:0/4084914320 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:01.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:01.582+0000 7ff5095f6640 1 -- 172.21.15.47:0/4084914320 >> v2:172.21.15.47:3300/0 conn(0x7ff504163610 msgr2=0x7ff5040cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-26T23:18:01.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:01.582+0000 7ff5095f6640 1 -- 172.21.15.47:0/4084914320 >> v2:172.21.15.47:3300/0 conn(0x7ff504163610 msgr2=0x7ff5040cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T23:18:01.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:01.582+0000 7ff5095f6640 1 --2- 172.21.15.47:0/4084914320 >> v2:172.21.15.47:3300/0 conn(0x7ff504163610 0x7ff5040cfda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T23:18:01.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:01.582+0000 7ff508df5640 1 --2- 172.21.15.47:0/4084914320 >> v2:172.21.15.203:3300/0 conn(0x7ff50415f5a0 0x7ff5040d5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:01.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:01.582+0000 7ff5095f6640 1 --2- 172.21.15.47:0/4084914320 >> v2:172.21.15.47:3300/0 conn(0x7ff504163610 0x7ff5040cfda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T23:18:01.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:01.582+0000 7ff508df5640 1 -- 172.21.15.47:0/4084914320 >> v2:172.21.15.203:3300/0 conn(0x7ff50415f5a0 msgr2=0x7ff5040d5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-26T23:18:01.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:01.582+0000 7ff508df5640 1 -- 172.21.15.47:0/4084914320 >> v2:172.21.15.203:3300/0 conn(0x7ff50415f5a0 msgr2=0x7ff5040d5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T23:18:01.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:01.582+0000 7ff508df5640 1 --2- 172.21.15.47:0/4084914320 >> v2:172.21.15.203:3300/0 conn(0x7ff50415f5a0 0x7ff5040d5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T23:18:01.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:01.582+0000 7ff508df5640 1 --2- 172.21.15.47:0/4084914320 >> v2:172.21.15.203:3300/0 conn(0x7ff50415f5a0 0x7ff5040d5cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T23:18:01.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:01.783+0000 7ff5095f6640 1 --2- 172.21.15.47:0/4084914320 >> v2:172.21.15.47:3300/0 conn(0x7ff504163610 0x7ff5040cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:01.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:01.783+0000 7ff508df5640 1 --2- 172.21.15.47:0/4084914320 >> v2:172.21.15.203:3300/0 conn(0x7ff50415f5a0 0x7ff5040d5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:01.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:01.783+0000 7ff5095f6640 1 -- 172.21.15.47:0/4084914320 >> v2:172.21.15.47:3300/0 conn(0x7ff504163610 msgr2=0x7ff5040cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-26T23:18:01.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:01.783+0000 7ff5095f6640 1 -- 172.21.15.47:0/4084914320 >> v2:172.21.15.47:3300/0 conn(0x7ff504163610 msgr2=0x7ff5040cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T23:18:01.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:01.783+0000 7ff5095f6640 1 --2- 172.21.15.47:0/4084914320 >> v2:172.21.15.47:3300/0 conn(0x7ff504163610 0x7ff5040cfda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T23:18:01.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:01.783+0000 7ff5095f6640 1 --2- 172.21.15.47:0/4084914320 >> v2:172.21.15.47:3300/0 conn(0x7ff504163610 0x7ff5040cfda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T23:18:01.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:01.783+0000 7ff508df5640 1 -- 172.21.15.47:0/4084914320 >> v2:172.21.15.203:3300/0 conn(0x7ff50415f5a0 msgr2=0x7ff5040d5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-26T23:18:01.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:01.783+0000 7ff508df5640 1 -- 172.21.15.47:0/4084914320 >> v2:172.21.15.203:3300/0 conn(0x7ff50415f5a0 msgr2=0x7ff5040d5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T23:18:01.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:01.783+0000 7ff508df5640 1 --2- 172.21.15.47:0/4084914320 >> v2:172.21.15.203:3300/0 conn(0x7ff50415f5a0 0x7ff5040d5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T23:18:01.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:01.783+0000 7ff508df5640 1 --2- 172.21.15.47:0/4084914320 >> v2:172.21.15.203:3300/0 conn(0x7ff50415f5a0 0x7ff5040d5cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T23:18:02.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:02.184+0000 7ff5095f6640 1 --2- 172.21.15.47:0/4084914320 >> v2:172.21.15.47:3300/0 conn(0x7ff504163610 0x7ff5040cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:02.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:02.184+0000 7ff508df5640 1 --2- 172.21.15.47:0/4084914320 >> v2:172.21.15.203:3300/0 conn(0x7ff50415f5a0 0x7ff5040d5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:02.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:02.184+0000 7ff5095f6640 1 -- 172.21.15.47:0/4084914320 >> v2:172.21.15.47:3300/0 conn(0x7ff504163610 msgr2=0x7ff5040cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-26T23:18:02.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:02.184+0000 7ff5095f6640 1 -- 172.21.15.47:0/4084914320 >> v2:172.21.15.47:3300/0 conn(0x7ff504163610 msgr2=0x7ff5040cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T23:18:02.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:02.184+0000 7ff5095f6640 1 --2- 172.21.15.47:0/4084914320 >> v2:172.21.15.47:3300/0 conn(0x7ff504163610 0x7ff5040cfda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T23:18:02.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:02.184+0000 7ff508df5640 1 -- 172.21.15.47:0/4084914320 >> v2:172.21.15.203:3300/0 conn(0x7ff50415f5a0 msgr2=0x7ff5040d5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-26T23:18:02.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:02.184+0000 7ff5095f6640 1 --2- 172.21.15.47:0/4084914320 >> v2:172.21.15.47:3300/0 conn(0x7ff504163610 0x7ff5040cfda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T23:18:02.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:02.184+0000 7ff508df5640 1 -- 172.21.15.47:0/4084914320 >> v2:172.21.15.203:3300/0 conn(0x7ff50415f5a0 msgr2=0x7ff5040d5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T23:18:02.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:02.184+0000 7ff508df5640 1 --2- 172.21.15.47:0/4084914320 >> v2:172.21.15.203:3300/0 conn(0x7ff50415f5a0 0x7ff5040d5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T23:18:02.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:02.184+0000 7ff508df5640 1 --2- 172.21.15.47:0/4084914320 >> v2:172.21.15.203:3300/0 conn(0x7ff50415f5a0 0x7ff5040d5cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T23:18:02.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:02.985+0000 7ff5095f6640 1 --2- 172.21.15.47:0/4084914320 >> v2:172.21.15.47:3300/0 conn(0x7ff504163610 0x7ff5040cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:02.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:02.985+0000 7ff508df5640 1 --2- 172.21.15.47:0/4084914320 >> v2:172.21.15.203:3300/0 conn(0x7ff50415f5a0 0x7ff5040d5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:02.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:02.986+0000 7ff5095f6640 1 -- 172.21.15.47:0/4084914320 >> v2:172.21.15.47:3300/0 conn(0x7ff504163610 msgr2=0x7ff5040cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-26T23:18:02.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:02.986+0000 7ff5095f6640 1 -- 172.21.15.47:0/4084914320 >> v2:172.21.15.47:3300/0 conn(0x7ff504163610 msgr2=0x7ff5040cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T23:18:02.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:02.986+0000 7ff5095f6640 1 --2- 172.21.15.47:0/4084914320 >> v2:172.21.15.47:3300/0 conn(0x7ff504163610 0x7ff5040cfda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T23:18:02.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:02.986+0000 7ff508df5640 1 -- 172.21.15.47:0/4084914320 >> v2:172.21.15.203:3300/0 conn(0x7ff50415f5a0 msgr2=0x7ff5040d5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-26T23:18:02.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:02.986+0000 7ff508df5640 1 -- 172.21.15.47:0/4084914320 >> v2:172.21.15.203:3300/0 conn(0x7ff50415f5a0 msgr2=0x7ff5040d5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T23:18:02.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:02.986+0000 7ff508df5640 1 --2- 172.21.15.47:0/4084914320 >> v2:172.21.15.203:3300/0 conn(0x7ff50415f5a0 0x7ff5040d5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T23:18:02.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:02.986+0000 7ff5095f6640 1 --2- 172.21.15.47:0/4084914320 >> v2:172.21.15.47:3300/0 conn(0x7ff504163610 0x7ff5040cfda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-26T23:18:02.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:02.986+0000 7ff508df5640 1 --2- 172.21.15.47:0/4084914320 >> v2:172.21.15.203:3300/0 conn(0x7ff50415f5a0 0x7ff5040d5cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-26T23:18:04.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:04.583+0000 7ff5027fc640 1 -- 172.21.15.47:0/4084914320 >> v2:172.21.15.203:3300/0 conn(0x7ff50415f5a0 msgr2=0x7ff5040d5cb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T23:18:04.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:04.583+0000 7ff5027fc640 1 --2- 172.21.15.47:0/4084914320 >> v2:172.21.15.203:3300/0 conn(0x7ff50415f5a0 0x7ff5040d5cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:04.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:04.583+0000 7ff5027fc640 1 -- 172.21.15.47:0/4084914320 >> v2:172.21.15.47:3300/0 conn(0x7ff504163610 msgr2=0x7ff5040cfda0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T23:18:04.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:04.583+0000 7ff5027fc640 1 --2- 172.21.15.47:0/4084914320 >> v2:172.21.15.47:3300/0 conn(0x7ff504163610 0x7ff5040cfda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:04.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:04.583+0000 7ff5027fc640 1 -- 172.21.15.47:0/4084914320 >> v1:172.21.15.203:6789/0 conn(0x7ff5040d2830 legacy=0x7ff504162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:04.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:04.583+0000 7ff5027fc640 1 --2- 172.21.15.47:0/4084914320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4f0003690 0x7ff4f0005ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:04.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:04.583+0000 7ff5027fc640 1 -- 172.21.15.47:0/4084914320 --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5040d2030 con 0x7ff4f0006150 2024-04-26T23:18:04.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:04.583+0000 7ff5027fc640 1 -- 172.21.15.47:0/4084914320 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7ff5040d1d50 con 0x7ff4f0003690 2024-04-26T23:18:04.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:04.583+0000 7ff508df5640 1 --2- 172.21.15.47:0/4084914320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4f0003690 0x7ff4f0005ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:04.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:04.583+0000 7ff508df5640 1 -- 172.21.15.47:0/4084914320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4f0003690 msgr2=0x7ff4f0005ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-26T23:18:04.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:04.583+0000 7ff508df5640 1 -- 172.21.15.47:0/4084914320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4f0003690 msgr2=0x7ff4f0005ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T23:18:04.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:04.583+0000 7ff508df5640 1 --2- 172.21.15.47:0/4084914320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4f0003690 0x7ff4f0005ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T23:18:04.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:04.583+0000 7ff508df5640 1 --2- 172.21.15.47:0/4084914320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4f0003690 0x7ff4f0005ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T23:18:04.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:04.784+0000 7ff508df5640 1 --2- 172.21.15.47:0/4084914320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4f0003690 0x7ff4f0005ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:04.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:04.784+0000 7ff508df5640 1 -- 172.21.15.47:0/4084914320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4f0003690 msgr2=0x7ff4f0005ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-26T23:18:04.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:04.784+0000 7ff508df5640 1 -- 172.21.15.47:0/4084914320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4f0003690 msgr2=0x7ff4f0005ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T23:18:04.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:04.784+0000 7ff508df5640 1 --2- 172.21.15.47:0/4084914320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4f0003690 0x7ff4f0005ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T23:18:04.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:04.784+0000 7ff508df5640 1 --2- 172.21.15.47:0/4084914320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4f0003690 0x7ff4f0005ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T23:18:05.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:05.185+0000 7ff508df5640 1 --2- 172.21.15.47:0/4084914320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4f0003690 0x7ff4f0005ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:05.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:05.185+0000 7ff508df5640 1 -- 172.21.15.47:0/4084914320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4f0003690 msgr2=0x7ff4f0005ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-26T23:18:05.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:05.185+0000 7ff508df5640 1 -- 172.21.15.47:0/4084914320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4f0003690 msgr2=0x7ff4f0005ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T23:18:05.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:05.185+0000 7ff508df5640 1 --2- 172.21.15.47:0/4084914320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4f0003690 0x7ff4f0005ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T23:18:05.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:05.185+0000 7ff508df5640 1 --2- 172.21.15.47:0/4084914320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4f0003690 0x7ff4f0005ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T23:18:05.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:05.986+0000 7ff508df5640 1 --2- 172.21.15.47:0/4084914320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4f0003690 0x7ff4f0005ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:05.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:05.987+0000 7ff508df5640 1 -- 172.21.15.47:0/4084914320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4f0003690 msgr2=0x7ff4f0005ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-26T23:18:05.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:05.987+0000 7ff508df5640 1 -- 172.21.15.47:0/4084914320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4f0003690 msgr2=0x7ff4f0005ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T23:18:05.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:05.987+0000 7ff508df5640 1 --2- 172.21.15.47:0/4084914320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4f0003690 0x7ff4f0005ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T23:18:05.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:05.987+0000 7ff508df5640 1 --2- 172.21.15.47:0/4084914320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4f0003690 0x7ff4f0005ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-26T23:18:06.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.747+0000 7ff5037fe640 1 -- 172.21.15.47:0/4084914320 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1410550853 0 0) 0x7ff5040d2030 con 0x7ff4f0006150 2024-04-26T23:18:06.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.747+0000 7ff5037fe640 1 -- 172.21.15.47:0/4084914320 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff4ec003520 con 0x7ff4f0006150 2024-04-26T23:18:06.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.747+0000 7ff5037fe640 1 -- 172.21.15.47:0/4084914320 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3506680672 0 0) 0x7ff4ec003520 con 0x7ff4f0006150 2024-04-26T23:18:06.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.747+0000 7ff5037fe640 1 -- 172.21.15.47:0/4084914320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4f0003690 msgr2=0x7ff4f0005ac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T23:18:06.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.747+0000 7ff5037fe640 1 --2- 172.21.15.47:0/4084914320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4f0003690 0x7ff4f0005ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:06.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.747+0000 7ff5037fe640 1 -- 172.21.15.47:0/4084914320 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4f000af90 con 0x7ff4f0006150 2024-04-26T23:18:06.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.747+0000 7ff5037fe640 1 -- 172.21.15.47:0/4084914320 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7ff4f80016f0 con 0x7ff4f0006150 2024-04-26T23:18:06.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.747+0000 7ff5037fe640 1 -- 172.21.15.47:0/4084914320 >> v1:172.21.15.47:6789/0 conn(0x7ff4f0006150 legacy=0x7ff4f00085f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:06.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.747+0000 7ff5095f6640 1 -- 172.21.15.47:0/4084914320 reap_dead start 2024-04-26T23:18:06.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.747+0000 7ff5037fe640 1 --2- 172.21.15.47:0/4084914320 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4ec0039b0 0x7ff4ec003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:06.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.747+0000 7ff5037fe640 1 --2- 172.21.15.47:0/4084914320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4ec004800 0x7ff4ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:06.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.747+0000 7ff5037fe640 1 --2- 172.21.15.47:0/4084914320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4ec005290 0x7ff4ec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:06.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.748+0000 7ff5037fe640 1 -- 172.21.15.47:0/4084914320 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff5040d1d50 con 0x7ff4ec0039b0 2024-04-26T23:18:06.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.748+0000 7ff5037fe640 1 -- 172.21.15.47:0/4084914320 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ec00c1e0 con 0x7ff4ec005290 2024-04-26T23:18:06.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.748+0000 7ff5037fe640 1 -- 172.21.15.47:0/4084914320 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7ff4ec00c3e0 con 0x7ff4ec004800 2024-04-26T23:18:06.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.748+0000 7ff508df5640 1 --2- 172.21.15.47:0/4084914320 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4ec0039b0 0x7ff4ec003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:06.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.748+0000 7ff503fff640 1 --2- 172.21.15.47:0/4084914320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4ec004800 0x7ff4ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:06.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.748+0000 7ff5095f6640 1 --2- 172.21.15.47:0/4084914320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4ec005290 0x7ff4ec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:06.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.748+0000 7ff508df5640 1 -- 172.21.15.47:0/4084914320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4ec004800 msgr2=0x7ff4ec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:06.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.748+0000 7ff508df5640 1 --2- 172.21.15.47:0/4084914320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4ec004800 0x7ff4ec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:06.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.748+0000 7ff508df5640 1 -- 172.21.15.47:0/4084914320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4ec005290 msgr2=0x7ff4ec00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:06.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.748+0000 7ff508df5640 1 --2- 172.21.15.47:0/4084914320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4ec005290 0x7ff4ec00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:06.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.748+0000 7ff508df5640 1 -- 172.21.15.47:0/4084914320 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4ec003520 con 0x7ff4ec0039b0 2024-04-26T23:18:06.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.749+0000 7ff508df5640 1 --2- 172.21.15.47:0/4084914320 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4ec0039b0 0x7ff4ec003db0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4019cc0 tx=0x7ff4ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9ddc6ec76e9f5d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:06.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.749+0000 7ff5037fe640 1 -- 172.21.15.47:0/4084914320 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f4032070 con 0x7ff4ec0039b0 2024-04-26T23:18:06.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.749+0000 7ff5037fe640 1 -- 172.21.15.47:0/4084914320 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4f402cd00 con 0x7ff4ec0039b0 2024-04-26T23:18:06.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.749+0000 7ff5037fe640 1 -- 172.21.15.47:0/4084914320 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f4019940 con 0x7ff4ec0039b0 2024-04-26T23:18:06.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.750+0000 7ff50b059640 1 -- 172.21.15.47:0/4084914320 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4ec0039b0 msgr2=0x7ff4ec003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:06.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.750+0000 7ff50b059640 1 --2- 172.21.15.47:0/4084914320 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4ec0039b0 0x7ff4ec003db0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4019cc0 tx=0x7ff4ec005830 comp rx=0 tx=0).stop 2024-04-26T23:18:06.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.750+0000 7ff50b059640 1 -- 172.21.15.47:0/4084914320 shutdown_connections 2024-04-26T23:18:06.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.750+0000 7ff50b059640 1 --2- 172.21.15.47:0/4084914320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4ec005290 0x7ff4ec00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:06.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.750+0000 7ff50b059640 1 --2- 172.21.15.47:0/4084914320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4ec004800 0x7ff4ec004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:06.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.750+0000 7ff50b059640 1 --2- 172.21.15.47:0/4084914320 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4ec0039b0 0x7ff4ec003db0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:06.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.750+0000 7ff50b059640 1 -- 172.21.15.47:0/4084914320 >> 172.21.15.47:0/4084914320 conn(0x7ff5040d5190 msgr2=0x7ff5040d55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:06.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.750+0000 7ff50b059640 1 -- 172.21.15.47:0/4084914320 shutdown_connections 2024-04-26T23:18:06.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.750+0000 7ff50b059640 1 -- 172.21.15.47:0/4084914320 wait complete. 2024-04-26T23:18:06.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.752+0000 7ff50b059640 1 Processor -- start 2024-04-26T23:18:06.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.752+0000 7ff50b059640 1 -- start start 2024-04-26T23:18:06.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.753+0000 7ff50b059640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4ec0039b0 0x7ff5040cf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:06.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.753+0000 7ff50b059640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4ec004800 0x7ff5040a2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:06.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.753+0000 7ff50b059640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4ec005290 0x7ff5041f8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:06.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.753+0000 7ff50b059640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff5040d3c20 con 0x7ff4ec0039b0 2024-04-26T23:18:06.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.753+0000 7ff50b059640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff5040d4190 con 0x7ff4ec004800 2024-04-26T23:18:06.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.753+0000 7ff50b059640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7ff5040d3da0 con 0x7ff4ec005290 2024-04-26T23:18:06.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.753+0000 7ff508df5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4ec0039b0 0x7ff5040cf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:06.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.753+0000 7ff508df5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4ec0039b0 0x7ff5040cf1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44606/0 (socket says 172.21.15.47:44606) 2024-04-26T23:18:06.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.753+0000 7ff508df5640 1 -- 172.21.15.47:0/3881649695 learned_addr learned my addr 172.21.15.47:0/3881649695 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:06.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.753+0000 7ff5095f6640 1 --2- 172.21.15.47:0/3881649695 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4ec005290 0x7ff5041f8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:06.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.753+0000 7ff503fff640 1 --2- 172.21.15.47:0/3881649695 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4ec004800 0x7ff5040a2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:06.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.753+0000 7ff5095f6640 1 -- 172.21.15.47:0/3881649695 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4ec005290 msgr2=0x7ff5041f8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-04-26T23:18:06.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.753+0000 7ff5095f6640 1 -- 172.21.15.47:0/3881649695 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4ec005290 msgr2=0x7ff5041f8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T23:18:06.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.753+0000 7ff5095f6640 1 --2- 172.21.15.47:0/3881649695 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4ec005290 0x7ff5041f8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T23:18:06.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.753+0000 7ff5095f6640 1 --2- 172.21.15.47:0/3881649695 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4ec005290 0x7ff5041f8390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T23:18:06.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.753+0000 7ff508df5640 1 -- 172.21.15.47:0/3881649695 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4ec005290 msgr2=0x7ff5041f8390 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T23:18:06.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.753+0000 7ff508df5640 1 --2- 172.21.15.47:0/3881649695 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4ec005290 0x7ff5041f8390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:06.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.753+0000 7ff508df5640 1 -- 172.21.15.47:0/3881649695 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4ec004800 msgr2=0x7ff5040a2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:06.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.753+0000 7ff508df5640 1 --2- 172.21.15.47:0/3881649695 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4ec004800 0x7ff5040a2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:06.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.753+0000 7ff508df5640 1 -- 172.21.15.47:0/3881649695 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5041f88d0 con 0x7ff4ec0039b0 2024-04-26T23:18:06.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.753+0000 7ff503fff640 1 --2- 172.21.15.47:0/3881649695 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4ec004800 0x7ff5040a2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:18:06.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.754+0000 7ff508df5640 1 --2- 172.21.15.47:0/3881649695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4ec0039b0 0x7ff5040cf1d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4038040 tx=0x7ff4f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:06.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.754+0000 7ff5017fa640 1 -- 172.21.15.47:0/3881649695 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f4044020 con 0x7ff4ec0039b0 2024-04-26T23:18:06.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.754+0000 7ff5017fa640 1 -- 172.21.15.47:0/3881649695 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4f4003610 con 0x7ff4ec0039b0 2024-04-26T23:18:06.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.754+0000 7ff50b059640 1 -- 172.21.15.47:0/3881649695 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5041f8ba0 con 0x7ff4ec0039b0 2024-04-26T23:18:06.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.754+0000 7ff5017fa640 1 -- 172.21.15.47:0/3881649695 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f4019940 con 0x7ff4ec0039b0 2024-04-26T23:18:06.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.754+0000 7ff50b059640 1 -- 172.21.15.47:0/3881649695 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5041f9160 con 0x7ff4ec0039b0 2024-04-26T23:18:06.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.754+0000 7ff5017fa640 1 -- 172.21.15.47:0/3881649695 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff4f4047020 con 0x7ff4ec0039b0 2024-04-26T23:18:06.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.754+0000 7ff5017fa640 1 -- 172.21.15.47:0/3881649695 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff4f403f8e0 con 0x7ff4ec0039b0 2024-04-26T23:18:06.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.755+0000 7ff50b059640 1 -- 172.21.15.47:0/3881649695 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5040694f0 con 0x7ff4ec0039b0 2024-04-26T23:18:06.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.758+0000 7ff5017fa640 1 -- 172.21.15.47:0/3881649695 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7ff4f404e030 con 0x7ff4ec0039b0 2024-04-26T23:18:06.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.826+0000 7ff50b059640 1 -- 172.21.15.47:0/3881649695 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7ff50415d3b0 con 0x7ff4ec0039b0 2024-04-26T23:18:06.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.828+0000 7ff5017fa640 1 -- 172.21.15.47:0/3881649695 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7ff4f403f440 con 0x7ff4ec0039b0 2024-04-26T23:18:06.828 INFO:teuthology.orchestra.run.smithi047.stderr:adjusted tunables profile to default 2024-04-26T23:18:06.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.828+0000 7ff50b059640 1 -- 172.21.15.47:0/3881649695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4ec0039b0 msgr2=0x7ff5040cf1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:06.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.828+0000 7ff50b059640 1 --2- 172.21.15.47:0/3881649695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4ec0039b0 0x7ff5040cf1d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4038040 tx=0x7ff4f40046c0 comp rx=0 tx=0).stop 2024-04-26T23:18:06.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.829+0000 7ff50b059640 1 -- 172.21.15.47:0/3881649695 shutdown_connections 2024-04-26T23:18:06.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.829+0000 7ff50b059640 1 --2- 172.21.15.47:0/3881649695 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4ec005290 0x7ff5041f8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:06.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.829+0000 7ff50b059640 1 --2- 172.21.15.47:0/3881649695 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4ec004800 0x7ff5040a2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:06.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.829+0000 7ff50b059640 1 --2- 172.21.15.47:0/3881649695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4ec0039b0 0x7ff5040cf1d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:06.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.829+0000 7ff50b059640 1 -- 172.21.15.47:0/3881649695 >> 172.21.15.47:0/3881649695 conn(0x7ff5040d5190 msgr2=0x7ff5040ba9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:06.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.829+0000 7ff50b059640 1 -- 172.21.15.47:0/3881649695 shutdown_connections 2024-04-26T23:18:06.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:06.829+0000 7ff50b059640 1 -- 172.21.15.47:0/3881649695 wait complete. 2024-04-26T23:18:06.841 INFO:tasks.ceph:check_enable_crimson: False 2024-04-26T23:18:06.841 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-26T23:18:06.841 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:18:06.841 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-26T23:18:06.864 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-26T23:18:06.864+0000 7f68c4e8c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T23:18:06.867 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:18:06.867 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-26T23:18:06.932 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:18:06.932 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-26T23:18:06.997 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:18:06.997 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-26T23:18:07.013 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-26T23:18:07.012+0000 7f68c4e8c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T23:18:07.061 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-26T23:18:07.061 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-26T23:18:07.066 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-26T23:18:07.065+0000 7f68c4e8c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T23:18:07.085 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-26T23:18:07.085 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-26T23:18:07.149 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-26T23:18:07.149 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-26T23:18:07.213 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-26T23:18:07.213 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-26T23:18:07.277 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --cluster ceph osd new ee091567-fea0-4a14-bf4e-0e16c20bb2f4 0 2024-04-26T23:18:07.320 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-26T23:18:07.320+0000 7f68c4e8c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T23:18:07.371 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-26T23:18:07.368+0000 7f68c4e8c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T23:18:07.415 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.414+0000 7f0c7c693640 1 Processor -- start 2024-04-26T23:18:07.415 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.414+0000 7f0c7c693640 1 -- start start 2024-04-26T23:18:07.417 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.415+0000 7f0c7c693640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0c74072fb0 0x7f0c740b56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:07.417 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.415+0000 7f0c7c693640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c74073390 0x7f0c7406f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:07.417 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.415+0000 7f0c7c693640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c740b5e30 con 0x7f0c7406fc80 2024-04-26T23:18:07.417 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.415+0000 7f0c7c693640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f0c740b5c00 con 0x7f0c74072fb0 2024-04-26T23:18:07.417 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.415+0000 7f0c7c693640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f0c7406f2b0 con 0x7f0c74073390 2024-04-26T23:18:07.417 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.415+0000 7f0c7a42f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0c74072fb0 0x7f0c740b56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:07.417 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.415+0000 7f0c79c2e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c74073390 0x7f0c7406f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:07.417 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.415+0000 7f0c7a42f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0c74072fb0 0x7f0c740b56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:35720/0 (socket says 172.21.15.203:35720) 2024-04-26T23:18:07.417 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.415+0000 7f0c7a42f640 1 -- 172.21.15.203:0/3202001346 learned_addr learned my addr 172.21.15.203:0/3202001346 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-26T23:18:07.418 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.415+0000 7f0c79c2e640 1 --2- 172.21.15.203:0/3202001346 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c74073390 0x7f0c7406f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.203:33498/0 (socket says 172.21.15.203:33498) 2024-04-26T23:18:07.418 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.415+0000 7f0c79c2e640 1 -- 172.21.15.203:0/3202001346 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c74073390 msgr2=0x7f0c7406f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-26T23:18:07.418 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.415+0000 7f0c79c2e640 1 -- 172.21.15.203:0/3202001346 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c74073390 msgr2=0x7f0c7406f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T23:18:07.418 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.415+0000 7f0c79c2e640 1 --2- 172.21.15.203:0/3202001346 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c74073390 0x7f0c7406f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T23:18:07.418 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.415+0000 7f0c79c2e640 1 --2- 172.21.15.203:0/3202001346 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c74073390 0x7f0c7406f740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T23:18:07.418 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.415+0000 7f0c7a42f640 1 -- 172.21.15.203:0/3202001346 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c74073390 msgr2=0x7f0c7406f740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T23:18:07.418 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.415+0000 7f0c7a42f640 1 --2- 172.21.15.203:0/3202001346 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c74073390 0x7f0c7406f740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:07.418 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.415+0000 7f0c7a42f640 1 -- 172.21.15.203:0/3202001346 >> v1:172.21.15.47:6789/0 conn(0x7f0c7406fc80 legacy=0x7f0c74070110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:07.418 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.415+0000 7f0c7a42f640 1 -- 172.21.15.203:0/3202001346 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c740b6110 con 0x7f0c74072fb0 2024-04-26T23:18:07.418 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.416+0000 7f0c7a42f640 1 --2- 172.21.15.203:0/3202001346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c74072fb0 0x7f0c740b56c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0c64009870 tx=0x7f0c6402a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b1de90164e2e577 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:07.418 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.416+0000 7f0c7942d640 1 -- 172.21.15.203:0/3202001346 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c64037070 con 0x7f0c74072fb0 2024-04-26T23:18:07.418 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.416+0000 7f0c7942d640 1 -- 172.21.15.203:0/3202001346 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c64032440 con 0x7f0c74072fb0 2024-04-26T23:18:07.418 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.416+0000 7f0c7942d640 1 -- 172.21.15.203:0/3202001346 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c64032810 con 0x7f0c74072fb0 2024-04-26T23:18:07.421 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.417+0000 7f0c7c693640 1 -- 172.21.15.203:0/3202001346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c74072fb0 msgr2=0x7f0c740b56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:07.422 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.417+0000 7f0c7c693640 1 --2- 172.21.15.203:0/3202001346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c74072fb0 0x7f0c740b56c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0c64009870 tx=0x7f0c6402a210 comp rx=0 tx=0).stop 2024-04-26T23:18:07.422 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.417+0000 7f0c7c693640 1 -- 172.21.15.203:0/3202001346 shutdown_connections 2024-04-26T23:18:07.422 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.417+0000 7f0c7c693640 1 --2- 172.21.15.203:0/3202001346 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c74073390 0x7f0c7406f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:07.422 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.417+0000 7f0c7c693640 1 --2- 172.21.15.203:0/3202001346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c74072fb0 0x7f0c740b56c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:07.422 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.417+0000 7f0c7c693640 1 -- 172.21.15.203:0/3202001346 >> 172.21.15.203:0/3202001346 conn(0x7f0c740b39b0 msgr2=0x7f0c740b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:07.422 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.417+0000 7f0c7c693640 1 -- 172.21.15.203:0/3202001346 shutdown_connections 2024-04-26T23:18:07.422 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.417+0000 7f0c7c693640 1 -- 172.21.15.203:0/3202001346 wait complete. 2024-04-26T23:18:07.422 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.418+0000 7f0c7c693640 1 Processor -- start 2024-04-26T23:18:07.422 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.418+0000 7f0c7c693640 1 -- start start 2024-04-26T23:18:07.422 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.419+0000 7f0c7c693640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c7406fc80 0x7f0c741f5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:07.422 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.419+0000 7f0c7c693640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c74072fb0 0x7f0c741f3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:07.422 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.419+0000 7f0c7c693640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c74073390 0x7f0c74153970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:07.422 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.419+0000 7f0c7c693640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0c7415c160 con 0x7f0c74073390 2024-04-26T23:18:07.423 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.419+0000 7f0c7c693640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0c740737f0 con 0x7f0c74072fb0 2024-04-26T23:18:07.423 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.419+0000 7f0c7c693640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f0c7415c2e0 con 0x7f0c7406fc80 2024-04-26T23:18:07.423 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.419+0000 7f0c7a42f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c7406fc80 0x7f0c741f5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:07.423 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.419+0000 7f0c79c2e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c74072fb0 0x7f0c741f3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:07.423 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.419+0000 7f0c7a42f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c7406fc80 0x7f0c741f5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.203:33504/0 (socket says 172.21.15.203:33504) 2024-04-26T23:18:07.423 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.419+0000 7f0c79c2e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c74072fb0 0x7f0c741f3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:35732/0 (socket says 172.21.15.203:35732) 2024-04-26T23:18:07.423 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.419+0000 7f0c7a42f640 1 -- 172.21.15.203:0/1435415633 learned_addr learned my addr 172.21.15.203:0/1435415633 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-26T23:18:07.423 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.419+0000 7f0c7a42f640 1 -- 172.21.15.203:0/1435415633 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c7406fc80 msgr2=0x7f0c741f5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-04-26T23:18:07.423 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.419+0000 7f0c7a42f640 1 -- 172.21.15.203:0/1435415633 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c7406fc80 msgr2=0x7f0c741f5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T23:18:07.423 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.419+0000 7f0c7a42f640 1 --2- 172.21.15.203:0/1435415633 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c7406fc80 0x7f0c741f5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T23:18:07.423 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.419+0000 7f0c7a42f640 1 --2- 172.21.15.203:0/1435415633 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c7406fc80 0x7f0c741f5ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T23:18:07.423 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.419+0000 7f0c7ac30640 1 --2- 172.21.15.203:0/1435415633 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c74073390 0x7f0c74153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:07.423 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.419+0000 7f0c79c2e640 1 -- 172.21.15.203:0/1435415633 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c7406fc80 msgr2=0x7f0c741f5ee0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T23:18:07.423 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.419+0000 7f0c79c2e640 1 --2- 172.21.15.203:0/1435415633 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c7406fc80 0x7f0c741f5ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:07.423 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.419+0000 7f0c79c2e640 1 -- 172.21.15.203:0/1435415633 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c74073390 msgr2=0x7f0c74153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:07.424 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.419+0000 7f0c79c2e640 1 --2- 172.21.15.203:0/1435415633 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c74073390 0x7f0c74153970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:07.424 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.419+0000 7f0c79c2e640 1 -- 172.21.15.203:0/1435415633 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c741f6530 con 0x7f0c74072fb0 2024-04-26T23:18:07.424 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.419+0000 7f0c79c2e640 1 --2- 172.21.15.203:0/1435415633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c74072fb0 0x7f0c741f3390 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0c7000d5a0 tx=0x7f0c7000da60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:07.424 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.420+0000 7f0c6affd640 1 -- 172.21.15.203:0/1435415633 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c700125f0 con 0x7f0c74072fb0 2024-04-26T23:18:07.424 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.420+0000 7f0c6affd640 1 -- 172.21.15.203:0/1435415633 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c70019070 con 0x7f0c74072fb0 2024-04-26T23:18:07.424 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.420+0000 7f0c7c693640 1 -- 172.21.15.203:0/1435415633 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c741f6810 con 0x7f0c74072fb0 2024-04-26T23:18:07.424 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.420+0000 7f0c6affd640 1 -- 172.21.15.203:0/1435415633 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c700129d0 con 0x7f0c74072fb0 2024-04-26T23:18:07.424 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.420+0000 7f0c7c693640 1 -- 172.21.15.203:0/1435415633 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c740b1920 con 0x7f0c74072fb0 2024-04-26T23:18:07.424 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.420+0000 7f0c6affd640 1 -- 172.21.15.203:0/1435415633 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0c70012b70 con 0x7f0c74072fb0 2024-04-26T23:18:07.424 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.420+0000 7f0c6affd640 1 -- 172.21.15.203:0/1435415633 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f0c70014590 con 0x7f0c74072fb0 2024-04-26T23:18:07.424 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.421+0000 7f0c7c693640 1 -- 172.21.15.203:0/1435415633 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c740694f0 con 0x7f0c74072fb0 2024-04-26T23:18:07.425 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.425+0000 7f0c6affd640 1 -- 172.21.15.203:0/1435415633 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f0c700147e0 con 0x7f0c74072fb0 2024-04-26T23:18:07.498 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.497+0000 7f0c7c693640 1 -- 172.21.15.203:0/1435415633 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ee091567-fea0-4a14-bf4e-0e16c20bb2f4", "id": 0} v 0) -- 0x7f0c741f6d30 con 0x7f0c74072fb0 2024-04-26T23:18:07.504 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.503+0000 7f0c6affd640 1 -- 172.21.15.203:0/1435415633 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ee091567-fea0-4a14-bf4e-0e16c20bb2f4", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f0c70014a90 con 0x7f0c74072fb0 2024-04-26T23:18:07.504 INFO:teuthology.orchestra.run.smithi203.stdout:0 2024-04-26T23:18:07.505 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.504+0000 7f0c7c693640 1 -- 172.21.15.203:0/1435415633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c74072fb0 msgr2=0x7f0c741f3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:07.505 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.504+0000 7f0c7c693640 1 --2- 172.21.15.203:0/1435415633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c74072fb0 0x7f0c741f3390 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0c7000d5a0 tx=0x7f0c7000da60 comp rx=0 tx=0).stop 2024-04-26T23:18:07.506 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.504+0000 7f0c7c693640 1 -- 172.21.15.203:0/1435415633 shutdown_connections 2024-04-26T23:18:07.506 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.504+0000 7f0c7c693640 1 --2- 172.21.15.203:0/1435415633 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c74073390 0x7f0c74153970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:07.506 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.504+0000 7f0c7c693640 1 --2- 172.21.15.203:0/1435415633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c74072fb0 0x7f0c741f3390 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:07.506 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.504+0000 7f0c7c693640 1 --2- 172.21.15.203:0/1435415633 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c7406fc80 0x7f0c741f5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:07.506 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.504+0000 7f0c7c693640 1 -- 172.21.15.203:0/1435415633 >> 172.21.15.203:0/1435415633 conn(0x7f0c740b39b0 msgr2=0x7f0c740d1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:07.506 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.504+0000 7f0c7c693640 1 -- 172.21.15.203:0/1435415633 shutdown_connections 2024-04-26T23:18:07.506 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.504+0000 7f0c7c693640 1 -- 172.21.15.203:0/1435415633 wait complete. 2024-04-26T23:18:07.515 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --cluster ceph osd new c14440a0-bba5-4931-a6db-b736c08e869b 1 2024-04-26T23:18:07.653 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.652+0000 7fb48f14e640 1 Processor -- start 2024-04-26T23:18:07.653 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.652+0000 7fb48f14e640 1 -- start start 2024-04-26T23:18:07.657 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.653+0000 7fb48f14e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb48806c240 0x7fb48806c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:07.657 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.653+0000 7fb48f14e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb488160b20 0x7fb488160f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:07.657 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.653+0000 7fb48f14e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb48806cb60 con 0x7fb4880ce310 2024-04-26T23:18:07.658 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.653+0000 7fb48f14e640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fb48815d530 con 0x7fb488160b20 2024-04-26T23:18:07.658 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.653+0000 7fb48f14e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb48815d7f0 con 0x7fb48806c240 2024-04-26T23:18:07.658 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.653+0000 7fb48ceea640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb48806c240 0x7fb48806c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:07.658 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.653+0000 7fb487fff640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fb4880ce310 0x7fb488071040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:56010/0 (socket says 172.21.15.203:56010) 2024-04-26T23:18:07.658 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.653+0000 7fb487fff640 1 -- 172.21.15.203:0/2478557492 learned_addr learned my addr 172.21.15.203:0/2478557492 (peer_addr_for_me v1:172.21.15.203:0/0) 2024-04-26T23:18:07.658 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.653+0000 7fb48ceea640 1 -- 172.21.15.203:0/2478557492 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb48806c240 msgr2=0x7fb48806c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-26T23:18:07.658 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.653+0000 7fb48ceea640 1 -- 172.21.15.203:0/2478557492 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb48806c240 msgr2=0x7fb48806c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T23:18:07.658 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.653+0000 7fb48ceea640 1 --2- 172.21.15.203:0/2478557492 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb48806c240 0x7fb48806c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T23:18:07.658 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.653+0000 7fb48ceea640 1 --2- 172.21.15.203:0/2478557492 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb48806c240 0x7fb48806c620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T23:18:07.658 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.653+0000 7fb48d6eb640 1 --2- 172.21.15.203:0/2478557492 >> v2:172.21.15.47:3300/0 conn(0x7fb488160b20 0x7fb488160f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:07.658 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.653+0000 7fb4877fe640 1 -- 172.21.15.203:0/2478557492 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1034554606 0 0) 0x7fb48806cb60 con 0x7fb4880ce310 2024-04-26T23:18:07.658 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.653+0000 7fb4877fe640 1 -- 172.21.15.203:0/2478557492 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb478003520 con 0x7fb4880ce310 2024-04-26T23:18:07.658 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.653+0000 7fb4877fe640 1 -- 172.21.15.203:0/2478557492 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2643667319 0 0) 0x7fb478003520 con 0x7fb4880ce310 2024-04-26T23:18:07.659 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.653+0000 7fb4877fe640 1 -- 172.21.15.203:0/2478557492 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb48806c240 msgr2=0x7fb48806c620 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T23:18:07.659 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.653+0000 7fb4877fe640 1 --2- 172.21.15.203:0/2478557492 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb48806c240 0x7fb48806c620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:07.659 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.653+0000 7fb4877fe640 1 -- 172.21.15.203:0/2478557492 >> v2:172.21.15.47:3300/0 conn(0x7fb488160b20 msgr2=0x7fb488160f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:07.659 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.653+0000 7fb4877fe640 1 --2- 172.21.15.203:0/2478557492 >> v2:172.21.15.47:3300/0 conn(0x7fb488160b20 0x7fb488160f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:07.659 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.653+0000 7fb4877fe640 1 -- 172.21.15.203:0/2478557492 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb48815dad0 con 0x7fb4880ce310 2024-04-26T23:18:07.659 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.653+0000 7fb4877fe640 1 -- 172.21.15.203:0/2478557492 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7fb474001670 con 0x7fb4880ce310 2024-04-26T23:18:07.659 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.653+0000 7fb4877fe640 1 -- 172.21.15.203:0/2478557492 >> v1:172.21.15.203:6789/0 conn(0x7fb4880ce310 legacy=0x7fb488071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:07.659 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.653+0000 7fb4877fe640 1 --2- 172.21.15.203:0/2478557492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4780039b0 0x7fb478003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:07.659 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.653+0000 7fb4877fe640 1 --2- 172.21.15.203:0/2478557492 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb48806c240 0x7fb478004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:07.659 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.653+0000 7fb4877fe640 1 --2- 172.21.15.203:0/2478557492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb478004d40 0x7fb4780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:07.659 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.654+0000 7fb4877fe640 1 -- 172.21.15.203:0/2478557492 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb48815d530 con 0x7fb4780039b0 2024-04-26T23:18:07.659 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.654+0000 7fb4877fe640 1 -- 172.21.15.203:0/2478557492 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb48815d7f0 con 0x7fb478004d40 2024-04-26T23:18:07.659 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.654+0000 7fb4877fe640 1 -- 172.21.15.203:0/2478557492 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb47800b9c0 con 0x7fb48806c240 2024-04-26T23:18:07.659 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.654+0000 7fb487fff640 1 --2- 172.21.15.203:0/2478557492 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb48806c240 0x7fb478004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:07.659 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.654+0000 7fb48d6eb640 1 --2- 172.21.15.203:0/2478557492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb478004d40 0x7fb4780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:07.660 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.654+0000 7fb48ceea640 1 --2- 172.21.15.203:0/2478557492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4780039b0 0x7fb478003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:07.660 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.654+0000 7fb487fff640 1 -- 172.21.15.203:0/2478557492 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb48806c240 msgr2=0x7fb478004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-26T23:18:07.660 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.654+0000 7fb487fff640 1 -- 172.21.15.203:0/2478557492 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb48806c240 msgr2=0x7fb478004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T23:18:07.660 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.654+0000 7fb487fff640 1 --2- 172.21.15.203:0/2478557492 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb48806c240 0x7fb478004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T23:18:07.660 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.654+0000 7fb487fff640 1 --2- 172.21.15.203:0/2478557492 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb48806c240 0x7fb478004800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T23:18:07.660 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.654+0000 7fb48d6eb640 1 -- 172.21.15.203:0/2478557492 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb48806c240 msgr2=0x7fb478004800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T23:18:07.660 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.654+0000 7fb48d6eb640 1 --2- 172.21.15.203:0/2478557492 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb48806c240 0x7fb478004800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:07.660 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.654+0000 7fb48d6eb640 1 -- 172.21.15.203:0/2478557492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4780039b0 msgr2=0x7fb478003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:07.660 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.654+0000 7fb48d6eb640 1 --2- 172.21.15.203:0/2478557492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4780039b0 0x7fb478003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:07.660 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.654+0000 7fb48d6eb640 1 -- 172.21.15.203:0/2478557492 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb478003520 con 0x7fb478004d40 2024-04-26T23:18:07.660 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.655+0000 7fb48d6eb640 1 --2- 172.21.15.203:0/2478557492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb478004d40 0x7fb4780051c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb480009fc0 tx=0x7fb478005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62623f88b6eae41e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:07.660 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.655+0000 7fb4877fe640 1 -- 172.21.15.203:0/2478557492 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb480009c10 con 0x7fb478004d40 2024-04-26T23:18:07.660 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.655+0000 7fb4877fe640 1 -- 172.21.15.203:0/2478557492 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb480036070 con 0x7fb478004d40 2024-04-26T23:18:07.660 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.655+0000 7fb4877fe640 1 -- 172.21.15.203:0/2478557492 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48002d070 con 0x7fb478004d40 2024-04-26T23:18:07.660 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.655+0000 7fb48f14e640 1 -- 172.21.15.203:0/2478557492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb478004d40 msgr2=0x7fb4780051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:07.661 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.655+0000 7fb48f14e640 1 --2- 172.21.15.203:0/2478557492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb478004d40 0x7fb4780051c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb480009fc0 tx=0x7fb478005830 comp rx=0 tx=0).stop 2024-04-26T23:18:07.661 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.655+0000 7fb48d6eb640 1 -- 172.21.15.203:0/2478557492 reap_dead start 2024-04-26T23:18:07.661 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.655+0000 7fb48f14e640 1 -- 172.21.15.203:0/2478557492 shutdown_connections 2024-04-26T23:18:07.661 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.655+0000 7fb48f14e640 1 -- 172.21.15.203:0/2478557492 >> 172.21.15.203:0/2478557492 conn(0x7fb4880d1cb0 msgr2=0x7fb4880d2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:07.661 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.655+0000 7fb48f14e640 1 -- 172.21.15.203:0/2478557492 shutdown_connections 2024-04-26T23:18:07.661 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.655+0000 7fb48f14e640 1 -- 172.21.15.203:0/2478557492 wait complete. 2024-04-26T23:18:07.661 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.656+0000 7fb48f14e640 1 Processor -- start 2024-04-26T23:18:07.661 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.656+0000 7fb48f14e640 1 -- start start 2024-04-26T23:18:07.663 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.657+0000 7fb48f14e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb478004d40 0x7fb488156270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:07.664 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.657+0000 7fb48f14e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4881f6200 0x7fb4881f4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:07.664 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.657+0000 7fb48f14e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4881f5530 0x7fb4880cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:07.664 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.657+0000 7fb48f14e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb4880b7850 con 0x7fb4881f5530 2024-04-26T23:18:07.664 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.657+0000 7fb48f14e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb48815e090 con 0x7fb4881f6200 2024-04-26T23:18:07.664 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.657+0000 7fb48f14e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb488071750 con 0x7fb478004d40 2024-04-26T23:18:07.664 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.657+0000 7fb48ceea640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb478004d40 0x7fb488156270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:07.664 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.657+0000 7fb487fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4881f6200 0x7fb4881f4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:07.664 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.657+0000 7fb48ceea640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb478004d40 0x7fb488156270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.203:33528/0 (socket says 172.21.15.203:33528) 2024-04-26T23:18:07.664 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.657+0000 7fb48ceea640 1 -- 172.21.15.203:0/21292198 learned_addr learned my addr 172.21.15.203:0/21292198 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-26T23:18:07.664 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.658+0000 7fb48d6eb640 1 --2- 172.21.15.203:0/21292198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4881f5530 0x7fb4880cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:07.664 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.658+0000 7fb48ceea640 1 -- 172.21.15.203:0/21292198 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb478004d40 msgr2=0x7fb488156270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-04-26T23:18:07.664 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.658+0000 7fb48ceea640 1 -- 172.21.15.203:0/21292198 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb478004d40 msgr2=0x7fb488156270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T23:18:07.664 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.658+0000 7fb48ceea640 1 --2- 172.21.15.203:0/21292198 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb478004d40 0x7fb488156270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T23:18:07.664 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.658+0000 7fb48ceea640 1 --2- 172.21.15.203:0/21292198 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb478004d40 0x7fb488156270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T23:18:07.664 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.658+0000 7fb487fff640 1 -- 172.21.15.203:0/21292198 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb478004d40 msgr2=0x7fb488156270 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T23:18:07.665 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.658+0000 7fb487fff640 1 --2- 172.21.15.203:0/21292198 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb478004d40 0x7fb488156270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:07.665 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.658+0000 7fb487fff640 1 -- 172.21.15.203:0/21292198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4881f5530 msgr2=0x7fb4880cafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:07.665 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.658+0000 7fb487fff640 1 --2- 172.21.15.203:0/21292198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4881f5530 0x7fb4880cafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:07.665 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.658+0000 7fb487fff640 1 -- 172.21.15.203:0/21292198 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4881567b0 con 0x7fb4881f6200 2024-04-26T23:18:07.665 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.658+0000 7fb487fff640 1 --2- 172.21.15.203:0/21292198 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4881f6200 0x7fb4881f4ff0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb474001da0 tx=0x7fb47400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:07.665 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.658+0000 7fb4857fa640 1 -- 172.21.15.203:0/21292198 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47400c870 con 0x7fb4881f6200 2024-04-26T23:18:07.665 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.658+0000 7fb4857fa640 1 -- 172.21.15.203:0/21292198 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb474018070 con 0x7fb4881f6200 2024-04-26T23:18:07.665 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.658+0000 7fb48f14e640 1 -- 172.21.15.203:0/21292198 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4880aa640 con 0x7fb4881f6200 2024-04-26T23:18:07.665 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.658+0000 7fb4857fa640 1 -- 172.21.15.203:0/21292198 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47400cc50 con 0x7fb4881f6200 2024-04-26T23:18:07.665 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.658+0000 7fb48f14e640 1 -- 172.21.15.203:0/21292198 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4881f5de0 con 0x7fb4881f6200 2024-04-26T23:18:07.665 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.658+0000 7fb4857fa640 1 -- 172.21.15.203:0/21292198 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb47401d460 con 0x7fb4881f6200 2024-04-26T23:18:07.665 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.658+0000 7fb4857fa640 1 -- 172.21.15.203:0/21292198 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fb4740136e0 con 0x7fb4881f6200 2024-04-26T23:18:07.665 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.659+0000 7fb48f14e640 1 -- 172.21.15.203:0/21292198 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4880694f0 con 0x7fb4881f6200 2024-04-26T23:18:07.665 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.663+0000 7fb4857fa640 1 -- 172.21.15.203:0/21292198 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fb47400cdf0 con 0x7fb4881f6200 2024-04-26T23:18:07.671 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-26T23:18:07.670+0000 7fe93fb2c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T23:18:07.736 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.736+0000 7fb48f14e640 1 -- 172.21.15.203:0/21292198 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c14440a0-bba5-4931-a6db-b736c08e869b", "id": 1} v 0) -- 0x7fb4880d2bc0 con 0x7fb4881f6200 2024-04-26T23:18:07.742 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.741+0000 7fb4857fa640 1 -- 172.21.15.203:0/21292198 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c14440a0-bba5-4931-a6db-b736c08e869b", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fb474013930 con 0x7fb4881f6200 2024-04-26T23:18:07.742 INFO:teuthology.orchestra.run.smithi203.stdout:1 2024-04-26T23:18:07.743 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.742+0000 7fb48f14e640 1 -- 172.21.15.203:0/21292198 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4881f6200 msgr2=0x7fb4881f4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:07.743 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.742+0000 7fb48f14e640 1 --2- 172.21.15.203:0/21292198 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4881f6200 0x7fb4881f4ff0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb474001da0 tx=0x7fb47400ecd0 comp rx=0 tx=0).stop 2024-04-26T23:18:07.743 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.742+0000 7fb48f14e640 1 -- 172.21.15.203:0/21292198 shutdown_connections 2024-04-26T23:18:07.743 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.742+0000 7fb48f14e640 1 --2- 172.21.15.203:0/21292198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4881f5530 0x7fb4880cafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:07.743 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.742+0000 7fb48f14e640 1 --2- 172.21.15.203:0/21292198 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4881f6200 0x7fb4881f4ff0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:07.743 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.742+0000 7fb48f14e640 1 --2- 172.21.15.203:0/21292198 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb478004d40 0x7fb488156270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:07.744 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.742+0000 7fb48f14e640 1 -- 172.21.15.203:0/21292198 >> 172.21.15.203:0/21292198 conn(0x7fb4880d1cb0 msgr2=0x7fb4880b5c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:07.744 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.742+0000 7fb48f14e640 1 -- 172.21.15.203:0/21292198 shutdown_connections 2024-04-26T23:18:07.744 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.742+0000 7fb48f14e640 1 -- 172.21.15.203:0/21292198 wait complete. 2024-04-26T23:18:07.754 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --cluster ceph osd new 3940f8de-c9b2-4cd3-a911-9c301729eb30 2 2024-04-26T23:18:07.829 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-26T23:18:07.829+0000 7fe93fb2c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T23:18:07.852 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.851+0000 7f0c59a15640 1 Processor -- start 2024-04-26T23:18:07.852 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.851+0000 7f0c59a15640 1 -- start start 2024-04-26T23:18:07.855 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.852+0000 7f0c59a15640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0c5415bcc0 0x7f0c540cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:07.855 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.852+0000 7f0c59a15640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c54071110 0x7f0c54071570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:07.855 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.852+0000 7f0c59a15640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c540cfbf0 con 0x7f0c54072f50 2024-04-26T23:18:07.856 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.852+0000 7f0c59a15640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f0c540d0800 con 0x7f0c5415bcc0 2024-04-26T23:18:07.856 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.852+0000 7f0c59a15640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f0c540cfed0 con 0x7f0c54071110 2024-04-26T23:18:07.856 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.852+0000 7f0c52ffd640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0c5415bcc0 0x7f0c540cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:07.856 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.852+0000 7f0c4bfff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c54071110 0x7f0c54071570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:07.856 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.852+0000 7f0c52ffd640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0c5415bcc0 0x7f0c540cc300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:35766/0 (socket says 172.21.15.203:35766) 2024-04-26T23:18:07.856 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.852+0000 7f0c4bfff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c54071110 0x7f0c54071570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.203:33538/0 (socket says 172.21.15.203:33538) 2024-04-26T23:18:07.856 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.852+0000 7f0c52ffd640 1 -- 172.21.15.203:0/4076310212 learned_addr learned my addr 172.21.15.203:0/4076310212 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-26T23:18:07.856 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.853+0000 7f0c4bfff640 1 -- 172.21.15.203:0/4076310212 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c54071110 msgr2=0x7f0c54071570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-26T23:18:07.856 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.853+0000 7f0c4bfff640 1 -- 172.21.15.203:0/4076310212 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c54071110 msgr2=0x7f0c54071570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T23:18:07.856 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.853+0000 7f0c4bfff640 1 --2- 172.21.15.203:0/4076310212 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c54071110 0x7f0c54071570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T23:18:07.856 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.853+0000 7f0c4bfff640 1 --2- 172.21.15.203:0/4076310212 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c54071110 0x7f0c54071570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T23:18:07.856 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.853+0000 7f0c52ffd640 1 -- 172.21.15.203:0/4076310212 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c54071110 msgr2=0x7f0c54071570 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T23:18:07.856 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.853+0000 7f0c52ffd640 1 --2- 172.21.15.203:0/4076310212 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c54071110 0x7f0c54071570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:07.856 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.853+0000 7f0c52ffd640 1 -- 172.21.15.203:0/4076310212 >> v1:172.21.15.47:6789/0 conn(0x7f0c54072f50 legacy=0x7f0c540d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:07.857 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.853+0000 7f0c52ffd640 1 -- 172.21.15.203:0/4076310212 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c540d01b0 con 0x7f0c5415bcc0 2024-04-26T23:18:07.857 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.853+0000 7f0c52ffd640 1 --2- 172.21.15.203:0/4076310212 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c5415bcc0 0x7f0c540cc300 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0c4400bb10 tx=0x7f0c4402c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=914edcfbdbe33570 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:07.857 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.853+0000 7f0c527fc640 1 -- 172.21.15.203:0/4076310212 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c44036020 con 0x7f0c5415bcc0 2024-04-26T23:18:07.857 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.853+0000 7f0c527fc640 1 -- 172.21.15.203:0/4076310212 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c44032820 con 0x7f0c5415bcc0 2024-04-26T23:18:07.857 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.853+0000 7f0c527fc640 1 -- 172.21.15.203:0/4076310212 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c44032bf0 con 0x7f0c5415bcc0 2024-04-26T23:18:07.857 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.854+0000 7f0c59a15640 1 -- 172.21.15.203:0/4076310212 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c5415bcc0 msgr2=0x7f0c540cc300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:07.857 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.854+0000 7f0c59a15640 1 --2- 172.21.15.203:0/4076310212 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c5415bcc0 0x7f0c540cc300 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0c4400bb10 tx=0x7f0c4402c920 comp rx=0 tx=0).stop 2024-04-26T23:18:07.857 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.854+0000 7f0c59a15640 1 -- 172.21.15.203:0/4076310212 shutdown_connections 2024-04-26T23:18:07.857 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.854+0000 7f0c59a15640 1 --2- 172.21.15.203:0/4076310212 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c54071110 0x7f0c54071570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:07.857 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.854+0000 7f0c59a15640 1 --2- 172.21.15.203:0/4076310212 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c5415bcc0 0x7f0c540cc300 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:07.857 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.854+0000 7f0c59a15640 1 -- 172.21.15.203:0/4076310212 >> 172.21.15.203:0/4076310212 conn(0x7f0c540d1cb0 msgr2=0x7f0c540d2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:07.857 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.854+0000 7f0c59a15640 1 -- 172.21.15.203:0/4076310212 shutdown_connections 2024-04-26T23:18:07.857 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.854+0000 7f0c59a15640 1 -- 172.21.15.203:0/4076310212 wait complete. 2024-04-26T23:18:07.860 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.855+0000 7f0c59a15640 1 Processor -- start 2024-04-26T23:18:07.860 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.855+0000 7f0c59a15640 1 -- start start 2024-04-26T23:18:07.860 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.856+0000 7f0c59a15640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c54071110 0x7f0c54154fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:07.860 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.856+0000 7f0c59a15640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c54072f50 0x7f0c540c4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:07.860 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.856+0000 7f0c59a15640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c5415bcc0 0x7f0c540c5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:07.860 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.856+0000 7f0c59a15640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0c540cc840 con 0x7f0c5415bcc0 2024-04-26T23:18:07.860 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.856+0000 7f0c59a15640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0c540cf880 con 0x7f0c54071110 2024-04-26T23:18:07.860 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.856+0000 7f0c59a15640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f0c540cc9c0 con 0x7f0c54072f50 2024-04-26T23:18:07.860 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.856+0000 7f0c52ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c54071110 0x7f0c54154fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:07.861 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.856+0000 7f0c4bfff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c54072f50 0x7f0c540c4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:07.861 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.856+0000 7f0c52ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c54071110 0x7f0c54154fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:35776/0 (socket says 172.21.15.203:35776) 2024-04-26T23:18:07.861 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.856+0000 7f0c52ffd640 1 -- 172.21.15.203:0/1800339131 learned_addr learned my addr 172.21.15.203:0/1800339131 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-26T23:18:07.861 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.856+0000 7f0c4bfff640 1 -- 172.21.15.203:0/1800339131 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c54072f50 msgr2=0x7f0c540c4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-04-26T23:18:07.861 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.856+0000 7f0c4bfff640 1 -- 172.21.15.203:0/1800339131 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c54072f50 msgr2=0x7f0c540c4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T23:18:07.861 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.856+0000 7f0c4bfff640 1 --2- 172.21.15.203:0/1800339131 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c54072f50 0x7f0c540c4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T23:18:07.861 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.856+0000 7f0c537fe640 1 --2- 172.21.15.203:0/1800339131 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c5415bcc0 0x7f0c540c5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:07.861 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.856+0000 7f0c4bfff640 1 --2- 172.21.15.203:0/1800339131 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c54072f50 0x7f0c540c4f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T23:18:07.861 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.856+0000 7f0c52ffd640 1 -- 172.21.15.203:0/1800339131 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c54072f50 msgr2=0x7f0c540c4f10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T23:18:07.861 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.856+0000 7f0c52ffd640 1 --2- 172.21.15.203:0/1800339131 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c54072f50 0x7f0c540c4f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:07.861 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.856+0000 7f0c52ffd640 1 -- 172.21.15.203:0/1800339131 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c5415bcc0 msgr2=0x7f0c540c5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:07.861 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.856+0000 7f0c52ffd640 1 --2- 172.21.15.203:0/1800339131 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c5415bcc0 0x7f0c540c5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:07.861 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.856+0000 7f0c52ffd640 1 -- 172.21.15.203:0/1800339131 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c540cae10 con 0x7f0c54071110 2024-04-26T23:18:07.861 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.856+0000 7f0c52ffd640 1 --2- 172.21.15.203:0/1800339131 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c54071110 0x7f0c54154fd0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0c4400b2c0 tx=0x7f0c44004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:07.861 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.857+0000 7f0c50ff9640 1 -- 172.21.15.203:0/1800339131 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4404a070 con 0x7f0c54071110 2024-04-26T23:18:07.862 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.857+0000 7f0c50ff9640 1 -- 172.21.15.203:0/1800339131 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c44032620 con 0x7f0c54071110 2024-04-26T23:18:07.862 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.857+0000 7f0c50ff9640 1 -- 172.21.15.203:0/1800339131 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c44045410 con 0x7f0c54071110 2024-04-26T23:18:07.862 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.857+0000 7f0c59a15640 1 -- 172.21.15.203:0/1800339131 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c540cb0f0 con 0x7f0c54071110 2024-04-26T23:18:07.862 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.857+0000 7f0c59a15640 1 -- 172.21.15.203:0/1800339131 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c540cb6b0 con 0x7f0c54071110 2024-04-26T23:18:07.862 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.857+0000 7f0c50ff9640 1 -- 172.21.15.203:0/1800339131 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0c4404d070 con 0x7f0c54071110 2024-04-26T23:18:07.862 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.857+0000 7f0c50ff9640 1 -- 172.21.15.203:0/1800339131 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f0c4404e7d0 con 0x7f0c54071110 2024-04-26T23:18:07.862 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.858+0000 7f0c59a15640 1 -- 172.21.15.203:0/1800339131 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c540694f0 con 0x7f0c54071110 2024-04-26T23:18:07.862 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.860+0000 7f0c50ff9640 1 -- 172.21.15.203:0/1800339131 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f0c4402f070 con 0x7f0c54071110 2024-04-26T23:18:07.885 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-26T23:18:07.885+0000 7fe93fb2c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T23:18:07.933 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.932+0000 7f0c59a15640 1 -- 172.21.15.203:0/1800339131 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd new", "uuid": "3940f8de-c9b2-4cd3-a911-9c301729eb30", "id": 2} v 0) -- 0x7f0c541e4b80 con 0x7f0c54071110 2024-04-26T23:18:07.936 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.935+0000 7f0c50ff9640 1 -- 172.21.15.203:0/1800339131 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3940f8de-c9b2-4cd3-a911-9c301729eb30", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f0c440327d0 con 0x7f0c54071110 2024-04-26T23:18:07.936 INFO:teuthology.orchestra.run.smithi203.stdout:2 2024-04-26T23:18:07.937 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.936+0000 7f0c59a15640 1 -- 172.21.15.203:0/1800339131 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c54071110 msgr2=0x7f0c54154fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:07.937 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.936+0000 7f0c59a15640 1 --2- 172.21.15.203:0/1800339131 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c54071110 0x7f0c54154fd0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0c4400b2c0 tx=0x7f0c44004730 comp rx=0 tx=0).stop 2024-04-26T23:18:07.937 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.936+0000 7f0c59a15640 1 -- 172.21.15.203:0/1800339131 shutdown_connections 2024-04-26T23:18:07.937 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.936+0000 7f0c59a15640 1 --2- 172.21.15.203:0/1800339131 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c5415bcc0 0x7f0c540c5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:07.937 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.936+0000 7f0c59a15640 1 --2- 172.21.15.203:0/1800339131 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c54072f50 0x7f0c540c4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:07.937 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.936+0000 7f0c59a15640 1 --2- 172.21.15.203:0/1800339131 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c54071110 0x7f0c54154fd0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:07.937 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.936+0000 7f0c59a15640 1 -- 172.21.15.203:0/1800339131 >> 172.21.15.203:0/1800339131 conn(0x7f0c540d1cb0 msgr2=0x7f0c540705e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:07.937 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.936+0000 7f0c59a15640 1 -- 172.21.15.203:0/1800339131 shutdown_connections 2024-04-26T23:18:07.938 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:07.936+0000 7f0c59a15640 1 -- 172.21.15.203:0/1800339131 wait complete. 2024-04-26T23:18:07.947 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --cluster ceph osd new 1286a67e-8958-426a-8c2e-bc85498c619c 3 2024-04-26T23:18:08.045 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.044+0000 7fe73766d640 1 Processor -- start 2024-04-26T23:18:08.045 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.044+0000 7fe73766d640 1 -- start start 2024-04-26T23:18:08.049 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.045+0000 7fe73766d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe7300b4d40 0x7fe7300b6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:08.049 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.045+0000 7fe73766d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7300b6c50 0x7fe7300aed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:08.050 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.045+0000 7fe73766d640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe730072fb0 con 0x7fe7300b0b70 2024-04-26T23:18:08.050 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.045+0000 7fe73766d640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fe730073bc0 con 0x7fe7300b4d40 2024-04-26T23:18:08.050 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.045+0000 7fe73766d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fe730073290 con 0x7fe7300b6c50 2024-04-26T23:18:08.050 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.045+0000 7fe735c0a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7300b6c50 0x7fe7300aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:08.050 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.045+0000 7fe735c0a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7300b6c50 0x7fe7300aed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.203:33556/0 (socket says 172.21.15.203:33556) 2024-04-26T23:18:08.050 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.045+0000 7fe735c0a640 1 -- 172.21.15.203:0/1370214191 learned_addr learned my addr 172.21.15.203:0/1370214191 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-26T23:18:08.050 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.046+0000 7fe734c08640 1 --2- 172.21.15.203:0/1370214191 >> v2:172.21.15.47:3300/0 conn(0x7fe7300b4d40 0x7fe7300b6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:08.050 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.046+0000 7fe735c0a640 1 -- 172.21.15.203:0/1370214191 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7300b6c50 msgr2=0x7fe7300aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-26T23:18:08.051 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.046+0000 7fe735c0a640 1 -- 172.21.15.203:0/1370214191 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7300b6c50 msgr2=0x7fe7300aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T23:18:08.051 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.046+0000 7fe735c0a640 1 --2- 172.21.15.203:0/1370214191 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7300b6c50 0x7fe7300aed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T23:18:08.051 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.046+0000 7fe735c0a640 1 --2- 172.21.15.203:0/1370214191 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7300b6c50 0x7fe7300aed90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T23:18:08.051 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.046+0000 7fe727fff640 1 -- 172.21.15.203:0/1370214191 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3986769013 0 0) 0x7fe730072fb0 con 0x7fe7300b0b70 2024-04-26T23:18:08.051 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.046+0000 7fe727fff640 1 -- 172.21.15.203:0/1370214191 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe71c003520 con 0x7fe7300b0b70 2024-04-26T23:18:08.051 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.046+0000 7fe734c08640 1 -- 172.21.15.203:0/1370214191 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7300b6c50 msgr2=0x7fe7300aed90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T23:18:08.051 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.046+0000 7fe734c08640 1 --2- 172.21.15.203:0/1370214191 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7300b6c50 0x7fe7300aed90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:08.051 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.046+0000 7fe734c08640 1 -- 172.21.15.203:0/1370214191 >> v1:172.21.15.47:6789/0 conn(0x7fe7300b0b70 legacy=0x7fe7300b0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:08.051 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.046+0000 7fe734c08640 1 -- 172.21.15.203:0/1370214191 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe730073570 con 0x7fe7300b4d40 2024-04-26T23:18:08.051 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.047+0000 7fe734c08640 1 --2- 172.21.15.203:0/1370214191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7300b4d40 0x7fe7300b6710 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe7200097a0 tx=0x7fe72002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bce3d1a42e2495ff server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:08.051 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.047+0000 7fe727fff640 1 -- 172.21.15.203:0/1370214191 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe720037070 con 0x7fe7300b4d40 2024-04-26T23:18:08.051 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.047+0000 7fe727fff640 1 -- 172.21.15.203:0/1370214191 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe720032440 con 0x7fe7300b4d40 2024-04-26T23:18:08.051 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.047+0000 7fe727fff640 1 -- 172.21.15.203:0/1370214191 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe720032810 con 0x7fe7300b4d40 2024-04-26T23:18:08.051 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.047+0000 7fe73766d640 1 -- 172.21.15.203:0/1370214191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7300b4d40 msgr2=0x7fe7300b6710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:08.052 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.047+0000 7fe73766d640 1 --2- 172.21.15.203:0/1370214191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7300b4d40 0x7fe7300b6710 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe7200097a0 tx=0x7fe72002a180 comp rx=0 tx=0).stop 2024-04-26T23:18:08.052 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.047+0000 7fe73766d640 1 -- 172.21.15.203:0/1370214191 shutdown_connections 2024-04-26T23:18:08.052 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.047+0000 7fe73766d640 1 --2- 172.21.15.203:0/1370214191 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7300b6c50 0x7fe7300aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:08.052 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.047+0000 7fe73766d640 1 --2- 172.21.15.203:0/1370214191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7300b4d40 0x7fe7300b6710 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:08.052 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.047+0000 7fe73766d640 1 -- 172.21.15.203:0/1370214191 >> 172.21.15.203:0/1370214191 conn(0x7fe7300b39b0 msgr2=0x7fe7300b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:08.052 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.048+0000 7fe73766d640 1 -- 172.21.15.203:0/1370214191 shutdown_connections 2024-04-26T23:18:08.052 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.048+0000 7fe73766d640 1 -- 172.21.15.203:0/1370214191 wait complete. 2024-04-26T23:18:08.052 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.049+0000 7fe73766d640 1 Processor -- start 2024-04-26T23:18:08.052 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.049+0000 7fe73766d640 1 -- start start 2024-04-26T23:18:08.055 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.049+0000 7fe73766d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7300b0b70 0x7fe730155740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:08.055 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.049+0000 7fe73766d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7300b4d40 0x7fe7301f4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:08.055 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.049+0000 7fe73766d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7300b6c50 0x7fe7301f54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:08.055 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.049+0000 7fe73766d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe7301604e0 con 0x7fe7300b4d40 2024-04-26T23:18:08.055 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.049+0000 7fe73766d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe730160360 con 0x7fe7300b6c50 2024-04-26T23:18:08.055 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.049+0000 7fe73766d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fe7300af2d0 con 0x7fe7300b0b70 2024-04-26T23:18:08.055 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.049+0000 7fe735409640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7300b0b70 0x7fe730155740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:08.055 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.049+0000 7fe735409640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7300b0b70 0x7fe730155740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.203:33566/0 (socket says 172.21.15.203:33566) 2024-04-26T23:18:08.055 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.049+0000 7fe735409640 1 -- 172.21.15.203:0/1877648060 learned_addr learned my addr 172.21.15.203:0/1877648060 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-26T23:18:08.055 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.049+0000 7fe735c0a640 1 --2- 172.21.15.203:0/1877648060 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7300b6c50 0x7fe7301f54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:08.055 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.050+0000 7fe735409640 1 -- 172.21.15.203:0/1877648060 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7300b0b70 msgr2=0x7fe730155740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-04-26T23:18:08.056 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.050+0000 7fe735409640 1 -- 172.21.15.203:0/1877648060 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7300b0b70 msgr2=0x7fe730155740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T23:18:08.056 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.050+0000 7fe735409640 1 --2- 172.21.15.203:0/1877648060 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7300b0b70 0x7fe730155740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T23:18:08.056 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.050+0000 7fe735409640 1 --2- 172.21.15.203:0/1877648060 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7300b0b70 0x7fe730155740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T23:18:08.056 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.050+0000 7fe734c08640 1 --2- 172.21.15.203:0/1877648060 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7300b4d40 0x7fe7301f4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:08.056 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.050+0000 7fe735c0a640 1 -- 172.21.15.203:0/1877648060 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7300b0b70 msgr2=0x7fe730155740 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T23:18:08.056 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.050+0000 7fe735c0a640 1 --2- 172.21.15.203:0/1877648060 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7300b0b70 0x7fe730155740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:08.056 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.050+0000 7fe735c0a640 1 -- 172.21.15.203:0/1877648060 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7300b4d40 msgr2=0x7fe7301f4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:08.056 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.050+0000 7fe735c0a640 1 --2- 172.21.15.203:0/1877648060 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7300b4d40 0x7fe7301f4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:08.056 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.050+0000 7fe735c0a640 1 -- 172.21.15.203:0/1877648060 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe73015dad0 con 0x7fe7300b6c50 2024-04-26T23:18:08.056 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.050+0000 7fe735c0a640 1 --2- 172.21.15.203:0/1877648060 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7300b6c50 0x7fe7301f54d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe728009fd0 tx=0x7fe728007ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:08.056 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.050+0000 7fe725ffb640 1 -- 172.21.15.203:0/1877648060 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72801a070 con 0x7fe7300b6c50 2024-04-26T23:18:08.056 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.050+0000 7fe725ffb640 1 -- 172.21.15.203:0/1877648060 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe72800dbe0 con 0x7fe7300b6c50 2024-04-26T23:18:08.057 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.050+0000 7fe73766d640 1 -- 172.21.15.203:0/1877648060 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe73015ddb0 con 0x7fe7300b6c50 2024-04-26T23:18:08.057 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.050+0000 7fe725ffb640 1 -- 172.21.15.203:0/1877648060 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe728015400 con 0x7fe7300b6c50 2024-04-26T23:18:08.057 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.050+0000 7fe73766d640 1 -- 172.21.15.203:0/1877648060 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe730143c50 con 0x7fe7300b6c50 2024-04-26T23:18:08.057 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.050+0000 7fe725ffb640 1 -- 172.21.15.203:0/1877648060 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe72801d070 con 0x7fe7300b6c50 2024-04-26T23:18:08.057 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.051+0000 7fe725ffb640 1 -- 172.21.15.203:0/1877648060 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fe728015a30 con 0x7fe7300b6c50 2024-04-26T23:18:08.057 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.051+0000 7fe73766d640 1 -- 172.21.15.203:0/1877648060 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7300694f0 con 0x7fe7300b6c50 2024-04-26T23:18:08.057 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.053+0000 7fe725ffb640 1 -- 172.21.15.203:0/1877648060 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fe728024070 con 0x7fe7300b6c50 2024-04-26T23:18:08.127 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.126+0000 7fe73766d640 1 -- 172.21.15.203:0/1877648060 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd new", "uuid": "1286a67e-8958-426a-8c2e-bc85498c619c", "id": 3} v 0) -- 0x7fe7300af0b0 con 0x7fe7300b6c50 2024-04-26T23:18:08.127 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-26T23:18:08.127+0000 7fe93fb2c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T23:18:08.130 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.129+0000 7fe725ffb640 1 -- 172.21.15.203:0/1877648060 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1286a67e-8958-426a-8c2e-bc85498c619c", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7fe728010070 con 0x7fe7300b6c50 2024-04-26T23:18:08.130 INFO:teuthology.orchestra.run.smithi203.stdout:3 2024-04-26T23:18:08.131 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.130+0000 7fe73766d640 1 -- 172.21.15.203:0/1877648060 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7300b6c50 msgr2=0x7fe7301f54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:08.131 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.130+0000 7fe73766d640 1 --2- 172.21.15.203:0/1877648060 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7300b6c50 0x7fe7301f54d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe728009fd0 tx=0x7fe728007ef0 comp rx=0 tx=0).stop 2024-04-26T23:18:08.131 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.130+0000 7fe73766d640 1 -- 172.21.15.203:0/1877648060 shutdown_connections 2024-04-26T23:18:08.131 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.130+0000 7fe73766d640 1 --2- 172.21.15.203:0/1877648060 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7300b6c50 0x7fe7301f54d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:08.132 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.130+0000 7fe73766d640 1 --2- 172.21.15.203:0/1877648060 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7300b4d40 0x7fe7301f4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:08.132 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.130+0000 7fe73766d640 1 --2- 172.21.15.203:0/1877648060 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7300b0b70 0x7fe730155740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:08.132 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.130+0000 7fe73766d640 1 -- 172.21.15.203:0/1877648060 >> 172.21.15.203:0/1877648060 conn(0x7fe7300b39b0 msgr2=0x7fe7300afef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:08.132 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.130+0000 7fe73766d640 1 -- 172.21.15.203:0/1877648060 shutdown_connections 2024-04-26T23:18:08.132 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.130+0000 7fe73766d640 1 -- 172.21.15.203:0/1877648060 wait complete. 2024-04-26T23:18:08.132 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-26T23:18:08.132+0000 7f68c4e8c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T23:18:08.141 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --cluster ceph osd new d8ee2c20-e5b7-46c2-9999-b234070126cf 4 2024-04-26T23:18:08.175 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-26T23:18:08.175+0000 7fe93fb2c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T23:18:08.225 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-26T23:18:08.225+0000 7f68c4e8c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T23:18:08.239 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.239+0000 7f7099903640 1 Processor -- start 2024-04-26T23:18:08.240 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.239+0000 7f7099903640 1 -- start start 2024-04-26T23:18:08.242 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.239+0000 7f7099903640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f70940d06f0 0x7f70940d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:08.243 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.239+0000 7f7099903640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70940b2850 0x7f70940b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:08.243 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.239+0000 7f7099903640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70940d0ba0 con 0x7f709415f3a0 2024-04-26T23:18:08.243 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.239+0000 7f7099903640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f70940d0e20 con 0x7f70940d06f0 2024-04-26T23:18:08.243 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.239+0000 7f7099903640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f70940b1d70 con 0x7f70940b2850 2024-04-26T23:18:08.243 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.240+0000 7f70937fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70940b2850 0x7f70940b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:08.243 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.240+0000 7f70937fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70940b2850 0x7f70940b2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.203:33568/0 (socket says 172.21.15.203:33568) 2024-04-26T23:18:08.243 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.240+0000 7f70937fe640 1 -- 172.21.15.203:0/3052548209 learned_addr learned my addr 172.21.15.203:0/3052548209 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-26T23:18:08.243 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.240+0000 7f70927fc640 1 --2- 172.21.15.203:0/3052548209 >> v2:172.21.15.47:3300/0 conn(0x7f70940d06f0 0x7f70940d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:08.243 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.240+0000 7f70937fe640 1 -- 172.21.15.203:0/3052548209 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70940b2850 msgr2=0x7f70940b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-26T23:18:08.244 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.240+0000 7f70937fe640 1 -- 172.21.15.203:0/3052548209 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70940b2850 msgr2=0x7f70940b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T23:18:08.244 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.240+0000 7f70937fe640 1 --2- 172.21.15.203:0/3052548209 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70940b2850 0x7f70940b2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T23:18:08.244 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.240+0000 7f70937fe640 1 --2- 172.21.15.203:0/3052548209 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70940b2850 0x7f70940b2cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T23:18:08.244 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.240+0000 7f7091ffb640 1 -- 172.21.15.203:0/3052548209 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3670453437 0 0) 0x7f7084001110 con 0x7f709415f3a0 2024-04-26T23:18:08.244 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.240+0000 7f7091ffb640 1 -- 172.21.15.203:0/3052548209 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f707c003520 con 0x7f709415f3a0 2024-04-26T23:18:08.244 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.240+0000 7f70927fc640 1 -- 172.21.15.203:0/3052548209 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70940b2850 msgr2=0x7f70940b2cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T23:18:08.244 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.240+0000 7f70927fc640 1 --2- 172.21.15.203:0/3052548209 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70940b2850 0x7f70940b2cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:08.244 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.240+0000 7f70927fc640 1 -- 172.21.15.203:0/3052548209 >> v1:172.21.15.47:6789/0 conn(0x7f709415f3a0 legacy=0x7f70940d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:08.244 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.240+0000 7f70927fc640 1 -- 172.21.15.203:0/3052548209 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70940b2050 con 0x7f70940d06f0 2024-04-26T23:18:08.244 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.241+0000 7f70927fc640 1 --2- 172.21.15.203:0/3052548209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70940d06f0 0x7f70940d1e80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f70880097a0 tx=0x7f708802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17b1d98bdbbbe06d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:08.244 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.241+0000 7f7091ffb640 1 -- 172.21.15.203:0/3052548209 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7088037070 con 0x7f70940d06f0 2024-04-26T23:18:08.245 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.241+0000 7f7091ffb640 1 -- 172.21.15.203:0/3052548209 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7088032440 con 0x7f70940d06f0 2024-04-26T23:18:08.245 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.241+0000 7f7091ffb640 1 -- 172.21.15.203:0/3052548209 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7088032810 con 0x7f70940d06f0 2024-04-26T23:18:08.245 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.242+0000 7f7099903640 1 -- 172.21.15.203:0/3052548209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70940d06f0 msgr2=0x7f70940d1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:08.245 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.242+0000 7f7099903640 1 --2- 172.21.15.203:0/3052548209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70940d06f0 0x7f70940d1e80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f70880097a0 tx=0x7f708802a180 comp rx=0 tx=0).stop 2024-04-26T23:18:08.245 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.242+0000 7f7099903640 1 -- 172.21.15.203:0/3052548209 shutdown_connections 2024-04-26T23:18:08.245 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.242+0000 7f7099903640 1 --2- 172.21.15.203:0/3052548209 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70940b2850 0x7f70940b2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:08.245 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.242+0000 7f7099903640 1 --2- 172.21.15.203:0/3052548209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70940d06f0 0x7f70940d1e80 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:08.245 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.242+0000 7f7099903640 1 -- 172.21.15.203:0/3052548209 >> 172.21.15.203:0/3052548209 conn(0x7f70940d4f90 msgr2=0x7f70940d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:08.245 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.242+0000 7f7099903640 1 -- 172.21.15.203:0/3052548209 shutdown_connections 2024-04-26T23:18:08.245 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.242+0000 7f7099903640 1 -- 172.21.15.203:0/3052548209 wait complete. 2024-04-26T23:18:08.249 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.243+0000 7f7099903640 1 Processor -- start 2024-04-26T23:18:08.249 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.243+0000 7f7099903640 1 -- start start 2024-04-26T23:18:08.249 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.244+0000 7f7099903640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70940b2850 0x7f70940cec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:08.249 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.244+0000 7f7099903640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70940d06f0 0x7f70941f49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:08.249 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.244+0000 7f7099903640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f709415f3a0 0x7f70941f4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:08.249 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.244+0000 7f7099903640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f709415f900 con 0x7f70940d06f0 2024-04-26T23:18:08.249 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.244+0000 7f7099903640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f709415f780 con 0x7f70940b2850 2024-04-26T23:18:08.249 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.244+0000 7f7099903640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f70940d23c0 con 0x7f709415f3a0 2024-04-26T23:18:08.249 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.244+0000 7f7092ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70940b2850 0x7f70940cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:08.249 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.244+0000 7f70937fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f709415f3a0 0x7f70941f4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:08.250 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.244+0000 7f7092ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70940b2850 0x7f70940cec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:35802/0 (socket says 172.21.15.203:35802) 2024-04-26T23:18:08.250 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.244+0000 7f7092ffd640 1 -- 172.21.15.203:0/3003990575 learned_addr learned my addr 172.21.15.203:0/3003990575 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-26T23:18:08.250 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.244+0000 7f70927fc640 1 --2- 172.21.15.203:0/3003990575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70940d06f0 0x7f70941f49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:08.250 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.244+0000 7f70937fe640 1 -- 172.21.15.203:0/3003990575 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f709415f3a0 msgr2=0x7f70941f4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-04-26T23:18:08.250 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.244+0000 7f70937fe640 1 -- 172.21.15.203:0/3003990575 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f709415f3a0 msgr2=0x7f70941f4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T23:18:08.250 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.244+0000 7f70937fe640 1 --2- 172.21.15.203:0/3003990575 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f709415f3a0 0x7f70941f4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T23:18:08.250 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.244+0000 7f70937fe640 1 --2- 172.21.15.203:0/3003990575 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f709415f3a0 0x7f70941f4f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T23:18:08.250 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.244+0000 7f7092ffd640 1 -- 172.21.15.203:0/3003990575 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f709415f3a0 msgr2=0x7f70941f4f30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T23:18:08.250 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.244+0000 7f7092ffd640 1 --2- 172.21.15.203:0/3003990575 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f709415f3a0 0x7f70941f4f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:08.250 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.244+0000 7f7092ffd640 1 -- 172.21.15.203:0/3003990575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70940d06f0 msgr2=0x7f70941f49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:08.251 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.244+0000 7f7092ffd640 1 --2- 172.21.15.203:0/3003990575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70940d06f0 0x7f70941f49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:08.251 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.244+0000 7f7092ffd640 1 -- 172.21.15.203:0/3003990575 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70941f5550 con 0x7f70940b2850 2024-04-26T23:18:08.251 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.244+0000 7f7092ffd640 1 --2- 172.21.15.203:0/3003990575 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70940b2850 0x7f70940cec90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f708400c730 tx=0x7f708400cbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:08.251 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.244+0000 7f707bfff640 1 -- 172.21.15.203:0/3003990575 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70840052d0 con 0x7f70940b2850 2024-04-26T23:18:08.251 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.245+0000 7f707bfff640 1 -- 172.21.15.203:0/3003990575 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7084005490 con 0x7f70940b2850 2024-04-26T23:18:08.251 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.245+0000 7f7099903640 1 -- 172.21.15.203:0/3003990575 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70941f5830 con 0x7f70940b2850 2024-04-26T23:18:08.251 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.245+0000 7f707bfff640 1 -- 172.21.15.203:0/3003990575 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f708400a880 con 0x7f70940b2850 2024-04-26T23:18:08.251 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.245+0000 7f7099903640 1 -- 172.21.15.203:0/3003990575 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70941f60d0 con 0x7f70940b2850 2024-04-26T23:18:08.251 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.245+0000 7f707bfff640 1 -- 172.21.15.203:0/3003990575 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f708400aa20 con 0x7f70940b2850 2024-04-26T23:18:08.251 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.245+0000 7f707bfff640 1 -- 172.21.15.203:0/3003990575 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f7084012930 con 0x7f70940b2850 2024-04-26T23:18:08.251 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.246+0000 7f7099903640 1 -- 172.21.15.203:0/3003990575 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70941f5a00 con 0x7f70940b2850 2024-04-26T23:18:08.252 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.249+0000 7f707bfff640 1 -- 172.21.15.203:0/3003990575 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f70940d0ba0 con 0x7f70940b2850 2024-04-26T23:18:08.272 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-26T23:18:08.272+0000 7f68c4e8c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T23:18:08.323 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.322+0000 7f7099903640 1 -- 172.21.15.203:0/3003990575 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d8ee2c20-e5b7-46c2-9999-b234070126cf", "id": 4} v 0) -- 0x7f70940694f0 con 0x7f70940b2850 2024-04-26T23:18:08.326 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.326+0000 7f707bfff640 1 -- 172.21.15.203:0/3003990575 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d8ee2c20-e5b7-46c2-9999-b234070126cf", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f7084003e10 con 0x7f70940b2850 2024-04-26T23:18:08.326 INFO:teuthology.orchestra.run.smithi203.stdout:4 2024-04-26T23:18:08.327 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.327+0000 7f7099903640 1 -- 172.21.15.203:0/3003990575 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70940b2850 msgr2=0x7f70940cec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:08.328 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.327+0000 7f7099903640 1 --2- 172.21.15.203:0/3003990575 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70940b2850 0x7f70940cec90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f708400c730 tx=0x7f708400cbf0 comp rx=0 tx=0).stop 2024-04-26T23:18:08.328 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.327+0000 7f7099903640 1 -- 172.21.15.203:0/3003990575 shutdown_connections 2024-04-26T23:18:08.328 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.327+0000 7f7099903640 1 --2- 172.21.15.203:0/3003990575 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f709415f3a0 0x7f70941f4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:08.328 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.327+0000 7f7099903640 1 --2- 172.21.15.203:0/3003990575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70940d06f0 0x7f70941f49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:08.328 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.327+0000 7f7099903640 1 --2- 172.21.15.203:0/3003990575 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70940b2850 0x7f70940cec90 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:08.328 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.327+0000 7f7099903640 1 -- 172.21.15.203:0/3003990575 >> 172.21.15.203:0/3003990575 conn(0x7f70940d4f90 msgr2=0x7f70940b4830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:08.328 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.327+0000 7f7099903640 1 -- 172.21.15.203:0/3003990575 shutdown_connections 2024-04-26T23:18:08.328 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.327+0000 7f7099903640 1 -- 172.21.15.203:0/3003990575 wait complete. 2024-04-26T23:18:08.338 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --cluster ceph osd new b453a8f0-f324-4840-9ae0-d67bdadb9f8b 5 2024-04-26T23:18:08.436 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.435+0000 7fa655d33640 1 Processor -- start 2024-04-26T23:18:08.436 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.435+0000 7fa655d33640 1 -- start start 2024-04-26T23:18:08.438 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.436+0000 7fa655d33640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa6500b0b70 0x7fa6500b0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:08.438 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.436+0000 7fa655d33640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6500b1490 con 0x7fa6500b4d40 2024-04-26T23:18:08.438 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.436+0000 7fa655d33640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa650072fb0 con 0x7fa6500aed90 2024-04-26T23:18:08.438 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.436+0000 7fa655d33640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fa650073bc0 con 0x7fa6500b0b70 2024-04-26T23:18:08.438 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.436+0000 7fa64f7fe640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa6500b0b70 0x7fa6500b0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:08.438 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.436+0000 7fa64ffff640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fa6500aed90 0x7fa6500af240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:56024/0 (socket says 172.21.15.203:56024) 2024-04-26T23:18:08.438 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.436+0000 7fa64ffff640 1 -- 172.21.15.203:0/3430591888 learned_addr learned my addr 172.21.15.203:0/3430591888 (peer_addr_for_me v1:172.21.15.203:0/0) 2024-04-26T23:18:08.438 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.436+0000 7fa64f7fe640 1 -- 172.21.15.203:0/3430591888 >> v1:172.21.15.203:6789/0 conn(0x7fa6500aed90 legacy=0x7fa6500af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:08.438 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.436+0000 7fa64f7fe640 1 -- 172.21.15.203:0/3430591888 >> v1:172.21.15.47:6789/0 conn(0x7fa6500b4d40 legacy=0x7fa6500b6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:08.438 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.436+0000 7fa64f7fe640 1 -- 172.21.15.203:0/3430591888 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa650073290 con 0x7fa6500b0b70 2024-04-26T23:18:08.439 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.437+0000 7fa64f7fe640 1 --2- 172.21.15.203:0/3430591888 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6500b0b70 0x7fa6500b0f50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa6400097a0 tx=0x7fa64002a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da5f9a440b96d424 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:08.439 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.437+0000 7fa64effd640 1 -- 172.21.15.203:0/3430591888 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa640037070 con 0x7fa6500b0b70 2024-04-26T23:18:08.439 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.437+0000 7fa64effd640 1 -- 172.21.15.203:0/3430591888 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa640032440 con 0x7fa6500b0b70 2024-04-26T23:18:08.439 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.437+0000 7fa64effd640 1 -- 172.21.15.203:0/3430591888 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa640032810 con 0x7fa6500b0b70 2024-04-26T23:18:08.440 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.438+0000 7fa655d33640 1 -- 172.21.15.203:0/3430591888 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6500b0b70 msgr2=0x7fa6500b0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:08.440 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.438+0000 7fa655d33640 1 --2- 172.21.15.203:0/3430591888 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6500b0b70 0x7fa6500b0f50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa6400097a0 tx=0x7fa64002a230 comp rx=0 tx=0).stop 2024-04-26T23:18:08.440 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.438+0000 7fa655d33640 1 -- 172.21.15.203:0/3430591888 shutdown_connections 2024-04-26T23:18:08.440 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.438+0000 7fa655d33640 1 --2- 172.21.15.203:0/3430591888 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6500b0b70 0x7fa6500b0f50 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:08.440 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.438+0000 7fa655d33640 1 -- 172.21.15.203:0/3430591888 >> 172.21.15.203:0/3430591888 conn(0x7fa6500b39b0 msgr2=0x7fa6500b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:08.440 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.438+0000 7fa655d33640 1 -- 172.21.15.203:0/3430591888 shutdown_connections 2024-04-26T23:18:08.441 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.438+0000 7fa655d33640 1 -- 172.21.15.203:0/3430591888 wait complete. 2024-04-26T23:18:08.441 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.439+0000 7fa655d33640 1 Processor -- start 2024-04-26T23:18:08.441 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.439+0000 7fa655d33640 1 -- start start 2024-04-26T23:18:08.444 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.440+0000 7fa655d33640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6500aed90 0x7fa650156590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:08.444 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.440+0000 7fa655d33640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6500b0b70 0x7fa6501501a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:08.444 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.440+0000 7fa655d33640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6500b4d40 0x7fa65015c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:08.444 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.440+0000 7fa655d33640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa6500b6e20 con 0x7fa6500b0b70 2024-04-26T23:18:08.444 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.440+0000 7fa655d33640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa650073850 con 0x7fa6500aed90 2024-04-26T23:18:08.444 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.440+0000 7fa655d33640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa650160360 con 0x7fa6500b4d40 2024-04-26T23:18:08.444 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.440+0000 7fa64f7fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6500aed90 0x7fa650156590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:08.444 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.440+0000 7fa64ffff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6500b4d40 0x7fa65015c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:08.444 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.440+0000 7fa64f7fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6500aed90 0x7fa650156590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:35824/0 (socket says 172.21.15.203:35824) 2024-04-26T23:18:08.445 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.440+0000 7fa64f7fe640 1 -- 172.21.15.203:0/1636859695 learned_addr learned my addr 172.21.15.203:0/1636859695 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-26T23:18:08.445 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.440+0000 7fa64ffff640 1 -- 172.21.15.203:0/1636859695 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6500b4d40 msgr2=0x7fa65015c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-04-26T23:18:08.445 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.440+0000 7fa64ffff640 1 -- 172.21.15.203:0/1636859695 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6500b4d40 msgr2=0x7fa65015c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T23:18:08.445 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.440+0000 7fa64ffff640 1 --2- 172.21.15.203:0/1636859695 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6500b4d40 0x7fa65015c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T23:18:08.445 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.440+0000 7fa64ffff640 1 --2- 172.21.15.203:0/1636859695 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6500b4d40 0x7fa65015c7b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T23:18:08.445 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.440+0000 7fa647fff640 1 --2- 172.21.15.203:0/1636859695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6500b0b70 0x7fa6501501a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:08.445 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.440+0000 7fa64f7fe640 1 -- 172.21.15.203:0/1636859695 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6500b4d40 msgr2=0x7fa65015c7b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T23:18:08.445 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.440+0000 7fa64f7fe640 1 --2- 172.21.15.203:0/1636859695 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6500b4d40 0x7fa65015c7b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:08.445 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.440+0000 7fa64f7fe640 1 -- 172.21.15.203:0/1636859695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6500b0b70 msgr2=0x7fa6501501a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:08.445 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.440+0000 7fa64f7fe640 1 --2- 172.21.15.203:0/1636859695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6500b0b70 0x7fa6501501a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:08.445 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.440+0000 7fa64f7fe640 1 -- 172.21.15.203:0/1636859695 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa65015d190 con 0x7fa6500aed90 2024-04-26T23:18:08.445 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.440+0000 7fa647fff640 1 --2- 172.21.15.203:0/1636859695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6500b0b70 0x7fa6501501a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:18:08.445 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.440+0000 7fa64f7fe640 1 --2- 172.21.15.203:0/1636859695 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6500aed90 0x7fa650156590 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa640035040 tx=0x7fa6400046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:08.445 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.440+0000 7fa64cff9640 1 -- 172.21.15.203:0/1636859695 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa640037040 con 0x7fa6500aed90 2024-04-26T23:18:08.445 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.441+0000 7fa64cff9640 1 -- 172.21.15.203:0/1636859695 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa640030770 con 0x7fa6500aed90 2024-04-26T23:18:08.446 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.441+0000 7fa655d33640 1 -- 172.21.15.203:0/1636859695 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa65015d470 con 0x7fa6500aed90 2024-04-26T23:18:08.446 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.441+0000 7fa64cff9640 1 -- 172.21.15.203:0/1636859695 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa640030b80 con 0x7fa6500aed90 2024-04-26T23:18:08.446 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.441+0000 7fa655d33640 1 -- 172.21.15.203:0/1636859695 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6501506e0 con 0x7fa6500aed90 2024-04-26T23:18:08.446 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.441+0000 7fa64cff9640 1 -- 172.21.15.203:0/1636859695 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa640044020 con 0x7fa6500aed90 2024-04-26T23:18:08.446 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.441+0000 7fa64cff9640 1 -- 172.21.15.203:0/1636859695 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7fa64003dd50 con 0x7fa6500aed90 2024-04-26T23:18:08.446 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.441+0000 7fa655d33640 1 -- 172.21.15.203:0/1636859695 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6500694f0 con 0x7fa6500aed90 2024-04-26T23:18:08.446 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.443+0000 7fa64cff9640 1 -- 172.21.15.203:0/1636859695 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fa640032e20 con 0x7fa6500aed90 2024-04-26T23:18:08.448 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-26T23:18:08.447+0000 7f68c4e8c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T23:18:08.497 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-26T23:18:08.496+0000 7f68c4e8c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T23:18:08.517 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.517+0000 7fa655d33640 1 -- 172.21.15.203:0/1636859695 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b453a8f0-f324-4840-9ae0-d67bdadb9f8b", "id": 5} v 0) -- 0x7fa650154150 con 0x7fa6500aed90 2024-04-26T23:18:08.520 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.520+0000 7fa64cff9640 1 -- 172.21.15.203:0/1636859695 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b453a8f0-f324-4840-9ae0-d67bdadb9f8b", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7fa64003d7d0 con 0x7fa6500aed90 2024-04-26T23:18:08.521 INFO:teuthology.orchestra.run.smithi203.stdout:5 2024-04-26T23:18:08.522 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.521+0000 7fa655d33640 1 -- 172.21.15.203:0/1636859695 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6500aed90 msgr2=0x7fa650156590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:08.522 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.521+0000 7fa655d33640 1 --2- 172.21.15.203:0/1636859695 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6500aed90 0x7fa650156590 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa640035040 tx=0x7fa6400046c0 comp rx=0 tx=0).stop 2024-04-26T23:18:08.522 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.521+0000 7fa655d33640 1 -- 172.21.15.203:0/1636859695 shutdown_connections 2024-04-26T23:18:08.522 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.521+0000 7fa655d33640 1 --2- 172.21.15.203:0/1636859695 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6500b4d40 0x7fa65015c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:08.522 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.521+0000 7fa655d33640 1 --2- 172.21.15.203:0/1636859695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6500b0b70 0x7fa6501501a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:08.522 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.521+0000 7fa655d33640 1 --2- 172.21.15.203:0/1636859695 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6500aed90 0x7fa650156590 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:08.522 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.521+0000 7fa655d33640 1 -- 172.21.15.203:0/1636859695 >> 172.21.15.203:0/1636859695 conn(0x7fa6500b39b0 msgr2=0x7fa65006f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:08.522 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.521+0000 7fa655d33640 1 -- 172.21.15.203:0/1636859695 shutdown_connections 2024-04-26T23:18:08.522 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.521+0000 7fa655d33640 1 -- 172.21.15.203:0/1636859695 wait complete. 2024-04-26T23:18:08.532 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --cluster ceph osd new a6d31951-2d4b-4e8d-a510-6283b01dc158 6 2024-04-26T23:18:08.553 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-26T23:18:08.552+0000 7f68c4e8c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T23:18:08.630 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.629+0000 7f9718a64640 1 Processor -- start 2024-04-26T23:18:08.630 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.629+0000 7f9718a64640 1 -- start start 2024-04-26T23:18:08.631 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.630+0000 7f9718a64640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f97141613f0 0x7f9714071080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:08.631 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.630+0000 7f9718a64640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97140702c0 con 0x7f971415d1a0 2024-04-26T23:18:08.631 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.630+0000 7f9718a64640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97140705a0 con 0x7f97140d3b20 2024-04-26T23:18:08.631 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.630+0000 7f9718a64640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f97140cec30 con 0x7f97141613f0 2024-04-26T23:18:08.631 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.630+0000 7f9712d9d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f97141613f0 0x7f9714071080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:08.631 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.630+0000 7f9711d9b640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f97140d3b20 0x7f9714160ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:56030/0 (socket says 172.21.15.203:56030) 2024-04-26T23:18:08.631 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.630+0000 7f9711d9b640 1 -- 172.21.15.203:0/1114151373 learned_addr learned my addr 172.21.15.203:0/1114151373 (peer_addr_for_me v1:172.21.15.203:0/0) 2024-04-26T23:18:08.632 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.630+0000 7f9712d9d640 1 -- 172.21.15.203:0/1114151373 >> v1:172.21.15.203:6789/0 conn(0x7f97140d3b20 legacy=0x7f9714160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:08.632 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.630+0000 7f9712d9d640 1 -- 172.21.15.203:0/1114151373 >> v1:172.21.15.47:6789/0 conn(0x7f971415d1a0 legacy=0x7f97140ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:08.632 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.630+0000 7f9712d9d640 1 -- 172.21.15.203:0/1114151373 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9714070880 con 0x7f97141613f0 2024-04-26T23:18:08.633 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.631+0000 7f9712d9d640 1 --2- 172.21.15.203:0/1114151373 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97141613f0 0x7f9714071080 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f970c009d90 tx=0x7f970c02a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d2850040e9bdb8d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:08.633 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.631+0000 7f971159a640 1 -- 172.21.15.203:0/1114151373 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970c007740 con 0x7f97141613f0 2024-04-26T23:18:08.633 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.631+0000 7f971159a640 1 -- 172.21.15.203:0/1114151373 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f970c0078e0 con 0x7f97141613f0 2024-04-26T23:18:08.633 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.631+0000 7f971159a640 1 -- 172.21.15.203:0/1114151373 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970c007cb0 con 0x7f97141613f0 2024-04-26T23:18:08.634 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.632+0000 7f9718a64640 1 -- 172.21.15.203:0/1114151373 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97141613f0 msgr2=0x7f9714071080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:08.634 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.632+0000 7f9718a64640 1 --2- 172.21.15.203:0/1114151373 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97141613f0 0x7f9714071080 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f970c009d90 tx=0x7f970c02a6f0 comp rx=0 tx=0).stop 2024-04-26T23:18:08.634 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.632+0000 7f9718a64640 1 -- 172.21.15.203:0/1114151373 shutdown_connections 2024-04-26T23:18:08.634 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.632+0000 7f9718a64640 1 --2- 172.21.15.203:0/1114151373 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97141613f0 0x7f9714071080 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:08.634 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.632+0000 7f9718a64640 1 -- 172.21.15.203:0/1114151373 >> 172.21.15.203:0/1114151373 conn(0x7f97140d2e00 msgr2=0x7f97140d31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:08.634 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.632+0000 7f9718a64640 1 -- 172.21.15.203:0/1114151373 shutdown_connections 2024-04-26T23:18:08.634 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.632+0000 7f9718a64640 1 -- 172.21.15.203:0/1114151373 wait complete. 2024-04-26T23:18:08.637 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.633+0000 7f9718a64640 1 Processor -- start 2024-04-26T23:18:08.637 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.633+0000 7f9718a64640 1 -- start start 2024-04-26T23:18:08.637 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.634+0000 7f9718a64640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97140d3b20 0x7f9714149f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:08.637 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.634+0000 7f9718a64640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f971415d1a0 0x7f971415e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:08.637 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.634+0000 7f9718a64640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97141613f0 0x7f971414c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:08.637 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.634+0000 7f9718a64640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f97140d0d00 con 0x7f971415d1a0 2024-04-26T23:18:08.637 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.634+0000 7f9718a64640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f971415d580 con 0x7f97141613f0 2024-04-26T23:18:08.637 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.634+0000 7f9718a64640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f97140d0e80 con 0x7f97140d3b20 2024-04-26T23:18:08.637 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.634+0000 7f9712d9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97141613f0 0x7f971414c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:08.637 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.634+0000 7f971259c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97140d3b20 0x7f9714149f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:08.638 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.634+0000 7f9712d9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97141613f0 0x7f971414c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:35832/0 (socket says 172.21.15.203:35832) 2024-04-26T23:18:08.638 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.634+0000 7f9712d9d640 1 -- 172.21.15.203:0/1414102954 learned_addr learned my addr 172.21.15.203:0/1414102954 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-26T23:18:08.638 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.634+0000 7f9711d9b640 1 --2- 172.21.15.203:0/1414102954 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f971415d1a0 0x7f971415e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:08.638 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.634+0000 7f971259c640 1 -- 172.21.15.203:0/1414102954 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97140d3b20 msgr2=0x7f9714149f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-04-26T23:18:08.638 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.634+0000 7f971259c640 1 -- 172.21.15.203:0/1414102954 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97140d3b20 msgr2=0x7f9714149f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T23:18:08.638 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.634+0000 7f971259c640 1 --2- 172.21.15.203:0/1414102954 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97140d3b20 0x7f9714149f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T23:18:08.638 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.634+0000 7f971259c640 1 --2- 172.21.15.203:0/1414102954 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97140d3b20 0x7f9714149f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T23:18:08.638 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.634+0000 7f9712d9d640 1 -- 172.21.15.203:0/1414102954 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97140d3b20 msgr2=0x7f9714149f70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T23:18:08.638 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.634+0000 7f9712d9d640 1 --2- 172.21.15.203:0/1414102954 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97140d3b20 0x7f9714149f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:08.638 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.634+0000 7f9712d9d640 1 -- 172.21.15.203:0/1414102954 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f971415d1a0 msgr2=0x7f971415e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:08.638 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.634+0000 7f9712d9d640 1 --2- 172.21.15.203:0/1414102954 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f971415d1a0 0x7f971415e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:08.638 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.634+0000 7f9712d9d640 1 -- 172.21.15.203:0/1414102954 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f971414cb60 con 0x7f97141613f0 2024-04-26T23:18:08.638 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.634+0000 7f9712d9d640 1 --2- 172.21.15.203:0/1414102954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97141613f0 0x7f971414c620 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f970c009eb0 tx=0x7f970c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:08.638 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.634+0000 7f9702ffd640 1 -- 172.21.15.203:0/1414102954 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970c03f070 con 0x7f97141613f0 2024-04-26T23:18:08.639 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.635+0000 7f9702ffd640 1 -- 172.21.15.203:0/1414102954 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f970c031c90 con 0x7f97141613f0 2024-04-26T23:18:08.639 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.635+0000 7f9718a64640 1 -- 172.21.15.203:0/1414102954 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f971414ce10 con 0x7f97141613f0 2024-04-26T23:18:08.639 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.635+0000 7f9702ffd640 1 -- 172.21.15.203:0/1414102954 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970c044680 con 0x7f97141613f0 2024-04-26T23:18:08.639 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.635+0000 7f9718a64640 1 -- 172.21.15.203:0/1414102954 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f971414d3d0 con 0x7f97141613f0 2024-04-26T23:18:08.639 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.635+0000 7f9702ffd640 1 -- 172.21.15.203:0/1414102954 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f970c044820 con 0x7f97141613f0 2024-04-26T23:18:08.639 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.635+0000 7f9702ffd640 1 -- 172.21.15.203:0/1414102954 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f970c03bcd0 con 0x7f97141613f0 2024-04-26T23:18:08.639 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.635+0000 7f9718a64640 1 -- 172.21.15.203:0/1414102954 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97140694f0 con 0x7f97141613f0 2024-04-26T23:18:08.639 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.637+0000 7f9702ffd640 1 -- 172.21.15.203:0/1414102954 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f970c007840 con 0x7f97141613f0 2024-04-26T23:18:08.672 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-26T23:18:08.671+0000 7f68c4e8c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T23:18:08.711 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.711+0000 7f9718a64640 1 -- 172.21.15.203:0/1414102954 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a6d31951-2d4b-4e8d-a510-6283b01dc158", "id": 6} v 0) -- 0x7f971415c8c0 con 0x7f97141613f0 2024-04-26T23:18:08.715 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.714+0000 7f9702ffd640 1 -- 172.21.15.203:0/1414102954 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a6d31951-2d4b-4e8d-a510-6283b01dc158", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f970c03b6e0 con 0x7f97141613f0 2024-04-26T23:18:08.715 INFO:teuthology.orchestra.run.smithi203.stdout:6 2024-04-26T23:18:08.716 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.715+0000 7f9718a64640 1 -- 172.21.15.203:0/1414102954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97141613f0 msgr2=0x7f971414c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:08.716 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.715+0000 7f9718a64640 1 --2- 172.21.15.203:0/1414102954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97141613f0 0x7f971414c620 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f970c009eb0 tx=0x7f970c004a40 comp rx=0 tx=0).stop 2024-04-26T23:18:08.716 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.715+0000 7f9718a64640 1 -- 172.21.15.203:0/1414102954 shutdown_connections 2024-04-26T23:18:08.716 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.715+0000 7f9718a64640 1 --2- 172.21.15.203:0/1414102954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97141613f0 0x7f971414c620 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:08.716 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.715+0000 7f9718a64640 1 --2- 172.21.15.203:0/1414102954 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f971415d1a0 0x7f971415e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:08.716 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.715+0000 7f9718a64640 1 --2- 172.21.15.203:0/1414102954 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97140d3b20 0x7f9714149f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:08.716 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.715+0000 7f9718a64640 1 -- 172.21.15.203:0/1414102954 >> 172.21.15.203:0/1414102954 conn(0x7f97140d2e00 msgr2=0x7f971406d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:08.717 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.715+0000 7f9718a64640 1 -- 172.21.15.203:0/1414102954 shutdown_connections 2024-04-26T23:18:08.717 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.715+0000 7f9718a64640 1 -- 172.21.15.203:0/1414102954 wait complete. 2024-04-26T23:18:08.726 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --cluster ceph osd new 7119b8c0-fe96-402d-9e75-7b509d9d44d9 7 2024-04-26T23:18:08.863 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.863+0000 7f0155a08640 1 Processor -- start 2024-04-26T23:18:08.864 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.863+0000 7f0155a08640 1 -- start start 2024-04-26T23:18:08.865 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.863+0000 7f0155a08640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f015015f3a0 0x7f01500d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:08.865 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.863+0000 7f0155a08640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f01500d2330 0x7f01500b2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:08.865 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.863+0000 7f0155a08640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01500d0e20 con 0x7f01500d1770 2024-04-26T23:18:08.865 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.863+0000 7f0155a08640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f01500b1d70 con 0x7f01500d2330 2024-04-26T23:18:08.866 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.863+0000 7f0155a08640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f01500b2050 con 0x7f015015f3a0 2024-04-26T23:18:08.866 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.864+0000 7f014effd640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f015015f3a0 0x7f01500d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:08.866 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.864+0000 7f014effd640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f015015f3a0 0x7f01500d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:35844/0 (socket says 172.21.15.203:35844) 2024-04-26T23:18:08.866 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.864+0000 7f014effd640 1 -- 172.21.15.203:0/2741939353 learned_addr learned my addr 172.21.15.203:0/2741939353 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-26T23:18:08.866 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.864+0000 7f014f7fe640 1 --2- 172.21.15.203:0/2741939353 >> v2:172.21.15.47:3300/0 conn(0x7f01500d2330 0x7f01500b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:08.866 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.864+0000 7f014effd640 1 -- 172.21.15.203:0/2741939353 >> v2:172.21.15.47:3300/0 conn(0x7f01500d2330 msgr2=0x7f01500b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:08.866 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.864+0000 7f014effd640 1 --2- 172.21.15.203:0/2741939353 >> v2:172.21.15.47:3300/0 conn(0x7f01500d2330 0x7f01500b2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:08.866 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.864+0000 7f014effd640 1 -- 172.21.15.203:0/2741939353 >> v1:172.21.15.47:6789/0 conn(0x7f01500d1770 legacy=0x7f01500d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:08.866 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.864+0000 7f014effd640 1 -- 172.21.15.203:0/2741939353 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01500b2330 con 0x7f015015f3a0 2024-04-26T23:18:08.866 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.865+0000 7f014effd640 1 --2- 172.21.15.203:0/2741939353 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f015015f3a0 0x7f01500d06f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0150073b50 tx=0x7f014002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=721f83521d50a902 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:08.866 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.865+0000 7f014e7fc640 1 -- 172.21.15.203:0/2741939353 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0140037070 con 0x7f015015f3a0 2024-04-26T23:18:08.866 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.865+0000 7f014e7fc640 1 -- 172.21.15.203:0/2741939353 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0140032440 con 0x7f015015f3a0 2024-04-26T23:18:08.867 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.865+0000 7f014e7fc640 1 -- 172.21.15.203:0/2741939353 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0140032810 con 0x7f015015f3a0 2024-04-26T23:18:08.870 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.865+0000 7f0155a08640 1 -- 172.21.15.203:0/2741939353 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f015015f3a0 msgr2=0x7f01500d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:08.870 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.865+0000 7f0155a08640 1 --2- 172.21.15.203:0/2741939353 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f015015f3a0 0x7f01500d06f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0150073b50 tx=0x7f014002a270 comp rx=0 tx=0).stop 2024-04-26T23:18:08.870 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.865+0000 7f0155a08640 1 -- 172.21.15.203:0/2741939353 shutdown_connections 2024-04-26T23:18:08.870 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.865+0000 7f0155a08640 1 --2- 172.21.15.203:0/2741939353 >> v2:172.21.15.47:3300/0 conn(0x7f01500d2330 0x7f01500b2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:08.870 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.865+0000 7f0155a08640 1 --2- 172.21.15.203:0/2741939353 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f015015f3a0 0x7f01500d06f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:08.870 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.865+0000 7f0155a08640 1 -- 172.21.15.203:0/2741939353 >> 172.21.15.203:0/2741939353 conn(0x7f01500d4f90 msgr2=0x7f01500d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:08.871 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.865+0000 7f0155a08640 1 -- 172.21.15.203:0/2741939353 shutdown_connections 2024-04-26T23:18:08.871 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.865+0000 7f0155a08640 1 -- 172.21.15.203:0/2741939353 wait complete. 2024-04-26T23:18:08.871 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.867+0000 7f0155a08640 1 Processor -- start 2024-04-26T23:18:08.871 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.867+0000 7f0155a08640 1 -- start start 2024-04-26T23:18:08.871 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.867+0000 7f0155a08640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01500d1770 0x7f01500cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:08.871 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.867+0000 7f0155a08640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01500d2330 0x7f0150159c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:08.871 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.867+0000 7f0155a08640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f015015f3a0 0x7f015015a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:08.871 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.867+0000 7f0155a08640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f01500b2f10 con 0x7f01500d1770 2024-04-26T23:18:08.871 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.867+0000 7f0155a08640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f01500b2d90 con 0x7f015015f3a0 2024-04-26T23:18:08.871 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.867+0000 7f0155a08640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f01500b3090 con 0x7f01500d2330 2024-04-26T23:18:08.871 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.867+0000 7f014f7fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f015015f3a0 0x7f015015a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:08.871 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.867+0000 7f0147fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01500d2330 0x7f0150159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:08.871 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.867+0000 7f014effd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01500d1770 0x7f01500cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:08.871 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.867+0000 7f0147fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01500d2330 0x7f0150159c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.203:33580/0 (socket says 172.21.15.203:33580) 2024-04-26T23:18:08.871 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.867+0000 7f014f7fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f015015f3a0 0x7f015015a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:35850/0 (socket says 172.21.15.203:35850) 2024-04-26T23:18:08.872 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.867+0000 7f0147fff640 1 -- 172.21.15.203:0/2042573495 learned_addr learned my addr 172.21.15.203:0/2042573495 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-26T23:18:08.872 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.868+0000 7f0147fff640 1 -- 172.21.15.203:0/2042573495 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01500d2330 msgr2=0x7f0150159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-04-26T23:18:08.872 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.868+0000 7f0147fff640 1 -- 172.21.15.203:0/2042573495 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01500d2330 msgr2=0x7f0150159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T23:18:08.872 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.868+0000 7f0147fff640 1 --2- 172.21.15.203:0/2042573495 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01500d2330 0x7f0150159c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T23:18:08.872 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.868+0000 7f0147fff640 1 --2- 172.21.15.203:0/2042573495 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01500d2330 0x7f0150159c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T23:18:08.872 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.868+0000 7f014f7fe640 1 -- 172.21.15.203:0/2042573495 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01500d2330 msgr2=0x7f0150159c20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T23:18:08.872 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.868+0000 7f014f7fe640 1 --2- 172.21.15.203:0/2042573495 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01500d2330 0x7f0150159c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:08.872 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.868+0000 7f014f7fe640 1 -- 172.21.15.203:0/2042573495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01500d1770 msgr2=0x7f01500cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:08.872 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.868+0000 7f014f7fe640 1 --2- 172.21.15.203:0/2042573495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01500d1770 0x7f01500cdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:08.872 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.868+0000 7f014f7fe640 1 -- 172.21.15.203:0/2042573495 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01500ccb10 con 0x7f015015f3a0 2024-04-26T23:18:08.872 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.868+0000 7f014f7fe640 1 --2- 172.21.15.203:0/2042573495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f015015f3a0 0x7f015015a160 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0148007dc0 tx=0x7f014800e4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:08.872 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.868+0000 7f014cff9640 1 -- 172.21.15.203:0/2042573495 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014800edd0 con 0x7f015015f3a0 2024-04-26T23:18:08.872 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.868+0000 7f014cff9640 1 -- 172.21.15.203:0/2042573495 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0148003990 con 0x7f015015f3a0 2024-04-26T23:18:08.872 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.868+0000 7f0155a08640 1 -- 172.21.15.203:0/2042573495 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01500ccd80 con 0x7f015015f3a0 2024-04-26T23:18:08.872 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.868+0000 7f014cff9640 1 -- 172.21.15.203:0/2042573495 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0148003d80 con 0x7f015015f3a0 2024-04-26T23:18:08.873 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.868+0000 7f0155a08640 1 -- 172.21.15.203:0/2042573495 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01500cd340 con 0x7f015015f3a0 2024-04-26T23:18:08.873 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.868+0000 7f014cff9640 1 -- 172.21.15.203:0/2042573495 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f014801b020 con 0x7f015015f3a0 2024-04-26T23:18:08.873 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.868+0000 7f014cff9640 1 -- 172.21.15.203:0/2042573495 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f014801da70 con 0x7f015015f3a0 2024-04-26T23:18:08.873 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.869+0000 7f0155a08640 1 -- 172.21.15.203:0/2042573495 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01500694f0 con 0x7f015015f3a0 2024-04-26T23:18:08.873 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.873+0000 7f014cff9640 1 -- 172.21.15.203:0/2042573495 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f0148014420 con 0x7f015015f3a0 2024-04-26T23:18:08.914 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-26T23:18:08.913+0000 7fe93fb2c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T23:18:08.944 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.944+0000 7f0155a08640 1 -- 172.21.15.203:0/2042573495 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7119b8c0-fe96-402d-9e75-7b509d9d44d9", "id": 7} v 0) -- 0x7f01500b6860 con 0x7f015015f3a0 2024-04-26T23:18:08.953 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.952+0000 7f014cff9640 1 -- 172.21.15.203:0/2042573495 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7119b8c0-fe96-402d-9e75-7b509d9d44d9", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f0148014600 con 0x7f015015f3a0 2024-04-26T23:18:08.953 INFO:teuthology.orchestra.run.smithi203.stdout:7 2024-04-26T23:18:08.954 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.953+0000 7f0155a08640 1 -- 172.21.15.203:0/2042573495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f015015f3a0 msgr2=0x7f015015a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:08.954 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.953+0000 7f0155a08640 1 --2- 172.21.15.203:0/2042573495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f015015f3a0 0x7f015015a160 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0148007dc0 tx=0x7f014800e4b0 comp rx=0 tx=0).stop 2024-04-26T23:18:08.955 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.953+0000 7f0155a08640 1 -- 172.21.15.203:0/2042573495 shutdown_connections 2024-04-26T23:18:08.955 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.953+0000 7f0155a08640 1 --2- 172.21.15.203:0/2042573495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f015015f3a0 0x7f015015a160 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:08.955 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.953+0000 7f0155a08640 1 --2- 172.21.15.203:0/2042573495 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01500d2330 0x7f0150159c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:08.955 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.953+0000 7f0155a08640 1 --2- 172.21.15.203:0/2042573495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01500d1770 0x7f01500cdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:08.955 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.953+0000 7f0155a08640 1 -- 172.21.15.203:0/2042573495 >> 172.21.15.203:0/2042573495 conn(0x7f01500d4f90 msgr2=0x7f015006e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:08.955 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.953+0000 7f0155a08640 1 -- 172.21.15.203:0/2042573495 shutdown_connections 2024-04-26T23:18:08.955 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:18:08.953+0000 7f0155a08640 1 -- 172.21.15.203:0/2042573495 wait complete. 2024-04-26T23:18:08.965 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-26T23:18:08.965 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-26T23:18:08.967 INFO:tasks.ceph.osd.0:Started 2024-04-26T23:18:08.968 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-26T23:18:08.968 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-26T23:18:08.972 INFO:tasks.ceph.osd.1:Started 2024-04-26T23:18:08.973 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-26T23:18:08.973 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-26T23:18:08.975 INFO:tasks.ceph.osd.2:Started 2024-04-26T23:18:08.975 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-26T23:18:08.975 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-26T23:18:08.977 INFO:tasks.ceph.osd.3:Started 2024-04-26T23:18:08.978 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-26T23:18:08.978 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-26T23:18:08.980 INFO:tasks.ceph.osd.4:Started 2024-04-26T23:18:08.980 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-26T23:18:08.980 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-26T23:18:08.982 INFO:tasks.ceph.osd.5:Started 2024-04-26T23:18:08.982 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-26T23:18:08.982 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-26T23:18:08.984 INFO:tasks.ceph.osd.6:Started 2024-04-26T23:18:08.984 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-26T23:18:08.984 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-26T23:18:08.986 INFO:tasks.ceph.osd.7:Started 2024-04-26T23:18:08.987 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:18:09.028 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-26T23:18:09.026+0000 7fe93fb2c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T23:18:09.090 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-26T23:18:09.088+0000 7fe93fb2c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T23:18:09.100 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-26T23:18:09.099+0000 7f68c4e8c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T23:18:09.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.106+0000 7f859f637640 1 Processor -- start 2024-04-26T23:18:09.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.107+0000 7f859f637640 1 -- start start 2024-04-26T23:18:09.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.107+0000 7f859f637640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f859815f520 0x7f859816a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:09.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.107+0000 7f859f637640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85980cfde0 con 0x7f85980d3d10 2024-04-26T23:18:09.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.107+0000 7f859f637640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85980d00c0 con 0x7f859815f140 2024-04-26T23:18:09.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.107+0000 7f859f637640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f85980b8eb0 con 0x7f859815f520 2024-04-26T23:18:09.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.108+0000 7f859cbd2640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f85980d3d10 0x7f85980d1990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38584/0 (socket says 172.21.15.47:38584) 2024-04-26T23:18:09.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.108+0000 7f859cbd2640 1 -- 172.21.15.47:0/2660221424 learned_addr learned my addr 172.21.15.47:0/2660221424 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:18:09.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.108+0000 7f859dbd4640 1 --2- 172.21.15.47:0/2660221424 >> v2:172.21.15.203:3300/0 conn(0x7f859815f520 0x7f859816a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:09.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.108+0000 7f858ffff640 1 -- 172.21.15.47:0/2660221424 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2559939706 0 0) 0x7f85980cfde0 con 0x7f85980d3d10 2024-04-26T23:18:09.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.108+0000 7f858ffff640 1 -- 172.21.15.47:0/2660221424 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8584003520 con 0x7f85980d3d10 2024-04-26T23:18:09.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.108+0000 7f859dbd4640 1 -- 172.21.15.47:0/2660221424 >> v1:172.21.15.203:6789/0 conn(0x7f859815f140 legacy=0x7f85980d3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:09.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.108+0000 7f859dbd4640 1 -- 172.21.15.47:0/2660221424 >> v1:172.21.15.47:6789/0 conn(0x7f85980d3d10 legacy=0x7f85980d1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:09.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.108+0000 7f859dbd4640 1 -- 172.21.15.47:0/2660221424 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85980d03a0 con 0x7f859815f520 2024-04-26T23:18:09.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.109+0000 7f859dbd4640 1 --2- 172.21.15.47:0/2660221424 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f859815f520 0x7f859816a160 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8590009dc0 tx=0x7f859002a730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=673cd595f8e33180 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:09.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.109+0000 7f858ffff640 1 -- 172.21.15.47:0/2660221424 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8590007740 con 0x7f859815f520 2024-04-26T23:18:09.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.109+0000 7f858ffff640 1 -- 172.21.15.47:0/2660221424 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85900078e0 con 0x7f859815f520 2024-04-26T23:18:09.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.109+0000 7f858ffff640 1 -- 172.21.15.47:0/2660221424 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8590007cb0 con 0x7f859815f520 2024-04-26T23:18:09.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.110+0000 7f859f637640 1 -- 172.21.15.47:0/2660221424 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f859815f520 msgr2=0x7f859816a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:09.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.110+0000 7f859f637640 1 --2- 172.21.15.47:0/2660221424 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f859815f520 0x7f859816a160 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8590009dc0 tx=0x7f859002a730 comp rx=0 tx=0).stop 2024-04-26T23:18:09.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.110+0000 7f859f637640 1 -- 172.21.15.47:0/2660221424 shutdown_connections 2024-04-26T23:18:09.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.110+0000 7f859f637640 1 --2- 172.21.15.47:0/2660221424 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f859815f520 0x7f859816a160 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:09.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.110+0000 7f859f637640 1 -- 172.21.15.47:0/2660221424 >> 172.21.15.47:0/2660221424 conn(0x7f859809e320 msgr2=0x7f85980d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:09.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.110+0000 7f859f637640 1 -- 172.21.15.47:0/2660221424 shutdown_connections 2024-04-26T23:18:09.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.110+0000 7f859f637640 1 -- 172.21.15.47:0/2660221424 wait complete. 2024-04-26T23:18:09.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.111+0000 7f859f637640 1 Processor -- start 2024-04-26T23:18:09.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.111+0000 7f859f637640 1 -- start start 2024-04-26T23:18:09.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.112+0000 7f859f637640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85980d3d10 0x7f85980cead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:09.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.112+0000 7f859f637640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f859815f140 0x7f85980cc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:09.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.112+0000 7f859f637640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f859815f520 0x7f85980b9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:09.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.112+0000 7f859f637640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f85980d0e80 con 0x7f85980d3d10 2024-04-26T23:18:09.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.112+0000 7f859f637640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f85980d0960 con 0x7f859815f140 2024-04-26T23:18:09.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.112+0000 7f859f637640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f859815acb0 con 0x7f859815f520 2024-04-26T23:18:09.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.112+0000 7f859d3d3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85980d3d10 0x7f85980cead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:09.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.112+0000 7f859d3d3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85980d3d10 0x7f85980cead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44652/0 (socket says 172.21.15.47:44652) 2024-04-26T23:18:09.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.112+0000 7f859d3d3640 1 -- 172.21.15.47:0/3560566910 learned_addr learned my addr 172.21.15.47:0/3560566910 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:09.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.112+0000 7f859cbd2640 1 --2- 172.21.15.47:0/3560566910 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f859815f140 0x7f85980cc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:09.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.112+0000 7f859dbd4640 1 --2- 172.21.15.47:0/3560566910 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f859815f520 0x7f85980b9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:09.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.112+0000 7f859dbd4640 1 -- 172.21.15.47:0/3560566910 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f859815f520 msgr2=0x7f85980b9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-04-26T23:18:09.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.112+0000 7f859dbd4640 1 -- 172.21.15.47:0/3560566910 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f859815f520 msgr2=0x7f85980b9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T23:18:09.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.112+0000 7f859dbd4640 1 --2- 172.21.15.47:0/3560566910 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f859815f520 0x7f85980b9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T23:18:09.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.112+0000 7f859dbd4640 1 --2- 172.21.15.47:0/3560566910 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f859815f520 0x7f85980b9f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T23:18:09.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.112+0000 7f859d3d3640 1 -- 172.21.15.47:0/3560566910 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f859815f520 msgr2=0x7f85980b9f50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T23:18:09.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.112+0000 7f859d3d3640 1 --2- 172.21.15.47:0/3560566910 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f859815f520 0x7f85980b9f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:09.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.112+0000 7f859d3d3640 1 -- 172.21.15.47:0/3560566910 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f859815f140 msgr2=0x7f85980cc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:09.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.112+0000 7f859d3d3640 1 --2- 172.21.15.47:0/3560566910 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f859815f140 0x7f85980cc5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:09.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.112+0000 7f859d3d3640 1 -- 172.21.15.47:0/3560566910 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85980cd9c0 con 0x7f85980d3d10 2024-04-26T23:18:09.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.112+0000 7f859cbd2640 1 --2- 172.21.15.47:0/3560566910 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f859815f140 0x7f85980cc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:18:09.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.112+0000 7f859d3d3640 1 --2- 172.21.15.47:0/3560566910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85980d3d10 0x7f85980cead0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f858800c430 tx=0x7f858800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:09.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.112+0000 7f858dffb640 1 -- 172.21.15.47:0/3560566910 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85880052d0 con 0x7f85980d3d10 2024-04-26T23:18:09.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.112+0000 7f858dffb640 1 -- 172.21.15.47:0/3560566910 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8588005490 con 0x7f85980d3d10 2024-04-26T23:18:09.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.112+0000 7f859f637640 1 -- 172.21.15.47:0/3560566910 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85980cce40 con 0x7f85980d3d10 2024-04-26T23:18:09.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.112+0000 7f858dffb640 1 -- 172.21.15.47:0/3560566910 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8588007260 con 0x7f85980d3d10 2024-04-26T23:18:09.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.113+0000 7f858dffb640 1 -- 172.21.15.47:0/3560566910 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8588007400 con 0x7f85980d3d10 2024-04-26T23:18:09.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.113+0000 7f859f637640 1 -- 172.21.15.47:0/3560566910 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85980cd400 con 0x7f85980d3d10 2024-04-26T23:18:09.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.113+0000 7f858dffb640 1 -- 172.21.15.47:0/3560566910 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f8588003650 con 0x7f85980d3d10 2024-04-26T23:18:09.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.114+0000 7f859f637640 1 -- 172.21.15.47:0/3560566910 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85980d5f40 con 0x7f85980d3d10 2024-04-26T23:18:09.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.116+0000 7f858dffb640 1 -- 172.21.15.47:0/3560566910 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f85980d5f40 con 0x7f85980d3d10 2024-04-26T23:18:09.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.178+0000 7f859f637640 1 -- 172.21.15.47:0/3560566910 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f859815ae40 con 0x7f85980d3d10 2024-04-26T23:18:09.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.178+0000 7f858dffb640 1 -- 172.21.15.47:0/3560566910 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f8588003b40 con 0x7f85980d3d10 2024-04-26T23:18:09.180 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:18:09.180 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":10,"fsid":"399006d3-4db1-43e6-ba81-a79a276eabfd","created":"2024-04-26T23:18:06.731211+0000","modified":"2024-04-26T23:18:08.945859+0000","last_up_change":"0.000000","last_in_change":"2024-04-26T23:18:08.945859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ee091567-fea0-4a14-bf4e-0e16c20bb2f4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c14440a0-bba5-4931-a6db-b736c08e869b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"3940f8de-c9b2-4cd3-a911-9c301729eb30","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"1286a67e-8958-426a-8c2e-bc85498c619c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d8ee2c20-e5b7-46c2-9999-b234070126cf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b453a8f0-f324-4840-9ae0-d67bdadb9f8b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"a6d31951-2d4b-4e8d-a510-6283b01dc158","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"7119b8c0-fe96-402d-9e75-7b509d9d44d9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:18:09.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.179+0000 7f859f637640 1 -- 172.21.15.47:0/3560566910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85980d3d10 msgr2=0x7f85980cead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:09.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.179+0000 7f859f637640 1 --2- 172.21.15.47:0/3560566910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85980d3d10 0x7f85980cead0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f858800c430 tx=0x7f858800c8f0 comp rx=0 tx=0).stop 2024-04-26T23:18:09.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.179+0000 7f859f637640 1 -- 172.21.15.47:0/3560566910 shutdown_connections 2024-04-26T23:18:09.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.179+0000 7f859f637640 1 --2- 172.21.15.47:0/3560566910 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f859815f520 0x7f85980b9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:09.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.179+0000 7f859f637640 1 --2- 172.21.15.47:0/3560566910 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f859815f140 0x7f85980cc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:09.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.179+0000 7f859f637640 1 --2- 172.21.15.47:0/3560566910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85980d3d10 0x7f85980cead0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:09.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.179+0000 7f859f637640 1 -- 172.21.15.47:0/3560566910 >> 172.21.15.47:0/3560566910 conn(0x7f859809e320 msgr2=0x7f8598161940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:09.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.180+0000 7f859f637640 1 -- 172.21.15.47:0/3560566910 shutdown_connections 2024-04-26T23:18:09.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:09.180+0000 7f859f637640 1 -- 172.21.15.47:0/3560566910 wait complete. 2024-04-26T23:18:09.192 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-26T23:18:09.192 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-26T23:18:09.266 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-26T23:18:09.264+0000 7fe93fb2c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T23:18:09.315 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-26T23:18:09.314+0000 7fe93fb2c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T23:18:09.359 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-26T23:18:09.359+0000 7f68c4e8c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T23:18:09.370 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-26T23:18:09.368+0000 7fe93fb2c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T23:18:09.411 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-26T23:18:09.410+0000 7f68c4e8c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T23:18:09.487 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-26T23:18:09.486+0000 7fe93fb2c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T23:18:09.527 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-26T23:18:09.527+0000 7f68c4e8c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T23:18:09.574 INFO:tasks.ceph.osd.2.smithi047.stderr:2024-04-26T23:18:09.572+0000 7efc5d85d640 -1 Falling back to public interface 2024-04-26T23:18:09.576 INFO:tasks.ceph.osd.5.smithi203.stderr:2024-04-26T23:18:09.575+0000 7f5980686640 -1 Falling back to public interface 2024-04-26T23:18:09.583 INFO:tasks.ceph.osd.4.smithi203.stderr:2024-04-26T23:18:09.582+0000 7faff7819640 -1 Falling back to public interface 2024-04-26T23:18:09.592 INFO:tasks.ceph.osd.7.smithi203.stderr:2024-04-26T23:18:09.591+0000 7f71dc9c4640 -1 Falling back to public interface 2024-04-26T23:18:09.601 INFO:tasks.ceph.osd.1.smithi047.stderr:2024-04-26T23:18:09.600+0000 7fe217bd6640 -1 Falling back to public interface 2024-04-26T23:18:09.606 INFO:tasks.ceph.osd.0.smithi047.stderr:2024-04-26T23:18:09.604+0000 7f8192b33640 -1 Falling back to public interface 2024-04-26T23:18:09.606 INFO:tasks.ceph.osd.3.smithi047.stderr:2024-04-26T23:18:09.604+0000 7ff7ec6e3640 -1 Falling back to public interface 2024-04-26T23:18:09.607 INFO:tasks.ceph.osd.0.smithi047.stderr:2024-04-26T23:18:09.605+0000 7f8191ae2640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6816/0: (98) Address already in use 2024-04-26T23:18:09.608 INFO:tasks.ceph.osd.6.smithi203.stderr:2024-04-26T23:18:09.608+0000 7fda5e9d8640 -1 Falling back to public interface 2024-04-26T23:18:09.814 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-26T23:18:09.813+0000 7f68c4e8c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T23:18:09.862 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-26T23:18:09.861+0000 7f68c4e8c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T23:18:09.886 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-26T23:18:09.884+0000 7fe93fb2c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T23:18:09.998 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-26T23:18:10.018 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-26T23:18:10.018+0000 7f68c4e8c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T23:18:10.091 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.089+0000 7f7775cd9640 1 Processor -- start 2024-04-26T23:18:10.091 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.090+0000 7f7775cd9640 1 -- start start 2024-04-26T23:18:10.094 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.090+0000 7f7775cd9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77700b0510 0x7f7770073720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:10.094 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.090+0000 7f7775cd9640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7770073d00 con 0x7f777015f040 2024-04-26T23:18:10.094 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.090+0000 7f7775cd9640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77700aea20 con 0x7f777006ca30 2024-04-26T23:18:10.094 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.090+0000 7f7775cd9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f7770073fe0 con 0x7f77700b0510 2024-04-26T23:18:10.094 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.090+0000 7f776f7fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f777015f040 0x7f777006c320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35890/0 (socket says 172.21.15.47:35890) 2024-04-26T23:18:10.094 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.090+0000 7f776f7fe640 1 -- 172.21.15.47:0/3207605015 learned_addr learned my addr 172.21.15.47:0/3207605015 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:18:10.095 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.091+0000 7f776ffff640 1 --2- 172.21.15.47:0/3207605015 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77700b0510 0x7f7770073720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:10.095 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.091+0000 7f776e7fc640 1 -- 172.21.15.47:0/3207605015 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3200433866 0 0) 0x7f7770073d00 con 0x7f777015f040 2024-04-26T23:18:10.095 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.091+0000 7f776e7fc640 1 -- 172.21.15.47:0/3207605015 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7758003520 con 0x7f777015f040 2024-04-26T23:18:10.095 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.091+0000 7f776ffff640 1 -- 172.21.15.47:0/3207605015 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77700b0510 msgr2=0x7f7770073720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-26T23:18:10.095 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.091+0000 7f776ffff640 1 -- 172.21.15.47:0/3207605015 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77700b0510 msgr2=0x7f7770073720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T23:18:10.095 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.091+0000 7f776ffff640 1 --2- 172.21.15.47:0/3207605015 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77700b0510 0x7f7770073720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T23:18:10.095 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.091+0000 7f776ffff640 1 --2- 172.21.15.47:0/3207605015 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77700b0510 0x7f7770073720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T23:18:10.095 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.091+0000 7f776e7fc640 1 -- 172.21.15.47:0/3207605015 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2709705440 0 0) 0x7f77700aea20 con 0x7f777006ca30 2024-04-26T23:18:10.095 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.091+0000 7f776e7fc640 1 -- 172.21.15.47:0/3207605015 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7770073d00 con 0x7f777006ca30 2024-04-26T23:18:10.095 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.091+0000 7f776e7fc640 1 -- 172.21.15.47:0/3207605015 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1872707622 0 0) 0x7f7758003520 con 0x7f777015f040 2024-04-26T23:18:10.095 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.091+0000 7f776e7fc640 1 -- 172.21.15.47:0/3207605015 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77700b0510 msgr2=0x7f7770073720 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T23:18:10.095 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.091+0000 7f776e7fc640 1 --2- 172.21.15.47:0/3207605015 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77700b0510 0x7f7770073720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:10.095 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.091+0000 7f776e7fc640 1 -- 172.21.15.47:0/3207605015 >> v1:172.21.15.203:6789/0 conn(0x7f777006ca30 legacy=0x7f7770073010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:10.095 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.091+0000 7f776e7fc640 1 -- 172.21.15.47:0/3207605015 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77700aed00 con 0x7f777015f040 2024-04-26T23:18:10.096 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.091+0000 7f776e7fc640 1 -- 172.21.15.47:0/3207605015 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7f7760001670 con 0x7f777015f040 2024-04-26T23:18:10.096 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.091+0000 7f776e7fc640 1 -- 172.21.15.47:0/3207605015 >> v1:172.21.15.47:6789/0 conn(0x7f777015f040 legacy=0x7f777006c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:10.096 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.091+0000 7f776e7fc640 1 --2- 172.21.15.47:0/3207605015 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77700b0510 0x7f7758003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:10.096 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.091+0000 7f776e7fc640 1 --2- 172.21.15.47:0/3207605015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77580040d0 0x7f7758004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:10.096 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.091+0000 7f776e7fc640 1 --2- 172.21.15.47:0/3207605015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7758004d40 0x7f77580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:10.096 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.091+0000 7f776e7fc640 1 -- 172.21.15.47:0/3207605015 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7770073fe0 con 0x7f7758004d40 2024-04-26T23:18:10.096 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.091+0000 7f776e7fc640 1 -- 172.21.15.47:0/3207605015 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f775800bbd0 con 0x7f77580040d0 2024-04-26T23:18:10.096 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.091+0000 7f776e7fc640 1 -- 172.21.15.47:0/3207605015 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f775800bdd0 con 0x7f77700b0510 2024-04-26T23:18:10.096 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.091+0000 7f776ffff640 1 --2- 172.21.15.47:0/3207605015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7758004d40 0x7f77580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:10.096 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.091+0000 7f776f7fe640 1 --2- 172.21.15.47:0/3207605015 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77700b0510 0x7f7758003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:10.096 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.091+0000 7f776effd640 1 --2- 172.21.15.47:0/3207605015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77580040d0 0x7f7758004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:10.096 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.092+0000 7f776ffff640 1 -- 172.21.15.47:0/3207605015 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77700b0510 msgr2=0x7f7758003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:10.096 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.092+0000 7f776ffff640 1 --2- 172.21.15.47:0/3207605015 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77700b0510 0x7f7758003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:10.096 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.092+0000 7f776ffff640 1 -- 172.21.15.47:0/3207605015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77580040d0 msgr2=0x7f7758004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:10.096 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.092+0000 7f776ffff640 1 --2- 172.21.15.47:0/3207605015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77580040d0 0x7f7758004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:10.097 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.092+0000 7f776ffff640 1 -- 172.21.15.47:0/3207605015 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7758003520 con 0x7f7758004d40 2024-04-26T23:18:10.097 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.093+0000 7f776ffff640 1 --2- 172.21.15.47:0/3207605015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7758004d40 0x7f77580051c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7768009fd0 tx=0x7f7758005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afb357d20fca9a1e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:10.097 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.093+0000 7f776e7fc640 1 -- 172.21.15.47:0/3207605015 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7768033070 con 0x7f7758004d40 2024-04-26T23:18:10.097 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.093+0000 7f776e7fc640 1 -- 172.21.15.47:0/3207605015 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f776802f650 con 0x7f7758004d40 2024-04-26T23:18:10.097 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.093+0000 7f776e7fc640 1 -- 172.21.15.47:0/3207605015 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7768009bd0 con 0x7f7758004d40 2024-04-26T23:18:10.100 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.093+0000 7f7775cd9640 1 -- 172.21.15.47:0/3207605015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7758004d40 msgr2=0x7f77580051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:10.100 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.093+0000 7f7775cd9640 1 --2- 172.21.15.47:0/3207605015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7758004d40 0x7f77580051c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7768009fd0 tx=0x7f7758005830 comp rx=0 tx=0).stop 2024-04-26T23:18:10.100 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.093+0000 7f776ffff640 1 -- 172.21.15.47:0/3207605015 reap_dead start 2024-04-26T23:18:10.100 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.093+0000 7f7775cd9640 1 -- 172.21.15.47:0/3207605015 shutdown_connections 2024-04-26T23:18:10.100 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.093+0000 7f7775cd9640 1 -- 172.21.15.47:0/3207605015 >> 172.21.15.47:0/3207605015 conn(0x7f77700d1d00 msgr2=0x7f77700d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:10.100 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.093+0000 7f7775cd9640 1 -- 172.21.15.47:0/3207605015 shutdown_connections 2024-04-26T23:18:10.100 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.093+0000 7f7775cd9640 1 -- 172.21.15.47:0/3207605015 wait complete. 2024-04-26T23:18:10.101 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.095+0000 7f7775cd9640 1 Processor -- start 2024-04-26T23:18:10.101 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.095+0000 7f7775cd9640 1 -- start start 2024-04-26T23:18:10.101 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.095+0000 7f7775cd9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7770154550 0x7f77700b6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:10.101 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.095+0000 7f7775cd9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7770160b00 0x7f77700b1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:10.101 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.095+0000 7f7775cd9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77700b2020 0x7f777015f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:10.101 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.095+0000 7f7775cd9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7770073720 con 0x7f7770160b00 2024-04-26T23:18:10.101 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.095+0000 7f7775cd9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f77700af070 con 0x7f77700b2020 2024-04-26T23:18:10.101 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.095+0000 7f7775cd9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f77700738a0 con 0x7f7770154550 2024-04-26T23:18:10.101 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.096+0000 7f776effd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7770160b00 0x7f77700b1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:10.101 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.096+0000 7f776effd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7770160b00 0x7f77700b1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48056/0 (socket says 172.21.15.47:48056) 2024-04-26T23:18:10.101 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.096+0000 7f776effd640 1 -- 172.21.15.47:0/1597285957 learned_addr learned my addr 172.21.15.47:0/1597285957 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:10.102 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.096+0000 7f776ffff640 1 --2- 172.21.15.47:0/1597285957 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77700b2020 0x7f777015f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:10.102 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.096+0000 7f776f7fe640 1 --2- 172.21.15.47:0/1597285957 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7770154550 0x7f77700b6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:10.102 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.096+0000 7f776f7fe640 1 -- 172.21.15.47:0/1597285957 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7770154550 msgr2=0x7f77700b6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-04-26T23:18:10.102 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.096+0000 7f776f7fe640 1 -- 172.21.15.47:0/1597285957 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7770154550 msgr2=0x7f77700b6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T23:18:10.102 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.096+0000 7f776f7fe640 1 --2- 172.21.15.47:0/1597285957 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7770154550 0x7f77700b6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T23:18:10.102 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.096+0000 7f776f7fe640 1 --2- 172.21.15.47:0/1597285957 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7770154550 0x7f77700b6360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T23:18:10.102 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.096+0000 7f776effd640 1 -- 172.21.15.47:0/1597285957 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7770154550 msgr2=0x7f77700b6360 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T23:18:10.102 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.096+0000 7f776effd640 1 --2- 172.21.15.47:0/1597285957 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7770154550 0x7f77700b6360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:10.102 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.096+0000 7f776effd640 1 -- 172.21.15.47:0/1597285957 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77700b2020 msgr2=0x7f777015f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:10.102 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.096+0000 7f776effd640 1 --2- 172.21.15.47:0/1597285957 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77700b2020 0x7f777015f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:10.102 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.096+0000 7f776effd640 1 -- 172.21.15.47:0/1597285957 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7770161270 con 0x7f7770160b00 2024-04-26T23:18:10.102 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.096+0000 7f776effd640 1 --2- 172.21.15.47:0/1597285957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7770160b00 0x7f77700b1ae0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f776400f7d0 tx=0x7f776400fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:10.102 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.096+0000 7f7757fff640 1 -- 172.21.15.47:0/1597285957 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77640052e0 con 0x7f7770160b00 2024-04-26T23:18:10.102 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.096+0000 7f7757fff640 1 -- 172.21.15.47:0/1597285957 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7764018070 con 0x7f7770160b00 2024-04-26T23:18:10.103 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.096+0000 7f7775cd9640 1 -- 172.21.15.47:0/1597285957 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7770161550 con 0x7f7770160b00 2024-04-26T23:18:10.103 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.096+0000 7f7757fff640 1 -- 172.21.15.47:0/1597285957 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776400d6b0 con 0x7f7770160b00 2024-04-26T23:18:10.103 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.096+0000 7f7775cd9640 1 -- 172.21.15.47:0/1597285957 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7770161b10 con 0x7f7770160b00 2024-04-26T23:18:10.103 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.097+0000 7f7757fff640 1 -- 172.21.15.47:0/1597285957 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f776400d870 con 0x7f7770160b00 2024-04-26T23:18:10.103 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.097+0000 7f7757fff640 1 -- 172.21.15.47:0/1597285957 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f776401db30 con 0x7f7770160b00 2024-04-26T23:18:10.103 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.098+0000 7f7775cd9640 1 -- 172.21.15.47:0/1597285957 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77700b2430 con 0x7f7770160b00 2024-04-26T23:18:10.103 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.100+0000 7f7757fff640 1 -- 172.21.15.47:0/1597285957 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f77700b2430 con 0x7f7770160b00 2024-04-26T23:18:10.154 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-26T23:18:10.154+0000 7f68c4e8c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T23:18:10.160 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-26T23:18:10.159+0000 7fe93fb2c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T23:18:10.164 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.163+0000 7f7775cd9640 1 -- 172.21.15.47:0/1597285957 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7770076610 con 0x7f7770160b00 2024-04-26T23:18:10.164 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.163+0000 7f7757fff640 1 -- 172.21.15.47:0/1597285957 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f7764003e10 con 0x7f7770160b00 2024-04-26T23:18:10.165 INFO:teuthology.misc.health.smithi047.stdout: 2024-04-26T23:18:10.165 INFO:teuthology.misc.health.smithi047.stdout:{"epoch":10,"fsid":"399006d3-4db1-43e6-ba81-a79a276eabfd","created":"2024-04-26T23:18:06.731211+0000","modified":"2024-04-26T23:18:08.945859+0000","last_up_change":"0.000000","last_in_change":"2024-04-26T23:18:08.945859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ee091567-fea0-4a14-bf4e-0e16c20bb2f4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c14440a0-bba5-4931-a6db-b736c08e869b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"3940f8de-c9b2-4cd3-a911-9c301729eb30","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"1286a67e-8958-426a-8c2e-bc85498c619c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d8ee2c20-e5b7-46c2-9999-b234070126cf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b453a8f0-f324-4840-9ae0-d67bdadb9f8b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"a6d31951-2d4b-4e8d-a510-6283b01dc158","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"7119b8c0-fe96-402d-9e75-7b509d9d44d9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:18:10.166 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.164+0000 7f7775cd9640 1 -- 172.21.15.47:0/1597285957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7770160b00 msgr2=0x7f77700b1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:10.166 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.164+0000 7f7775cd9640 1 --2- 172.21.15.47:0/1597285957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7770160b00 0x7f77700b1ae0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f776400f7d0 tx=0x7f776400fc90 comp rx=0 tx=0).stop 2024-04-26T23:18:10.166 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.164+0000 7f7775cd9640 1 -- 172.21.15.47:0/1597285957 shutdown_connections 2024-04-26T23:18:10.166 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.164+0000 7f7775cd9640 1 --2- 172.21.15.47:0/1597285957 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77700b2020 0x7f777015f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:10.166 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.164+0000 7f7775cd9640 1 --2- 172.21.15.47:0/1597285957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7770160b00 0x7f77700b1ae0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:10.166 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.164+0000 7f7775cd9640 1 --2- 172.21.15.47:0/1597285957 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7770154550 0x7f77700b6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:10.166 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.164+0000 7f7775cd9640 1 -- 172.21.15.47:0/1597285957 >> 172.21.15.47:0/1597285957 conn(0x7f77700d1d00 msgr2=0x7f777006f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:10.166 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.164+0000 7f7775cd9640 1 -- 172.21.15.47:0/1597285957 shutdown_connections 2024-04-26T23:18:10.166 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:10.164+0000 7f7775cd9640 1 -- 172.21.15.47:0/1597285957 wait complete. 2024-04-26T23:18:10.175 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-26T23:18:10.204 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-26T23:18:10.203+0000 7f68c4e8c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T23:18:10.215 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-26T23:18:10.214+0000 7fe93fb2c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T23:18:10.272 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-26T23:18:10.271+0000 7f68c4e8c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T23:18:10.316 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-26T23:18:10.316+0000 7f68c4e8c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T23:18:10.337 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-26T23:18:10.335+0000 7fe93fb2c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T23:18:10.629 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-26T23:18:10.628+0000 7fe93fb2c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T23:18:10.679 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-26T23:18:10.677+0000 7fe93fb2c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T23:18:10.796 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-26T23:18:10.795+0000 7f68c4e8c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T23:18:10.842 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-26T23:18:10.841+0000 7fe93fb2c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T23:18:10.910 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-26T23:18:10.909+0000 7f68c4e8c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T23:18:10.987 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-26T23:18:10.985+0000 7fe93fb2c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T23:18:11.039 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-26T23:18:11.037+0000 7fe93fb2c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T23:18:11.112 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-26T23:18:11.110+0000 7fe93fb2c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T23:18:11.159 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-26T23:18:11.158+0000 7fe93fb2c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T23:18:11.660 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-26T23:18:11.659+0000 7fe93fb2c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T23:18:11.781 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-26T23:18:11.780+0000 7fe93fb2c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T23:18:11.924 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-26T23:18:11.924+0000 7f6865a8e640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-26T23:18:11.925 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-26T23:18:11.924+0000 7f6865a8e640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-26T23:18:13.056 INFO:tasks.ceph.osd.2.smithi047.stderr:2024-04-26T23:18:13.054+0000 7efc5d85d640 -1 osd.2 0 log_to_monitors true 2024-04-26T23:18:13.071 INFO:tasks.ceph.osd.7.smithi203.stderr:2024-04-26T23:18:13.070+0000 7f71dc9c4640 -1 osd.7 0 log_to_monitors true 2024-04-26T23:18:13.075 INFO:tasks.ceph.osd.5.smithi203.stderr:2024-04-26T23:18:13.075+0000 7f5980686640 -1 osd.5 0 log_to_monitors true 2024-04-26T23:18:13.091 INFO:tasks.ceph.osd.4.smithi203.stderr:2024-04-26T23:18:13.090+0000 7faff7819640 -1 osd.4 0 log_to_monitors true 2024-04-26T23:18:13.119 INFO:tasks.ceph.osd.6.smithi203.stderr:2024-04-26T23:18:13.119+0000 7fda5e9d8640 -1 osd.6 0 log_to_monitors true 2024-04-26T23:18:13.122 INFO:tasks.ceph.osd.3.smithi047.stderr:2024-04-26T23:18:13.121+0000 7ff7ec6e3640 -1 osd.3 0 log_to_monitors true 2024-04-26T23:18:13.137 INFO:tasks.ceph.osd.0.smithi047.stderr:2024-04-26T23:18:13.135+0000 7f8192b33640 -1 osd.0 0 log_to_monitors true 2024-04-26T23:18:13.137 INFO:tasks.ceph.osd.1.smithi047.stderr:2024-04-26T23:18:13.135+0000 7fe217bd6640 -1 osd.1 0 log_to_monitors true 2024-04-26T23:18:16.130 INFO:tasks.ceph.osd.2.smithi047.stderr:2024-04-26T23:18:16.123+0000 7efc597f3640 -1 osd.2 0 waiting for initial osdmap 2024-04-26T23:18:16.137 INFO:tasks.ceph.osd.2.smithi047.stderr:2024-04-26T23:18:16.135+0000 7efc51613640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T23:18:16.182 INFO:tasks.ceph.osd.3.smithi047.stderr:2024-04-26T23:18:16.180+0000 7ff7e8679640 -1 osd.3 0 waiting for initial osdmap 2024-04-26T23:18:16.193 INFO:tasks.ceph.osd.3.smithi047.stderr:2024-04-26T23:18:16.191+0000 7ff7e0499640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T23:18:16.195 INFO:tasks.ceph.osd.0.smithi047.stderr:2024-04-26T23:18:16.193+0000 7f818f2dd640 -1 osd.0 0 waiting for initial osdmap 2024-04-26T23:18:16.209 INFO:tasks.ceph.osd.0.smithi047.stderr:2024-04-26T23:18:16.207+0000 7f81868e9640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T23:18:16.227 INFO:tasks.ceph.osd.1.smithi047.stderr:2024-04-26T23:18:16.225+0000 7fe214380640 -1 osd.1 0 waiting for initial osdmap 2024-04-26T23:18:16.236 INFO:tasks.ceph.osd.1.smithi047.stderr:2024-04-26T23:18:16.235+0000 7fe20b98c640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T23:18:16.787 INFO:tasks.ceph.osd.7.smithi203.stderr:2024-04-26T23:18:16.786+0000 7f71d916e640 -1 osd.7 0 waiting for initial osdmap 2024-04-26T23:18:16.803 INFO:tasks.ceph.osd.7.smithi203.stderr:2024-04-26T23:18:16.802+0000 7f71d077a640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T23:18:16.872 INFO:tasks.ceph.osd.5.smithi203.stderr:2024-04-26T23:18:16.871+0000 7f597ce30640 -1 osd.5 0 waiting for initial osdmap 2024-04-26T23:18:16.886 INFO:tasks.ceph.osd.5.smithi203.stderr:2024-04-26T23:18:16.885+0000 7f597443c640 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T23:18:16.921 INFO:tasks.ceph.osd.4.smithi203.stderr:2024-04-26T23:18:16.920+0000 7faff3207640 -1 osd.4 0 waiting for initial osdmap 2024-04-26T23:18:16.924 INFO:tasks.ceph.osd.6.smithi203.stderr:2024-04-26T23:18:16.923+0000 7fda5a96e640 -1 osd.6 0 waiting for initial osdmap 2024-04-26T23:18:16.934 INFO:tasks.ceph.osd.4.smithi203.stderr:2024-04-26T23:18:16.933+0000 7fafeb027640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T23:18:16.947 INFO:tasks.ceph.osd.6.smithi203.stderr:2024-04-26T23:18:16.946+0000 7fda5278e640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T23:18:16.982 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-26T23:18:17.089 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.088+0000 7f7ef532b640 1 Processor -- start 2024-04-26T23:18:17.090 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.088+0000 7f7ef532b640 1 -- start start 2024-04-26T23:18:17.091 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.089+0000 7f7ef532b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7ef00b64e0 0x7f7ef00d1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:17.091 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.089+0000 7f7ef532b640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7ef00b40b0 0x7f7ef015fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:17.091 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.089+0000 7f7ef532b640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ef00d25f0 con 0x7f7ef015f830 2024-04-26T23:18:17.091 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.089+0000 7f7ef532b640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f7ef00d1080 con 0x7f7ef00b64e0 2024-04-26T23:18:17.091 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.089+0000 7f7ef532b640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f7ef00d3aa0 con 0x7f7ef00b40b0 2024-04-26T23:18:17.092 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.089+0000 7f7ee7fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7ef00b64e0 0x7f7ef00d1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:17.092 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.089+0000 7f7eeeffd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7ef015f830 0x7f7ef00d1790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35892/0 (socket says 172.21.15.47:35892) 2024-04-26T23:18:17.092 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.089+0000 7f7eeeffd640 1 -- 172.21.15.47:0/707357168 learned_addr learned my addr 172.21.15.47:0/707357168 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:18:17.092 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.089+0000 7f7eef7fe640 1 --2- 172.21.15.47:0/707357168 >> v2:172.21.15.203:3300/0 conn(0x7f7ef00b40b0 0x7f7ef015fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:17.092 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.089+0000 7f7ee7fff640 1 -- 172.21.15.47:0/707357168 >> v2:172.21.15.203:3300/0 conn(0x7f7ef00b40b0 msgr2=0x7f7ef015fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:17.092 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.089+0000 7f7ee7fff640 1 --2- 172.21.15.47:0/707357168 >> v2:172.21.15.203:3300/0 conn(0x7f7ef00b40b0 0x7f7ef015fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:17.092 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.089+0000 7f7ee7fff640 1 -- 172.21.15.47:0/707357168 >> v1:172.21.15.47:6789/0 conn(0x7f7ef015f830 legacy=0x7f7ef00d1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:17.092 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.089+0000 7f7ee7fff640 1 -- 172.21.15.47:0/707357168 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef00d3d80 con 0x7f7ef00b64e0 2024-04-26T23:18:17.094 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.090+0000 7f7ee7fff640 1 --2- 172.21.15.47:0/707357168 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ef00b64e0 0x7f7ef00d1ea0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7ef00b6470 tx=0x7f7ed802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac59898c23472472 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:17.094 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.090+0000 7f7eee7fc640 1 -- 172.21.15.47:0/707357168 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed8037070 con 0x7f7ef00b64e0 2024-04-26T23:18:17.094 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.090+0000 7f7eee7fc640 1 -- 172.21.15.47:0/707357168 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ed8032440 con 0x7f7ef00b64e0 2024-04-26T23:18:17.094 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.090+0000 7f7eee7fc640 1 -- 172.21.15.47:0/707357168 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed802ae60 con 0x7f7ef00b64e0 2024-04-26T23:18:17.094 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.091+0000 7f7ef532b640 1 -- 172.21.15.47:0/707357168 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ef00b64e0 msgr2=0x7f7ef00d1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:17.094 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.091+0000 7f7ef532b640 1 --2- 172.21.15.47:0/707357168 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ef00b64e0 0x7f7ef00d1ea0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7ef00b6470 tx=0x7f7ed802a110 comp rx=0 tx=0).stop 2024-04-26T23:18:17.094 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.091+0000 7f7ef532b640 1 -- 172.21.15.47:0/707357168 shutdown_connections 2024-04-26T23:18:17.094 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.091+0000 7f7ef532b640 1 --2- 172.21.15.47:0/707357168 >> v2:172.21.15.203:3300/0 conn(0x7f7ef00b40b0 0x7f7ef015fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:17.094 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.091+0000 7f7ef532b640 1 --2- 172.21.15.47:0/707357168 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ef00b64e0 0x7f7ef00d1ea0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:17.094 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.091+0000 7f7ef532b640 1 -- 172.21.15.47:0/707357168 >> 172.21.15.47:0/707357168 conn(0x7f7ef00d4fd0 msgr2=0x7f7ef00d53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:17.094 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.091+0000 7f7ef532b640 1 -- 172.21.15.47:0/707357168 shutdown_connections 2024-04-26T23:18:17.094 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.091+0000 7f7ef532b640 1 -- 172.21.15.47:0/707357168 wait complete. 2024-04-26T23:18:17.095 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.092+0000 7f7ef532b640 1 Processor -- start 2024-04-26T23:18:17.095 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.092+0000 7f7ef532b640 1 -- start start 2024-04-26T23:18:17.097 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.093+0000 7f7ef532b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ef00b40b0 0x7f7ef015c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:17.097 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.093+0000 7f7ef532b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ef00b64e0 0x7f7ef00b8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:17.097 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.093+0000 7f7ef532b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ef015f830 0x7f7ef00b9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:17.097 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.093+0000 7f7ef532b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef00b47a0 con 0x7f7ef00b40b0 2024-04-26T23:18:17.097 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.093+0000 7f7ef532b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef00b4620 con 0x7f7ef015f830 2024-04-26T23:18:17.097 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.093+0000 7f7ef532b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f7ef0160150 con 0x7f7ef00b64e0 2024-04-26T23:18:17.097 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.093+0000 7f7eeeffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ef00b40b0 0x7f7ef015c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:17.098 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.093+0000 7f7eeeffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ef00b40b0 0x7f7ef015c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48098/0 (socket says 172.21.15.47:48098) 2024-04-26T23:18:17.098 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.093+0000 7f7eeeffd640 1 -- 172.21.15.47:0/692339296 learned_addr learned my addr 172.21.15.47:0/692339296 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:17.098 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.093+0000 7f7ee7fff640 1 --2- 172.21.15.47:0/692339296 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ef00b64e0 0x7f7ef00b8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:17.098 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.093+0000 7f7eef7fe640 1 --2- 172.21.15.47:0/692339296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ef015f830 0x7f7ef00b9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:17.098 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.093+0000 7f7eeeffd640 1 -- 172.21.15.47:0/692339296 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ef00b64e0 msgr2=0x7f7ef00b8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:17.098 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.093+0000 7f7eeeffd640 1 --2- 172.21.15.47:0/692339296 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ef00b64e0 0x7f7ef00b8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:17.098 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.093+0000 7f7eeeffd640 1 -- 172.21.15.47:0/692339296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ef015f830 msgr2=0x7f7ef00b9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:17.099 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.093+0000 7f7eeeffd640 1 --2- 172.21.15.47:0/692339296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ef015f830 0x7f7ef00b9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:17.099 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.093+0000 7f7eeeffd640 1 -- 172.21.15.47:0/692339296 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef00b9940 con 0x7f7ef00b40b0 2024-04-26T23:18:17.099 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.094+0000 7f7eeeffd640 1 --2- 172.21.15.47:0/692339296 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ef00b40b0 0x7f7ef015c150 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7ee000c3c0 tx=0x7f7ee000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:17.099 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.094+0000 7f7ee77fe640 1 -- 172.21.15.47:0/692339296 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee0017070 con 0x7f7ef00b40b0 2024-04-26T23:18:17.099 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.094+0000 7f7ee77fe640 1 -- 172.21.15.47:0/692339296 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ee0005ce0 con 0x7f7ef00b40b0 2024-04-26T23:18:17.099 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.094+0000 7f7ef532b640 1 -- 172.21.15.47:0/692339296 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ef00b9c20 con 0x7f7ef00b40b0 2024-04-26T23:18:17.099 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.094+0000 7f7ee77fe640 1 -- 172.21.15.47:0/692339296 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee0007310 con 0x7f7ef00b40b0 2024-04-26T23:18:17.099 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.094+0000 7f7ef532b640 1 -- 172.21.15.47:0/692339296 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ef00ba1e0 con 0x7f7ef00b40b0 2024-04-26T23:18:17.099 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.095+0000 7f7ef532b640 1 -- 172.21.15.47:0/692339296 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ef00ba620 con 0x7f7ef00b40b0 2024-04-26T23:18:17.099 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.096+0000 7f7ee77fe640 1 -- 172.21.15.47:0/692339296 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7f7ee001a070 con 0x7f7ef00b40b0 2024-04-26T23:18:17.100 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.096+0000 7f7ee77fe640 1 --2- 172.21.15.47:0/692339296 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f7ed006c890 0x7f7ed006ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:17.100 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.096+0000 7f7ee77fe640 1 -- 172.21.15.47:0/692339296 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 3351+0+0 (secure 0 0 0) 0x7f7ee008a480 con 0x7f7ef00b40b0 2024-04-26T23:18:17.100 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.096+0000 7f7ee7fff640 1 --2- 172.21.15.47:0/692339296 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f7ed006c890 0x7f7ed006ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:17.100 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.097+0000 7f7ee7fff640 1 --2- 172.21.15.47:0/692339296 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f7ed006c890 0x7f7ed006ed50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8034000 tx=0x7f7ed802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:17.103 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.101+0000 7f7ee77fe640 1 -- 172.21.15.47:0/692339296 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7ee0058b60 con 0x7f7ef00b40b0 2024-04-26T23:18:17.297 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.294+0000 7f7ef532b640 1 -- 172.21.15.47:0/692339296 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7ef0076610 con 0x7f7ef00b40b0 2024-04-26T23:18:17.297 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.295+0000 7f7ee77fe640 1 -- 172.21.15.47:0/692339296 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+8965 (secure 0 0 0) 0x7f7ee005ce60 con 0x7f7ef00b40b0 2024-04-26T23:18:17.297 INFO:teuthology.misc.health.smithi047.stdout: 2024-04-26T23:18:17.298 INFO:teuthology.misc.health.smithi047.stdout:{"epoch":13,"fsid":"399006d3-4db1-43e6-ba81-a79a276eabfd","created":"2024-04-26T23:18:06.731211+0000","modified":"2024-04-26T23:18:16.797588+0000","last_up_change":"2024-04-26T23:18:16.797588+0000","last_in_change":"2024-04-26T23:18:08.945859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ee091567-fea0-4a14-bf4e-0e16c20bb2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6819","nonce":326520779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6821","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6823","nonce":326520779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6829","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6831","nonce":326520779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6825","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6827","nonce":326520779}]},"public_addr":"172.21.15.47:6819/326520779","cluster_addr":"172.21.15.47:6823/326520779","heartbeat_back_addr":"172.21.15.47:6831/326520779","heartbeat_front_addr":"172.21.15.47:6827/326520779","state":["exists","up"]},{"osd":1,"uuid":"c14440a0-bba5-4931-a6db-b736c08e869b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6809","nonce":1912610931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6811","nonce":1912610931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6815","nonce":1912610931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6813","nonce":1912610931}]},"public_addr":"172.21.15.47:6809/1912610931","cluster_addr":"172.21.15.47:6811/1912610931","heartbeat_back_addr":"172.21.15.47:6815/1912610931","heartbeat_front_addr":"172.21.15.47:6813/1912610931","state":["exists","up"]},{"osd":2,"uuid":"3940f8de-c9b2-4cd3-a911-9c301729eb30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6801","nonce":2569869305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6803","nonce":2569869305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6807","nonce":2569869305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6805","nonce":2569869305}]},"public_addr":"172.21.15.47:6801/2569869305","cluster_addr":"172.21.15.47:6803/2569869305","heartbeat_back_addr":"172.21.15.47:6807/2569869305","heartbeat_front_addr":"172.21.15.47:6805/2569869305","state":["exists","up"]},{"osd":3,"uuid":"1286a67e-8958-426a-8c2e-bc85498c619c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6818","nonce":2791411683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6822","nonce":2791411683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6830","nonce":2791411683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6826","nonce":2791411683}]},"public_addr":"172.21.15.47:6818/2791411683","cluster_addr":"172.21.15.47:6822/2791411683","heartbeat_back_addr":"172.21.15.47:6830/2791411683","heartbeat_front_addr":"172.21.15.47:6826/2791411683","state":["exists","up"]},{"osd":4,"uuid":"d8ee2c20-e5b7-46c2-9999-b234070126cf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b453a8f0-f324-4840-9ae0-d67bdadb9f8b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"a6d31951-2d4b-4e8d-a510-6283b01dc158","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"7119b8c0-fe96-402d-9e75-7b509d9d44d9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:18:17.299 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.297+0000 7f7ef532b640 1 -- 172.21.15.47:0/692339296 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f7ed006c890 msgr2=0x7f7ed006ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:17.300 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.297+0000 7f7ef532b640 1 --2- 172.21.15.47:0/692339296 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f7ed006c890 0x7f7ed006ed50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8034000 tx=0x7f7ed802e000 comp rx=0 tx=0).stop 2024-04-26T23:18:17.300 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.297+0000 7f7ef532b640 1 -- 172.21.15.47:0/692339296 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ef00b40b0 msgr2=0x7f7ef015c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:17.300 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.297+0000 7f7ef532b640 1 --2- 172.21.15.47:0/692339296 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ef00b40b0 0x7f7ef015c150 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7ee000c3c0 tx=0x7f7ee000c880 comp rx=0 tx=0).stop 2024-04-26T23:18:17.300 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.297+0000 7f7ef532b640 1 -- 172.21.15.47:0/692339296 shutdown_connections 2024-04-26T23:18:17.300 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.297+0000 7f7ef532b640 1 --2- 172.21.15.47:0/692339296 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f7ed006c890 0x7f7ed006ed50 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:17.300 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.297+0000 7f7ef532b640 1 --2- 172.21.15.47:0/692339296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ef015f830 0x7f7ef00b9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:17.300 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.297+0000 7f7ef532b640 1 --2- 172.21.15.47:0/692339296 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ef00b64e0 0x7f7ef00b8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:17.300 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.297+0000 7f7ef532b640 1 --2- 172.21.15.47:0/692339296 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ef00b40b0 0x7f7ef015c150 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:17.300 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.297+0000 7f7ef532b640 1 -- 172.21.15.47:0/692339296 >> 172.21.15.47:0/692339296 conn(0x7f7ef00d4fd0 msgr2=0x7f7ef006ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:17.300 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.297+0000 7f7ef532b640 1 -- 172.21.15.47:0/692339296 shutdown_connections 2024-04-26T23:18:17.300 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:17.297+0000 7f7ef532b640 1 -- 172.21.15.47:0/692339296 wait complete. 2024-04-26T23:18:17.309 DEBUG:teuthology.misc:4 of 8 OSDs are up 2024-04-26T23:18:24.117 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-26T23:18:24.221 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.219+0000 7fdb4e037640 1 Processor -- start 2024-04-26T23:18:24.221 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.219+0000 7fdb4e037640 1 -- start start 2024-04-26T23:18:24.222 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.220+0000 7fdb4e037640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdb4815f3d0 0x7fdb480b24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:24.222 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.220+0000 7fdb4e037640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb480d0ed0 con 0x7fdb480d3450 2024-04-26T23:18:24.222 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.220+0000 7fdb4e037640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb480b2b90 con 0x7fdb480d2c50 2024-04-26T23:18:24.222 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.220+0000 7fdb4e037640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fdb4815f7b0 con 0x7fdb4815f3d0 2024-04-26T23:18:24.222 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.220+0000 7fdb477fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdb4815f3d0 0x7fdb480b24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:24.222 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.220+0000 7fdb47fff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fdb480d3450 0x7fdb480b7d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47180/0 (socket says 172.21.15.47:47180) 2024-04-26T23:18:24.223 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.220+0000 7fdb47fff640 1 -- 172.21.15.47:0/137120586 learned_addr learned my addr 172.21.15.47:0/137120586 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:18:24.223 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.220+0000 7fdb477fe640 1 -- 172.21.15.47:0/137120586 >> v1:172.21.15.203:6789/0 conn(0x7fdb480d2c50 legacy=0x7fdb480b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:24.223 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.220+0000 7fdb477fe640 1 -- 172.21.15.47:0/137120586 >> v1:172.21.15.47:6789/0 conn(0x7fdb480d3450 legacy=0x7fdb480b7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:24.223 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.220+0000 7fdb477fe640 1 -- 172.21.15.47:0/137120586 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb480b2e70 con 0x7fdb4815f3d0 2024-04-26T23:18:24.223 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.220+0000 7fdb467fc640 1 -- 172.21.15.47:0/137120586 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2302566708 0 0) 0x7fdb480d0ed0 con 0x7fdb480d3450 2024-04-26T23:18:24.224 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.221+0000 7fdb477fe640 1 --2- 172.21.15.47:0/137120586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb4815f3d0 0x7fdb480b24d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdb380097a0 tx=0x7fdb3802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f199ef0e1bd70e01 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:24.224 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.221+0000 7fdb467fc640 1 -- 172.21.15.47:0/137120586 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb38037070 con 0x7fdb4815f3d0 2024-04-26T23:18:24.224 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.221+0000 7fdb467fc640 1 -- 172.21.15.47:0/137120586 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb38032440 con 0x7fdb4815f3d0 2024-04-26T23:18:24.224 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.221+0000 7fdb467fc640 1 -- 172.21.15.47:0/137120586 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb38032810 con 0x7fdb4815f3d0 2024-04-26T23:18:24.224 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.222+0000 7fdb4e037640 1 -- 172.21.15.47:0/137120586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb4815f3d0 msgr2=0x7fdb480b24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:24.224 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.222+0000 7fdb4e037640 1 --2- 172.21.15.47:0/137120586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb4815f3d0 0x7fdb480b24d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdb380097a0 tx=0x7fdb3802a230 comp rx=0 tx=0).stop 2024-04-26T23:18:24.224 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.222+0000 7fdb4e037640 1 -- 172.21.15.47:0/137120586 shutdown_connections 2024-04-26T23:18:24.224 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.222+0000 7fdb4e037640 1 --2- 172.21.15.47:0/137120586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb4815f3d0 0x7fdb480b24d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:24.224 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.222+0000 7fdb4e037640 1 -- 172.21.15.47:0/137120586 >> 172.21.15.47:0/137120586 conn(0x7fdb480d4f00 msgr2=0x7fdb480d5310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:24.224 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.222+0000 7fdb4e037640 1 -- 172.21.15.47:0/137120586 shutdown_connections 2024-04-26T23:18:24.224 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.222+0000 7fdb4e037640 1 -- 172.21.15.47:0/137120586 wait complete. 2024-04-26T23:18:24.225 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.223+0000 7fdb4e037640 1 Processor -- start 2024-04-26T23:18:24.225 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.223+0000 7fdb4e037640 1 -- start start 2024-04-26T23:18:24.227 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.224+0000 7fdb4e037640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb480d2c50 0x7fdb480b66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:24.227 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.224+0000 7fdb4e037640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb480d3450 0x7fdb480cc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:24.227 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.224+0000 7fdb4e037640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb4815f3d0 0x7fdb480cea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:24.227 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.224+0000 7fdb4e037640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fdb480b3430 con 0x7fdb480d3450 2024-04-26T23:18:24.227 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.224+0000 7fdb4e037640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdb480b37a0 con 0x7fdb4815f3d0 2024-04-26T23:18:24.227 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.224+0000 7fdb4e037640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fdb480d3900 con 0x7fdb480d2c50 2024-04-26T23:18:24.227 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.224+0000 7fdb46ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb480d3450 0x7fdb480cc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:24.227 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.224+0000 7fdb46ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb480d3450 0x7fdb480cc1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40130/0 (socket says 172.21.15.47:40130) 2024-04-26T23:18:24.228 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.224+0000 7fdb46ffd640 1 -- 172.21.15.47:0/1639934184 learned_addr learned my addr 172.21.15.47:0/1639934184 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:24.228 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.224+0000 7fdb47fff640 1 --2- 172.21.15.47:0/1639934184 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb4815f3d0 0x7fdb480cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:24.228 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.224+0000 7fdb477fe640 1 --2- 172.21.15.47:0/1639934184 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb480d2c50 0x7fdb480b66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:24.228 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.225+0000 7fdb46ffd640 1 -- 172.21.15.47:0/1639934184 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb480d2c50 msgr2=0x7fdb480b66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:24.228 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.225+0000 7fdb46ffd640 1 --2- 172.21.15.47:0/1639934184 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb480d2c50 0x7fdb480b66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:24.228 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.225+0000 7fdb46ffd640 1 -- 172.21.15.47:0/1639934184 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb4815f3d0 msgr2=0x7fdb480cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:24.228 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.225+0000 7fdb46ffd640 1 --2- 172.21.15.47:0/1639934184 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb4815f3d0 0x7fdb480cea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:24.228 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.225+0000 7fdb46ffd640 1 -- 172.21.15.47:0/1639934184 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb480b53b0 con 0x7fdb480d3450 2024-04-26T23:18:24.228 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.225+0000 7fdb46ffd640 1 --2- 172.21.15.47:0/1639934184 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb480d3450 0x7fdb480cc1b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c00c480 tx=0x7fdb3c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:24.228 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.225+0000 7fdb2ffff640 1 -- 172.21.15.47:0/1639934184 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c0052d0 con 0x7fdb480d3450 2024-04-26T23:18:24.228 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.225+0000 7fdb2ffff640 1 -- 172.21.15.47:0/1639934184 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb3c005470 con 0x7fdb480d3450 2024-04-26T23:18:24.228 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.225+0000 7fdb4e037640 1 -- 172.21.15.47:0/1639934184 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb480b5630 con 0x7fdb480d3450 2024-04-26T23:18:24.228 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.225+0000 7fdb2ffff640 1 -- 172.21.15.47:0/1639934184 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c012870 con 0x7fdb480d3450 2024-04-26T23:18:24.228 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.225+0000 7fdb4e037640 1 -- 172.21.15.47:0/1639934184 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb480b5bf0 con 0x7fdb480d3450 2024-04-26T23:18:24.229 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.226+0000 7fdb4e037640 1 -- 172.21.15.47:0/1639934184 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb480d0c50 con 0x7fdb480d3450 2024-04-26T23:18:24.229 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.226+0000 7fdb2ffff640 1 -- 172.21.15.47:0/1639934184 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdb3c012a10 con 0x7fdb480d3450 2024-04-26T23:18:24.229 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.227+0000 7fdb2ffff640 1 --2- 172.21.15.47:0/1639934184 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdb3406c8f0 0x7fdb3406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:24.230 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.227+0000 7fdb2ffff640 1 -- 172.21.15.47:0/1639934184 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fdb3c08b2b0 con 0x7fdb480d3450 2024-04-26T23:18:24.230 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.227+0000 7fdb477fe640 1 --2- 172.21.15.47:0/1639934184 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdb3406c8f0 0x7fdb3406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:24.230 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.228+0000 7fdb477fe640 1 --2- 172.21.15.47:0/1639934184 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdb3406c8f0 0x7fdb3406edb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdb38034000 tx=0x7fdb3802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:24.233 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.232+0000 7fdb2ffff640 1 -- 172.21.15.47:0/1639934184 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdb480b53b0 con 0x7fdb480d3450 2024-04-26T23:18:24.418 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.416+0000 7fdb4e037640 1 -- 172.21.15.47:0/1639934184 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdb480b7900 con 0x7fdb480d3450 2024-04-26T23:18:24.418 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.417+0000 7fdb2ffff640 1 -- 172.21.15.47:0/1639934184 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12848 (secure 0 0 0) 0x7fdb3c059470 con 0x7fdb480d3450 2024-04-26T23:18:24.419 INFO:teuthology.misc.health.smithi047.stdout: 2024-04-26T23:18:24.419 INFO:teuthology.misc.health.smithi047.stdout:{"epoch":16,"fsid":"399006d3-4db1-43e6-ba81-a79a276eabfd","created":"2024-04-26T23:18:06.731211+0000","modified":"2024-04-26T23:18:19.825334+0000","last_up_change":"2024-04-26T23:18:17.808736+0000","last_in_change":"2024-04-26T23:18:08.945859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:18:16.937492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ee091567-fea0-4a14-bf4e-0e16c20bb2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6819","nonce":326520779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6821","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6823","nonce":326520779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6829","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6831","nonce":326520779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6825","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6827","nonce":326520779}]},"public_addr":"172.21.15.47:6819/326520779","cluster_addr":"172.21.15.47:6823/326520779","heartbeat_back_addr":"172.21.15.47:6831/326520779","heartbeat_front_addr":"172.21.15.47:6827/326520779","state":["exists","up"]},{"osd":1,"uuid":"c14440a0-bba5-4931-a6db-b736c08e869b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6809","nonce":1912610931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6811","nonce":1912610931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6815","nonce":1912610931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6813","nonce":1912610931}]},"public_addr":"172.21.15.47:6809/1912610931","cluster_addr":"172.21.15.47:6811/1912610931","heartbeat_back_addr":"172.21.15.47:6815/1912610931","heartbeat_front_addr":"172.21.15.47:6813/1912610931","state":["exists","up"]},{"osd":2,"uuid":"3940f8de-c9b2-4cd3-a911-9c301729eb30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6801","nonce":2569869305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6803","nonce":2569869305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6807","nonce":2569869305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6805","nonce":2569869305}]},"public_addr":"172.21.15.47:6801/2569869305","cluster_addr":"172.21.15.47:6803/2569869305","heartbeat_back_addr":"172.21.15.47:6807/2569869305","heartbeat_front_addr":"172.21.15.47:6805/2569869305","state":["exists","up"]},{"osd":3,"uuid":"1286a67e-8958-426a-8c2e-bc85498c619c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6818","nonce":2791411683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6822","nonce":2791411683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6830","nonce":2791411683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6826","nonce":2791411683}]},"public_addr":"172.21.15.47:6818/2791411683","cluster_addr":"172.21.15.47:6822/2791411683","heartbeat_back_addr":"172.21.15.47:6830/2791411683","heartbeat_front_addr":"172.21.15.47:6826/2791411683","state":["exists","up"]},{"osd":4,"uuid":"d8ee2c20-e5b7-46c2-9999-b234070126cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6809","nonce":441313104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6811","nonce":441313104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6815","nonce":441313104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6813","nonce":441313104}]},"public_addr":"172.21.15.203:6809/441313104","cluster_addr":"172.21.15.203:6811/441313104","heartbeat_back_addr":"172.21.15.203:6815/441313104","heartbeat_front_addr":"172.21.15.203:6813/441313104","state":["exists","up"]},{"osd":5,"uuid":"b453a8f0-f324-4840-9ae0-d67bdadb9f8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6801","nonce":56838444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6803","nonce":56838444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6807","nonce":56838444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6805","nonce":56838444}]},"public_addr":"172.21.15.203:6801/56838444","cluster_addr":"172.21.15.203:6803/56838444","heartbeat_back_addr":"172.21.15.203:6807/56838444","heartbeat_front_addr":"172.21.15.203:6805/56838444","state":["exists","up"]},{"osd":6,"uuid":"a6d31951-2d4b-4e8d-a510-6283b01dc158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6825","nonce":2738517080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6827","nonce":2738517080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6831","nonce":2738517080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6829","nonce":2738517080}]},"public_addr":"172.21.15.203:6825/2738517080","cluster_addr":"172.21.15.203:6827/2738517080","heartbeat_back_addr":"172.21.15.203:6831/2738517080","heartbeat_front_addr":"172.21.15.203:6829/2738517080","state":["exists","up"]},{"osd":7,"uuid":"7119b8c0-fe96-402d-9e75-7b509d9d44d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6817","nonce":1315565058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6819","nonce":1315565058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6823","nonce":1315565058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6821","nonce":1315565058}]},"public_addr":"172.21.15.203:6817/1315565058","cluster_addr":"172.21.15.203:6819/1315565058","heartbeat_back_addr":"172.21.15.203:6823/1315565058","heartbeat_front_addr":"172.21.15.203:6821/1315565058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.121921+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.137494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.084814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.111593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.072430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.125066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.097867+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.057061+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:18:24.421 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.419+0000 7fdb4e037640 1 -- 172.21.15.47:0/1639934184 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdb3406c8f0 msgr2=0x7fdb3406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:24.421 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.419+0000 7fdb4e037640 1 --2- 172.21.15.47:0/1639934184 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdb3406c8f0 0x7fdb3406edb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdb38034000 tx=0x7fdb3802e000 comp rx=0 tx=0).stop 2024-04-26T23:18:24.421 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.419+0000 7fdb4e037640 1 -- 172.21.15.47:0/1639934184 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb480d3450 msgr2=0x7fdb480cc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:24.421 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.419+0000 7fdb4e037640 1 --2- 172.21.15.47:0/1639934184 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb480d3450 0x7fdb480cc1b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c00c480 tx=0x7fdb3c00c940 comp rx=0 tx=0).stop 2024-04-26T23:18:24.421 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.419+0000 7fdb4e037640 1 -- 172.21.15.47:0/1639934184 shutdown_connections 2024-04-26T23:18:24.422 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.419+0000 7fdb4e037640 1 --2- 172.21.15.47:0/1639934184 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdb3406c8f0 0x7fdb3406edb0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:24.422 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.419+0000 7fdb4e037640 1 --2- 172.21.15.47:0/1639934184 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb4815f3d0 0x7fdb480cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:24.422 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.419+0000 7fdb4e037640 1 --2- 172.21.15.47:0/1639934184 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb480d3450 0x7fdb480cc1b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:24.422 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.419+0000 7fdb4e037640 1 --2- 172.21.15.47:0/1639934184 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb480d2c50 0x7fdb480b66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:24.422 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.419+0000 7fdb4e037640 1 -- 172.21.15.47:0/1639934184 >> 172.21.15.47:0/1639934184 conn(0x7fdb480d4f00 msgr2=0x7fdb48160080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:24.422 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.419+0000 7fdb4e037640 1 -- 172.21.15.47:0/1639934184 shutdown_connections 2024-04-26T23:18:24.422 INFO:teuthology.misc.health.smithi047.stderr:2024-04-26T23:18:24.420+0000 7fdb4e037640 1 -- 172.21.15.47:0/1639934184 wait complete. 2024-04-26T23:18:24.430 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-26T23:18:24.431 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-26T23:18:24.431 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T23:18:24.431 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T23:18:24.433 INFO:tasks.ceph.mds.a:Started 2024-04-26T23:18:24.433 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T23:18:24.433 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T23:18:24.435 INFO:tasks.ceph.mds.c:Started 2024-04-26T23:18:24.436 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T23:18:24.436 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T23:18:24.439 INFO:tasks.ceph.mds.b:Started 2024-04-26T23:18:24.439 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T23:18:24.439 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T23:18:24.441 INFO:tasks.ceph.mds.d:Started 2024-04-26T23:18:24.441 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-26T23:18:24.441 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:18:24.509 INFO:tasks.ceph.mds.c.smithi047.stdout:starting mds.c at 2024-04-26T23:18:24.509 INFO:tasks.ceph.mds.a.smithi047.stdout:starting mds.a at 2024-04-26T23:18:24.524 INFO:tasks.ceph.mds.b.smithi203.stdout:starting mds.b at 2024-04-26T23:18:24.524 INFO:tasks.ceph.mds.d.smithi203.stdout:starting mds.d at 2024-04-26T23:18:24.524 INFO:tasks.ceph.mds.b.smithi203.stderr:2024-04-26T23:18:24.523+0000 7f082451f640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-04-26T23:18:24.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.545+0000 7fbac6389640 1 Processor -- start 2024-04-26T23:18:24.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.545+0000 7fbac6389640 1 -- start start 2024-04-26T23:18:24.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.546+0000 7fbac6389640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbac00b2340 0x7fbac00cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:24.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.546+0000 7fbac6389640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbac00cd930 0x7fbac00d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:24.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.546+0000 7fbac6389640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbac00d10d0 con 0x7fbac0162810 2024-04-26T23:18:24.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.546+0000 7fbac6389640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fbac00d0750 con 0x7fbac00cd930 2024-04-26T23:18:24.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.546+0000 7fbac6389640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fbac00d16a0 con 0x7fbac00b2340 2024-04-26T23:18:24.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.546+0000 7fbabffff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fbac0162810 0x7fbac0070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47196/0 (socket says 172.21.15.47:47196) 2024-04-26T23:18:24.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.546+0000 7fbabffff640 1 -- 172.21.15.47:0/2201011819 learned_addr learned my addr 172.21.15.47:0/2201011819 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:18:24.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.546+0000 7fbabf7fe640 1 --2- 172.21.15.47:0/2201011819 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbac00b2340 0x7fbac00cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:24.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.546+0000 7fbac4926640 1 --2- 172.21.15.47:0/2201011819 >> v2:172.21.15.203:3300/0 conn(0x7fbac00cd930 0x7fbac00d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:24.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.547+0000 7fbabeffd640 1 -- 172.21.15.47:0/2201011819 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 879244094 0 0) 0x7fbac00d10d0 con 0x7fbac0162810 2024-04-26T23:18:24.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.547+0000 7fbabeffd640 1 -- 172.21.15.47:0/2201011819 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbaa8003520 con 0x7fbac0162810 2024-04-26T23:18:24.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.547+0000 7fbabf7fe640 1 -- 172.21.15.47:0/2201011819 >> v2:172.21.15.203:3300/0 conn(0x7fbac00cd930 msgr2=0x7fbac00d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:24.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.547+0000 7fbabf7fe640 1 --2- 172.21.15.47:0/2201011819 >> v2:172.21.15.203:3300/0 conn(0x7fbac00cd930 0x7fbac00d1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:24.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.547+0000 7fbabf7fe640 1 -- 172.21.15.47:0/2201011819 >> v1:172.21.15.47:6789/0 conn(0x7fbac0162810 legacy=0x7fbac0070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:24.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.547+0000 7fbabf7fe640 1 -- 172.21.15.47:0/2201011819 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbac00d1370 con 0x7fbac00b2340 2024-04-26T23:18:24.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.547+0000 7fbabf7fe640 1 --2- 172.21.15.47:0/2201011819 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbac00b2340 0x7fbac00cd3f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbab40097a0 tx=0x7fbab402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50b3f08ae81c028f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:24.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.548+0000 7fbabeffd640 1 -- 172.21.15.47:0/2201011819 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab4037070 con 0x7fbac00b2340 2024-04-26T23:18:24.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.548+0000 7fbabeffd640 1 -- 172.21.15.47:0/2201011819 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbab4032440 con 0x7fbac00b2340 2024-04-26T23:18:24.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.548+0000 7fbabeffd640 1 -- 172.21.15.47:0/2201011819 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab40327f0 con 0x7fbac00b2340 2024-04-26T23:18:24.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.548+0000 7fbac6389640 1 -- 172.21.15.47:0/2201011819 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbac00b2340 msgr2=0x7fbac00cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:24.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.548+0000 7fbac6389640 1 --2- 172.21.15.47:0/2201011819 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbac00b2340 0x7fbac00cd3f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbab40097a0 tx=0x7fbab402a180 comp rx=0 tx=0).stop 2024-04-26T23:18:24.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.548+0000 7fbac6389640 1 -- 172.21.15.47:0/2201011819 shutdown_connections 2024-04-26T23:18:24.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.548+0000 7fbac6389640 1 --2- 172.21.15.47:0/2201011819 >> v2:172.21.15.203:3300/0 conn(0x7fbac00cd930 0x7fbac00d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:24.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.548+0000 7fbac6389640 1 --2- 172.21.15.47:0/2201011819 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbac00b2340 0x7fbac00cd3f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:24.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.548+0000 7fbac6389640 1 -- 172.21.15.47:0/2201011819 >> 172.21.15.47:0/2201011819 conn(0x7fbac00d2da0 msgr2=0x7fbac00d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:24.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.548+0000 7fbac6389640 1 -- 172.21.15.47:0/2201011819 shutdown_connections 2024-04-26T23:18:24.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.548+0000 7fbac6389640 1 -- 172.21.15.47:0/2201011819 wait complete. 2024-04-26T23:18:24.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.549+0000 7fbac6389640 1 Processor -- start 2024-04-26T23:18:24.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.550+0000 7fbac6389640 1 -- start start 2024-04-26T23:18:24.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.550+0000 7fbac6389640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbac00b2340 0x7fbac01532d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:24.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.550+0000 7fbac6389640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbac00cd930 0x7fbac00cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:24.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.550+0000 7fbac6389640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbac0162810 0x7fbac00b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:24.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.550+0000 7fbac6389640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fbac00d2070 con 0x7fbac00b2340 2024-04-26T23:18:24.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.550+0000 7fbac6389640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbac00d0470 con 0x7fbac00cd930 2024-04-26T23:18:24.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.550+0000 7fbac6389640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fbac00d21f0 con 0x7fbac0162810 2024-04-26T23:18:24.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.550+0000 7fbabffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbac00b2340 0x7fbac01532d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:24.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.550+0000 7fbabffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbac00b2340 0x7fbac01532d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40168/0 (socket says 172.21.15.47:40168) 2024-04-26T23:18:24.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.550+0000 7fbabffff640 1 -- 172.21.15.47:0/1745291673 learned_addr learned my addr 172.21.15.47:0/1745291673 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:24.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.551+0000 7fbabf7fe640 1 --2- 172.21.15.47:0/1745291673 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbac00cd930 0x7fbac00cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:24.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.551+0000 7fbac4926640 1 --2- 172.21.15.47:0/1745291673 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbac0162810 0x7fbac00b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:24.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.551+0000 7fbabffff640 1 -- 172.21.15.47:0/1745291673 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbac0162810 msgr2=0x7fbac00b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:24.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.551+0000 7fbabffff640 1 --2- 172.21.15.47:0/1745291673 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbac0162810 0x7fbac00b3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:24.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.551+0000 7fbabffff640 1 -- 172.21.15.47:0/1745291673 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbac00cd930 msgr2=0x7fbac00cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:24.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.551+0000 7fbabffff640 1 --2- 172.21.15.47:0/1745291673 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbac00cd930 0x7fbac00cb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:24.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.551+0000 7fbabffff640 1 -- 172.21.15.47:0/1745291673 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbac00cbec0 con 0x7fbac00b2340 2024-04-26T23:18:24.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.551+0000 7fbac4926640 1 --2- 172.21.15.47:0/1745291673 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbac0162810 0x7fbac00b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:18:24.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.551+0000 7fbabf7fe640 1 --2- 172.21.15.47:0/1745291673 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbac00cd930 0x7fbac00cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:18:24.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.551+0000 7fbabffff640 1 --2- 172.21.15.47:0/1745291673 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbac00b2340 0x7fbac01532d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbab000c410 tx=0x7fbab000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:24.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.551+0000 7fbabcff9640 1 -- 172.21.15.47:0/1745291673 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab0017070 con 0x7fbac00b2340 2024-04-26T23:18:24.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.551+0000 7fbac6389640 1 -- 172.21.15.47:0/1745291673 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbac00cc1a0 con 0x7fbac00b2340 2024-04-26T23:18:24.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.551+0000 7fbabcff9640 1 -- 172.21.15.47:0/1745291673 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbab0005ce0 con 0x7fbac00b2340 2024-04-26T23:18:24.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.551+0000 7fbabcff9640 1 -- 172.21.15.47:0/1745291673 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab00072f0 con 0x7fbac00b2340 2024-04-26T23:18:24.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.551+0000 7fbac6389640 1 -- 172.21.15.47:0/1745291673 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbac00cc760 con 0x7fbac00b2340 2024-04-26T23:18:24.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.552+0000 7fbac6389640 1 -- 172.21.15.47:0/1745291673 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbac00694f0 con 0x7fbac00b2340 2024-04-26T23:18:24.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.553+0000 7fbabcff9640 1 -- 172.21.15.47:0/1745291673 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbab001a070 con 0x7fbac00b2340 2024-04-26T23:18:24.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.553+0000 7fbabcff9640 1 --2- 172.21.15.47:0/1745291673 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fbaac06c8f0 0x7fbaac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:24.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.553+0000 7fbabcff9640 1 -- 172.21.15.47:0/1745291673 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fbab008ad70 con 0x7fbac00b2340 2024-04-26T23:18:24.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.553+0000 7fbabf7fe640 1 --2- 172.21.15.47:0/1745291673 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fbaac06c8f0 0x7fbaac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:24.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.554+0000 7fbabf7fe640 1 --2- 172.21.15.47:0/1745291673 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fbaac06c8f0 0x7fbaac06edb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbab4034000 tx=0x7fbab402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:24.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.557+0000 7fbabcff9640 1 -- 172.21.15.47:0/1745291673 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbab0059490 con 0x7fbac00b2340 2024-04-26T23:18:24.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.745+0000 7fbac6389640 1 -- 172.21.15.47:0/1745291673 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbac0163a80 con 0x7fbac00b2340 2024-04-26T23:18:24.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.746+0000 7fbabcff9640 1 -- 172.21.15.47:0/1745291673 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12848 (secure 0 0 0) 0x7fbab0058d90 con 0x7fbac00b2340 2024-04-26T23:18:24.747 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:18:24.747 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":16,"fsid":"399006d3-4db1-43e6-ba81-a79a276eabfd","created":"2024-04-26T23:18:06.731211+0000","modified":"2024-04-26T23:18:19.825334+0000","last_up_change":"2024-04-26T23:18:17.808736+0000","last_in_change":"2024-04-26T23:18:08.945859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:18:16.937492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ee091567-fea0-4a14-bf4e-0e16c20bb2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6819","nonce":326520779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6821","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6823","nonce":326520779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6829","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6831","nonce":326520779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6825","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6827","nonce":326520779}]},"public_addr":"172.21.15.47:6819/326520779","cluster_addr":"172.21.15.47:6823/326520779","heartbeat_back_addr":"172.21.15.47:6831/326520779","heartbeat_front_addr":"172.21.15.47:6827/326520779","state":["exists","up"]},{"osd":1,"uuid":"c14440a0-bba5-4931-a6db-b736c08e869b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6809","nonce":1912610931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6811","nonce":1912610931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6815","nonce":1912610931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6813","nonce":1912610931}]},"public_addr":"172.21.15.47:6809/1912610931","cluster_addr":"172.21.15.47:6811/1912610931","heartbeat_back_addr":"172.21.15.47:6815/1912610931","heartbeat_front_addr":"172.21.15.47:6813/1912610931","state":["exists","up"]},{"osd":2,"uuid":"3940f8de-c9b2-4cd3-a911-9c301729eb30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6801","nonce":2569869305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6803","nonce":2569869305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6807","nonce":2569869305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6805","nonce":2569869305}]},"public_addr":"172.21.15.47:6801/2569869305","cluster_addr":"172.21.15.47:6803/2569869305","heartbeat_back_addr":"172.21.15.47:6807/2569869305","heartbeat_front_addr":"172.21.15.47:6805/2569869305","state":["exists","up"]},{"osd":3,"uuid":"1286a67e-8958-426a-8c2e-bc85498c619c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6818","nonce":2791411683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6822","nonce":2791411683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6830","nonce":2791411683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6826","nonce":2791411683}]},"public_addr":"172.21.15.47:6818/2791411683","cluster_addr":"172.21.15.47:6822/2791411683","heartbeat_back_addr":"172.21.15.47:6830/2791411683","heartbeat_front_addr":"172.21.15.47:6826/2791411683","state":["exists","up"]},{"osd":4,"uuid":"d8ee2c20-e5b7-46c2-9999-b234070126cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6809","nonce":441313104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6811","nonce":441313104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6815","nonce":441313104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6813","nonce":441313104}]},"public_addr":"172.21.15.203:6809/441313104","cluster_addr":"172.21.15.203:6811/441313104","heartbeat_back_addr":"172.21.15.203:6815/441313104","heartbeat_front_addr":"172.21.15.203:6813/441313104","state":["exists","up"]},{"osd":5,"uuid":"b453a8f0-f324-4840-9ae0-d67bdadb9f8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6801","nonce":56838444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6803","nonce":56838444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6807","nonce":56838444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6805","nonce":56838444}]},"public_addr":"172.21.15.203:6801/56838444","cluster_addr":"172.21.15.203:6803/56838444","heartbeat_back_addr":"172.21.15.203:6807/56838444","heartbeat_front_addr":"172.21.15.203:6805/56838444","state":["exists","up"]},{"osd":6,"uuid":"a6d31951-2d4b-4e8d-a510-6283b01dc158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6825","nonce":2738517080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6827","nonce":2738517080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6831","nonce":2738517080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6829","nonce":2738517080}]},"public_addr":"172.21.15.203:6825/2738517080","cluster_addr":"172.21.15.203:6827/2738517080","heartbeat_back_addr":"172.21.15.203:6831/2738517080","heartbeat_front_addr":"172.21.15.203:6829/2738517080","state":["exists","up"]},{"osd":7,"uuid":"7119b8c0-fe96-402d-9e75-7b509d9d44d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6817","nonce":1315565058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6819","nonce":1315565058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6823","nonce":1315565058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6821","nonce":1315565058}]},"public_addr":"172.21.15.203:6817/1315565058","cluster_addr":"172.21.15.203:6819/1315565058","heartbeat_back_addr":"172.21.15.203:6823/1315565058","heartbeat_front_addr":"172.21.15.203:6821/1315565058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.121921+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.137494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.084814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.111593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.072430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.125066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.097867+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.057061+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:18:24.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.748+0000 7fbac6389640 1 -- 172.21.15.47:0/1745291673 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fbaac06c8f0 msgr2=0x7fbaac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:24.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.748+0000 7fbac6389640 1 --2- 172.21.15.47:0/1745291673 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fbaac06c8f0 0x7fbaac06edb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbab4034000 tx=0x7fbab402e000 comp rx=0 tx=0).stop 2024-04-26T23:18:24.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.748+0000 7fbac6389640 1 -- 172.21.15.47:0/1745291673 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbac00b2340 msgr2=0x7fbac01532d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:24.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.748+0000 7fbac6389640 1 --2- 172.21.15.47:0/1745291673 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbac00b2340 0x7fbac01532d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbab000c410 tx=0x7fbab000c8d0 comp rx=0 tx=0).stop 2024-04-26T23:18:24.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.748+0000 7fbac6389640 1 -- 172.21.15.47:0/1745291673 shutdown_connections 2024-04-26T23:18:24.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.748+0000 7fbac6389640 1 --2- 172.21.15.47:0/1745291673 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fbaac06c8f0 0x7fbaac06edb0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:24.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.748+0000 7fbac6389640 1 --2- 172.21.15.47:0/1745291673 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbac0162810 0x7fbac00b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:24.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.748+0000 7fbac6389640 1 --2- 172.21.15.47:0/1745291673 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbac00cd930 0x7fbac00cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:24.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.748+0000 7fbac6389640 1 --2- 172.21.15.47:0/1745291673 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbac00b2340 0x7fbac01532d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:24.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.748+0000 7fbac6389640 1 -- 172.21.15.47:0/1745291673 >> 172.21.15.47:0/1745291673 conn(0x7fbac00d2da0 msgr2=0x7fbac006dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:24.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.748+0000 7fbac6389640 1 -- 172.21.15.47:0/1745291673 shutdown_connections 2024-04-26T23:18:24.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.748+0000 7fbac6389640 1 -- 172.21.15.47:0/1745291673 wait complete. 2024-04-26T23:18:24.762 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:18:16.937492+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:18:24.763 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:18:24.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.864+0000 7fdaa8452640 1 Processor -- start 2024-04-26T23:18:24.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.865+0000 7fdaa8452640 1 -- start start 2024-04-26T23:18:24.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.865+0000 7fdaa8452640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdaa00b5b10 0x7fdaa016a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:24.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.865+0000 7fdaa8452640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdaa00d0a10 con 0x7fdaa00d1a00 2024-04-26T23:18:24.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.865+0000 7fdaa8452640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdaa00d0cf0 con 0x7fdaa00b40d0 2024-04-26T23:18:24.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.865+0000 7fdaa8452640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fdaa00d0fd0 con 0x7fdaa00b5b10 2024-04-26T23:18:24.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.866+0000 7fdaa61ee640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fdaa00d1a00 0x7fdaa00d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47202/0 (socket says 172.21.15.47:47202) 2024-04-26T23:18:24.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.866+0000 7fdaa61ee640 1 -- 172.21.15.47:0/1834693496 learned_addr learned my addr 172.21.15.47:0/1834693496 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:18:24.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.866+0000 7fdaa69ef640 1 --2- 172.21.15.47:0/1834693496 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdaa00b5b10 0x7fdaa016a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:24.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.866+0000 7fdaa51ec640 1 -- 172.21.15.47:0/1834693496 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 55427002 0 0) 0x7fdaa00d0a10 con 0x7fdaa00d1a00 2024-04-26T23:18:24.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.866+0000 7fdaa51ec640 1 -- 172.21.15.47:0/1834693496 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda8c003520 con 0x7fdaa00d1a00 2024-04-26T23:18:24.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.866+0000 7fdaa51ec640 1 -- 172.21.15.47:0/1834693496 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3132366772 0 0) 0x7fdaa00d0cf0 con 0x7fdaa00b40d0 2024-04-26T23:18:24.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.866+0000 7fdaa51ec640 1 -- 172.21.15.47:0/1834693496 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdaa00d0a10 con 0x7fdaa00b40d0 2024-04-26T23:18:24.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.866+0000 7fdaa51ec640 1 -- 172.21.15.47:0/1834693496 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 393315723 0 0) 0x7fda8c003520 con 0x7fdaa00d1a00 2024-04-26T23:18:24.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.866+0000 7fdaa69ef640 1 -- 172.21.15.47:0/1834693496 >> v1:172.21.15.203:6789/0 conn(0x7fdaa00b40d0 legacy=0x7fdaa01625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:24.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.866+0000 7fdaa69ef640 1 -- 172.21.15.47:0/1834693496 >> v1:172.21.15.47:6789/0 conn(0x7fdaa00d1a00 legacy=0x7fdaa00d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:24.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.866+0000 7fdaa69ef640 1 -- 172.21.15.47:0/1834693496 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaa00d40a0 con 0x7fdaa00b5b10 2024-04-26T23:18:24.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.867+0000 7fdaa69ef640 1 --2- 172.21.15.47:0/1834693496 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdaa00b5b10 0x7fdaa016a2e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fda98009dc0 tx=0x7fda9802a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24ea41fc8b7bf1a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:24.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.867+0000 7fdaa51ec640 1 -- 172.21.15.47:0/1834693496 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda94001670 con 0x7fdaa00b5b10 2024-04-26T23:18:24.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.867+0000 7fdaa51ec640 1 -- 172.21.15.47:0/1834693496 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda98007540 con 0x7fdaa00b5b10 2024-04-26T23:18:24.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.867+0000 7fdaa51ec640 1 -- 172.21.15.47:0/1834693496 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda980078f0 con 0x7fdaa00b5b10 2024-04-26T23:18:24.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.868+0000 7fdaa8452640 1 -- 172.21.15.47:0/1834693496 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdaa00b5b10 msgr2=0x7fdaa016a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:24.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.868+0000 7fdaa8452640 1 --2- 172.21.15.47:0/1834693496 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdaa00b5b10 0x7fdaa016a2e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fda98009dc0 tx=0x7fda9802a5d0 comp rx=0 tx=0).stop 2024-04-26T23:18:24.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.868+0000 7fdaa8452640 1 -- 172.21.15.47:0/1834693496 shutdown_connections 2024-04-26T23:18:24.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.868+0000 7fdaa8452640 1 --2- 172.21.15.47:0/1834693496 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdaa00b5b10 0x7fdaa016a2e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:24.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.868+0000 7fdaa8452640 1 -- 172.21.15.47:0/1834693496 >> 172.21.15.47:0/1834693496 conn(0x7fdaa00d4fd0 msgr2=0x7fdaa00d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:24.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.868+0000 7fdaa8452640 1 -- 172.21.15.47:0/1834693496 shutdown_connections 2024-04-26T23:18:24.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.868+0000 7fdaa8452640 1 -- 172.21.15.47:0/1834693496 wait complete. 2024-04-26T23:18:24.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.869+0000 7fdaa8452640 1 Processor -- start 2024-04-26T23:18:24.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.869+0000 7fdaa8452640 1 -- start start 2024-04-26T23:18:24.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.870+0000 7fdaa8452640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdaa00b40d0 0x7fdaa0158830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:24.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.870+0000 7fdaa8452640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdaa00b5b10 0x7fdaa00c86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:24.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.870+0000 7fdaa8452640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdaa00d1a00 0x7fdaa00cc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:24.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.870+0000 7fdaa8452640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa0162e80 con 0x7fdaa00b5b10 2024-04-26T23:18:24.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.870+0000 7fdaa8452640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa0162d00 con 0x7fdaa00d1a00 2024-04-26T23:18:24.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.870+0000 7fdaa8452640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fdaa0163000 con 0x7fdaa00b40d0 2024-04-26T23:18:24.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.870+0000 7fdaa59ed640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdaa00b5b10 0x7fdaa00c86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:24.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.870+0000 7fdaa59ed640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdaa00b5b10 0x7fdaa00c86f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40182/0 (socket says 172.21.15.47:40182) 2024-04-26T23:18:24.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.870+0000 7fdaa59ed640 1 -- 172.21.15.47:0/1279336328 learned_addr learned my addr 172.21.15.47:0/1279336328 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:24.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.870+0000 7fdaa61ee640 1 --2- 172.21.15.47:0/1279336328 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdaa00b40d0 0x7fdaa0158830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:24.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.870+0000 7fdaa69ef640 1 --2- 172.21.15.47:0/1279336328 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdaa00d1a00 0x7fdaa00cc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:24.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.870+0000 7fdaa59ed640 1 -- 172.21.15.47:0/1279336328 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdaa00b40d0 msgr2=0x7fdaa0158830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:24.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.870+0000 7fdaa59ed640 1 --2- 172.21.15.47:0/1279336328 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdaa00b40d0 0x7fdaa0158830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:24.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.870+0000 7fdaa59ed640 1 -- 172.21.15.47:0/1279336328 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdaa00d1a00 msgr2=0x7fdaa00cc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:24.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.870+0000 7fdaa59ed640 1 --2- 172.21.15.47:0/1279336328 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdaa00d1a00 0x7fdaa00cc0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:24.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.870+0000 7fdaa59ed640 1 -- 172.21.15.47:0/1279336328 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaa00ca210 con 0x7fdaa00b5b10 2024-04-26T23:18:24.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.870+0000 7fdaa59ed640 1 --2- 172.21.15.47:0/1279336328 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdaa00b5b10 0x7fdaa00c86f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fda9c00c490 tx=0x7fda9c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:24.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.871+0000 7fda92ffd640 1 -- 172.21.15.47:0/1279336328 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9c0052d0 con 0x7fdaa00b5b10 2024-04-26T23:18:24.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.871+0000 7fda92ffd640 1 -- 172.21.15.47:0/1279336328 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda9c005470 con 0x7fdaa00b5b10 2024-04-26T23:18:24.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.871+0000 7fdaa8452640 1 -- 172.21.15.47:0/1279336328 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaa00ca4b0 con 0x7fdaa00b5b10 2024-04-26T23:18:24.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.871+0000 7fda92ffd640 1 -- 172.21.15.47:0/1279336328 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9c012850 con 0x7fdaa00b5b10 2024-04-26T23:18:24.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.871+0000 7fdaa8452640 1 -- 172.21.15.47:0/1279336328 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaa00caa70 con 0x7fdaa00b5b10 2024-04-26T23:18:24.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.872+0000 7fdaa8452640 1 -- 172.21.15.47:0/1279336328 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaa00694f0 con 0x7fdaa00b5b10 2024-04-26T23:18:24.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.872+0000 7fda92ffd640 1 -- 172.21.15.47:0/1279336328 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fda9c0129f0 con 0x7fdaa00b5b10 2024-04-26T23:18:24.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.872+0000 7fda92ffd640 1 --2- 172.21.15.47:0/1279336328 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fda8406c8f0 0x7fda8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:24.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.873+0000 7fda92ffd640 1 -- 172.21.15.47:0/1279336328 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fda9c08b380 con 0x7fdaa00b5b10 2024-04-26T23:18:24.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.873+0000 7fdaa61ee640 1 --2- 172.21.15.47:0/1279336328 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fda8406c8f0 0x7fda8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:24.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.873+0000 7fdaa61ee640 1 --2- 172.21.15.47:0/1279336328 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fda8406c8f0 0x7fda8406edb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fda940017e0 tx=0x7fda94001610 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:24.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:24.877+0000 7fda92ffd640 1 -- 172.21.15.47:0/1279336328 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fda9c0594b0 con 0x7fdaa00b5b10 2024-04-26T23:18:25.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.062+0000 7fdaa8452640 1 -- 172.21.15.47:0/1279336328 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdaa00d20a0 con 0x7fdaa00b5b10 2024-04-26T23:18:25.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.063+0000 7fda92ffd640 1 -- 172.21.15.47:0/1279336328 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7fda9c05d7b0 con 0x7fdaa00b5b10 2024-04-26T23:18:25.064 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-26T23:18:25.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.064+0000 7fdaa8452640 1 -- 172.21.15.47:0/1279336328 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fda8406c8f0 msgr2=0x7fda8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:25.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.064+0000 7fdaa8452640 1 --2- 172.21.15.47:0/1279336328 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fda8406c8f0 0x7fda8406edb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fda940017e0 tx=0x7fda94001610 comp rx=0 tx=0).stop 2024-04-26T23:18:25.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.065+0000 7fdaa8452640 1 -- 172.21.15.47:0/1279336328 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdaa00b5b10 msgr2=0x7fdaa00c86f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:25.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.065+0000 7fdaa8452640 1 --2- 172.21.15.47:0/1279336328 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdaa00b5b10 0x7fdaa00c86f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fda9c00c490 tx=0x7fda9c00c950 comp rx=0 tx=0).stop 2024-04-26T23:18:25.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.065+0000 7fdaa8452640 1 -- 172.21.15.47:0/1279336328 shutdown_connections 2024-04-26T23:18:25.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.065+0000 7fdaa8452640 1 --2- 172.21.15.47:0/1279336328 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fda8406c8f0 0x7fda8406edb0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:25.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.065+0000 7fdaa8452640 1 --2- 172.21.15.47:0/1279336328 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdaa00d1a00 0x7fdaa00cc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:25.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.065+0000 7fdaa8452640 1 --2- 172.21.15.47:0/1279336328 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdaa00b5b10 0x7fdaa00c86f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:25.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.065+0000 7fdaa8452640 1 --2- 172.21.15.47:0/1279336328 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdaa00b40d0 0x7fdaa0158830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:25.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.065+0000 7fdaa8452640 1 -- 172.21.15.47:0/1279336328 >> 172.21.15.47:0/1279336328 conn(0x7fdaa00d4fd0 msgr2=0x7fdaa01613a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:25.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.065+0000 7fdaa8452640 1 -- 172.21.15.47:0/1279336328 shutdown_connections 2024-04-26T23:18:25.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.065+0000 7fdaa8452640 1 -- 172.21.15.47:0/1279336328 wait complete. 2024-04-26T23:18:25.077 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:18:25.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.175+0000 7f9437ec5640 1 Processor -- start 2024-04-26T23:18:25.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.176+0000 7f9437ec5640 1 -- start start 2024-04-26T23:18:25.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.176+0000 7f9437ec5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94301608f0 0x7f94300d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:25.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.176+0000 7f9437ec5640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f94300d0a90 0x7f94300cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:25.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.176+0000 7f9437ec5640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94300cde20 con 0x7f94300cfdd0 2024-04-26T23:18:25.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.176+0000 7f9437ec5640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f94300cea30 con 0x7f94300d0a90 2024-04-26T23:18:25.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.176+0000 7f9437ec5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f94300ce100 con 0x7f94301608f0 2024-04-26T23:18:25.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.177+0000 7f9435c61640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94301608f0 0x7f94300d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:25.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.177+0000 7f9435460640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f94300d0a90 0x7f94300cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:25.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.177+0000 7f9435460640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f94300d0a90 0x7f94300cf890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.47:58084/0 (socket says 172.21.15.47:58084) 2024-04-26T23:18:25.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.177+0000 7f9435460640 1 -- 172.21.15.47:0/3254783207 learned_addr learned my addr 172.21.15.47:0/3254783207 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:25.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.177+0000 7f9435c61640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94301608f0 0x7f94300d1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.47:56292/0 (socket says 172.21.15.47:56292) 2024-04-26T23:18:25.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.177+0000 7f9436462640 1 --1- 172.21.15.47:0/3254783207 >> v1:172.21.15.203:6789/0 conn(0x7f94300cfdd0 0x7f9430160e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.47:59122/0 (socket says 172.21.15.47:59122) 2024-04-26T23:18:25.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.177+0000 7f9435c61640 1 -- 172.21.15.47:0/3254783207 >> v2:172.21.15.203:3300/0 conn(0x7f94300d0a90 msgr2=0x7f94300cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:25.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.177+0000 7f9435c61640 1 --2- 172.21.15.47:0/3254783207 >> v2:172.21.15.203:3300/0 conn(0x7f94300d0a90 0x7f94300cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:25.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.177+0000 7f9435c61640 1 -- 172.21.15.47:0/3254783207 >> v1:172.21.15.203:6789/0 conn(0x7f94300cfdd0 legacy=0x7f9430160e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:25.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.177+0000 7f9434c5f640 1 -- 172.21.15.47:0/3254783207 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1749438617 0 0) 0x7f94300cde20 con 0x7f94300cfdd0 2024-04-26T23:18:25.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.177+0000 7f9435c61640 1 -- 172.21.15.47:0/3254783207 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94300ce3e0 con 0x7f94301608f0 2024-04-26T23:18:25.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.177+0000 7f9435460640 1 --2- 172.21.15.47:0/3254783207 >> v2:172.21.15.203:3300/0 conn(0x7f94300d0a90 0x7f94300cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:18:25.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.178+0000 7f9435c61640 1 --2- 172.21.15.47:0/3254783207 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94301608f0 0x7f94300d1bf0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f942c00dbc0 tx=0x7f942c02e480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fed6fab9ac75c10 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:25.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.178+0000 7f9434c5f640 1 -- 172.21.15.47:0/3254783207 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f942c03a070 con 0x7f94301608f0 2024-04-26T23:18:25.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.178+0000 7f9434c5f640 1 -- 172.21.15.47:0/3254783207 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f942c00b650 con 0x7f94301608f0 2024-04-26T23:18:25.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.178+0000 7f9434c5f640 1 -- 172.21.15.47:0/3254783207 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f942c00ba20 con 0x7f94301608f0 2024-04-26T23:18:25.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.179+0000 7f9437ec5640 1 -- 172.21.15.47:0/3254783207 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94301608f0 msgr2=0x7f94300d1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:25.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.179+0000 7f9437ec5640 1 --2- 172.21.15.47:0/3254783207 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94301608f0 0x7f94300d1bf0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f942c00dbc0 tx=0x7f942c02e480 comp rx=0 tx=0).stop 2024-04-26T23:18:25.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.179+0000 7f9437ec5640 1 -- 172.21.15.47:0/3254783207 shutdown_connections 2024-04-26T23:18:25.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.179+0000 7f9437ec5640 1 --2- 172.21.15.47:0/3254783207 >> v2:172.21.15.203:3300/0 conn(0x7f94300d0a90 0x7f94300cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:25.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.179+0000 7f9437ec5640 1 --2- 172.21.15.47:0/3254783207 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94301608f0 0x7f94300d1bf0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:25.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.179+0000 7f9437ec5640 1 -- 172.21.15.47:0/3254783207 >> 172.21.15.47:0/3254783207 conn(0x7f94300d2ec0 msgr2=0x7f94300d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:25.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.179+0000 7f9437ec5640 1 -- 172.21.15.47:0/3254783207 shutdown_connections 2024-04-26T23:18:25.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.179+0000 7f9437ec5640 1 -- 172.21.15.47:0/3254783207 wait complete. 2024-04-26T23:18:25.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.180+0000 7f9437ec5640 1 Processor -- start 2024-04-26T23:18:25.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.180+0000 7f9437ec5640 1 -- start start 2024-04-26T23:18:25.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.181+0000 7f9437ec5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94300cfdd0 0x7f94300cada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:25.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.181+0000 7f9437ec5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94300d0a90 0x7f94300b7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:25.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.181+0000 7f9437ec5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94301608f0 0x7f94300afb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:25.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.181+0000 7f9437ec5640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f94300b3310 con 0x7f94301608f0 2024-04-26T23:18:25.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.181+0000 7f9437ec5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f94300d0ef0 con 0x7f94300d0a90 2024-04-26T23:18:25.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.181+0000 7f9437ec5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f94300ceec0 con 0x7f94300cfdd0 2024-04-26T23:18:25.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.181+0000 7f9436462640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94301608f0 0x7f94300afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:25.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.181+0000 7f9436462640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94301608f0 0x7f94300afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40186/0 (socket says 172.21.15.47:40186) 2024-04-26T23:18:25.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.181+0000 7f9436462640 1 -- 172.21.15.47:0/3817790507 learned_addr learned my addr 172.21.15.47:0/3817790507 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:25.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.181+0000 7f9435460640 1 --2- 172.21.15.47:0/3817790507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94300d0a90 0x7f94300b7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:25.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.181+0000 7f9435c61640 1 --2- 172.21.15.47:0/3817790507 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94300cfdd0 0x7f94300cada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:25.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.181+0000 7f9436462640 1 -- 172.21.15.47:0/3817790507 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94300cfdd0 msgr2=0x7f94300cada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:25.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.181+0000 7f9436462640 1 --2- 172.21.15.47:0/3817790507 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94300cfdd0 0x7f94300cada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:25.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.181+0000 7f9436462640 1 -- 172.21.15.47:0/3817790507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94300d0a90 msgr2=0x7f94300b7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:25.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.181+0000 7f9436462640 1 --2- 172.21.15.47:0/3817790507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94300d0a90 0x7f94300b7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:25.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.181+0000 7f9436462640 1 -- 172.21.15.47:0/3817790507 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94300b0650 con 0x7f94301608f0 2024-04-26T23:18:25.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.181+0000 7f9436462640 1 --2- 172.21.15.47:0/3817790507 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94301608f0 0x7f94300afb90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f942800c700 tx=0x7f942800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:25.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.181+0000 7f94267fc640 1 -- 172.21.15.47:0/3817790507 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9428018070 con 0x7f94301608f0 2024-04-26T23:18:25.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.181+0000 7f94267fc640 1 -- 172.21.15.47:0/3817790507 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f942800ce10 con 0x7f94301608f0 2024-04-26T23:18:25.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.181+0000 7f9437ec5640 1 -- 172.21.15.47:0/3817790507 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94300b0930 con 0x7f94301608f0 2024-04-26T23:18:25.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.181+0000 7f94267fc640 1 -- 172.21.15.47:0/3817790507 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f942800a860 con 0x7f94301608f0 2024-04-26T23:18:25.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.182+0000 7f9437ec5640 1 -- 172.21.15.47:0/3817790507 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9430156800 con 0x7f94301608f0 2024-04-26T23:18:25.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.183+0000 7f9437ec5640 1 -- 172.21.15.47:0/3817790507 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94300694f0 con 0x7f94301608f0 2024-04-26T23:18:25.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.183+0000 7f94267fc640 1 -- 172.21.15.47:0/3817790507 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f942801b070 con 0x7f94301608f0 2024-04-26T23:18:25.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.183+0000 7f94267fc640 1 --2- 172.21.15.47:0/3817790507 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f941406c920 0x7f941406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:25.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.183+0000 7f94267fc640 1 -- 172.21.15.47:0/3817790507 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f942808ad60 con 0x7f94301608f0 2024-04-26T23:18:25.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.184+0000 7f9435c61640 1 --2- 172.21.15.47:0/3817790507 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f941406c920 0x7f941406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:25.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.184+0000 7f9435c61640 1 --2- 172.21.15.47:0/3817790507 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f941406c920 0x7f941406ede0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f942c037000 tx=0x7f942c00d000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:25.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.188+0000 7f94267fc640 1 -- 172.21.15.47:0/3817790507 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f94280594e0 con 0x7f94301608f0 2024-04-26T23:18:25.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.413+0000 7f9437ec5640 1 -- 172.21.15.47:0/3817790507 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f94300c6600 con 0x7f94301608f0 2024-04-26T23:18:25.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.413+0000 7f94267fc640 1 -- 172.21.15.47:0/3817790507 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1392 (secure 0 0 0) 0x7f9428058de0 con 0x7f94301608f0 2024-04-26T23:18:25.415 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:18:25.415 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/1553696401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1553696401},{"type":"v1","addr":"172.21.15.47:6834","nonce":1553696401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-26T23:18:25.415 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 2 2024-04-26T23:18:25.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.415+0000 7f9437ec5640 1 -- 172.21.15.47:0/3817790507 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f941406c920 msgr2=0x7f941406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:25.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.415+0000 7f9437ec5640 1 --2- 172.21.15.47:0/3817790507 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f941406c920 0x7f941406ede0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f942c037000 tx=0x7f942c00d000 comp rx=0 tx=0).stop 2024-04-26T23:18:25.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.415+0000 7f9437ec5640 1 -- 172.21.15.47:0/3817790507 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94301608f0 msgr2=0x7f94300afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:25.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.415+0000 7f9437ec5640 1 --2- 172.21.15.47:0/3817790507 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94301608f0 0x7f94300afb90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f942800c700 tx=0x7f942800cbc0 comp rx=0 tx=0).stop 2024-04-26T23:18:25.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.415+0000 7f9437ec5640 1 -- 172.21.15.47:0/3817790507 shutdown_connections 2024-04-26T23:18:25.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.415+0000 7f9437ec5640 1 --2- 172.21.15.47:0/3817790507 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f941406c920 0x7f941406ede0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:25.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.415+0000 7f9437ec5640 1 --2- 172.21.15.47:0/3817790507 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94301608f0 0x7f94300afb90 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:25.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.415+0000 7f9437ec5640 1 --2- 172.21.15.47:0/3817790507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94300d0a90 0x7f94300b7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:25.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.415+0000 7f9437ec5640 1 --2- 172.21.15.47:0/3817790507 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94300cfdd0 0x7f94300cada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:25.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.415+0000 7f9437ec5640 1 -- 172.21.15.47:0/3817790507 >> 172.21.15.47:0/3817790507 conn(0x7f94300d2ec0 msgr2=0x7f94301590b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:25.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.415+0000 7f9437ec5640 1 -- 172.21.15.47:0/3817790507 shutdown_connections 2024-04-26T23:18:25.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:25.415+0000 7f9437ec5640 1 -- 172.21.15.47:0/3817790507 wait complete. 2024-04-26T23:18:27.428 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:18:27.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.540+0000 7f1b52a25640 1 Processor -- start 2024-04-26T23:18:27.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.540+0000 7f1b52a25640 1 -- start start 2024-04-26T23:18:27.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.541+0000 7f1b52a25640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1b4c15f3a0 0x7f1b4c0d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:27.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.541+0000 7f1b52a25640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1b4c0d2330 0x7f1b4c0b2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:27.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.541+0000 7f1b52a25640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b4c0d0e20 con 0x7f1b4c0d1770 2024-04-26T23:18:27.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.541+0000 7f1b52a25640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f1b4c0b1d70 con 0x7f1b4c15f3a0 2024-04-26T23:18:27.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.541+0000 7f1b52a25640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f1b4c0b2050 con 0x7f1b4c0d2330 2024-04-26T23:18:27.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.541+0000 7f1b4bfff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1b4c15f3a0 0x7f1b4c0d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:27.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.541+0000 7f1b4bfff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1b4c15f3a0 0x7f1b4c0d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40200/0 (socket says 172.21.15.47:40200) 2024-04-26T23:18:27.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.541+0000 7f1b4bfff640 1 -- 172.21.15.47:0/2679088812 learned_addr learned my addr 172.21.15.47:0/2679088812 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:27.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.541+0000 7f1b50fc2640 1 --2- 172.21.15.47:0/2679088812 >> v2:172.21.15.203:3300/0 conn(0x7f1b4c0d2330 0x7f1b4c0b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:27.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.541+0000 7f1b4bfff640 1 -- 172.21.15.47:0/2679088812 >> v2:172.21.15.203:3300/0 conn(0x7f1b4c0d2330 msgr2=0x7f1b4c0b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:27.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.541+0000 7f1b4bfff640 1 --2- 172.21.15.47:0/2679088812 >> v2:172.21.15.203:3300/0 conn(0x7f1b4c0d2330 0x7f1b4c0b2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:27.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.541+0000 7f1b4bfff640 1 -- 172.21.15.47:0/2679088812 >> v1:172.21.15.203:6789/0 conn(0x7f1b4c0d1770 legacy=0x7f1b4c0d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:27.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.541+0000 7f1b4bfff640 1 -- 172.21.15.47:0/2679088812 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b4c0b2330 con 0x7f1b4c15f3a0 2024-04-26T23:18:27.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.541+0000 7f1b50fc2640 1 --2- 172.21.15.47:0/2679088812 >> v2:172.21.15.203:3300/0 conn(0x7f1b4c0d2330 0x7f1b4c0b2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:18:27.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.542+0000 7f1b4bfff640 1 --2- 172.21.15.47:0/2679088812 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b4c15f3a0 0x7f1b4c0d06f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c073b50 tx=0x7f1b3402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9fc3bfbd59f1d67 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:27.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.542+0000 7f1b4affd640 1 -- 172.21.15.47:0/2679088812 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b34037070 con 0x7f1b4c15f3a0 2024-04-26T23:18:27.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.542+0000 7f1b4affd640 1 -- 172.21.15.47:0/2679088812 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b34032440 con 0x7f1b4c15f3a0 2024-04-26T23:18:27.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.542+0000 7f1b4affd640 1 -- 172.21.15.47:0/2679088812 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b34032810 con 0x7f1b4c15f3a0 2024-04-26T23:18:27.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.543+0000 7f1b52a25640 1 -- 172.21.15.47:0/2679088812 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b4c15f3a0 msgr2=0x7f1b4c0d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:27.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.543+0000 7f1b52a25640 1 --2- 172.21.15.47:0/2679088812 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b4c15f3a0 0x7f1b4c0d06f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c073b50 tx=0x7f1b3402a200 comp rx=0 tx=0).stop 2024-04-26T23:18:27.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.543+0000 7f1b52a25640 1 -- 172.21.15.47:0/2679088812 shutdown_connections 2024-04-26T23:18:27.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.543+0000 7f1b52a25640 1 --2- 172.21.15.47:0/2679088812 >> v2:172.21.15.203:3300/0 conn(0x7f1b4c0d2330 0x7f1b4c0b2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:27.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.543+0000 7f1b52a25640 1 --2- 172.21.15.47:0/2679088812 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b4c15f3a0 0x7f1b4c0d06f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:27.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.543+0000 7f1b52a25640 1 -- 172.21.15.47:0/2679088812 >> 172.21.15.47:0/2679088812 conn(0x7f1b4c0d4f90 msgr2=0x7f1b4c0d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:27.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.543+0000 7f1b52a25640 1 -- 172.21.15.47:0/2679088812 shutdown_connections 2024-04-26T23:18:27.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.543+0000 7f1b52a25640 1 -- 172.21.15.47:0/2679088812 wait complete. 2024-04-26T23:18:27.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.544+0000 7f1b52a25640 1 Processor -- start 2024-04-26T23:18:27.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.544+0000 7f1b52a25640 1 -- start start 2024-04-26T23:18:27.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.545+0000 7f1b52a25640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b4c0d1770 0x7f1b4c0cb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:27.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.545+0000 7f1b52a25640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b4c0d2330 0x7f1b4c15af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:27.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.545+0000 7f1b52a25640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b4c15f3a0 0x7f1b4c159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:27.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.545+0000 7f1b52a25640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4c0b2f10 con 0x7f1b4c15f3a0 2024-04-26T23:18:27.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.545+0000 7f1b52a25640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4c0b2d90 con 0x7f1b4c0d2330 2024-04-26T23:18:27.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.545+0000 7f1b52a25640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f1b4c0b3090 con 0x7f1b4c0d1770 2024-04-26T23:18:27.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.545+0000 7f1b50fc2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b4c15f3a0 0x7f1b4c159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:27.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.545+0000 7f1b50fc2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b4c15f3a0 0x7f1b4c159690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40212/0 (socket says 172.21.15.47:40212) 2024-04-26T23:18:27.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.545+0000 7f1b50fc2640 1 -- 172.21.15.47:0/3334588312 learned_addr learned my addr 172.21.15.47:0/3334588312 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:27.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.545+0000 7f1b4bfff640 1 --2- 172.21.15.47:0/3334588312 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b4c0d1770 0x7f1b4c0cb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:27.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.545+0000 7f1b4b7fe640 1 --2- 172.21.15.47:0/3334588312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b4c0d2330 0x7f1b4c15af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:27.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.545+0000 7f1b50fc2640 1 -- 172.21.15.47:0/3334588312 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b4c0d1770 msgr2=0x7f1b4c0cb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:27.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.545+0000 7f1b50fc2640 1 --2- 172.21.15.47:0/3334588312 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b4c0d1770 0x7f1b4c0cb7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:27.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.545+0000 7f1b50fc2640 1 -- 172.21.15.47:0/3334588312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b4c0d2330 msgr2=0x7f1b4c15af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:27.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.545+0000 7f1b50fc2640 1 --2- 172.21.15.47:0/3334588312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b4c0d2330 0x7f1b4c15af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:27.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.545+0000 7f1b50fc2640 1 -- 172.21.15.47:0/3334588312 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b4c15b7d0 con 0x7f1b4c15f3a0 2024-04-26T23:18:27.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.545+0000 7f1b4b7fe640 1 --2- 172.21.15.47:0/3334588312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b4c0d2330 0x7f1b4c15af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:18:27.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.545+0000 7f1b50fc2640 1 --2- 172.21.15.47:0/3334588312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b4c15f3a0 0x7f1b4c159690 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1b4400cf20 tx=0x7f1b4400a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:27.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.546+0000 7f1b48ff9640 1 -- 172.21.15.47:0/3334588312 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4401a070 con 0x7f1b4c15f3a0 2024-04-26T23:18:27.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.546+0000 7f1b48ff9640 1 -- 172.21.15.47:0/3334588312 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b4400d840 con 0x7f1b4c15f3a0 2024-04-26T23:18:27.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.546+0000 7f1b48ff9640 1 -- 172.21.15.47:0/3334588312 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4400dc30 con 0x7f1b4c15f3a0 2024-04-26T23:18:27.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.546+0000 7f1b52a25640 1 -- 172.21.15.47:0/3334588312 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b4c15bab0 con 0x7f1b4c15f3a0 2024-04-26T23:18:27.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.546+0000 7f1b52a25640 1 -- 172.21.15.47:0/3334588312 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b4c15c070 con 0x7f1b4c15f3a0 2024-04-26T23:18:27.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.547+0000 7f1b52a25640 1 -- 172.21.15.47:0/3334588312 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b4c0694f0 con 0x7f1b4c15f3a0 2024-04-26T23:18:27.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.547+0000 7f1b48ff9640 1 -- 172.21.15.47:0/3334588312 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1b4401d070 con 0x7f1b4c15f3a0 2024-04-26T23:18:27.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.548+0000 7f1b48ff9640 1 --2- 172.21.15.47:0/3334588312 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1b2c06c8f0 0x7f1b2c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:27.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.548+0000 7f1b48ff9640 1 -- 172.21.15.47:0/3334588312 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f1b4408c090 con 0x7f1b4c15f3a0 2024-04-26T23:18:27.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.548+0000 7f1b4bfff640 1 --2- 172.21.15.47:0/3334588312 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1b2c06c8f0 0x7f1b2c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:27.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.549+0000 7f1b4bfff640 1 --2- 172.21.15.47:0/3334588312 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1b2c06c8f0 0x7f1b2c06edb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1b34034000 tx=0x7f1b3402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:27.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.552+0000 7f1b48ff9640 1 -- 172.21.15.47:0/3334588312 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1b4405a8a0 con 0x7f1b4c15f3a0 2024-04-26T23:18:27.661 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:18:27.664 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:18:27.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.779+0000 7f1b52a25640 1 -- 172.21.15.47:0/3334588312 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1b4c0b6860 con 0x7f1b4c15f3a0 2024-04-26T23:18:27.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.779+0000 7f1b48ff9640 1 -- 172.21.15.47:0/3334588312 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3867 (secure 0 0 0) 0x7f1b4405a1a0 con 0x7f1b4c15f3a0 2024-04-26T23:18:27.781 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:18:27.781 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/1553696401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1553696401},{"type":"v1","addr":"172.21.15.47:6834","nonce":1553696401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/3792767564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3792767564},{"type":"v1","addr":"172.21.15.203:6836","nonce":3792767564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4088100766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":4088100766},{"type":"v1","addr":"172.21.15.47:6835","nonce":4088100766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/997115011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":997115011},{"type":"v1","addr":"172.21.15.203:6837","nonce":997115011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-26T23:18:27.781 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 3 2024-04-26T23:18:27.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.781+0000 7f1b52a25640 1 -- 172.21.15.47:0/3334588312 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1b2c06c8f0 msgr2=0x7f1b2c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:27.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.781+0000 7f1b52a25640 1 --2- 172.21.15.47:0/3334588312 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1b2c06c8f0 0x7f1b2c06edb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1b34034000 tx=0x7f1b3402e000 comp rx=0 tx=0).stop 2024-04-26T23:18:27.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.781+0000 7f1b52a25640 1 -- 172.21.15.47:0/3334588312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b4c15f3a0 msgr2=0x7f1b4c159690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:27.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.781+0000 7f1b52a25640 1 --2- 172.21.15.47:0/3334588312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b4c15f3a0 0x7f1b4c159690 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1b4400cf20 tx=0x7f1b4400a7d0 comp rx=0 tx=0).stop 2024-04-26T23:18:27.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.781+0000 7f1b52a25640 1 -- 172.21.15.47:0/3334588312 shutdown_connections 2024-04-26T23:18:27.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.781+0000 7f1b52a25640 1 --2- 172.21.15.47:0/3334588312 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1b2c06c8f0 0x7f1b2c06edb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:27.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.781+0000 7f1b52a25640 1 --2- 172.21.15.47:0/3334588312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b4c15f3a0 0x7f1b4c159690 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:27.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.781+0000 7f1b52a25640 1 --2- 172.21.15.47:0/3334588312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b4c0d2330 0x7f1b4c15af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:27.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.781+0000 7f1b52a25640 1 --2- 172.21.15.47:0/3334588312 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b4c0d1770 0x7f1b4c0cb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:27.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.781+0000 7f1b52a25640 1 -- 172.21.15.47:0/3334588312 >> 172.21.15.47:0/3334588312 conn(0x7f1b4c0d4f90 msgr2=0x7f1b4c0b4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:27.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.781+0000 7f1b52a25640 1 -- 172.21.15.47:0/3334588312 shutdown_connections 2024-04-26T23:18:27.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.782+0000 7f1b52a25640 1 -- 172.21.15.47:0/3334588312 wait complete. 2024-04-26T23:18:27.794 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:18:27.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.893+0000 7f3fd9df0640 1 Processor -- start 2024-04-26T23:18:27.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.893+0000 7f3fd9df0640 1 -- start start 2024-04-26T23:18:27.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.894+0000 7f3fd9df0640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3fd406c950 0x7f3fd4160ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:27.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.894+0000 7f3fd9df0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fd4070930 0x7f3fd40aed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:27.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.894+0000 7f3fd9df0640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3fd40b0990 con 0x7f3fd415d100 2024-04-26T23:18:27.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.894+0000 7f3fd9df0640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f3fd40af2d0 con 0x7f3fd406c950 2024-04-26T23:18:27.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.894+0000 7f3fd9df0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3fd40b0c70 con 0x7f3fd4070930 2024-04-26T23:18:27.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.894+0000 7f3fd37fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f3fd415d100 0x7f3fd406c240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47216/0 (socket says 172.21.15.47:47216) 2024-04-26T23:18:27.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.894+0000 7f3fd37fe640 1 -- 172.21.15.47:0/3728239405 learned_addr learned my addr 172.21.15.47:0/3728239405 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:18:27.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.895+0000 7f3fd27fc640 1 -- 172.21.15.47:0/3728239405 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2345888755 0 0) 0x7f3fd40b0990 con 0x7f3fd415d100 2024-04-26T23:18:27.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.895+0000 7f3fd3fff640 1 --2- 172.21.15.47:0/3728239405 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fd4070930 0x7f3fd40aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:27.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.895+0000 7f3fd27fc640 1 -- 172.21.15.47:0/3728239405 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3fbc003520 con 0x7f3fd415d100 2024-04-26T23:18:27.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.895+0000 7f3fd2ffd640 1 --2- 172.21.15.47:0/3728239405 >> v2:172.21.15.203:3300/0 conn(0x7f3fd406c950 0x7f3fd4160ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:27.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.895+0000 7f3fd27fc640 1 -- 172.21.15.47:0/3728239405 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4140936585 0 0) 0x7f3fbc003520 con 0x7f3fd415d100 2024-04-26T23:18:27.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.895+0000 7f3fd27fc640 1 -- 172.21.15.47:0/3728239405 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fd4070930 msgr2=0x7f3fd40aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:27.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.895+0000 7f3fd27fc640 1 --2- 172.21.15.47:0/3728239405 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fd4070930 0x7f3fd40aed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:27.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.895+0000 7f3fd27fc640 1 -- 172.21.15.47:0/3728239405 >> v2:172.21.15.203:3300/0 conn(0x7f3fd406c950 msgr2=0x7f3fd4160ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:27.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.895+0000 7f3fd27fc640 1 --2- 172.21.15.47:0/3728239405 >> v2:172.21.15.203:3300/0 conn(0x7f3fd406c950 0x7f3fd4160ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:27.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.895+0000 7f3fd27fc640 1 -- 172.21.15.47:0/3728239405 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fd40b0f50 con 0x7f3fd415d100 2024-04-26T23:18:27.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.895+0000 7f3fd27fc640 1 -- 172.21.15.47:0/3728239405 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7f3fc4001670 con 0x7f3fd415d100 2024-04-26T23:18:27.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.895+0000 7f3fd27fc640 1 -- 172.21.15.47:0/3728239405 >> v1:172.21.15.47:6789/0 conn(0x7f3fd415d100 legacy=0x7f3fd406c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:27.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.895+0000 7f3fd27fc640 1 --2- 172.21.15.47:0/3728239405 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fbc0039b0 0x7f3fbc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:27.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.895+0000 7f3fd27fc640 1 --2- 172.21.15.47:0/3728239405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3fbc004800 0x7f3fbc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:27.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.895+0000 7f3fd27fc640 1 --2- 172.21.15.47:0/3728239405 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fd4070930 0x7f3fbc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:27.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.895+0000 7f3fd27fc640 1 -- 172.21.15.47:0/3728239405 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd40b0c70 con 0x7f3fbc004800 2024-04-26T23:18:27.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.895+0000 7f3fd27fc640 1 -- 172.21.15.47:0/3728239405 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3fbc00b9c0 con 0x7f3fbc0039b0 2024-04-26T23:18:27.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.895+0000 7f3fd27fc640 1 -- 172.21.15.47:0/3728239405 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3fbc00bb70 con 0x7f3fd4070930 2024-04-26T23:18:27.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.895+0000 7f3fd2ffd640 1 --2- 172.21.15.47:0/3728239405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3fbc004800 0x7f3fbc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:27.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.895+0000 7f3fd37fe640 1 --2- 172.21.15.47:0/3728239405 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fbc0039b0 0x7f3fbc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:27.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.895+0000 7f3fd3fff640 1 --2- 172.21.15.47:0/3728239405 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fd4070930 0x7f3fbc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:27.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.895+0000 7f3fd2ffd640 1 -- 172.21.15.47:0/3728239405 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fd4070930 msgr2=0x7f3fbc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:27.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.895+0000 7f3fd2ffd640 1 --2- 172.21.15.47:0/3728239405 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fd4070930 0x7f3fbc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:27.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.895+0000 7f3fd2ffd640 1 -- 172.21.15.47:0/3728239405 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fbc0039b0 msgr2=0x7f3fbc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:27.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.895+0000 7f3fd2ffd640 1 --2- 172.21.15.47:0/3728239405 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fbc0039b0 0x7f3fbc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:27.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.895+0000 7f3fd2ffd640 1 -- 172.21.15.47:0/3728239405 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fbc003520 con 0x7f3fbc004800 2024-04-26T23:18:27.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.896+0000 7f3fd2ffd640 1 --2- 172.21.15.47:0/3728239405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3fbc004800 0x7f3fbc004c80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3fc8009ac0 tx=0x7f3fbc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e640447371e6f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:27.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.896+0000 7f3fd27fc640 1 -- 172.21.15.47:0/3728239405 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc8036070 con 0x7f3fbc004800 2024-04-26T23:18:27.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.896+0000 7f3fd27fc640 1 -- 172.21.15.47:0/3728239405 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fc8031400 con 0x7f3fbc004800 2024-04-26T23:18:27.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.896+0000 7f3fd27fc640 1 -- 172.21.15.47:0/3728239405 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc8009690 con 0x7f3fbc004800 2024-04-26T23:18:27.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.897+0000 7f3fd9df0640 1 -- 172.21.15.47:0/3728239405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3fbc004800 msgr2=0x7f3fbc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:27.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.897+0000 7f3fd9df0640 1 --2- 172.21.15.47:0/3728239405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3fbc004800 0x7f3fbc004c80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3fc8009ac0 tx=0x7f3fbc005830 comp rx=0 tx=0).stop 2024-04-26T23:18:27.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.897+0000 7f3fd3fff640 1 -- 172.21.15.47:0/3728239405 reap_dead start 2024-04-26T23:18:27.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.897+0000 7f3fd9df0640 1 -- 172.21.15.47:0/3728239405 shutdown_connections 2024-04-26T23:18:27.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.897+0000 7f3fd9df0640 1 -- 172.21.15.47:0/3728239405 >> 172.21.15.47:0/3728239405 conn(0x7f3fd40b39b0 msgr2=0x7f3fd40b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:27.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.897+0000 7f3fd9df0640 1 -- 172.21.15.47:0/3728239405 shutdown_connections 2024-04-26T23:18:27.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.897+0000 7f3fd9df0640 1 -- 172.21.15.47:0/3728239405 wait complete. 2024-04-26T23:18:27.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.898+0000 7f3fd9df0640 1 Processor -- start 2024-04-26T23:18:27.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.898+0000 7f3fd9df0640 1 -- start start 2024-04-26T23:18:27.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.899+0000 7f3fd9df0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fbc004800 0x7f3fd4154270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:27.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.899+0000 7f3fd9df0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fd415a6f0 0x7f3fd4152aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:27.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.899+0000 7f3fd9df0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3fd4146650 0x7f3fd415b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:27.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.899+0000 7f3fd9df0640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd40aed90 con 0x7f3fd4146650 2024-04-26T23:18:27.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.899+0000 7f3fd9df0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd40b1510 con 0x7f3fbc004800 2024-04-26T23:18:27.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.899+0000 7f3fd9df0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3fd40aef10 con 0x7f3fd415a6f0 2024-04-26T23:18:27.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.899+0000 7f3fd3fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3fd4146650 0x7f3fd415b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:27.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.899+0000 7f3fd3fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3fd4146650 0x7f3fd415b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40232/0 (socket says 172.21.15.47:40232) 2024-04-26T23:18:27.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.899+0000 7f3fd3fff640 1 -- 172.21.15.47:0/2843339923 learned_addr learned my addr 172.21.15.47:0/2843339923 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:27.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.899+0000 7f3fd37fe640 1 --2- 172.21.15.47:0/2843339923 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fbc004800 0x7f3fd4154270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:27.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.899+0000 7f3fd2ffd640 1 --2- 172.21.15.47:0/2843339923 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fd415a6f0 0x7f3fd4152aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:27.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.899+0000 7f3fd3fff640 1 -- 172.21.15.47:0/2843339923 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fd415a6f0 msgr2=0x7f3fd4152aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:27.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.899+0000 7f3fd3fff640 1 --2- 172.21.15.47:0/2843339923 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fd415a6f0 0x7f3fd4152aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:27.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.899+0000 7f3fd3fff640 1 -- 172.21.15.47:0/2843339923 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fbc004800 msgr2=0x7f3fd4154270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:27.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.899+0000 7f3fd3fff640 1 --2- 172.21.15.47:0/2843339923 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fbc004800 0x7f3fd4154270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:27.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.899+0000 7f3fd3fff640 1 -- 172.21.15.47:0/2843339923 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fd41531d0 con 0x7f3fd4146650 2024-04-26T23:18:27.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.900+0000 7f3fd3fff640 1 --2- 172.21.15.47:0/2843339923 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3fd4146650 0x7f3fd415b1b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3fd40a0060 tx=0x7f3fcc010530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:27.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.900+0000 7f3fbbfff640 1 -- 172.21.15.47:0/2843339923 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc010d60 con 0x7f3fd4146650 2024-04-26T23:18:27.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.900+0000 7f3fbbfff640 1 -- 172.21.15.47:0/2843339923 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fcc01c070 con 0x7f3fd4146650 2024-04-26T23:18:27.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.900+0000 7f3fbbfff640 1 -- 172.21.15.47:0/2843339923 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc017670 con 0x7f3fd4146650 2024-04-26T23:18:27.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.900+0000 7f3fd9df0640 1 -- 172.21.15.47:0/2843339923 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fd41534b0 con 0x7f3fd4146650 2024-04-26T23:18:27.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.900+0000 7f3fd9df0640 1 -- 172.21.15.47:0/2843339923 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fd4153a70 con 0x7f3fd4146650 2024-04-26T23:18:27.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.901+0000 7f3fd9df0640 1 -- 172.21.15.47:0/2843339923 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fd415aad0 con 0x7f3fd4146650 2024-04-26T23:18:27.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.901+0000 7f3fbbfff640 1 -- 172.21.15.47:0/2843339923 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3fcc017810 con 0x7f3fd4146650 2024-04-26T23:18:27.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.902+0000 7f3fbbfff640 1 --2- 172.21.15.47:0/2843339923 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3fc006c8f0 0x7f3fc006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:27.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.902+0000 7f3fbbfff640 1 -- 172.21.15.47:0/2843339923 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f3fcc08c400 con 0x7f3fd4146650 2024-04-26T23:18:27.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.902+0000 7f3fd37fe640 1 --2- 172.21.15.47:0/2843339923 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3fc006c8f0 0x7f3fc006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:27.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.902+0000 7f3fd37fe640 1 --2- 172.21.15.47:0/2843339923 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3fc006c8f0 0x7f3fc006edb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4000fb0 tx=0x7f3fc4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:27.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:27.906+0000 7f3fbbfff640 1 -- 172.21.15.47:0/2843339923 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3fcc026020 con 0x7f3fd4146650 2024-04-26T23:18:28.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.091+0000 7f3fd9df0640 1 -- 172.21.15.47:0/2843339923 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3fd4161100 con 0x7f3fd4146650 2024-04-26T23:18:28.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.092+0000 7f3fbbfff640 1 -- 172.21.15.47:0/2843339923 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12848 (secure 0 0 0) 0x7f3fcc05ab80 con 0x7f3fd4146650 2024-04-26T23:18:28.094 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:18:28.094 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":16,"fsid":"399006d3-4db1-43e6-ba81-a79a276eabfd","created":"2024-04-26T23:18:06.731211+0000","modified":"2024-04-26T23:18:19.825334+0000","last_up_change":"2024-04-26T23:18:17.808736+0000","last_in_change":"2024-04-26T23:18:08.945859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:18:16.937492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ee091567-fea0-4a14-bf4e-0e16c20bb2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6819","nonce":326520779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6821","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6823","nonce":326520779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6829","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6831","nonce":326520779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6825","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6827","nonce":326520779}]},"public_addr":"172.21.15.47:6819/326520779","cluster_addr":"172.21.15.47:6823/326520779","heartbeat_back_addr":"172.21.15.47:6831/326520779","heartbeat_front_addr":"172.21.15.47:6827/326520779","state":["exists","up"]},{"osd":1,"uuid":"c14440a0-bba5-4931-a6db-b736c08e869b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6809","nonce":1912610931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6811","nonce":1912610931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6815","nonce":1912610931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6813","nonce":1912610931}]},"public_addr":"172.21.15.47:6809/1912610931","cluster_addr":"172.21.15.47:6811/1912610931","heartbeat_back_addr":"172.21.15.47:6815/1912610931","heartbeat_front_addr":"172.21.15.47:6813/1912610931","state":["exists","up"]},{"osd":2,"uuid":"3940f8de-c9b2-4cd3-a911-9c301729eb30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6801","nonce":2569869305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6803","nonce":2569869305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6807","nonce":2569869305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6805","nonce":2569869305}]},"public_addr":"172.21.15.47:6801/2569869305","cluster_addr":"172.21.15.47:6803/2569869305","heartbeat_back_addr":"172.21.15.47:6807/2569869305","heartbeat_front_addr":"172.21.15.47:6805/2569869305","state":["exists","up"]},{"osd":3,"uuid":"1286a67e-8958-426a-8c2e-bc85498c619c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6818","nonce":2791411683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6822","nonce":2791411683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6830","nonce":2791411683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6826","nonce":2791411683}]},"public_addr":"172.21.15.47:6818/2791411683","cluster_addr":"172.21.15.47:6822/2791411683","heartbeat_back_addr":"172.21.15.47:6830/2791411683","heartbeat_front_addr":"172.21.15.47:6826/2791411683","state":["exists","up"]},{"osd":4,"uuid":"d8ee2c20-e5b7-46c2-9999-b234070126cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6809","nonce":441313104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6811","nonce":441313104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6815","nonce":441313104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6813","nonce":441313104}]},"public_addr":"172.21.15.203:6809/441313104","cluster_addr":"172.21.15.203:6811/441313104","heartbeat_back_addr":"172.21.15.203:6815/441313104","heartbeat_front_addr":"172.21.15.203:6813/441313104","state":["exists","up"]},{"osd":5,"uuid":"b453a8f0-f324-4840-9ae0-d67bdadb9f8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6801","nonce":56838444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6803","nonce":56838444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6807","nonce":56838444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6805","nonce":56838444}]},"public_addr":"172.21.15.203:6801/56838444","cluster_addr":"172.21.15.203:6803/56838444","heartbeat_back_addr":"172.21.15.203:6807/56838444","heartbeat_front_addr":"172.21.15.203:6805/56838444","state":["exists","up"]},{"osd":6,"uuid":"a6d31951-2d4b-4e8d-a510-6283b01dc158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6825","nonce":2738517080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6827","nonce":2738517080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6831","nonce":2738517080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6829","nonce":2738517080}]},"public_addr":"172.21.15.203:6825/2738517080","cluster_addr":"172.21.15.203:6827/2738517080","heartbeat_back_addr":"172.21.15.203:6831/2738517080","heartbeat_front_addr":"172.21.15.203:6829/2738517080","state":["exists","up"]},{"osd":7,"uuid":"7119b8c0-fe96-402d-9e75-7b509d9d44d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6817","nonce":1315565058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6819","nonce":1315565058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6823","nonce":1315565058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6821","nonce":1315565058}]},"public_addr":"172.21.15.203:6817/1315565058","cluster_addr":"172.21.15.203:6819/1315565058","heartbeat_back_addr":"172.21.15.203:6823/1315565058","heartbeat_front_addr":"172.21.15.203:6821/1315565058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.121921+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.137494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.084814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.111593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.072430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.125066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.097867+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.057061+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:18:28.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.094+0000 7f3fd9df0640 1 -- 172.21.15.47:0/2843339923 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3fc006c8f0 msgr2=0x7f3fc006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:28.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.094+0000 7f3fd9df0640 1 --2- 172.21.15.47:0/2843339923 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3fc006c8f0 0x7f3fc006edb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4000fb0 tx=0x7f3fc4009040 comp rx=0 tx=0).stop 2024-04-26T23:18:28.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.094+0000 7f3fd9df0640 1 -- 172.21.15.47:0/2843339923 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3fd4146650 msgr2=0x7f3fd415b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:28.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.094+0000 7f3fd9df0640 1 --2- 172.21.15.47:0/2843339923 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3fd4146650 0x7f3fd415b1b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3fd40a0060 tx=0x7f3fcc010530 comp rx=0 tx=0).stop 2024-04-26T23:18:28.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.094+0000 7f3fd9df0640 1 -- 172.21.15.47:0/2843339923 shutdown_connections 2024-04-26T23:18:28.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.094+0000 7f3fd9df0640 1 --2- 172.21.15.47:0/2843339923 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3fc006c8f0 0x7f3fc006edb0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:28.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.094+0000 7f3fd9df0640 1 --2- 172.21.15.47:0/2843339923 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3fd4146650 0x7f3fd415b1b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:28.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.094+0000 7f3fd9df0640 1 --2- 172.21.15.47:0/2843339923 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fd415a6f0 0x7f3fd4152aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:28.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.094+0000 7f3fd9df0640 1 --2- 172.21.15.47:0/2843339923 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fbc004800 0x7f3fd4154270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:28.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.094+0000 7f3fd9df0640 1 -- 172.21.15.47:0/2843339923 >> 172.21.15.47:0/2843339923 conn(0x7f3fd40b39b0 msgr2=0x7f3fd40d19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:28.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.094+0000 7f3fd9df0640 1 -- 172.21.15.47:0/2843339923 shutdown_connections 2024-04-26T23:18:28.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.094+0000 7f3fd9df0640 1 -- 172.21.15.47:0/2843339923 wait complete. 2024-04-26T23:18:28.107 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:18:16.937492+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:18:28.107 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:18:28.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.208+0000 7f041b89c640 1 Processor -- start 2024-04-26T23:18:28.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.209+0000 7f041b89c640 1 -- start start 2024-04-26T23:18:28.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.209+0000 7f041b89c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f04140b40d0 0x7f04141625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:28.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.209+0000 7f041b89c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0414162b30 0x7f04140b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:28.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.209+0000 7f041b89c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04140d0a10 con 0x7f04140d1a00 2024-04-26T23:18:28.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.209+0000 7f041b89c640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f04140d0cf0 con 0x7f04140b40d0 2024-04-26T23:18:28.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.209+0000 7f041b89c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f04140d0fd0 con 0x7f0414162b30 2024-04-26T23:18:28.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.209+0000 7f0418e37640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f04140b40d0 0x7f04141625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:28.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.210+0000 7f0418e37640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f04140b40d0 0x7f04141625f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40246/0 (socket says 172.21.15.47:40246) 2024-04-26T23:18:28.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.210+0000 7f0418e37640 1 -- 172.21.15.47:0/3707610316 learned_addr learned my addr 172.21.15.47:0/3707610316 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:28.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.210+0000 7f0419e39640 1 --2- 172.21.15.47:0/3707610316 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0414162b30 0x7f04140b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:28.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.210+0000 7f0418e37640 1 -- 172.21.15.47:0/3707610316 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0414162b30 msgr2=0x7f04140b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:28.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.210+0000 7f0418e37640 1 --2- 172.21.15.47:0/3707610316 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0414162b30 0x7f04140b4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:28.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.210+0000 7f0418e37640 1 -- 172.21.15.47:0/3707610316 >> v1:172.21.15.203:6789/0 conn(0x7f04140d1a00 legacy=0x7f04140d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:28.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.210+0000 7f0418e37640 1 -- 172.21.15.47:0/3707610316 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0414162f80 con 0x7f04140b40d0 2024-04-26T23:18:28.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.211+0000 7f0418e37640 1 --2- 172.21.15.47:0/3707610316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04140b40d0 0x7f04141625f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0404009870 tx=0x7f040402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=addb8f4f1bcf995c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:28.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.211+0000 7f040bfff640 1 -- 172.21.15.47:0/3707610316 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0404037070 con 0x7f04140b40d0 2024-04-26T23:18:28.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.211+0000 7f040bfff640 1 -- 172.21.15.47:0/3707610316 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0404032440 con 0x7f04140b40d0 2024-04-26T23:18:28.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.211+0000 7f040bfff640 1 -- 172.21.15.47:0/3707610316 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0404032810 con 0x7f04140b40d0 2024-04-26T23:18:28.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.211+0000 7f041b89c640 1 -- 172.21.15.47:0/3707610316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04140b40d0 msgr2=0x7f04141625f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:28.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.211+0000 7f041b89c640 1 --2- 172.21.15.47:0/3707610316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04140b40d0 0x7f04141625f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0404009870 tx=0x7f040402a1c0 comp rx=0 tx=0).stop 2024-04-26T23:18:28.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.211+0000 7f041b89c640 1 -- 172.21.15.47:0/3707610316 shutdown_connections 2024-04-26T23:18:28.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.211+0000 7f041b89c640 1 --2- 172.21.15.47:0/3707610316 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0414162b30 0x7f04140b4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:28.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.211+0000 7f041b89c640 1 --2- 172.21.15.47:0/3707610316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04140b40d0 0x7f04141625f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:28.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.211+0000 7f041b89c640 1 -- 172.21.15.47:0/3707610316 >> 172.21.15.47:0/3707610316 conn(0x7f04140d4fd0 msgr2=0x7f04140d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:28.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.211+0000 7f041b89c640 1 -- 172.21.15.47:0/3707610316 shutdown_connections 2024-04-26T23:18:28.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.211+0000 7f041b89c640 1 -- 172.21.15.47:0/3707610316 wait complete. 2024-04-26T23:18:28.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.212+0000 7f041b89c640 1 Processor -- start 2024-04-26T23:18:28.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.213+0000 7f041b89c640 1 -- start start 2024-04-26T23:18:28.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.213+0000 7f041b89c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04140b40d0 0x7f0414156730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:28.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.213+0000 7f041b89c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04140d1a00 0x7f04140cb470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:28.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.213+0000 7f041b89c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0414162b30 0x7f04140ba770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:28.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.213+0000 7f041b89c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f04140d2670 con 0x7f04140d1a00 2024-04-26T23:18:28.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.213+0000 7f041b89c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f04140d24f0 con 0x7f04140b40d0 2024-04-26T23:18:28.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.213+0000 7f041b89c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f04140b5040 con 0x7f0414162b30 2024-04-26T23:18:28.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.213+0000 7f0418e37640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04140d1a00 0x7f04140cb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:28.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.213+0000 7f0418e37640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04140d1a00 0x7f04140cb470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40248/0 (socket says 172.21.15.47:40248) 2024-04-26T23:18:28.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.213+0000 7f0418e37640 1 -- 172.21.15.47:0/2947180077 learned_addr learned my addr 172.21.15.47:0/2947180077 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:28.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.213+0000 7f0419638640 1 --2- 172.21.15.47:0/2947180077 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04140b40d0 0x7f0414156730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:28.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.213+0000 7f0419e39640 1 --2- 172.21.15.47:0/2947180077 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0414162b30 0x7f04140ba770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:28.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.214+0000 7f0418e37640 1 -- 172.21.15.47:0/2947180077 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0414162b30 msgr2=0x7f04140ba770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:28.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.214+0000 7f0418e37640 1 --2- 172.21.15.47:0/2947180077 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0414162b30 0x7f04140ba770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:28.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.214+0000 7f0418e37640 1 -- 172.21.15.47:0/2947180077 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04140b40d0 msgr2=0x7f0414156730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:28.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.214+0000 7f0418e37640 1 --2- 172.21.15.47:0/2947180077 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04140b40d0 0x7f0414156730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:28.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.214+0000 7f0418e37640 1 -- 172.21.15.47:0/2947180077 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04140b95b0 con 0x7f04140d1a00 2024-04-26T23:18:28.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.214+0000 7f0419e39640 1 --2- 172.21.15.47:0/2947180077 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0414162b30 0x7f04140ba770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:18:28.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.214+0000 7f0418e37640 1 --2- 172.21.15.47:0/2947180077 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04140d1a00 0x7f04140cb470 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0404000c00 tx=0x7f04040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:28.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.214+0000 7f0409ffb640 1 -- 172.21.15.47:0/2947180077 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0404037040 con 0x7f04140d1a00 2024-04-26T23:18:28.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.214+0000 7f0409ffb640 1 -- 172.21.15.47:0/2947180077 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0404042070 con 0x7f04140d1a00 2024-04-26T23:18:28.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.214+0000 7f041b89c640 1 -- 172.21.15.47:0/2947180077 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04140b9c90 con 0x7f04140d1a00 2024-04-26T23:18:28.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.214+0000 7f0409ffb640 1 -- 172.21.15.47:0/2947180077 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0404030880 con 0x7f04140d1a00 2024-04-26T23:18:28.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.214+0000 7f041b89c640 1 -- 172.21.15.47:0/2947180077 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04140ba250 con 0x7f04140d1a00 2024-04-26T23:18:28.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.215+0000 7f041b89c640 1 -- 172.21.15.47:0/2947180077 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04140694f0 con 0x7f04140d1a00 2024-04-26T23:18:28.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.216+0000 7f0409ffb640 1 -- 172.21.15.47:0/2947180077 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f040402ad10 con 0x7f04140d1a00 2024-04-26T23:18:28.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.216+0000 7f0409ffb640 1 --2- 172.21.15.47:0/2947180077 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f03f806c8f0 0x7f03f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:28.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.216+0000 7f0409ffb640 1 -- 172.21.15.47:0/2947180077 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f04040abc40 con 0x7f04140d1a00 2024-04-26T23:18:28.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.216+0000 7f0419638640 1 --2- 172.21.15.47:0/2947180077 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f03f806c8f0 0x7f03f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:28.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.217+0000 7f0419638640 1 --2- 172.21.15.47:0/2947180077 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f03f806c8f0 0x7f03f806edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0410001780 tx=0x7f0410009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:28.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.221+0000 7f0409ffb640 1 -- 172.21.15.47:0/2947180077 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f040407a450 con 0x7f04140d1a00 2024-04-26T23:18:28.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.404+0000 7f041b89c640 1 -- 172.21.15.47:0/2947180077 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f04140b2ed0 con 0x7f04140d1a00 2024-04-26T23:18:28.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.404+0000 7f0409ffb640 1 -- 172.21.15.47:0/2947180077 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f0404079d50 con 0x7f04140d1a00 2024-04-26T23:18:28.406 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-26T23:18:28.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.406+0000 7f041b89c640 1 -- 172.21.15.47:0/2947180077 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f03f806c8f0 msgr2=0x7f03f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:28.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.406+0000 7f041b89c640 1 --2- 172.21.15.47:0/2947180077 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f03f806c8f0 0x7f03f806edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0410001780 tx=0x7f0410009040 comp rx=0 tx=0).stop 2024-04-26T23:18:28.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.406+0000 7f041b89c640 1 -- 172.21.15.47:0/2947180077 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04140d1a00 msgr2=0x7f04140cb470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:28.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.406+0000 7f041b89c640 1 --2- 172.21.15.47:0/2947180077 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04140d1a00 0x7f04140cb470 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0404000c00 tx=0x7f04040046c0 comp rx=0 tx=0).stop 2024-04-26T23:18:28.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.406+0000 7f041b89c640 1 -- 172.21.15.47:0/2947180077 shutdown_connections 2024-04-26T23:18:28.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.406+0000 7f041b89c640 1 --2- 172.21.15.47:0/2947180077 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f03f806c8f0 0x7f03f806edb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:28.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.406+0000 7f041b89c640 1 --2- 172.21.15.47:0/2947180077 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0414162b30 0x7f04140ba770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:28.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.406+0000 7f041b89c640 1 --2- 172.21.15.47:0/2947180077 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04140d1a00 0x7f04140cb470 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:28.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.406+0000 7f041b89c640 1 --2- 172.21.15.47:0/2947180077 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04140b40d0 0x7f0414156730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:28.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.406+0000 7f041b89c640 1 -- 172.21.15.47:0/2947180077 >> 172.21.15.47:0/2947180077 conn(0x7f04140d4fd0 msgr2=0x7f04140d1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:28.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.406+0000 7f041b89c640 1 -- 172.21.15.47:0/2947180077 shutdown_connections 2024-04-26T23:18:28.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.406+0000 7f041b89c640 1 -- 172.21.15.47:0/2947180077 wait complete. 2024-04-26T23:18:28.418 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T23:18:28.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.517+0000 7f21b359e640 1 Processor -- start 2024-04-26T23:18:28.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.517+0000 7f21b359e640 1 -- start start 2024-04-26T23:18:28.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.518+0000 7f21b359e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f21b40d17d0 0x7f21b40b6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:28.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.518+0000 7f21b359e640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21b40b66c0 con 0x7f21b40d2400 2024-04-26T23:18:28.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.518+0000 7f21b359e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21b415f9c0 con 0x7f21b40d0c80 2024-04-26T23:18:28.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.518+0000 7f21b359e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f21b40d37b0 con 0x7f21b40d17d0 2024-04-26T23:18:28.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.518+0000 7f21b1d9b640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f21b40d2400 0x7f21b40d1bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47224/0 (socket says 172.21.15.47:47224) 2024-04-26T23:18:28.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.518+0000 7f21b1d9b640 1 -- 172.21.15.47:0/527214061 learned_addr learned my addr 172.21.15.47:0/527214061 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:18:28.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.518+0000 7f21b259c640 1 --2- 172.21.15.47:0/527214061 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f21b40d17d0 0x7f21b40b6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:28.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.518+0000 7f21b159a640 1 -- 172.21.15.47:0/527214061 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2643046766 0 0) 0x7f219c001110 con 0x7f21b40d2400 2024-04-26T23:18:28.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.518+0000 7f21b159a640 1 -- 172.21.15.47:0/527214061 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f21a0003520 con 0x7f21b40d2400 2024-04-26T23:18:28.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.519+0000 7f21b159a640 1 -- 172.21.15.47:0/527214061 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 80791354 0 0) 0x7f21a0003520 con 0x7f21b40d2400 2024-04-26T23:18:28.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.519+0000 7f21b159a640 1 -- 172.21.15.47:0/527214061 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f21b40d17d0 msgr2=0x7f21b40b6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:28.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.519+0000 7f21b159a640 1 --2- 172.21.15.47:0/527214061 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f21b40d17d0 0x7f21b40b6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:28.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.519+0000 7f21b159a640 1 -- 172.21.15.47:0/527214061 >> v1:172.21.15.203:6789/0 conn(0x7f21b40d0c80 legacy=0x7f21b40b5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:28.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.519+0000 7f21b159a640 1 -- 172.21.15.47:0/527214061 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21b40d5730 con 0x7f21b40d2400 2024-04-26T23:18:28.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.519+0000 7f21b159a640 1 -- 172.21.15.47:0/527214061 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7f219c001840 con 0x7f21b40d2400 2024-04-26T23:18:28.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.519+0000 7f21b259c640 1 --2- 172.21.15.47:0/527214061 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f21b40d17d0 0x7f21b40b6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:18:28.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.519+0000 7f21b159a640 1 -- 172.21.15.47:0/527214061 >> v1:172.21.15.47:6789/0 conn(0x7f21b40d2400 legacy=0x7f21b40d1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:28.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.519+0000 7f21b159a640 1 --2- 172.21.15.47:0/527214061 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21a00039e0 0x7f21a0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:28.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.519+0000 7f21b159a640 1 --2- 172.21.15.47:0/527214061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21a0004390 0x7f21a0004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:28.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.519+0000 7f21b159a640 1 --2- 172.21.15.47:0/527214061 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f21b40d17d0 0x7f21a0005120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:28.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.519+0000 7f21b159a640 1 -- 172.21.15.47:0/527214061 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f21b40d37b0 con 0x7f21a0004390 2024-04-26T23:18:28.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.519+0000 7f21b159a640 1 -- 172.21.15.47:0/527214061 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f21a000ba10 con 0x7f21a00039e0 2024-04-26T23:18:28.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.519+0000 7f21b159a640 1 -- 172.21.15.47:0/527214061 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f21a000bc10 con 0x7f21b40d17d0 2024-04-26T23:18:28.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.519+0000 7f21b1d9b640 1 --2- 172.21.15.47:0/527214061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21a0004390 0x7f21a0004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:28.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.519+0000 7f21b259c640 1 --2- 172.21.15.47:0/527214061 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21a00039e0 0x7f21a0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:28.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.519+0000 7f21b2d9d640 1 --2- 172.21.15.47:0/527214061 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f21b40d17d0 0x7f21a0005120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:28.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.519+0000 7f21b1d9b640 1 -- 172.21.15.47:0/527214061 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f21b40d17d0 msgr2=0x7f21a0005120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:28.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.519+0000 7f21b1d9b640 1 --2- 172.21.15.47:0/527214061 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f21b40d17d0 0x7f21a0005120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:28.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.519+0000 7f21b1d9b640 1 -- 172.21.15.47:0/527214061 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21a00039e0 msgr2=0x7f21a0003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:28.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.519+0000 7f21b1d9b640 1 --2- 172.21.15.47:0/527214061 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21a00039e0 0x7f21a0003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:28.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.519+0000 7f21b1d9b640 1 -- 172.21.15.47:0/527214061 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21a0003520 con 0x7f21a0004390 2024-04-26T23:18:28.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.520+0000 7f21b1d9b640 1 --2- 172.21.15.47:0/527214061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21a0004390 0x7f21a0004860 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f219c00c3e0 tx=0x7f21a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19a79a582e6e430d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:28.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.520+0000 7f21b159a640 1 -- 172.21.15.47:0/527214061 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f219c00c0e0 con 0x7f21a0004390 2024-04-26T23:18:28.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.520+0000 7f21b159a640 1 -- 172.21.15.47:0/527214061 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f219c034070 con 0x7f21a0004390 2024-04-26T23:18:28.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.520+0000 7f21b159a640 1 -- 172.21.15.47:0/527214061 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f219c005d10 con 0x7f21a0004390 2024-04-26T23:18:28.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.521+0000 7f21b359e640 1 -- 172.21.15.47:0/527214061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21a0004390 msgr2=0x7f21a0004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:28.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.521+0000 7f21b359e640 1 --2- 172.21.15.47:0/527214061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21a0004390 0x7f21a0004860 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f219c00c3e0 tx=0x7f21a0005830 comp rx=0 tx=0).stop 2024-04-26T23:18:28.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.521+0000 7f21b2d9d640 1 -- 172.21.15.47:0/527214061 reap_dead start 2024-04-26T23:18:28.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.521+0000 7f21b359e640 1 -- 172.21.15.47:0/527214061 shutdown_connections 2024-04-26T23:18:28.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.521+0000 7f21b359e640 1 -- 172.21.15.47:0/527214061 >> 172.21.15.47:0/527214061 conn(0x7f21b409e210 msgr2=0x7f21b40d4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:28.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.521+0000 7f21b359e640 1 -- 172.21.15.47:0/527214061 shutdown_connections 2024-04-26T23:18:28.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.521+0000 7f21b359e640 1 -- 172.21.15.47:0/527214061 wait complete. 2024-04-26T23:18:28.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.522+0000 7f21b359e640 1 Processor -- start 2024-04-26T23:18:28.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.522+0000 7f21b359e640 1 -- start start 2024-04-26T23:18:28.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.523+0000 7f21b359e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f21a0004390 0x7f21b415bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:28.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.523+0000 7f21b359e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21b40ce4b0 0x7f21b41f59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:28.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.523+0000 7f21b359e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21b40b7ac0 0x7f21b40b7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:28.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.523+0000 7f21b359e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f21b40d3a00 con 0x7f21b40b7ac0 2024-04-26T23:18:28.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.523+0000 7f21b359e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f21b40d3d70 con 0x7f21b40ce4b0 2024-04-26T23:18:28.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.523+0000 7f21b359e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f21b40d1110 con 0x7f21a0004390 2024-04-26T23:18:28.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.523+0000 7f21b2d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21b40b7ac0 0x7f21b40b7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:28.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.523+0000 7f21b2d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21b40b7ac0 0x7f21b40b7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40256/0 (socket says 172.21.15.47:40256) 2024-04-26T23:18:28.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.523+0000 7f21b2d9d640 1 -- 172.21.15.47:0/1968053366 learned_addr learned my addr 172.21.15.47:0/1968053366 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:28.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.523+0000 7f21b259c640 1 --2- 172.21.15.47:0/1968053366 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f21a0004390 0x7f21b415bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:28.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.523+0000 7f21b1d9b640 1 --2- 172.21.15.47:0/1968053366 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21b40ce4b0 0x7f21b41f59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:28.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.523+0000 7f21b2d9d640 1 -- 172.21.15.47:0/1968053366 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f21a0004390 msgr2=0x7f21b415bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:28.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.523+0000 7f21b2d9d640 1 --2- 172.21.15.47:0/1968053366 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f21a0004390 0x7f21b415bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:28.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.523+0000 7f21b2d9d640 1 -- 172.21.15.47:0/1968053366 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21b40ce4b0 msgr2=0x7f21b41f59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:28.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.523+0000 7f21b2d9d640 1 --2- 172.21.15.47:0/1968053366 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21b40ce4b0 0x7f21b41f59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:28.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.523+0000 7f21b2d9d640 1 -- 172.21.15.47:0/1968053366 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21b40ce930 con 0x7f21b40b7ac0 2024-04-26T23:18:28.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.523+0000 7f21b259c640 1 --2- 172.21.15.47:0/1968053366 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f21a0004390 0x7f21b415bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:18:28.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.523+0000 7f21b2d9d640 1 --2- 172.21.15.47:0/1968053366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21b40b7ac0 0x7f21b40b7f40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f21b4163040 tx=0x7f21a400ff70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:28.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.523+0000 7f21aaffd640 1 -- 172.21.15.47:0/1968053366 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a4003b80 con 0x7f21b40b7ac0 2024-04-26T23:18:28.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.523+0000 7f21aaffd640 1 -- 172.21.15.47:0/1968053366 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21a4018070 con 0x7f21b40b7ac0 2024-04-26T23:18:28.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.523+0000 7f21b359e640 1 -- 172.21.15.47:0/1968053366 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21b40a2950 con 0x7f21b40b7ac0 2024-04-26T23:18:28.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.524+0000 7f21aaffd640 1 -- 172.21.15.47:0/1968053366 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a40050d0 con 0x7f21b40b7ac0 2024-04-26T23:18:28.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.524+0000 7f21b359e640 1 -- 172.21.15.47:0/1968053366 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21b40cd6f0 con 0x7f21b40b7ac0 2024-04-26T23:18:28.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.525+0000 7f21b359e640 1 -- 172.21.15.47:0/1968053366 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21b4069370 con 0x7f21b40b7ac0 2024-04-26T23:18:28.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.525+0000 7f21aaffd640 1 -- 172.21.15.47:0/1968053366 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f21a4005270 con 0x7f21b40b7ac0 2024-04-26T23:18:28.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.525+0000 7f21aaffd640 1 --2- 172.21.15.47:0/1968053366 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f219406c920 0x7f219406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:28.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.525+0000 7f21aaffd640 1 -- 172.21.15.47:0/1968053366 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f21a408b890 con 0x7f21b40b7ac0 2024-04-26T23:18:28.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.526+0000 7f21b259c640 1 --2- 172.21.15.47:0/1968053366 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f219406c920 0x7f219406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:28.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.526+0000 7f21b259c640 1 --2- 172.21.15.47:0/1968053366 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f219406c920 0x7f219406ede0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f21ac009560 tx=0x7f21ac008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:28.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.530+0000 7f21aaffd640 1 -- 172.21.15.47:0/1968053366 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f21a401a310 con 0x7f21b40b7ac0 2024-04-26T23:18:28.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.715+0000 7f21b359e640 1 -- 172.21.15.47:0/1968053366 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f21b40d5db0 con 0x7f21b40b7ac0 2024-04-26T23:18:28.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.715+0000 7f21aaffd640 1 -- 172.21.15.47:0/1968053366 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7f21a405a010 con 0x7f21b40b7ac0 2024-04-26T23:18:28.717 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:18:28.717 INFO:teuthology.orchestra.run.smithi047.stdout:[ 2024-04-26T23:18:28.717 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-26T23:18:28.717 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 1, 2024-04-26T23:18:28.717 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": ".mgr" 2024-04-26T23:18:28.717 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-26T23:18:28.717 INFO:teuthology.orchestra.run.smithi047.stdout:] 2024-04-26T23:18:28.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.717+0000 7f21b359e640 1 -- 172.21.15.47:0/1968053366 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f219406c920 msgr2=0x7f219406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:28.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.717+0000 7f21b359e640 1 --2- 172.21.15.47:0/1968053366 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f219406c920 0x7f219406ede0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f21ac009560 tx=0x7f21ac008040 comp rx=0 tx=0).stop 2024-04-26T23:18:28.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.717+0000 7f21b359e640 1 -- 172.21.15.47:0/1968053366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21b40b7ac0 msgr2=0x7f21b40b7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:28.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.717+0000 7f21b359e640 1 --2- 172.21.15.47:0/1968053366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21b40b7ac0 0x7f21b40b7f40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f21b4163040 tx=0x7f21a400ff70 comp rx=0 tx=0).stop 2024-04-26T23:18:28.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.717+0000 7f21b359e640 1 -- 172.21.15.47:0/1968053366 shutdown_connections 2024-04-26T23:18:28.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.717+0000 7f21b359e640 1 --2- 172.21.15.47:0/1968053366 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f219406c920 0x7f219406ede0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:28.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.717+0000 7f21b359e640 1 --2- 172.21.15.47:0/1968053366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21b40b7ac0 0x7f21b40b7f40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:28.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.717+0000 7f21b359e640 1 --2- 172.21.15.47:0/1968053366 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21b40ce4b0 0x7f21b41f59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:28.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.717+0000 7f21b359e640 1 --2- 172.21.15.47:0/1968053366 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f21a0004390 0x7f21b415bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:28.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.717+0000 7f21b359e640 1 -- 172.21.15.47:0/1968053366 >> 172.21.15.47:0/1968053366 conn(0x7f21b409e210 msgr2=0x7f21b40cf9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:28.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.717+0000 7f21b359e640 1 -- 172.21.15.47:0/1968053366 shutdown_connections 2024-04-26T23:18:28.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.717+0000 7f21b359e640 1 -- 172.21.15.47:0/1968053366 wait complete. 2024-04-26T23:18:28.729 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T23:18:28.729 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T23:18:28.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.832+0000 7f3ab8074640 1 Processor -- start 2024-04-26T23:18:28.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.832+0000 7f3ab8074640 1 -- start start 2024-04-26T23:18:28.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.833+0000 7f3ab8074640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ab0162c40 0x7f3ab00b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:28.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.833+0000 7f3ab8074640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3ab00b3f80 0x7f3ab00d3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:28.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.833+0000 7f3ab8074640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ab00d5880 con 0x7f3ab0162860 2024-04-26T23:18:28.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.833+0000 7f3ab8074640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f3ab00d5b60 con 0x7f3ab00b3f80 2024-04-26T23:18:28.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.833+0000 7f3ab8074640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3ab00d5e40 con 0x7f3ab0162c40 2024-04-26T23:18:28.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.833+0000 7f3ab5e10640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f3ab0162860 0x7f3ab00d2440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47238/0 (socket says 172.21.15.47:47238) 2024-04-26T23:18:28.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.833+0000 7f3ab5e10640 1 -- 172.21.15.47:0/1662953587 learned_addr learned my addr 172.21.15.47:0/1662953587 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:18:28.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.833+0000 7f3ab560f640 1 --2- 172.21.15.47:0/1662953587 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ab0162c40 0x7f3ab00b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:28.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.833+0000 7f3ab6611640 1 --2- 172.21.15.47:0/1662953587 >> v2:172.21.15.203:3300/0 conn(0x7f3ab00b3f80 0x7f3ab00d3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:28.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.833+0000 7f3ab4e0e640 1 -- 172.21.15.47:0/1662953587 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2970129960 0 0) 0x7f3ab00d5880 con 0x7f3ab0162860 2024-04-26T23:18:28.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.833+0000 7f3ab4e0e640 1 -- 172.21.15.47:0/1662953587 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3a9c003520 con 0x7f3ab0162860 2024-04-26T23:18:28.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.833+0000 7f3ab4e0e640 1 -- 172.21.15.47:0/1662953587 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 287107840 0 0) 0x7f3a9c003520 con 0x7f3ab0162860 2024-04-26T23:18:28.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.833+0000 7f3ab4e0e640 1 -- 172.21.15.47:0/1662953587 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ab0162c40 msgr2=0x7f3ab00b1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:28.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.833+0000 7f3ab4e0e640 1 --2- 172.21.15.47:0/1662953587 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ab0162c40 0x7f3ab00b1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:28.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.833+0000 7f3ab4e0e640 1 -- 172.21.15.47:0/1662953587 >> v2:172.21.15.203:3300/0 conn(0x7f3ab00b3f80 msgr2=0x7f3ab00d3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:28.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.833+0000 7f3ab4e0e640 1 --2- 172.21.15.47:0/1662953587 >> v2:172.21.15.203:3300/0 conn(0x7f3ab00b3f80 0x7f3ab00d3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:28.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.833+0000 7f3ab4e0e640 1 -- 172.21.15.47:0/1662953587 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ab00cfc70 con 0x7f3ab0162860 2024-04-26T23:18:28.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.833+0000 7f3ab4e0e640 1 -- 172.21.15.47:0/1662953587 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7f3aac001670 con 0x7f3ab0162860 2024-04-26T23:18:28.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.833+0000 7f3ab4e0e640 1 -- 172.21.15.47:0/1662953587 >> v1:172.21.15.47:6789/0 conn(0x7f3ab0162860 legacy=0x7f3ab00d2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:28.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.833+0000 7f3ab4e0e640 1 --2- 172.21.15.47:0/1662953587 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a9c0039b0 0x7f3a9c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:28.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.833+0000 7f3ab4e0e640 1 --2- 172.21.15.47:0/1662953587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a9c004800 0x7f3a9c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:28.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.833+0000 7f3ab4e0e640 1 --2- 172.21.15.47:0/1662953587 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ab0162c40 0x7f3a9c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:28.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.833+0000 7f3ab4e0e640 1 -- 172.21.15.47:0/1662953587 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab00d5b60 con 0x7f3a9c004800 2024-04-26T23:18:28.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.833+0000 7f3ab4e0e640 1 -- 172.21.15.47:0/1662953587 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab00d5e40 con 0x7f3a9c0039b0 2024-04-26T23:18:28.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.833+0000 7f3ab4e0e640 1 -- 172.21.15.47:0/1662953587 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3a9c00b9c0 con 0x7f3ab0162c40 2024-04-26T23:18:28.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.833+0000 7f3ab6611640 1 --2- 172.21.15.47:0/1662953587 >> v2:172.21.15.203:3300/0 conn(0x7f3ab00b3f80 0x7f3ab00d3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:18:28.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.833+0000 7f3ab560f640 1 --2- 172.21.15.47:0/1662953587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a9c004800 0x7f3a9c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:28.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.834+0000 7f3ab5e10640 1 --2- 172.21.15.47:0/1662953587 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a9c0039b0 0x7f3a9c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:28.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.834+0000 7f3ab6611640 1 --2- 172.21.15.47:0/1662953587 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ab0162c40 0x7f3a9c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:28.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.834+0000 7f3ab560f640 1 -- 172.21.15.47:0/1662953587 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ab0162c40 msgr2=0x7f3a9c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:28.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.834+0000 7f3ab560f640 1 --2- 172.21.15.47:0/1662953587 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ab0162c40 0x7f3a9c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:28.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.834+0000 7f3ab560f640 1 -- 172.21.15.47:0/1662953587 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a9c0039b0 msgr2=0x7f3a9c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:28.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.834+0000 7f3ab560f640 1 --2- 172.21.15.47:0/1662953587 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a9c0039b0 0x7f3a9c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:28.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.834+0000 7f3ab560f640 1 -- 172.21.15.47:0/1662953587 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a9c003520 con 0x7f3a9c004800 2024-04-26T23:18:28.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.834+0000 7f3ab560f640 1 --2- 172.21.15.47:0/1662953587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a9c004800 0x7f3a9c004c80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0009af0 tx=0x7f3a9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51cc659abd150a4f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:28.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.835+0000 7f3ab4e0e640 1 -- 172.21.15.47:0/1662953587 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0036070 con 0x7f3a9c004800 2024-04-26T23:18:28.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.835+0000 7f3ab4e0e640 1 -- 172.21.15.47:0/1662953587 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3aa0031400 con 0x7f3a9c004800 2024-04-26T23:18:28.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.835+0000 7f3ab4e0e640 1 -- 172.21.15.47:0/1662953587 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0009690 con 0x7f3a9c004800 2024-04-26T23:18:28.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.835+0000 7f3ab8074640 1 -- 172.21.15.47:0/1662953587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a9c004800 msgr2=0x7f3a9c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:28.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.835+0000 7f3ab8074640 1 --2- 172.21.15.47:0/1662953587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a9c004800 0x7f3a9c004c80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0009af0 tx=0x7f3a9c005830 comp rx=0 tx=0).stop 2024-04-26T23:18:28.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.835+0000 7f3ab6611640 1 -- 172.21.15.47:0/1662953587 reap_dead start 2024-04-26T23:18:28.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.835+0000 7f3ab8074640 1 -- 172.21.15.47:0/1662953587 shutdown_connections 2024-04-26T23:18:28.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.835+0000 7f3ab8074640 1 -- 172.21.15.47:0/1662953587 >> 172.21.15.47:0/1662953587 conn(0x7f3ab009e3e0 msgr2=0x7f3ab00d50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:28.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.835+0000 7f3ab8074640 1 -- 172.21.15.47:0/1662953587 shutdown_connections 2024-04-26T23:18:28.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.835+0000 7f3ab8074640 1 -- 172.21.15.47:0/1662953587 wait complete. 2024-04-26T23:18:28.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.836+0000 7f3ab8074640 1 Processor -- start 2024-04-26T23:18:28.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.836+0000 7f3ab8074640 1 -- start start 2024-04-26T23:18:28.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.837+0000 7f3ab8074640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a9c004800 0x7f3ab00c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:28.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.837+0000 7f3ab8074640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ab00c8db0 0x7f3ab0159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:28.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.837+0000 7f3ab8074640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ab00ce8a0 0x7f3ab00ced50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:28.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.837+0000 7f3ab8074640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab00b20d0 con 0x7f3ab00c8db0 2024-04-26T23:18:28.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.837+0000 7f3ab8074640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab00b1f50 con 0x7f3a9c004800 2024-04-26T23:18:28.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.837+0000 7f3ab8074640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3ab00d38d0 con 0x7f3ab00ce8a0 2024-04-26T23:18:28.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.837+0000 7f3ab560f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ab00c8db0 0x7f3ab0159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:28.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.837+0000 7f3ab560f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ab00c8db0 0x7f3ab0159620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40270/0 (socket says 172.21.15.47:40270) 2024-04-26T23:18:28.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.837+0000 7f3ab560f640 1 -- 172.21.15.47:0/1264572978 learned_addr learned my addr 172.21.15.47:0/1264572978 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:28.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.837+0000 7f3ab5e10640 1 --2- 172.21.15.47:0/1264572978 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a9c004800 0x7f3ab00c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:28.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.837+0000 7f3ab6611640 1 --2- 172.21.15.47:0/1264572978 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ab00ce8a0 0x7f3ab00ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:28.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.837+0000 7f3ab560f640 1 -- 172.21.15.47:0/1264572978 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ab00ce8a0 msgr2=0x7f3ab00ced50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:28.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.838+0000 7f3ab560f640 1 --2- 172.21.15.47:0/1264572978 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ab00ce8a0 0x7f3ab00ced50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:28.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.838+0000 7f3ab560f640 1 -- 172.21.15.47:0/1264572978 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a9c004800 msgr2=0x7f3ab00c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:28.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.838+0000 7f3ab560f640 1 --2- 172.21.15.47:0/1264572978 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a9c004800 0x7f3ab00c9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:28.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.838+0000 7f3ab560f640 1 -- 172.21.15.47:0/1264572978 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ab00cd860 con 0x7f3ab00c8db0 2024-04-26T23:18:28.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.838+0000 7f3ab560f640 1 --2- 172.21.15.47:0/1264572978 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ab00c8db0 0x7f3ab0159620 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0034040 tx=0x7f3aa0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:28.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.838+0000 7f3aa67fc640 1 -- 172.21.15.47:0/1264572978 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0041020 con 0x7f3ab00c8db0 2024-04-26T23:18:28.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.838+0000 7f3aa67fc640 1 -- 172.21.15.47:0/1264572978 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3aa003c470 con 0x7f3ab00c8db0 2024-04-26T23:18:28.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.838+0000 7f3ab8074640 1 -- 172.21.15.47:0/1264572978 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ab00cdae0 con 0x7f3ab00c8db0 2024-04-26T23:18:28.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.838+0000 7f3aa67fc640 1 -- 172.21.15.47:0/1264572978 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0036040 con 0x7f3ab00c8db0 2024-04-26T23:18:28.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.838+0000 7f3ab8074640 1 -- 172.21.15.47:0/1264572978 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ab00ce0a0 con 0x7f3ab00c8db0 2024-04-26T23:18:28.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.839+0000 7f3ab8074640 1 -- 172.21.15.47:0/1264572978 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ab0069600 con 0x7f3ab00c8db0 2024-04-26T23:18:28.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.839+0000 7f3aa67fc640 1 -- 172.21.15.47:0/1264572978 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3aa0044020 con 0x7f3ab00c8db0 2024-04-26T23:18:28.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.840+0000 7f3aa67fc640 1 --2- 172.21.15.47:0/1264572978 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3a9406c920 0x7f3a9406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:28.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.840+0000 7f3aa67fc640 1 -- 172.21.15.47:0/1264572978 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f3aa00ab4b0 con 0x7f3ab00c8db0 2024-04-26T23:18:28.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.840+0000 7f3ab5e10640 1 --2- 172.21.15.47:0/1264572978 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3a9406c920 0x7f3a9406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:28.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.841+0000 7f3ab5e10640 1 --2- 172.21.15.47:0/1264572978 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3a9406c920 0x7f3a9406ede0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3ab00b6090 tx=0x7f3aac0078f0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:28.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:28.844+0000 7f3aa67fc640 1 -- 172.21.15.47:0/1264572978 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3ab00cd860 con 0x7f3ab00c8db0 2024-04-26T23:18:29.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:29.029+0000 7f3ab8074640 1 -- 172.21.15.47:0/1264572978 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f3ab009e880 con 0x7f3ab00c8db0 2024-04-26T23:18:29.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:29.983+0000 7f3aa67fc640 1 -- 172.21.15.47:0/1264572978 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7f3aa0079670 con 0x7f3ab00c8db0 2024-04-26T23:18:29.984 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' created 2024-04-26T23:18:29.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:29.986+0000 7f3ab8074640 1 -- 172.21.15.47:0/1264572978 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3a9406c920 msgr2=0x7f3a9406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:29.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:29.986+0000 7f3ab8074640 1 --2- 172.21.15.47:0/1264572978 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3a9406c920 0x7f3a9406ede0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3ab00b6090 tx=0x7f3aac0078f0 comp rx=0 tx=0).stop 2024-04-26T23:18:29.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:29.986+0000 7f3ab8074640 1 -- 172.21.15.47:0/1264572978 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ab00c8db0 msgr2=0x7f3ab0159620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:29.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:29.986+0000 7f3ab8074640 1 --2- 172.21.15.47:0/1264572978 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ab00c8db0 0x7f3ab0159620 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0034040 tx=0x7f3aa0004730 comp rx=0 tx=0).stop 2024-04-26T23:18:29.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:29.986+0000 7f3ab8074640 1 -- 172.21.15.47:0/1264572978 shutdown_connections 2024-04-26T23:18:29.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:29.986+0000 7f3ab8074640 1 --2- 172.21.15.47:0/1264572978 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3a9406c920 0x7f3a9406ede0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:29.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:29.986+0000 7f3ab8074640 1 --2- 172.21.15.47:0/1264572978 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ab00ce8a0 0x7f3ab00ced50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:29.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:29.986+0000 7f3ab8074640 1 --2- 172.21.15.47:0/1264572978 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ab00c8db0 0x7f3ab0159620 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:29.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:29.986+0000 7f3ab8074640 1 --2- 172.21.15.47:0/1264572978 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a9c004800 0x7f3ab00c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:29.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:29.986+0000 7f3ab8074640 1 -- 172.21.15.47:0/1264572978 >> 172.21.15.47:0/1264572978 conn(0x7f3ab009e3e0 msgr2=0x7f3ab01608b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:29.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:29.987+0000 7f3ab8074640 1 -- 172.21.15.47:0/1264572978 shutdown_connections 2024-04-26T23:18:29.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:29.987+0000 7f3ab8074640 1 -- 172.21.15.47:0/1264572978 wait complete. 2024-04-26T23:18:30.005 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-26T23:18:30.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.117+0000 7fd37e5e2640 1 Processor -- start 2024-04-26T23:18:30.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.117+0000 7fd37e5e2640 1 -- start start 2024-04-26T23:18:30.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.118+0000 7fd37e5e2640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd3780d3ee0 0x7fd3780d1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:30.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.118+0000 7fd37e5e2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd3780c9350 0x7fd3780c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:30.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.118+0000 7fd37e5e2640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3780c8bc0 con 0x7fd378162a20 2024-04-26T23:18:30.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.118+0000 7fd37e5e2640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fd3780d07a0 con 0x7fd3780d3ee0 2024-04-26T23:18:30.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.118+0000 7fd37e5e2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd3780d09f0 con 0x7fd3780c9350 2024-04-26T23:18:30.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.118+0000 7fd3777fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd3780d3ee0 0x7fd3780d1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:30.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.118+0000 7fd3777fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd3780d3ee0 0x7fd3780d1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41448/0 (socket says 172.21.15.47:41448) 2024-04-26T23:18:30.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.118+0000 7fd3777fe640 1 -- 172.21.15.47:0/1618156163 learned_addr learned my addr 172.21.15.47:0/1618156163 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:30.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.118+0000 7fd37cb7f640 1 --2- 172.21.15.47:0/1618156163 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd3780c9350 0x7fd3780c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:30.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.119+0000 7fd3777fe640 1 -- 172.21.15.47:0/1618156163 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd3780c9350 msgr2=0x7fd3780c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:30.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.119+0000 7fd3777fe640 1 --2- 172.21.15.47:0/1618156163 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd3780c9350 0x7fd3780c97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:30.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.119+0000 7fd3777fe640 1 -- 172.21.15.47:0/1618156163 >> v1:172.21.15.203:6789/0 conn(0x7fd378162a20 legacy=0x7fd3780d1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:30.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.119+0000 7fd3777fe640 1 -- 172.21.15.47:0/1618156163 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3780d0cd0 con 0x7fd3780d3ee0 2024-04-26T23:18:30.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.119+0000 7fd37cb7f640 1 --2- 172.21.15.47:0/1618156163 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd3780c9350 0x7fd3780c97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:18:30.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.119+0000 7fd3777fe640 1 --2- 172.21.15.47:0/1618156163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3780d3ee0 0x7fd3780d1e10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd368009870 tx=0x7fd36802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b7125be950ac29 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:30.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.119+0000 7fd376ffd640 1 -- 172.21.15.47:0/1618156163 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd368037070 con 0x7fd3780d3ee0 2024-04-26T23:18:30.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.119+0000 7fd376ffd640 1 -- 172.21.15.47:0/1618156163 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd368032440 con 0x7fd3780d3ee0 2024-04-26T23:18:30.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.119+0000 7fd376ffd640 1 -- 172.21.15.47:0/1618156163 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd368032810 con 0x7fd3780d3ee0 2024-04-26T23:18:30.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.120+0000 7fd37e5e2640 1 -- 172.21.15.47:0/1618156163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3780d3ee0 msgr2=0x7fd3780d1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:30.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.120+0000 7fd37e5e2640 1 --2- 172.21.15.47:0/1618156163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3780d3ee0 0x7fd3780d1e10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd368009870 tx=0x7fd36802a1c0 comp rx=0 tx=0).stop 2024-04-26T23:18:30.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.120+0000 7fd37e5e2640 1 -- 172.21.15.47:0/1618156163 shutdown_connections 2024-04-26T23:18:30.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.120+0000 7fd37e5e2640 1 --2- 172.21.15.47:0/1618156163 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd3780c9350 0x7fd3780c97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:30.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.120+0000 7fd37e5e2640 1 --2- 172.21.15.47:0/1618156163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3780d3ee0 0x7fd3780d1e10 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:30.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.120+0000 7fd37e5e2640 1 -- 172.21.15.47:0/1618156163 >> 172.21.15.47:0/1618156163 conn(0x7fd3780d4fd0 msgr2=0x7fd3780d53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:30.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.120+0000 7fd37e5e2640 1 -- 172.21.15.47:0/1618156163 shutdown_connections 2024-04-26T23:18:30.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.120+0000 7fd37e5e2640 1 -- 172.21.15.47:0/1618156163 wait complete. 2024-04-26T23:18:30.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.121+0000 7fd37e5e2640 1 Processor -- start 2024-04-26T23:18:30.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.121+0000 7fd37e5e2640 1 -- start start 2024-04-26T23:18:30.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.122+0000 7fd37e5e2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3780c9350 0x7fd378159590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:30.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.122+0000 7fd37e5e2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd3780d3ee0 0x7fd3780b5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:30.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.122+0000 7fd37e5e2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd378162a20 0x7fd3780b85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:30.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.122+0000 7fd37e5e2640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd3780cbd90 con 0x7fd3780c9350 2024-04-26T23:18:30.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.122+0000 7fd37e5e2640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd3780d0fb0 con 0x7fd378162a20 2024-04-26T23:18:30.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.122+0000 7fd37e5e2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd378162e00 con 0x7fd3780d3ee0 2024-04-26T23:18:30.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.122+0000 7fd377fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3780c9350 0x7fd378159590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:30.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.122+0000 7fd377fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3780c9350 0x7fd378159590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41464/0 (socket says 172.21.15.47:41464) 2024-04-26T23:18:30.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.122+0000 7fd377fff640 1 -- 172.21.15.47:0/1001548526 learned_addr learned my addr 172.21.15.47:0/1001548526 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:30.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.122+0000 7fd3777fe640 1 --2- 172.21.15.47:0/1001548526 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd3780d3ee0 0x7fd3780b5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:30.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.122+0000 7fd37cb7f640 1 --2- 172.21.15.47:0/1001548526 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd378162a20 0x7fd3780b85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:30.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.123+0000 7fd377fff640 1 -- 172.21.15.47:0/1001548526 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd3780d3ee0 msgr2=0x7fd3780b5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:30.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.123+0000 7fd377fff640 1 --2- 172.21.15.47:0/1001548526 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd3780d3ee0 0x7fd3780b5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:30.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.123+0000 7fd377fff640 1 -- 172.21.15.47:0/1001548526 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd378162a20 msgr2=0x7fd3780b85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:30.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.123+0000 7fd377fff640 1 --2- 172.21.15.47:0/1001548526 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd378162a20 0x7fd3780b85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:30.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.123+0000 7fd377fff640 1 -- 172.21.15.47:0/1001548526 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3780b6a50 con 0x7fd3780c9350 2024-04-26T23:18:30.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.123+0000 7fd37cb7f640 1 --2- 172.21.15.47:0/1001548526 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd378162a20 0x7fd3780b85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:18:30.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.123+0000 7fd377fff640 1 --2- 172.21.15.47:0/1001548526 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3780c9350 0x7fd378159590 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd36000c400 tx=0x7fd36000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:30.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.123+0000 7fd374ff9640 1 -- 172.21.15.47:0/1001548526 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3600052d0 con 0x7fd3780c9350 2024-04-26T23:18:30.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.123+0000 7fd374ff9640 1 -- 172.21.15.47:0/1001548526 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd360005490 con 0x7fd3780c9350 2024-04-26T23:18:30.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.123+0000 7fd37e5e2640 1 -- 172.21.15.47:0/1001548526 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3780b6d30 con 0x7fd3780c9350 2024-04-26T23:18:30.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.123+0000 7fd374ff9640 1 -- 172.21.15.47:0/1001548526 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd360007260 con 0x7fd3780c9350 2024-04-26T23:18:30.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.123+0000 7fd37e5e2640 1 -- 172.21.15.47:0/1001548526 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3780b72f0 con 0x7fd3780c9350 2024-04-26T23:18:30.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.124+0000 7fd37e5e2640 1 -- 172.21.15.47:0/1001548526 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3780694f0 con 0x7fd3780c9350 2024-04-26T23:18:30.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.124+0000 7fd374ff9640 1 -- 172.21.15.47:0/1001548526 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd360007400 con 0x7fd3780c9350 2024-04-26T23:18:30.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.125+0000 7fd374ff9640 1 --2- 172.21.15.47:0/1001548526 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd35806c8f0 0x7fd35806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:30.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.125+0000 7fd374ff9640 1 -- 172.21.15.47:0/1001548526 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7fd36008ae50 con 0x7fd3780c9350 2024-04-26T23:18:30.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.125+0000 7fd3777fe640 1 --2- 172.21.15.47:0/1001548526 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd35806c8f0 0x7fd35806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:30.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.126+0000 7fd3777fe640 1 --2- 172.21.15.47:0/1001548526 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd35806c8f0 0x7fd35806edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd368034000 tx=0x7fd36802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:30.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.130+0000 7fd374ff9640 1 -- 172.21.15.47:0/1001548526 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd360058e10 con 0x7fd3780c9350 2024-04-26T23:18:30.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.313+0000 7fd37e5e2640 1 -- 172.21.15.47:0/1001548526 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fd3780d23f0 con 0x7fd3780c9350 2024-04-26T23:18:30.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.989+0000 7fd374ff9640 1 -- 172.21.15.47:0/1001548526 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v18) ==== 171+0+0 (secure 0 0 0) 0x7fd36005d110 con 0x7fd3780c9350 2024-04-26T23:18:30.990 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' created 2024-04-26T23:18:30.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.992+0000 7fd37e5e2640 1 -- 172.21.15.47:0/1001548526 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd35806c8f0 msgr2=0x7fd35806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:30.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.992+0000 7fd37e5e2640 1 --2- 172.21.15.47:0/1001548526 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd35806c8f0 0x7fd35806edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd368034000 tx=0x7fd36802e000 comp rx=0 tx=0).stop 2024-04-26T23:18:30.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.992+0000 7fd37e5e2640 1 -- 172.21.15.47:0/1001548526 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3780c9350 msgr2=0x7fd378159590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:30.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.992+0000 7fd37e5e2640 1 --2- 172.21.15.47:0/1001548526 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3780c9350 0x7fd378159590 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd36000c400 tx=0x7fd36000c8c0 comp rx=0 tx=0).stop 2024-04-26T23:18:30.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.992+0000 7fd37e5e2640 1 -- 172.21.15.47:0/1001548526 shutdown_connections 2024-04-26T23:18:30.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.992+0000 7fd37e5e2640 1 --2- 172.21.15.47:0/1001548526 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd35806c8f0 0x7fd35806edb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:30.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.992+0000 7fd37e5e2640 1 --2- 172.21.15.47:0/1001548526 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd378162a20 0x7fd3780b85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:30.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.992+0000 7fd37e5e2640 1 --2- 172.21.15.47:0/1001548526 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd3780d3ee0 0x7fd3780b5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:30.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.992+0000 7fd37e5e2640 1 --2- 172.21.15.47:0/1001548526 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3780c9350 0x7fd378159590 secure :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd36000c400 tx=0x7fd36000c8c0 comp rx=0 tx=0).stop 2024-04-26T23:18:30.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.992+0000 7fd37e5e2640 1 -- 172.21.15.47:0/1001548526 >> 172.21.15.47:0/1001548526 conn(0x7fd3780d4fd0 msgr2=0x7fd37806cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:30.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.993+0000 7fd37e5e2640 1 -- 172.21.15.47:0/1001548526 shutdown_connections 2024-04-26T23:18:30.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:30.993+0000 7fd37e5e2640 1 -- 172.21.15.47:0/1001548526 wait complete. 2024-04-26T23:18:31.023 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T23:18:31.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.130+0000 7f546ff9d640 1 Processor -- start 2024-04-26T23:18:31.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.130+0000 7f546ff9d640 1 -- start start 2024-04-26T23:18:31.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.131+0000 7f546ff9d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f54680d1a00 0x7f54680d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:31.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.131+0000 7f546ff9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f54680d2320 0x7f54681625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:31.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.131+0000 7f546ff9d640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54680d0a10 con 0x7f54680b40d0 2024-04-26T23:18:31.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.131+0000 7f546ff9d640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f54680d0cf0 con 0x7f54680d2320 2024-04-26T23:18:31.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.131+0000 7f546ff9d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f54680d0fd0 con 0x7f54680d1a00 2024-04-26T23:18:31.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.131+0000 7f546d538640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f54680d2320 0x7f54681625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:31.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.131+0000 7f546d538640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f54680d2320 0x7f54681625f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41470/0 (socket says 172.21.15.47:41470) 2024-04-26T23:18:31.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.131+0000 7f546d538640 1 -- 172.21.15.47:0/3778802721 learned_addr learned my addr 172.21.15.47:0/3778802721 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:31.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.131+0000 7f546dd39640 1 --2- 172.21.15.47:0/3778802721 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f54680d1a00 0x7f54680d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:31.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.131+0000 7f546d538640 1 -- 172.21.15.47:0/3778802721 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f54680d1a00 msgr2=0x7f54680d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:31.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.131+0000 7f546d538640 1 --2- 172.21.15.47:0/3778802721 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f54680d1a00 0x7f54680d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:31.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.131+0000 7f546d538640 1 -- 172.21.15.47:0/3778802721 >> v1:172.21.15.203:6789/0 conn(0x7f54680b40d0 legacy=0x7f54680d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:31.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.132+0000 7f546d538640 1 -- 172.21.15.47:0/3778802721 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54680d40a0 con 0x7f54680d2320 2024-04-26T23:18:31.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.132+0000 7f546dd39640 1 --2- 172.21.15.47:0/3778802721 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f54680d1a00 0x7f54680d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:18:31.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.132+0000 7f546d538640 1 --2- 172.21.15.47:0/3778802721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54680d2320 0x7f54681625f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f545c009870 tx=0x7f545c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63c0dcc986bc9d50 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:31.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.132+0000 7f546cd37640 1 -- 172.21.15.47:0/3778802721 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545c037070 con 0x7f54680d2320 2024-04-26T23:18:31.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.132+0000 7f546cd37640 1 -- 172.21.15.47:0/3778802721 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f545c032440 con 0x7f54680d2320 2024-04-26T23:18:31.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.132+0000 7f546cd37640 1 -- 172.21.15.47:0/3778802721 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545c032810 con 0x7f54680d2320 2024-04-26T23:18:31.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.133+0000 7f546ff9d640 1 -- 172.21.15.47:0/3778802721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54680d2320 msgr2=0x7f54681625f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:31.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.133+0000 7f546ff9d640 1 --2- 172.21.15.47:0/3778802721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54680d2320 0x7f54681625f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f545c009870 tx=0x7f545c02a180 comp rx=0 tx=0).stop 2024-04-26T23:18:31.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.133+0000 7f546ff9d640 1 -- 172.21.15.47:0/3778802721 shutdown_connections 2024-04-26T23:18:31.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.133+0000 7f546ff9d640 1 --2- 172.21.15.47:0/3778802721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54680d2320 0x7f54681625f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:31.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.133+0000 7f546ff9d640 1 --2- 172.21.15.47:0/3778802721 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f54680d1a00 0x7f54680d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:31.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.133+0000 7f546ff9d640 1 -- 172.21.15.47:0/3778802721 >> 172.21.15.47:0/3778802721 conn(0x7f54680d4fd0 msgr2=0x7f54680d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:31.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.133+0000 7f546ff9d640 1 -- 172.21.15.47:0/3778802721 shutdown_connections 2024-04-26T23:18:31.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.133+0000 7f546ff9d640 1 -- 172.21.15.47:0/3778802721 wait complete. 2024-04-26T23:18:31.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.134+0000 7f546ff9d640 1 Processor -- start 2024-04-26T23:18:31.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.134+0000 7f546ff9d640 1 -- start start 2024-04-26T23:18:31.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.135+0000 7f546ff9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54680b40d0 0x7f54680caee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:31.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.135+0000 7f546ff9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54680d1a00 0x7f54680b27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:31.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.135+0000 7f546ff9d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f54680d2320 0x7f5468153f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:31.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.135+0000 7f546ff9d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5468162b30 con 0x7f54680d1a00 2024-04-26T23:18:31.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.135+0000 7f546ff9d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f54680b5d60 con 0x7f54680b40d0 2024-04-26T23:18:31.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.135+0000 7f546ff9d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f5468162cb0 con 0x7f54680d2320 2024-04-26T23:18:31.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.135+0000 7f546d538640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54680d1a00 0x7f54680b27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:31.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.135+0000 7f546d538640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54680d1a00 0x7f54680b27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41476/0 (socket says 172.21.15.47:41476) 2024-04-26T23:18:31.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.135+0000 7f546d538640 1 -- 172.21.15.47:0/1052808075 learned_addr learned my addr 172.21.15.47:0/1052808075 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:31.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.135+0000 7f546dd39640 1 --2- 172.21.15.47:0/1052808075 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54680b40d0 0x7f54680caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:31.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.135+0000 7f546e53a640 1 --2- 172.21.15.47:0/1052808075 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f54680d2320 0x7f5468153f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:31.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.135+0000 7f546d538640 1 -- 172.21.15.47:0/1052808075 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f54680d2320 msgr2=0x7f5468153f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:31.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.135+0000 7f546d538640 1 --2- 172.21.15.47:0/1052808075 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f54680d2320 0x7f5468153f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:31.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.135+0000 7f546d538640 1 -- 172.21.15.47:0/1052808075 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54680b40d0 msgr2=0x7f54680caee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:31.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.135+0000 7f546d538640 1 --2- 172.21.15.47:0/1052808075 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54680b40d0 0x7f54680caee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:31.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.135+0000 7f546d538640 1 -- 172.21.15.47:0/1052808075 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54680b80f0 con 0x7f54680d1a00 2024-04-26T23:18:31.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.136+0000 7f546d538640 1 --2- 172.21.15.47:0/1052808075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54680d1a00 0x7f54680b27e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f545c0090b0 tx=0x7f545c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:31.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.136+0000 7f54667fc640 1 -- 172.21.15.47:0/1052808075 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545c041020 con 0x7f54680d1a00 2024-04-26T23:18:31.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.136+0000 7f54667fc640 1 -- 172.21.15.47:0/1052808075 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f545c030990 con 0x7f54680d1a00 2024-04-26T23:18:31.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.136+0000 7f546ff9d640 1 -- 172.21.15.47:0/1052808075 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54680b83d0 con 0x7f54680d1a00 2024-04-26T23:18:31.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.136+0000 7f54667fc640 1 -- 172.21.15.47:0/1052808075 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545c037040 con 0x7f54680d1a00 2024-04-26T23:18:31.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.136+0000 7f546ff9d640 1 -- 172.21.15.47:0/1052808075 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54680b8990 con 0x7f54680d1a00 2024-04-26T23:18:31.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.137+0000 7f546ff9d640 1 -- 172.21.15.47:0/1052808075 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54680694f0 con 0x7f54680d1a00 2024-04-26T23:18:31.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.137+0000 7f54667fc640 1 -- 172.21.15.47:0/1052808075 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f545c044020 con 0x7f54680d1a00 2024-04-26T23:18:31.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.138+0000 7f54667fc640 1 --2- 172.21.15.47:0/1052808075 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f545406c920 0x7f545406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:31.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.138+0000 7f54667fc640 1 -- 172.21.15.47:0/1052808075 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7f545c0ac310 con 0x7f54680d1a00 2024-04-26T23:18:31.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.138+0000 7f546dd39640 1 --2- 172.21.15.47:0/1052808075 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f545406c920 0x7f545406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:31.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.138+0000 7f546dd39640 1 --2- 172.21.15.47:0/1052808075 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f545406c920 0x7f545406ede0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f546000ab60 tx=0x7f5460008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:31.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.142+0000 7f54667fc640 1 -- 172.21.15.47:0/1052808075 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f545c04b030 con 0x7f54680d1a00 2024-04-26T23:18:31.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.366+0000 7f546ff9d640 1 -- 172.21.15.47:0/1052808075 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f54680cbbf0 con 0x7f54680d1a00 2024-04-26T23:18:31.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.990+0000 7f54667fc640 1 -- 172.21.15.47:0/1052808075 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f545c07a1e0 con 0x7f54680d1a00 2024-04-26T23:18:31.991 INFO:teuthology.orchestra.run.smithi047.stderr:new fs with metadata pool 2 and data pool 3 2024-04-26T23:18:31.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.995+0000 7f546ff9d640 1 -- 172.21.15.47:0/1052808075 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f545406c920 msgr2=0x7f545406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:31.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.995+0000 7f546ff9d640 1 --2- 172.21.15.47:0/1052808075 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f545406c920 0x7f545406ede0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f546000ab60 tx=0x7f5460008040 comp rx=0 tx=0).stop 2024-04-26T23:18:31.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.995+0000 7f546ff9d640 1 -- 172.21.15.47:0/1052808075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54680d1a00 msgr2=0x7f54680b27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:31.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.995+0000 7f546ff9d640 1 --2- 172.21.15.47:0/1052808075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54680d1a00 0x7f54680b27e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f545c0090b0 tx=0x7f545c0046c0 comp rx=0 tx=0).stop 2024-04-26T23:18:31.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.996+0000 7f546ff9d640 1 -- 172.21.15.47:0/1052808075 shutdown_connections 2024-04-26T23:18:31.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.997+0000 7f546ff9d640 1 --2- 172.21.15.47:0/1052808075 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f545406c920 0x7f545406ede0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:31.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.997+0000 7f546ff9d640 1 --2- 172.21.15.47:0/1052808075 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f54680d2320 0x7f5468153f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:31.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.997+0000 7f546ff9d640 1 --2- 172.21.15.47:0/1052808075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54680d1a00 0x7f54680b27e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:31.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.997+0000 7f546ff9d640 1 --2- 172.21.15.47:0/1052808075 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54680b40d0 0x7f54680caee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:31.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.997+0000 7f546ff9d640 1 -- 172.21.15.47:0/1052808075 >> 172.21.15.47:0/1052808075 conn(0x7f54680d4fd0 msgr2=0x7f546806da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:31.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:31.998+0000 7f546ff9d640 1 -- 172.21.15.47:0/1052808075 shutdown_connections 2024-04-26T23:18:32.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.000+0000 7f546ff9d640 1 -- 172.21.15.47:0/1052808075 wait complete. 2024-04-26T23:18:32.031 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-26T23:18:32.031 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-26T23:18:32.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.140+0000 7f3a792f1640 1 Processor -- start 2024-04-26T23:18:32.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.140+0000 7f3a792f1640 1 -- start start 2024-04-26T23:18:32.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.140+0000 7f3a792f1640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3a7415f340 0x7f3a740d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:32.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.140+0000 7f3a792f1640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a74163160 con 0x7f3a7415fe30 2024-04-26T23:18:32.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.141+0000 7f3a792f1640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a74163440 con 0x7f3a741606c0 2024-04-26T23:18:32.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.141+0000 7f3a792f1640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f3a74162f10 con 0x7f3a7415f340 2024-04-26T23:18:32.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.141+0000 7f3a737fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f3a7415fe30 0x7f3a740cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35576/0 (socket says 172.21.15.47:35576) 2024-04-26T23:18:32.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.141+0000 7f3a737fe640 1 -- 172.21.15.47:0/4148563854 learned_addr learned my addr 172.21.15.47:0/4148563854 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:18:32.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.141+0000 7f3a72ffd640 1 --2- 172.21.15.47:0/4148563854 >> v2:172.21.15.203:3300/0 conn(0x7f3a7415f340 0x7f3a740d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:32.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.141+0000 7f3a727fc640 1 -- 172.21.15.47:0/4148563854 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1344271599 0 0) 0x7f3a74163160 con 0x7f3a7415fe30 2024-04-26T23:18:32.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.141+0000 7f3a727fc640 1 -- 172.21.15.47:0/4148563854 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3a60003520 con 0x7f3a7415fe30 2024-04-26T23:18:32.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.141+0000 7f3a727fc640 1 -- 172.21.15.47:0/4148563854 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1680458577 0 0) 0x7f3a60003520 con 0x7f3a7415fe30 2024-04-26T23:18:32.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.141+0000 7f3a727fc640 1 -- 172.21.15.47:0/4148563854 >> v2:172.21.15.203:3300/0 conn(0x7f3a7415f340 msgr2=0x7f3a740d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:32.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.141+0000 7f3a727fc640 1 --2- 172.21.15.47:0/4148563854 >> v2:172.21.15.203:3300/0 conn(0x7f3a7415f340 0x7f3a740d2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:32.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.141+0000 7f3a727fc640 1 -- 172.21.15.47:0/4148563854 >> v1:172.21.15.203:6789/0 conn(0x7f3a741606c0 legacy=0x7f3a7415f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:32.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.141+0000 7f3a727fc640 1 -- 172.21.15.47:0/4148563854 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a74163720 con 0x7f3a7415fe30 2024-04-26T23:18:32.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.141+0000 7f3a727fc640 1 -- 172.21.15.47:0/4148563854 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7f3a6c0019f0 con 0x7f3a7415fe30 2024-04-26T23:18:32.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.141+0000 7f3a727fc640 1 -- 172.21.15.47:0/4148563854 >> v1:172.21.15.47:6789/0 conn(0x7f3a7415fe30 legacy=0x7f3a740cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:32.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.141+0000 7f3a727fc640 1 --2- 172.21.15.47:0/4148563854 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a600039b0 0x7f3a60003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:32.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.141+0000 7f3a727fc640 1 --2- 172.21.15.47:0/4148563854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a60004800 0x7f3a60004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:32.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.141+0000 7f3a727fc640 1 --2- 172.21.15.47:0/4148563854 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3a600051c0 0x7f3a6000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:32.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.141+0000 7f3a727fc640 1 -- 172.21.15.47:0/4148563854 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3a74162f10 con 0x7f3a600039b0 2024-04-26T23:18:32.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.141+0000 7f3a727fc640 1 -- 172.21.15.47:0/4148563854 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6000c060 con 0x7f3a60004800 2024-04-26T23:18:32.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.141+0000 7f3a727fc640 1 -- 172.21.15.47:0/4148563854 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3a6000c260 con 0x7f3a600051c0 2024-04-26T23:18:32.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.141+0000 7f3a72ffd640 1 --2- 172.21.15.47:0/4148563854 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a600039b0 0x7f3a60003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:32.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.141+0000 7f3a6a7fc640 1 --2- 172.21.15.47:0/4148563854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a60004800 0x7f3a60004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:32.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.142+0000 7f3a737fe640 1 --2- 172.21.15.47:0/4148563854 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3a600051c0 0x7f3a6000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:32.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.142+0000 7f3a72ffd640 1 -- 172.21.15.47:0/4148563854 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3a600051c0 msgr2=0x7f3a6000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:32.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.142+0000 7f3a72ffd640 1 --2- 172.21.15.47:0/4148563854 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3a600051c0 0x7f3a6000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:32.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.142+0000 7f3a72ffd640 1 -- 172.21.15.47:0/4148563854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a60004800 msgr2=0x7f3a60004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:32.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.142+0000 7f3a72ffd640 1 --2- 172.21.15.47:0/4148563854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a60004800 0x7f3a60004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:32.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.142+0000 7f3a72ffd640 1 -- 172.21.15.47:0/4148563854 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a60003520 con 0x7f3a600039b0 2024-04-26T23:18:32.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.142+0000 7f3a737fe640 1 -- 172.21.15.47:0/4148563854 reap_dead start 2024-04-26T23:18:32.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.142+0000 7f3a72ffd640 1 --2- 172.21.15.47:0/4148563854 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a600039b0 0x7f3a60003dd0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3a64009ac0 tx=0x7f3a60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72ed5f5eb06caaf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:32.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.143+0000 7f3a727fc640 1 -- 172.21.15.47:0/4148563854 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a64036070 con 0x7f3a600039b0 2024-04-26T23:18:32.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.143+0000 7f3a727fc640 1 -- 172.21.15.47:0/4148563854 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a64031400 con 0x7f3a600039b0 2024-04-26T23:18:32.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.143+0000 7f3a727fc640 1 -- 172.21.15.47:0/4148563854 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a64009690 con 0x7f3a600039b0 2024-04-26T23:18:32.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.143+0000 7f3a792f1640 1 -- 172.21.15.47:0/4148563854 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a600039b0 msgr2=0x7f3a60003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:32.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.143+0000 7f3a792f1640 1 --2- 172.21.15.47:0/4148563854 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a600039b0 0x7f3a60003dd0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3a64009ac0 tx=0x7f3a60005830 comp rx=0 tx=0).stop 2024-04-26T23:18:32.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.143+0000 7f3a792f1640 1 -- 172.21.15.47:0/4148563854 shutdown_connections 2024-04-26T23:18:32.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.143+0000 7f3a792f1640 1 --2- 172.21.15.47:0/4148563854 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a600039b0 0x7f3a60003dd0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:32.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.143+0000 7f3a792f1640 1 -- 172.21.15.47:0/4148563854 >> 172.21.15.47:0/4148563854 conn(0x7f3a7409ba50 msgr2=0x7f3a7409be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:32.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.143+0000 7f3a792f1640 1 -- 172.21.15.47:0/4148563854 shutdown_connections 2024-04-26T23:18:32.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.143+0000 7f3a792f1640 1 -- 172.21.15.47:0/4148563854 wait complete. 2024-04-26T23:18:32.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.144+0000 7f3a792f1640 1 Processor -- start 2024-04-26T23:18:32.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.144+0000 7f3a792f1640 1 -- start start 2024-04-26T23:18:32.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.145+0000 7f3a792f1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3a600039b0 0x7f3a740cec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:32.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.145+0000 7f3a792f1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a7415bd80 0x7f3a74159960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:32.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.145+0000 7f3a792f1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a74159ea0 0x7f3a741f55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:32.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.145+0000 7f3a792f1640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3a74160230 con 0x7f3a74159ea0 2024-04-26T23:18:32.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.145+0000 7f3a792f1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3a74163d70 con 0x7f3a7415bd80 2024-04-26T23:18:32.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.145+0000 7f3a792f1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3a7415f720 con 0x7f3a600039b0 2024-04-26T23:18:32.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.145+0000 7f3a737fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a74159ea0 0x7f3a741f55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:32.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.145+0000 7f3a737fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a74159ea0 0x7f3a741f55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41496/0 (socket says 172.21.15.47:41496) 2024-04-26T23:18:32.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.145+0000 7f3a737fe640 1 -- 172.21.15.47:0/2752049310 learned_addr learned my addr 172.21.15.47:0/2752049310 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:32.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.145+0000 7f3a72ffd640 1 --2- 172.21.15.47:0/2752049310 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3a600039b0 0x7f3a740cec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:32.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.145+0000 7f3a737fe640 1 -- 172.21.15.47:0/2752049310 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3a600039b0 msgr2=0x7f3a740cec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:32.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.145+0000 7f3a6a7fc640 1 --2- 172.21.15.47:0/2752049310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a7415bd80 0x7f3a74159960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:32.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.146+0000 7f3a737fe640 1 --2- 172.21.15.47:0/2752049310 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3a600039b0 0x7f3a740cec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:32.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.146+0000 7f3a737fe640 1 -- 172.21.15.47:0/2752049310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a7415bd80 msgr2=0x7f3a74159960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:32.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.146+0000 7f3a737fe640 1 --2- 172.21.15.47:0/2752049310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a7415bd80 0x7f3a74159960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:32.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.146+0000 7f3a737fe640 1 -- 172.21.15.47:0/2752049310 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a7415afe0 con 0x7f3a74159ea0 2024-04-26T23:18:32.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.146+0000 7f3a737fe640 1 --2- 172.21.15.47:0/2752049310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a74159ea0 0x7f3a741f55e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c0012e0 tx=0x7f3a6c007b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:32.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.146+0000 7f3a6bfff640 1 -- 172.21.15.47:0/2752049310 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c00ac80 con 0x7f3a74159ea0 2024-04-26T23:18:32.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.146+0000 7f3a6bfff640 1 -- 172.21.15.47:0/2752049310 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a6c019070 con 0x7f3a74159ea0 2024-04-26T23:18:32.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.146+0000 7f3a792f1640 1 -- 172.21.15.47:0/2752049310 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a7415ce40 con 0x7f3a74159ea0 2024-04-26T23:18:32.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.146+0000 7f3a6bfff640 1 -- 172.21.15.47:0/2752049310 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c013610 con 0x7f3a74159ea0 2024-04-26T23:18:32.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.146+0000 7f3a792f1640 1 -- 172.21.15.47:0/2752049310 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a7415b580 con 0x7f3a74159ea0 2024-04-26T23:18:32.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.147+0000 7f3a792f1640 1 -- 172.21.15.47:0/2752049310 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a74069600 con 0x7f3a74159ea0 2024-04-26T23:18:32.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.147+0000 7f3a6bfff640 1 -- 172.21.15.47:0/2752049310 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3a6c0137b0 con 0x7f3a74159ea0 2024-04-26T23:18:32.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.148+0000 7f3a6bfff640 1 --2- 172.21.15.47:0/2752049310 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3a5406c920 0x7f3a5406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:32.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.148+0000 7f3a6bfff640 1 -- 172.21.15.47:0/2752049310 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f3a6c08c110 con 0x7f3a74159ea0 2024-04-26T23:18:32.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.148+0000 7f3a72ffd640 1 --2- 172.21.15.47:0/2752049310 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3a5406c920 0x7f3a5406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:32.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.149+0000 7f3a72ffd640 1 --2- 172.21.15.47:0/2752049310 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3a5406c920 0x7f3a5406ede0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3a64033000 tx=0x7f3a64008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:32.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.152+0000 7f3a6bfff640 1 -- 172.21.15.47:0/2752049310 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3a6c05a540 con 0x7f3a74159ea0 2024-04-26T23:18:32.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.344+0000 7f3a792f1640 1 -- 172.21.15.47:0/2752049310 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f3a74068e00 con 0x7f3a74159ea0 2024-04-26T23:18:32.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.992+0000 7f3a6bfff640 1 -- 172.21.15.47:0/2752049310 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) ==== 157+0+0 (secure 0 0 0) 0x7f3a6c00ae20 con 0x7f3a74159ea0 2024-04-26T23:18:32.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.996+0000 7f3a792f1640 1 -- 172.21.15.47:0/2752049310 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3a5406c920 msgr2=0x7f3a5406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:32.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.996+0000 7f3a792f1640 1 --2- 172.21.15.47:0/2752049310 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3a5406c920 0x7f3a5406ede0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3a64033000 tx=0x7f3a64008030 comp rx=0 tx=0).stop 2024-04-26T23:18:32.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.996+0000 7f3a792f1640 1 -- 172.21.15.47:0/2752049310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a74159ea0 msgr2=0x7f3a741f55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:32.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.996+0000 7f3a792f1640 1 --2- 172.21.15.47:0/2752049310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a74159ea0 0x7f3a741f55e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c0012e0 tx=0x7f3a6c007b20 comp rx=0 tx=0).stop 2024-04-26T23:18:32.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.996+0000 7f3a792f1640 1 -- 172.21.15.47:0/2752049310 shutdown_connections 2024-04-26T23:18:32.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.996+0000 7f3a792f1640 1 --2- 172.21.15.47:0/2752049310 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3a5406c920 0x7f3a5406ede0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:32.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.996+0000 7f3a792f1640 1 --2- 172.21.15.47:0/2752049310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a74159ea0 0x7f3a741f55e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:32.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.996+0000 7f3a792f1640 1 --2- 172.21.15.47:0/2752049310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a7415bd80 0x7f3a74159960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:32.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.996+0000 7f3a792f1640 1 --2- 172.21.15.47:0/2752049310 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3a600039b0 0x7f3a740cec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:32.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.996+0000 7f3a792f1640 1 -- 172.21.15.47:0/2752049310 >> 172.21.15.47:0/2752049310 conn(0x7f3a7409ba50 msgr2=0x7f3a74161340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:32.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.997+0000 7f3a792f1640 1 -- 172.21.15.47:0/2752049310 shutdown_connections 2024-04-26T23:18:32.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:32.997+0000 7f3a792f1640 1 -- 172.21.15.47:0/2752049310 wait complete. 2024-04-26T23:18:33.010 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T23:18:33.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.121+0000 7f3f1ed11640 1 Processor -- start 2024-04-26T23:18:33.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.121+0000 7f3f1ed11640 1 -- start start 2024-04-26T23:18:33.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.121+0000 7f3f1ed11640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3f180d5710 0x7f3f180d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:33.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.121+0000 7f3f1ed11640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f180d5c10 con 0x7f3f180d1880 2024-04-26T23:18:33.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.121+0000 7f3f1ed11640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f180cfd40 con 0x7f3f180d0d60 2024-04-26T23:18:33.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.121+0000 7f3f1ed11640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f3f180cfb10 con 0x7f3f180d5710 2024-04-26T23:18:33.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.122+0000 7f3f17fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3f180d5710 0x7f3f180d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:33.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.122+0000 7f3f1caad640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f3f180d1880 0x7f3f180b60c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35580/0 (socket says 172.21.15.47:35580) 2024-04-26T23:18:33.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.122+0000 7f3f17fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3f180d5710 0x7f3f180d0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41502/0 (socket says 172.21.15.47:41502) 2024-04-26T23:18:33.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.122+0000 7f3f1caad640 1 -- 172.21.15.47:0/179137317 learned_addr learned my addr 172.21.15.47:0/179137317 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:18:33.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.122+0000 7f3f17fff640 1 -- 172.21.15.47:0/179137317 >> v1:172.21.15.203:6789/0 conn(0x7f3f180d0d60 legacy=0x7f3f180b7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:33.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.122+0000 7f3f17fff640 1 -- 172.21.15.47:0/179137317 >> v1:172.21.15.47:6789/0 conn(0x7f3f180d1880 legacy=0x7f3f180b60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:33.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.122+0000 7f3f17fff640 1 -- 172.21.15.47:0/179137317 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f180d0020 con 0x7f3f180d5710 2024-04-26T23:18:33.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.122+0000 7f3f177fe640 1 -- 172.21.15.47:0/179137317 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2065833996 0 0) 0x7f3f180d5c10 con 0x7f3f180d1880 2024-04-26T23:18:33.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.123+0000 7f3f17fff640 1 --2- 172.21.15.47:0/179137317 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f180d5710 0x7f3f180d0820 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3f040097a0 tx=0x7f3f0402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f98ab4531c62acbe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:33.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.123+0000 7f3f177fe640 1 -- 172.21.15.47:0/179137317 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f04037070 con 0x7f3f180d5710 2024-04-26T23:18:33.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.123+0000 7f3f177fe640 1 -- 172.21.15.47:0/179137317 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f04032440 con 0x7f3f180d5710 2024-04-26T23:18:33.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.123+0000 7f3f177fe640 1 -- 172.21.15.47:0/179137317 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f04032810 con 0x7f3f180d5710 2024-04-26T23:18:33.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.123+0000 7f3f1ed11640 1 -- 172.21.15.47:0/179137317 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f180d5710 msgr2=0x7f3f180d0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:33.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.123+0000 7f3f1ed11640 1 --2- 172.21.15.47:0/179137317 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f180d5710 0x7f3f180d0820 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3f040097a0 tx=0x7f3f0402a270 comp rx=0 tx=0).stop 2024-04-26T23:18:33.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.123+0000 7f3f1ed11640 1 -- 172.21.15.47:0/179137317 shutdown_connections 2024-04-26T23:18:33.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.123+0000 7f3f1ed11640 1 --2- 172.21.15.47:0/179137317 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f180d5710 0x7f3f180d0820 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:33.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.123+0000 7f3f1ed11640 1 -- 172.21.15.47:0/179137317 >> 172.21.15.47:0/179137317 conn(0x7f3f1809e360 msgr2=0x7f3f180d5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:33.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.123+0000 7f3f1ed11640 1 -- 172.21.15.47:0/179137317 shutdown_connections 2024-04-26T23:18:33.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.123+0000 7f3f1ed11640 1 -- 172.21.15.47:0/179137317 wait complete. 2024-04-26T23:18:33.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.124+0000 7f3f1ed11640 1 Processor -- start 2024-04-26T23:18:33.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.125+0000 7f3f1ed11640 1 -- start start 2024-04-26T23:18:33.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.125+0000 7f3f1ed11640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f180d1880 0x7f3f1806e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:33.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.125+0000 7f3f1ed11640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f180d5710 0x7f3f18157b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:33.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.125+0000 7f3f1ed11640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f18158050 0x7f3f18149cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:33.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.125+0000 7f3f1ed11640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3f180b8560 con 0x7f3f180d5710 2024-04-26T23:18:33.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.125+0000 7f3f1ed11640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3f180d5ef0 con 0x7f3f18158050 2024-04-26T23:18:33.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.125+0000 7f3f1ed11640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3f180b86e0 con 0x7f3f180d1880 2024-04-26T23:18:33.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.125+0000 7f3f17fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f180d5710 0x7f3f18157b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:33.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.126+0000 7f3f17fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f180d5710 0x7f3f18157b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41518/0 (socket says 172.21.15.47:41518) 2024-04-26T23:18:33.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.126+0000 7f3f17fff640 1 -- 172.21.15.47:0/1326561126 learned_addr learned my addr 172.21.15.47:0/1326561126 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:33.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.126+0000 7f3f1d2ae640 1 --2- 172.21.15.47:0/1326561126 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f18158050 0x7f3f18149cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:33.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.126+0000 7f3f1caad640 1 --2- 172.21.15.47:0/1326561126 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f180d1880 0x7f3f1806e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:33.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.126+0000 7f3f17fff640 1 -- 172.21.15.47:0/1326561126 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f180d1880 msgr2=0x7f3f1806e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:33.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.126+0000 7f3f17fff640 1 --2- 172.21.15.47:0/1326561126 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f180d1880 0x7f3f1806e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:33.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.126+0000 7f3f17fff640 1 -- 172.21.15.47:0/1326561126 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f18158050 msgr2=0x7f3f18149cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:33.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.126+0000 7f3f17fff640 1 --2- 172.21.15.47:0/1326561126 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f18158050 0x7f3f18149cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:33.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.126+0000 7f3f17fff640 1 -- 172.21.15.47:0/1326561126 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f1814a8d0 con 0x7f3f180d5710 2024-04-26T23:18:33.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.126+0000 7f3f1caad640 1 --2- 172.21.15.47:0/1326561126 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f180d1880 0x7f3f1806e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:18:33.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.126+0000 7f3f1d2ae640 1 --2- 172.21.15.47:0/1326561126 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f18158050 0x7f3f18149cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:18:33.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.126+0000 7f3f17fff640 1 --2- 172.21.15.47:0/1326561126 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f180d5710 0x7f3f18157b10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3f040097a0 tx=0x7f3f040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:33.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.126+0000 7f3f157fa640 1 -- 172.21.15.47:0/1326561126 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f04041070 con 0x7f3f180d5710 2024-04-26T23:18:33.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.126+0000 7f3f157fa640 1 -- 172.21.15.47:0/1326561126 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f0403c870 con 0x7f3f180d5710 2024-04-26T23:18:33.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.126+0000 7f3f157fa640 1 -- 172.21.15.47:0/1326561126 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f04037040 con 0x7f3f180d5710 2024-04-26T23:18:33.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.126+0000 7f3f1ed11640 1 -- 172.21.15.47:0/1326561126 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f1814ab70 con 0x7f3f180d5710 2024-04-26T23:18:33.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.126+0000 7f3f1ed11640 1 -- 172.21.15.47:0/1326561126 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f1814b130 con 0x7f3f180d5710 2024-04-26T23:18:33.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.127+0000 7f3f1ed11640 1 -- 172.21.15.47:0/1326561126 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f18069590 con 0x7f3f180d5710 2024-04-26T23:18:33.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.128+0000 7f3f157fa640 1 -- 172.21.15.47:0/1326561126 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3f04044070 con 0x7f3f180d5710 2024-04-26T23:18:33.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.128+0000 7f3f157fa640 1 --2- 172.21.15.47:0/1326561126 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3efc06c920 0x7f3efc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:33.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.128+0000 7f3f157fa640 1 -- 172.21.15.47:0/1326561126 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 6007+0+0 (secure 0 0 0) 0x7f3f040ac890 con 0x7f3f180d5710 2024-04-26T23:18:33.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.129+0000 7f3f1caad640 1 --2- 172.21.15.47:0/1326561126 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3efc06c920 0x7f3efc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:33.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.129+0000 7f3f1caad640 1 --2- 172.21.15.47:0/1326561126 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3efc06c920 0x7f3efc06ede0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c000f60 tx=0x7f3f0c007c10 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:33.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.133+0000 7f3f157fa640 1 -- 172.21.15.47:0/1326561126 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3f0407aaf0 con 0x7f3f180d5710 2024-04-26T23:18:33.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:33.318+0000 7f3f1ed11640 1 -- 172.21.15.47:0/1326561126 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f3f180cb610 con 0x7f3f180d5710 2024-04-26T23:18:35.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.016+0000 7f3f157fa640 1 -- 172.21.15.47:0/1326561126 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v22) ==== 228+0+0 (secure 0 0 0) 0x7f3f0407a3f0 con 0x7f3f180d5710 2024-04-26T23:18:35.017 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data_ec' created 2024-04-26T23:18:35.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.021+0000 7f3f1ed11640 1 -- 172.21.15.47:0/1326561126 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3efc06c920 msgr2=0x7f3efc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:35.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.021+0000 7f3f1ed11640 1 --2- 172.21.15.47:0/1326561126 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3efc06c920 0x7f3efc06ede0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c000f60 tx=0x7f3f0c007c10 comp rx=0 tx=0).stop 2024-04-26T23:18:35.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.021+0000 7f3f1ed11640 1 -- 172.21.15.47:0/1326561126 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f180d5710 msgr2=0x7f3f18157b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:35.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.021+0000 7f3f1ed11640 1 --2- 172.21.15.47:0/1326561126 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f180d5710 0x7f3f18157b10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3f040097a0 tx=0x7f3f040046c0 comp rx=0 tx=0).stop 2024-04-26T23:18:35.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.022+0000 7f3f1ed11640 1 -- 172.21.15.47:0/1326561126 shutdown_connections 2024-04-26T23:18:35.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.022+0000 7f3f1ed11640 1 --2- 172.21.15.47:0/1326561126 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3efc06c920 0x7f3efc06ede0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:35.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.022+0000 7f3f1ed11640 1 --2- 172.21.15.47:0/1326561126 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f18158050 0x7f3f18149cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:35.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.022+0000 7f3f1ed11640 1 --2- 172.21.15.47:0/1326561126 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f180d5710 0x7f3f18157b10 secure :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3f040097a0 tx=0x7f3f040046c0 comp rx=0 tx=0).stop 2024-04-26T23:18:35.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.022+0000 7f3f1ed11640 1 --2- 172.21.15.47:0/1326561126 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f180d1880 0x7f3f1806e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:35.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.022+0000 7f3f1ed11640 1 -- 172.21.15.47:0/1326561126 >> 172.21.15.47:0/1326561126 conn(0x7f3f1809e360 msgr2=0x7f3f180ba990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:35.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.024+0000 7f3f1ed11640 1 -- 172.21.15.47:0/1326561126 shutdown_connections 2024-04-26T23:18:35.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.026+0000 7f3f1ed11640 1 -- 172.21.15.47:0/1326561126 wait complete. 2024-04-26T23:18:35.041 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-26T23:18:35.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.151+0000 7fbafc524640 1 Processor -- start 2024-04-26T23:18:35.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.151+0000 7fbafc524640 1 -- start start 2024-04-26T23:18:35.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.152+0000 7fbafc524640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbaf40b5fd0 0x7fbaf40b4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:35.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.152+0000 7fbafc524640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbaf4162d80 con 0x7fbaf41629a0 2024-04-26T23:18:35.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.152+0000 7fbafc524640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbaf40d39d0 con 0x7fbaf40d1950 2024-04-26T23:18:35.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.152+0000 7fbafc524640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fbaf40d5f30 con 0x7fbaf40b5fd0 2024-04-26T23:18:35.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.152+0000 7fbaf9abf640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbaf40b5fd0 0x7fbaf40b4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:35.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.152+0000 7fbaf9abf640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbaf40b5fd0 0x7fbaf40b4890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41532/0 (socket says 172.21.15.47:41532) 2024-04-26T23:18:35.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.152+0000 7fbaf9abf640 1 -- 172.21.15.47:0/363534748 learned_addr learned my addr 172.21.15.47:0/363534748 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:35.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.152+0000 7fbafa2c0640 1 --1- 172.21.15.47:0/363534748 >> v1:172.21.15.47:6789/0 conn(0x7fbaf41629a0 0x7fbaf40d5820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35594/0 (socket says 172.21.15.47:35594) 2024-04-26T23:18:35.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.152+0000 7fbaf9abf640 1 -- 172.21.15.47:0/363534748 >> v1:172.21.15.203:6789/0 conn(0x7fbaf40d1950 legacy=0x7fbaf40d02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:35.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.152+0000 7fbaf9abf640 1 -- 172.21.15.47:0/363534748 >> v1:172.21.15.47:6789/0 conn(0x7fbaf41629a0 legacy=0x7fbaf40d5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:35.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.152+0000 7fbaf9abf640 1 -- 172.21.15.47:0/363534748 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaf40d3cb0 con 0x7fbaf40b5fd0 2024-04-26T23:18:35.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.153+0000 7fbaf9abf640 1 --2- 172.21.15.47:0/363534748 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbaf40b5fd0 0x7fbaf40b4890 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbae4009870 tx=0x7fbae402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6452f242f1f3ff85 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:35.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.153+0000 7fbaf92be640 1 -- 172.21.15.47:0/363534748 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae4037070 con 0x7fbaf40b5fd0 2024-04-26T23:18:35.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.153+0000 7fbaf92be640 1 -- 172.21.15.47:0/363534748 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbae4032440 con 0x7fbaf40b5fd0 2024-04-26T23:18:35.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.153+0000 7fbaf92be640 1 -- 172.21.15.47:0/363534748 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae4032810 con 0x7fbaf40b5fd0 2024-04-26T23:18:35.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.153+0000 7fbafc524640 1 -- 172.21.15.47:0/363534748 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbaf40b5fd0 msgr2=0x7fbaf40b4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:35.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.153+0000 7fbafc524640 1 --2- 172.21.15.47:0/363534748 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbaf40b5fd0 0x7fbaf40b4890 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbae4009870 tx=0x7fbae402a2a0 comp rx=0 tx=0).stop 2024-04-26T23:18:35.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.154+0000 7fbafc524640 1 -- 172.21.15.47:0/363534748 shutdown_connections 2024-04-26T23:18:35.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.154+0000 7fbafc524640 1 --2- 172.21.15.47:0/363534748 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbaf40b5fd0 0x7fbaf40b4890 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:35.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.154+0000 7fbafc524640 1 -- 172.21.15.47:0/363534748 >> 172.21.15.47:0/363534748 conn(0x7fbaf409e3e0 msgr2=0x7fbaf40d5110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:35.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.154+0000 7fbafc524640 1 -- 172.21.15.47:0/363534748 shutdown_connections 2024-04-26T23:18:35.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.154+0000 7fbafc524640 1 -- 172.21.15.47:0/363534748 wait complete. 2024-04-26T23:18:35.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.155+0000 7fbafc524640 1 Processor -- start 2024-04-26T23:18:35.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.155+0000 7fbafc524640 1 -- start start 2024-04-26T23:18:35.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.155+0000 7fbafc524640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbaf40b5fd0 0x7fbaf41f8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:35.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.155+0000 7fbafc524640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbaf40d1950 0x7fbaf40ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:35.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.155+0000 7fbafc524640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbaf41629a0 0x7fbaf40ce930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:35.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.155+0000 7fbafc524640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf40d3f90 con 0x7fbaf40d1950 2024-04-26T23:18:35.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.155+0000 7fbafc524640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf40d36a0 con 0x7fbaf41629a0 2024-04-26T23:18:35.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.155+0000 7fbafc524640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fbaf40d1de0 con 0x7fbaf40b5fd0 2024-04-26T23:18:35.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.156+0000 7fbaf9abf640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbaf40d1950 0x7fbaf40ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:35.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.156+0000 7fbaf9abf640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbaf40d1950 0x7fbaf40ce3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41540/0 (socket says 172.21.15.47:41540) 2024-04-26T23:18:35.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.156+0000 7fbaf9abf640 1 -- 172.21.15.47:0/3045179557 learned_addr learned my addr 172.21.15.47:0/3045179557 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:35.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.156+0000 7fbaf9abf640 1 -- 172.21.15.47:0/3045179557 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbaf40b5fd0 msgr2=0x7fbaf41f8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:35.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.156+0000 7fbaf9abf640 1 --2- 172.21.15.47:0/3045179557 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbaf40b5fd0 0x7fbaf41f8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:35.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.156+0000 7fbaf9abf640 1 -- 172.21.15.47:0/3045179557 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbaf41629a0 msgr2=0x7fbaf40ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:35.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.156+0000 7fbaf9abf640 1 --2- 172.21.15.47:0/3045179557 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbaf41629a0 0x7fbaf40ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:35.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.156+0000 7fbaf9abf640 1 -- 172.21.15.47:0/3045179557 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaf41f8b50 con 0x7fbaf40d1950 2024-04-26T23:18:35.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.156+0000 7fbaf9abf640 1 --2- 172.21.15.47:0/3045179557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbaf40d1950 0x7fbaf40ce3f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbae4009870 tx=0x7fbae40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:35.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.156+0000 7fbaeaffd640 1 -- 172.21.15.47:0/3045179557 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae4041020 con 0x7fbaf40d1950 2024-04-26T23:18:35.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.156+0000 7fbaeaffd640 1 -- 172.21.15.47:0/3045179557 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbae4030970 con 0x7fbaf40d1950 2024-04-26T23:18:35.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.156+0000 7fbaeaffd640 1 -- 172.21.15.47:0/3045179557 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae4037040 con 0x7fbaf40d1950 2024-04-26T23:18:35.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.156+0000 7fbafc524640 1 -- 172.21.15.47:0/3045179557 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaf40cd7e0 con 0x7fbaf40d1950 2024-04-26T23:18:35.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.156+0000 7fbafc524640 1 -- 172.21.15.47:0/3045179557 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaf4154310 con 0x7fbaf40d1950 2024-04-26T23:18:35.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.157+0000 7fbafc524640 1 -- 172.21.15.47:0/3045179557 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaf40d2140 con 0x7fbaf40d1950 2024-04-26T23:18:35.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.158+0000 7fbaeaffd640 1 -- 172.21.15.47:0/3045179557 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbae4044020 con 0x7fbaf40d1950 2024-04-26T23:18:35.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.158+0000 7fbaeaffd640 1 --2- 172.21.15.47:0/3045179557 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fbad806c920 0x7fbad806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:35.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.158+0000 7fbaeaffd640 1 -- 172.21.15.47:0/3045179557 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 6500+0+0 (secure 0 0 0) 0x7fbae40ac780 con 0x7fbaf40d1950 2024-04-26T23:18:35.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.158+0000 7fbafa2c0640 1 --2- 172.21.15.47:0/3045179557 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fbad806c920 0x7fbad806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:35.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.159+0000 7fbafa2c0640 1 --2- 172.21.15.47:0/3045179557 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fbad806c920 0x7fbad806ede0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0000f60 tx=0x7fbaf0007a00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:35.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.163+0000 7fbaeaffd640 1 -- 172.21.15.47:0/3045179557 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbaf40d2140 con 0x7fbaf40d1950 2024-04-26T23:18:35.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:35.346+0000 7fbafc524640 1 -- 172.21.15.47:0/3045179557 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7fbaf4069600 con 0x7fbaf40d1950 2024-04-26T23:18:35.523 INFO:tasks.ceph.mds.b.smithi203.stderr:2024-04-26T23:18:35.522+0000 7f081e513640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T23:18:36.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.018+0000 7fbaeaffd640 1 -- 172.21.15.47:0/3045179557 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v23) ==== 169+0+0 (secure 0 0 0) 0x7fbae404b030 con 0x7fbaf40d1950 2024-04-26T23:18:36.019 INFO:teuthology.orchestra.run.smithi047.stderr:set pool 4 allow_ec_overwrites to true 2024-04-26T23:18:36.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.022+0000 7fbafc524640 1 -- 172.21.15.47:0/3045179557 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fbad806c920 msgr2=0x7fbad806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:36.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.022+0000 7fbafc524640 1 --2- 172.21.15.47:0/3045179557 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fbad806c920 0x7fbad806ede0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0000f60 tx=0x7fbaf0007a00 comp rx=0 tx=0).stop 2024-04-26T23:18:36.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.023+0000 7fbafc524640 1 -- 172.21.15.47:0/3045179557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbaf40d1950 msgr2=0x7fbaf40ce3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:36.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.023+0000 7fbafc524640 1 --2- 172.21.15.47:0/3045179557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbaf40d1950 0x7fbaf40ce3f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbae4009870 tx=0x7fbae40046c0 comp rx=0 tx=0).stop 2024-04-26T23:18:36.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.034+0000 7fbafc524640 1 -- 172.21.15.47:0/3045179557 shutdown_connections 2024-04-26T23:18:36.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.034+0000 7fbafc524640 1 --2- 172.21.15.47:0/3045179557 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fbad806c920 0x7fbad806ede0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:36.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.034+0000 7fbafc524640 1 --2- 172.21.15.47:0/3045179557 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbaf41629a0 0x7fbaf40ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:36.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.034+0000 7fbafc524640 1 --2- 172.21.15.47:0/3045179557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbaf40d1950 0x7fbaf40ce3f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:36.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.034+0000 7fbafc524640 1 --2- 172.21.15.47:0/3045179557 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbaf40b5fd0 0x7fbaf41f8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:36.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.034+0000 7fbafc524640 1 -- 172.21.15.47:0/3045179557 >> 172.21.15.47:0/3045179557 conn(0x7fbaf409e3e0 msgr2=0x7fbaf406e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:36.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.035+0000 7fbafc524640 1 -- 172.21.15.47:0/3045179557 shutdown_connections 2024-04-26T23:18:36.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.035+0000 7fbafc524640 1 -- 172.21.15.47:0/3045179557 wait complete. 2024-04-26T23:18:36.059 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-26T23:18:36.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.165+0000 7f3631178640 1 Processor -- start 2024-04-26T23:18:36.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.165+0000 7f3631178640 1 -- start start 2024-04-26T23:18:36.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.166+0000 7f3631178640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f362c162880 0x7f362c0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:36.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.166+0000 7f3631178640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f362c0b5f20 0x7f362c0d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:36.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.166+0000 7f3631178640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f362c0b7e50 con 0x7f362c0d39f0 2024-04-26T23:18:36.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.166+0000 7f3631178640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f362c0d3e40 con 0x7f362c0b5f20 2024-04-26T23:18:36.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.166+0000 7f3631178640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f362c0d0580 con 0x7f362c162880 2024-04-26T23:18:36.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.166+0000 7f3623fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f362c0b5f20 0x7f362c0d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:36.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.166+0000 7f3623fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f362c0b5f20 0x7f362c0d34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41554/0 (socket says 172.21.15.47:41554) 2024-04-26T23:18:36.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.166+0000 7f3623fff640 1 -- 172.21.15.47:0/3113276491 learned_addr learned my addr 172.21.15.47:0/3113276491 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:36.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.166+0000 7f362ad9d640 1 --2- 172.21.15.47:0/3113276491 >> v2:172.21.15.203:3300/0 conn(0x7f362c162880 0x7f362c0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:36.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.166+0000 7f3623fff640 1 -- 172.21.15.47:0/3113276491 >> v2:172.21.15.203:3300/0 conn(0x7f362c162880 msgr2=0x7f362c0d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:36.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.166+0000 7f3623fff640 1 --2- 172.21.15.47:0/3113276491 >> v2:172.21.15.203:3300/0 conn(0x7f362c162880 0x7f362c0d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:36.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.166+0000 7f3623fff640 1 -- 172.21.15.47:0/3113276491 >> v1:172.21.15.203:6789/0 conn(0x7f362c0d39f0 legacy=0x7f362c0b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:36.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.166+0000 7f3623fff640 1 -- 172.21.15.47:0/3113276491 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f362c0d1800 con 0x7f362c0b5f20 2024-04-26T23:18:36.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.166+0000 7f362ad9d640 1 --2- 172.21.15.47:0/3113276491 >> v2:172.21.15.203:3300/0 conn(0x7f362c162880 0x7f362c0d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:18:36.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.167+0000 7f3623fff640 1 --2- 172.21.15.47:0/3113276491 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f362c0b5f20 0x7f362c0d34b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f362c08d490 tx=0x7f361402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76911c44876afb2a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:36.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.167+0000 7f362a59c640 1 -- 172.21.15.47:0/3113276491 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3614037070 con 0x7f362c0b5f20 2024-04-26T23:18:36.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.167+0000 7f362a59c640 1 -- 172.21.15.47:0/3113276491 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3614032440 con 0x7f362c0b5f20 2024-04-26T23:18:36.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.167+0000 7f362a59c640 1 -- 172.21.15.47:0/3113276491 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3614032810 con 0x7f362c0b5f20 2024-04-26T23:18:36.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.168+0000 7f3631178640 1 -- 172.21.15.47:0/3113276491 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f362c0b5f20 msgr2=0x7f362c0d34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:36.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.168+0000 7f3631178640 1 --2- 172.21.15.47:0/3113276491 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f362c0b5f20 0x7f362c0d34b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f362c08d490 tx=0x7f361402a270 comp rx=0 tx=0).stop 2024-04-26T23:18:36.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.168+0000 7f3631178640 1 -- 172.21.15.47:0/3113276491 shutdown_connections 2024-04-26T23:18:36.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.168+0000 7f3631178640 1 --2- 172.21.15.47:0/3113276491 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f362c0b5f20 0x7f362c0d34b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:36.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.168+0000 7f3631178640 1 --2- 172.21.15.47:0/3113276491 >> v2:172.21.15.203:3300/0 conn(0x7f362c162880 0x7f362c0d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:36.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.168+0000 7f3631178640 1 -- 172.21.15.47:0/3113276491 >> 172.21.15.47:0/3113276491 conn(0x7f362c0d4f60 msgr2=0x7f362c0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:36.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.168+0000 7f3631178640 1 -- 172.21.15.47:0/3113276491 shutdown_connections 2024-04-26T23:18:36.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.168+0000 7f3631178640 1 -- 172.21.15.47:0/3113276491 wait complete. 2024-04-26T23:18:36.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.169+0000 7f3631178640 1 Processor -- start 2024-04-26T23:18:36.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.169+0000 7f3631178640 1 -- start start 2024-04-26T23:18:36.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.170+0000 7f3631178640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f362c0b5f20 0x7f362c1f4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:36.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.170+0000 7f3631178640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f362c0d39f0 0x7f362c0cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:36.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.170+0000 7f3631178640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f362c162880 0x7f362c15c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:36.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.170+0000 7f3631178640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f362c0d0d90 con 0x7f362c162880 2024-04-26T23:18:36.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.170+0000 7f3631178640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f362c0d0ab0 con 0x7f362c0d39f0 2024-04-26T23:18:36.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.170+0000 7f3631178640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f362c162c60 con 0x7f362c0b5f20 2024-04-26T23:18:36.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.170+0000 7f362b59e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f362c162880 0x7f362c15c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:36.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.170+0000 7f362b59e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f362c162880 0x7f362c15c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41568/0 (socket says 172.21.15.47:41568) 2024-04-26T23:18:36.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.170+0000 7f362b59e640 1 -- 172.21.15.47:0/1902769970 learned_addr learned my addr 172.21.15.47:0/1902769970 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:36.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.170+0000 7f3623fff640 1 --2- 172.21.15.47:0/1902769970 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f362c0d39f0 0x7f362c0cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:36.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.170+0000 7f362ad9d640 1 --2- 172.21.15.47:0/1902769970 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f362c0b5f20 0x7f362c1f4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:36.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.170+0000 7f362b59e640 1 -- 172.21.15.47:0/1902769970 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f362c0b5f20 msgr2=0x7f362c1f4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:36.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.170+0000 7f362b59e640 1 --2- 172.21.15.47:0/1902769970 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f362c0b5f20 0x7f362c1f4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:36.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.170+0000 7f362b59e640 1 -- 172.21.15.47:0/1902769970 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f362c0d39f0 msgr2=0x7f362c0cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:36.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.170+0000 7f362b59e640 1 --2- 172.21.15.47:0/1902769970 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f362c0d39f0 0x7f362c0cdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:36.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.170+0000 7f362b59e640 1 -- 172.21.15.47:0/1902769970 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f362c0ce290 con 0x7f362c162880 2024-04-26T23:18:36.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.170+0000 7f362b59e640 1 --2- 172.21.15.47:0/1902769970 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f362c162880 0x7f362c15c800 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f362400c6e0 tx=0x7f362400cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:36.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.171+0000 7f36237fe640 1 -- 172.21.15.47:0/1902769970 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3624003b70 con 0x7f362c162880 2024-04-26T23:18:36.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.171+0000 7f36237fe640 1 -- 172.21.15.47:0/1902769970 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3624003d30 con 0x7f362c162880 2024-04-26T23:18:36.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.171+0000 7f36237fe640 1 -- 172.21.15.47:0/1902769970 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3624005310 con 0x7f362c162880 2024-04-26T23:18:36.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.171+0000 7f3631178640 1 -- 172.21.15.47:0/1902769970 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f362c1f59c0 con 0x7f362c162880 2024-04-26T23:18:36.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.171+0000 7f3631178640 1 -- 172.21.15.47:0/1902769970 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f362c1f5400 con 0x7f362c162880 2024-04-26T23:18:36.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.172+0000 7f3631178640 1 -- 172.21.15.47:0/1902769970 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f362c0694f0 con 0x7f362c162880 2024-04-26T23:18:36.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.172+0000 7f36237fe640 1 -- 172.21.15.47:0/1902769970 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f36240054b0 con 0x7f362c162880 2024-04-26T23:18:36.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.172+0000 7f36237fe640 1 --2- 172.21.15.47:0/1902769970 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f360c06c8f0 0x7f360c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:36.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.173+0000 7f36237fe640 1 -- 172.21.15.47:0/1902769970 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6500+0+0 (secure 0 0 0) 0x7f362408bad0 con 0x7f362c162880 2024-04-26T23:18:36.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.173+0000 7f362ad9d640 1 --2- 172.21.15.47:0/1902769970 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f360c06c8f0 0x7f360c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:36.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.173+0000 7f362ad9d640 1 --2- 172.21.15.47:0/1902769970 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f360c06c8f0 0x7f360c06edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f362c0d43e0 tx=0x7f361c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:36.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.177+0000 7f36237fe640 1 -- 172.21.15.47:0/1902769970 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3624059620 con 0x7f362c162880 2024-04-26T23:18:36.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:36.401+0000 7f3631178640 1 -- 172.21.15.47:0/1902769970 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f362c0cb4a0 con 0x7f362c162880 2024-04-26T23:18:37.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.030+0000 7f36237fe640 1 -- 172.21.15.47:0/1902769970 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) ==== 137+0+0 (secure 0 0 0) 0x7f362405d920 con 0x7f362c162880 2024-04-26T23:18:37.031 INFO:teuthology.orchestra.run.smithi047.stderr:added data pool 4 to fsmap 2024-04-26T23:18:37.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.033+0000 7f3631178640 1 -- 172.21.15.47:0/1902769970 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f360c06c8f0 msgr2=0x7f360c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:37.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.033+0000 7f3631178640 1 --2- 172.21.15.47:0/1902769970 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f360c06c8f0 0x7f360c06edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f362c0d43e0 tx=0x7f361c008040 comp rx=0 tx=0).stop 2024-04-26T23:18:37.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.033+0000 7f3631178640 1 -- 172.21.15.47:0/1902769970 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f362c162880 msgr2=0x7f362c15c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:37.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.033+0000 7f3631178640 1 --2- 172.21.15.47:0/1902769970 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f362c162880 0x7f362c15c800 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f362400c6e0 tx=0x7f362400cba0 comp rx=0 tx=0).stop 2024-04-26T23:18:37.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.033+0000 7f3631178640 1 -- 172.21.15.47:0/1902769970 shutdown_connections 2024-04-26T23:18:37.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.033+0000 7f3631178640 1 --2- 172.21.15.47:0/1902769970 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f360c06c8f0 0x7f360c06edb0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:37.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.033+0000 7f3631178640 1 --2- 172.21.15.47:0/1902769970 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f362c162880 0x7f362c15c800 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:37.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.033+0000 7f3631178640 1 --2- 172.21.15.47:0/1902769970 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f362c0d39f0 0x7f362c0cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:37.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.033+0000 7f3631178640 1 --2- 172.21.15.47:0/1902769970 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f362c0b5f20 0x7f362c1f4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:37.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.033+0000 7f3631178640 1 -- 172.21.15.47:0/1902769970 >> 172.21.15.47:0/1902769970 conn(0x7f362c0d4f60 msgr2=0x7f362c06e640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:37.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.033+0000 7f3631178640 1 -- 172.21.15.47:0/1902769970 shutdown_connections 2024-04-26T23:18:37.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.033+0000 7f3631178640 1 -- 172.21.15.47:0/1902769970 wait complete. 2024-04-26T23:18:37.047 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:18:37.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.153+0000 7f06b7e72640 1 Processor -- start 2024-04-26T23:18:37.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.153+0000 7f06b7e72640 1 -- start start 2024-04-26T23:18:37.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.154+0000 7f06b7e72640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f06b00cc480 0x7f06b0072240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:37.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.154+0000 7f06b7e72640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f06b0072780 0x7f06b0166ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:37.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.154+0000 7f06b7e72640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06b015d7c0 con 0x7f06b0071e60 2024-04-26T23:18:37.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.154+0000 7f06b7e72640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f06b015daa0 con 0x7f06b00cc480 2024-04-26T23:18:37.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.154+0000 7f06b7e72640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f06b015dd80 con 0x7f06b0072780 2024-04-26T23:18:37.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.154+0000 7f06b540d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f06b00cc480 0x7f06b0072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:37.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.154+0000 7f06b540d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f06b00cc480 0x7f06b0072240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41580/0 (socket says 172.21.15.47:41580) 2024-04-26T23:18:37.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.154+0000 7f06b540d640 1 -- 172.21.15.47:0/933131405 learned_addr learned my addr 172.21.15.47:0/933131405 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:37.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.154+0000 7f06b640f640 1 --2- 172.21.15.47:0/933131405 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f06b0072780 0x7f06b0166ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:37.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.154+0000 7f06b540d640 1 -- 172.21.15.47:0/933131405 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f06b0072780 msgr2=0x7f06b0166ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:37.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.154+0000 7f06b540d640 1 --2- 172.21.15.47:0/933131405 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f06b0072780 0x7f06b0166ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:37.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.154+0000 7f06b540d640 1 -- 172.21.15.47:0/933131405 >> v1:172.21.15.203:6789/0 conn(0x7f06b0071e60 legacy=0x7f06b006f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:37.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.154+0000 7f06b540d640 1 -- 172.21.15.47:0/933131405 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06b015e060 con 0x7f06b00cc480 2024-04-26T23:18:37.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.154+0000 7f06b640f640 1 --2- 172.21.15.47:0/933131405 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f06b0072780 0x7f06b0166ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:18:37.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.155+0000 7f06b540d640 1 --2- 172.21.15.47:0/933131405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06b00cc480 0x7f06b0072240 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f06a0009870 tx=0x7f06a002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1b0ec43460d4769 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:37.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.155+0000 7f06b4c0c640 1 -- 172.21.15.47:0/933131405 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a0037070 con 0x7f06b00cc480 2024-04-26T23:18:37.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.155+0000 7f06b4c0c640 1 -- 172.21.15.47:0/933131405 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06a0032440 con 0x7f06b00cc480 2024-04-26T23:18:37.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.155+0000 7f06b4c0c640 1 -- 172.21.15.47:0/933131405 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a0032810 con 0x7f06b00cc480 2024-04-26T23:18:37.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.156+0000 7f06b7e72640 1 -- 172.21.15.47:0/933131405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06b00cc480 msgr2=0x7f06b0072240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:37.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.156+0000 7f06b7e72640 1 --2- 172.21.15.47:0/933131405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06b00cc480 0x7f06b0072240 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f06a0009870 tx=0x7f06a002a270 comp rx=0 tx=0).stop 2024-04-26T23:18:37.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.156+0000 7f06b7e72640 1 -- 172.21.15.47:0/933131405 shutdown_connections 2024-04-26T23:18:37.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.156+0000 7f06b7e72640 1 --2- 172.21.15.47:0/933131405 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f06b0072780 0x7f06b0166ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:37.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.156+0000 7f06b7e72640 1 --2- 172.21.15.47:0/933131405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06b00cc480 0x7f06b0072240 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:37.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.156+0000 7f06b7e72640 1 -- 172.21.15.47:0/933131405 >> 172.21.15.47:0/933131405 conn(0x7f06b00d0be0 msgr2=0x7f06b00d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:37.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.156+0000 7f06b7e72640 1 -- 172.21.15.47:0/933131405 shutdown_connections 2024-04-26T23:18:37.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.156+0000 7f06b7e72640 1 -- 172.21.15.47:0/933131405 wait complete. 2024-04-26T23:18:37.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.157+0000 7f06b7e72640 1 Processor -- start 2024-04-26T23:18:37.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.157+0000 7f06b7e72640 1 -- start start 2024-04-26T23:18:37.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.158+0000 7f06b7e72640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06b0071e60 0x7f06b0148710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:37.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.158+0000 7f06b7e72640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f06b0072780 0x7f06b0149040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:37.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.158+0000 7f06b7e72640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06b00cc480 0x7f06b00ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:37.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.158+0000 7f06b7e72640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f06b015d3f0 con 0x7f06b0071e60 2024-04-26T23:18:37.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.158+0000 7f06b7e72640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f06b0072c10 con 0x7f06b00cc480 2024-04-26T23:18:37.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.158+0000 7f06b7e72640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f06b015d570 con 0x7f06b0072780 2024-04-26T23:18:37.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.158+0000 7f06b5c0e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06b0071e60 0x7f06b0148710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:37.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.158+0000 7f06b5c0e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06b0071e60 0x7f06b0148710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41596/0 (socket says 172.21.15.47:41596) 2024-04-26T23:18:37.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.158+0000 7f06b5c0e640 1 -- 172.21.15.47:0/699316689 learned_addr learned my addr 172.21.15.47:0/699316689 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:37.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.158+0000 7f06b540d640 1 --2- 172.21.15.47:0/699316689 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f06b0072780 0x7f06b0149040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:37.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.158+0000 7f06b640f640 1 --2- 172.21.15.47:0/699316689 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06b00cc480 0x7f06b00ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:37.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.158+0000 7f06b5c0e640 1 -- 172.21.15.47:0/699316689 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f06b0072780 msgr2=0x7f06b0149040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:37.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.158+0000 7f06b5c0e640 1 --2- 172.21.15.47:0/699316689 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f06b0072780 0x7f06b0149040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:37.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.158+0000 7f06b5c0e640 1 -- 172.21.15.47:0/699316689 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06b00cc480 msgr2=0x7f06b00ca7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:37.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.158+0000 7f06b5c0e640 1 --2- 172.21.15.47:0/699316689 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06b00cc480 0x7f06b00ca7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:37.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.158+0000 7f06b5c0e640 1 -- 172.21.15.47:0/699316689 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06b014beb0 con 0x7f06b0071e60 2024-04-26T23:18:37.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.158+0000 7f06b540d640 1 --2- 172.21.15.47:0/699316689 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f06b0072780 0x7f06b0149040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:18:37.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.158+0000 7f06b5c0e640 1 --2- 172.21.15.47:0/699316689 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06b0071e60 0x7f06b0148710 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f06ac00c400 tx=0x7f06ac00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:37.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.158+0000 7f06a67fc640 1 -- 172.21.15.47:0/699316689 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06ac0052d0 con 0x7f06b0071e60 2024-04-26T23:18:37.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.158+0000 7f06a67fc640 1 -- 172.21.15.47:0/699316689 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06ac005490 con 0x7f06b0071e60 2024-04-26T23:18:37.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.158+0000 7f06a67fc640 1 -- 172.21.15.47:0/699316689 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06ac007260 con 0x7f06b0071e60 2024-04-26T23:18:37.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.159+0000 7f06b7e72640 1 -- 172.21.15.47:0/699316689 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06b00c9ce0 con 0x7f06b0071e60 2024-04-26T23:18:37.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.159+0000 7f06b7e72640 1 -- 172.21.15.47:0/699316689 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06b00ca2a0 con 0x7f06b0071e60 2024-04-26T23:18:37.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.160+0000 7f06b7e72640 1 -- 172.21.15.47:0/699316689 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06b00694f0 con 0x7f06b0071e60 2024-04-26T23:18:37.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.160+0000 7f06a67fc640 1 -- 172.21.15.47:0/699316689 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f06ac007400 con 0x7f06b0071e60 2024-04-26T23:18:37.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.161+0000 7f06a67fc640 1 --2- 172.21.15.47:0/699316689 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f069406c920 0x7f069406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:37.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.161+0000 7f06a67fc640 1 -- 172.21.15.47:0/699316689 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f06ac08b2f0 con 0x7f06b0071e60 2024-04-26T23:18:37.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.161+0000 7f06b540d640 1 --2- 172.21.15.47:0/699316689 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f069406c920 0x7f069406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:37.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.162+0000 7f06b540d640 1 --2- 172.21.15.47:0/699316689 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f069406c920 0x7f069406ede0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f06a0034000 tx=0x7f06a002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:37.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.165+0000 7f06a67fc640 1 -- 172.21.15.47:0/699316689 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f06ac059470 con 0x7f06b0071e60 2024-04-26T23:18:37.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.389+0000 7f06b7e72640 1 -- 172.21.15.47:0/699316689 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f06b00b11b0 con 0x7f06b0071e60 2024-04-26T23:18:37.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.390+0000 7f06a67fc640 1 -- 172.21.15.47:0/699316689 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5180 (secure 0 0 0) 0x7f06ac058d70 con 0x7f06b0071e60 2024-04-26T23:18:37.391 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:18:37.391 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/1553696401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1553696401},{"type":"v1","addr":"172.21.15.47:6834","nonce":1553696401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/3792767564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3792767564},{"type":"v1","addr":"172.21.15.203:6836","nonce":3792767564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4088100766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":4088100766},{"type":"v1","addr":"172.21.15.47:6835","nonce":4088100766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:18:31.367292+0000","modified":"2024-04-26T23:18:36.402282+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/997115011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":997115011},{"type":"v1","addr":"172.21.15.203:6837","nonce":997115011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4282,"qdb_cluster":[4282]},"id":1}]} 2024-04-26T23:18:37.391 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 7 2024-04-26T23:18:37.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.392+0000 7f06b7e72640 1 -- 172.21.15.47:0/699316689 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f069406c920 msgr2=0x7f069406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:37.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.392+0000 7f06b7e72640 1 --2- 172.21.15.47:0/699316689 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f069406c920 0x7f069406ede0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f06a0034000 tx=0x7f06a002e000 comp rx=0 tx=0).stop 2024-04-26T23:18:37.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.392+0000 7f06b7e72640 1 -- 172.21.15.47:0/699316689 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06b0071e60 msgr2=0x7f06b0148710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:37.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.392+0000 7f06b7e72640 1 --2- 172.21.15.47:0/699316689 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06b0071e60 0x7f06b0148710 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f06ac00c400 tx=0x7f06ac00c8c0 comp rx=0 tx=0).stop 2024-04-26T23:18:37.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.392+0000 7f06b7e72640 1 -- 172.21.15.47:0/699316689 shutdown_connections 2024-04-26T23:18:37.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.392+0000 7f06b7e72640 1 --2- 172.21.15.47:0/699316689 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f069406c920 0x7f069406ede0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:37.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.392+0000 7f06b7e72640 1 --2- 172.21.15.47:0/699316689 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06b00cc480 0x7f06b00ca7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:37.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.392+0000 7f06b7e72640 1 --2- 172.21.15.47:0/699316689 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f06b0072780 0x7f06b0149040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:37.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.392+0000 7f06b7e72640 1 --2- 172.21.15.47:0/699316689 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06b0071e60 0x7f06b0148710 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:37.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.392+0000 7f06b7e72640 1 -- 172.21.15.47:0/699316689 >> 172.21.15.47:0/699316689 conn(0x7f06b00d0be0 msgr2=0x7f06b015ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:37.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.392+0000 7f06b7e72640 1 -- 172.21.15.47:0/699316689 shutdown_connections 2024-04-26T23:18:37.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.392+0000 7f06b7e72640 1 -- 172.21.15.47:0/699316689 wait complete. 2024-04-26T23:18:37.404 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:18:37.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.504+0000 7f59401c6640 1 Processor -- start 2024-04-26T23:18:37.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.504+0000 7f59401c6640 1 -- start start 2024-04-26T23:18:37.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.505+0000 7f59401c6640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f59380d3bc0 0x7f59380d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:37.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.505+0000 7f59401c6640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59380b7e50 con 0x7f59380b5f20 2024-04-26T23:18:37.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.505+0000 7f59401c6640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59380d1800 con 0x7f5938162880 2024-04-26T23:18:37.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.505+0000 7f59401c6640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f59380d0580 con 0x7f59380d3bc0 2024-04-26T23:18:37.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.505+0000 7f593d761640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f59380d3bc0 0x7f59380d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:37.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.505+0000 7f593e763640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f59380b5f20 0x7f59380b8300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35610/0 (socket says 172.21.15.47:35610) 2024-04-26T23:18:37.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.505+0000 7f593e763640 1 -- 172.21.15.47:0/2633075792 learned_addr learned my addr 172.21.15.47:0/2633075792 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:18:37.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.505+0000 7f593d761640 1 -- 172.21.15.47:0/2633075792 >> v1:172.21.15.203:6789/0 conn(0x7f5938162880 legacy=0x7f59380d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:37.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.505+0000 7f593d761640 1 -- 172.21.15.47:0/2633075792 >> v1:172.21.15.47:6789/0 conn(0x7f59380b5f20 legacy=0x7f59380b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:37.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.505+0000 7f593cf60640 1 -- 172.21.15.47:0/2633075792 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3988173459 0 0) 0x7f59380b7e50 con 0x7f59380b5f20 2024-04-26T23:18:37.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.505+0000 7f593d761640 1 -- 172.21.15.47:0/2633075792 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59380d07d0 con 0x7f59380d3bc0 2024-04-26T23:18:37.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.506+0000 7f593d761640 1 --2- 172.21.15.47:0/2633075792 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59380d3bc0 0x7f59380d2cb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f592c0097a0 tx=0x7f592c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62ef869f2b21f4b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:37.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.506+0000 7f593cf60640 1 -- 172.21.15.47:0/2633075792 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c037070 con 0x7f59380d3bc0 2024-04-26T23:18:37.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.506+0000 7f593cf60640 1 -- 172.21.15.47:0/2633075792 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f592c032440 con 0x7f59380d3bc0 2024-04-26T23:18:37.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.506+0000 7f593cf60640 1 -- 172.21.15.47:0/2633075792 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c032810 con 0x7f59380d3bc0 2024-04-26T23:18:37.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.507+0000 7f59401c6640 1 -- 172.21.15.47:0/2633075792 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59380d3bc0 msgr2=0x7f59380d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:37.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.507+0000 7f59401c6640 1 --2- 172.21.15.47:0/2633075792 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59380d3bc0 0x7f59380d2cb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f592c0097a0 tx=0x7f592c02a1e0 comp rx=0 tx=0).stop 2024-04-26T23:18:37.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.507+0000 7f59401c6640 1 -- 172.21.15.47:0/2633075792 shutdown_connections 2024-04-26T23:18:37.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.507+0000 7f59401c6640 1 --2- 172.21.15.47:0/2633075792 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59380d3bc0 0x7f59380d2cb0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:37.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.507+0000 7f59401c6640 1 -- 172.21.15.47:0/2633075792 >> 172.21.15.47:0/2633075792 conn(0x7f59380d4f60 msgr2=0x7f59380d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:37.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.507+0000 7f59401c6640 1 -- 172.21.15.47:0/2633075792 shutdown_connections 2024-04-26T23:18:37.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.507+0000 7f59401c6640 1 -- 172.21.15.47:0/2633075792 wait complete. 2024-04-26T23:18:37.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.508+0000 7f59401c6640 1 Processor -- start 2024-04-26T23:18:37.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.508+0000 7f59401c6640 1 -- start start 2024-04-26T23:18:37.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.509+0000 7f59401c6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59380b5f20 0x7f59380cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:37.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.509+0000 7f59401c6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59380d3bc0 0x7f59381f5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:37.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.509+0000 7f59401c6640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5938162880 0x7f593815c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:37.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.509+0000 7f59401c6640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5938162c60 con 0x7f59380d3bc0 2024-04-26T23:18:37.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.509+0000 7f59401c6640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f59380d0d90 con 0x7f59380b5f20 2024-04-26T23:18:37.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.509+0000 7f59401c6640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f5938162de0 con 0x7f5938162880 2024-04-26T23:18:37.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.509+0000 7f593d761640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59380d3bc0 0x7f59381f5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:37.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.509+0000 7f593d761640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59380d3bc0 0x7f59381f5380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41616/0 (socket says 172.21.15.47:41616) 2024-04-26T23:18:37.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.509+0000 7f593d761640 1 -- 172.21.15.47:0/1432138913 learned_addr learned my addr 172.21.15.47:0/1432138913 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:37.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.509+0000 7f593e763640 1 --2- 172.21.15.47:0/1432138913 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5938162880 0x7f593815c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:37.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.509+0000 7f593df62640 1 --2- 172.21.15.47:0/1432138913 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59380b5f20 0x7f59380cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:37.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.509+0000 7f593d761640 1 -- 172.21.15.47:0/1432138913 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5938162880 msgr2=0x7f593815c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:37.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.509+0000 7f593d761640 1 --2- 172.21.15.47:0/1432138913 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5938162880 0x7f593815c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:37.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.509+0000 7f593d761640 1 -- 172.21.15.47:0/1432138913 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59380b5f20 msgr2=0x7f59380cbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:37.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.509+0000 7f593d761640 1 --2- 172.21.15.47:0/1432138913 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59380b5f20 0x7f59380cbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:37.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.509+0000 7f593d761640 1 -- 172.21.15.47:0/1432138913 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f593809ee00 con 0x7f59380d3bc0 2024-04-26T23:18:37.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.509+0000 7f593d761640 1 --2- 172.21.15.47:0/1432138913 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59380d3bc0 0x7f59381f5380 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f592c0097a0 tx=0x7f592c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:37.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.510+0000 7f59367fc640 1 -- 172.21.15.47:0/1432138913 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c037040 con 0x7f59380d3bc0 2024-04-26T23:18:37.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.510+0000 7f59367fc640 1 -- 172.21.15.47:0/1432138913 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f592c004250 con 0x7f59380d3bc0 2024-04-26T23:18:37.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.510+0000 7f59401c6640 1 -- 172.21.15.47:0/1432138913 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f593809f0e0 con 0x7f59380d3bc0 2024-04-26T23:18:37.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.510+0000 7f59367fc640 1 -- 172.21.15.47:0/1432138913 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c032e20 con 0x7f59380d3bc0 2024-04-26T23:18:37.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.510+0000 7f59401c6640 1 -- 172.21.15.47:0/1432138913 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59380a2bd0 con 0x7f59380d3bc0 2024-04-26T23:18:37.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.511+0000 7f59401c6640 1 -- 172.21.15.47:0/1432138913 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59380694f0 con 0x7f59380d3bc0 2024-04-26T23:18:37.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.511+0000 7f59367fc640 1 -- 172.21.15.47:0/1432138913 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f592c030770 con 0x7f59380d3bc0 2024-04-26T23:18:37.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.512+0000 7f59367fc640 1 --2- 172.21.15.47:0/1432138913 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f592406c8f0 0x7f592406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:37.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.512+0000 7f59367fc640 1 -- 172.21.15.47:0/1432138913 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f592c0ad0e0 con 0x7f59380d3bc0 2024-04-26T23:18:37.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.512+0000 7f593df62640 1 --2- 172.21.15.47:0/1432138913 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f592406c8f0 0x7f592406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:37.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.512+0000 7f593df62640 1 --2- 172.21.15.47:0/1432138913 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f592406c8f0 0x7f592406edb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5930006fd0 tx=0x7f5930008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:37.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.516+0000 7f59367fc640 1 -- 172.21.15.47:0/1432138913 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f59380d1800 con 0x7f59380d3bc0 2024-04-26T23:18:37.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.704+0000 7f59401c6640 1 -- 172.21.15.47:0/1432138913 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f59380d38c0 con 0x7f59380d3bc0 2024-04-26T23:18:37.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.705+0000 7f59367fc640 1 -- 172.21.15.47:0/1432138913 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18478 (secure 0 0 0) 0x7f592c07b260 con 0x7f59380d3bc0 2024-04-26T23:18:37.710 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:18:37.710 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":24,"fsid":"399006d3-4db1-43e6-ba81-a79a276eabfd","created":"2024-04-26T23:18:06.731211+0000","modified":"2024-04-26T23:18:36.402132+0000","last_up_change":"2024-04-26T23:18:17.808736+0000","last_in_change":"2024-04-26T23:18:08.945859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:18:16.937492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:18:29.033575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:18:30.318499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:18:34.014536+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ee091567-fea0-4a14-bf4e-0e16c20bb2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6819","nonce":326520779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6821","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6823","nonce":326520779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6829","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6831","nonce":326520779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6825","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6827","nonce":326520779}]},"public_addr":"172.21.15.47:6819/326520779","cluster_addr":"172.21.15.47:6823/326520779","heartbeat_back_addr":"172.21.15.47:6831/326520779","heartbeat_front_addr":"172.21.15.47:6827/326520779","state":["exists","up"]},{"osd":1,"uuid":"c14440a0-bba5-4931-a6db-b736c08e869b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6809","nonce":1912610931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6811","nonce":1912610931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6815","nonce":1912610931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6813","nonce":1912610931}]},"public_addr":"172.21.15.47:6809/1912610931","cluster_addr":"172.21.15.47:6811/1912610931","heartbeat_back_addr":"172.21.15.47:6815/1912610931","heartbeat_front_addr":"172.21.15.47:6813/1912610931","state":["exists","up"]},{"osd":2,"uuid":"3940f8de-c9b2-4cd3-a911-9c301729eb30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6801","nonce":2569869305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6803","nonce":2569869305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6807","nonce":2569869305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6805","nonce":2569869305}]},"public_addr":"172.21.15.47:6801/2569869305","cluster_addr":"172.21.15.47:6803/2569869305","heartbeat_back_addr":"172.21.15.47:6807/2569869305","heartbeat_front_addr":"172.21.15.47:6805/2569869305","state":["exists","up"]},{"osd":3,"uuid":"1286a67e-8958-426a-8c2e-bc85498c619c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6818","nonce":2791411683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6822","nonce":2791411683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6830","nonce":2791411683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6826","nonce":2791411683}]},"public_addr":"172.21.15.47:6818/2791411683","cluster_addr":"172.21.15.47:6822/2791411683","heartbeat_back_addr":"172.21.15.47:6830/2791411683","heartbeat_front_addr":"172.21.15.47:6826/2791411683","state":["exists","up"]},{"osd":4,"uuid":"d8ee2c20-e5b7-46c2-9999-b234070126cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6809","nonce":441313104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6811","nonce":441313104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6815","nonce":441313104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6813","nonce":441313104}]},"public_addr":"172.21.15.203:6809/441313104","cluster_addr":"172.21.15.203:6811/441313104","heartbeat_back_addr":"172.21.15.203:6815/441313104","heartbeat_front_addr":"172.21.15.203:6813/441313104","state":["exists","up"]},{"osd":5,"uuid":"b453a8f0-f324-4840-9ae0-d67bdadb9f8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6801","nonce":56838444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6803","nonce":56838444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6807","nonce":56838444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6805","nonce":56838444}]},"public_addr":"172.21.15.203:6801/56838444","cluster_addr":"172.21.15.203:6803/56838444","heartbeat_back_addr":"172.21.15.203:6807/56838444","heartbeat_front_addr":"172.21.15.203:6805/56838444","state":["exists","up"]},{"osd":6,"uuid":"a6d31951-2d4b-4e8d-a510-6283b01dc158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6825","nonce":2738517080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6827","nonce":2738517080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6831","nonce":2738517080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6829","nonce":2738517080}]},"public_addr":"172.21.15.203:6825/2738517080","cluster_addr":"172.21.15.203:6827/2738517080","heartbeat_back_addr":"172.21.15.203:6831/2738517080","heartbeat_front_addr":"172.21.15.203:6829/2738517080","state":["exists","up"]},{"osd":7,"uuid":"7119b8c0-fe96-402d-9e75-7b509d9d44d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6817","nonce":1315565058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6819","nonce":1315565058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6823","nonce":1315565058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6821","nonce":1315565058}]},"public_addr":"172.21.15.203:6817/1315565058","cluster_addr":"172.21.15.203:6819/1315565058","heartbeat_back_addr":"172.21.15.203:6823/1315565058","heartbeat_front_addr":"172.21.15.203:6821/1315565058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.121921+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.137494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.084814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.111593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.072430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.125066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.097867+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.057061+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:18:37.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.707+0000 7f59401c6640 1 -- 172.21.15.47:0/1432138913 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f592406c8f0 msgr2=0x7f592406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:37.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.707+0000 7f59401c6640 1 --2- 172.21.15.47:0/1432138913 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f592406c8f0 0x7f592406edb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5930006fd0 tx=0x7f5930008040 comp rx=0 tx=0).stop 2024-04-26T23:18:37.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.707+0000 7f59401c6640 1 -- 172.21.15.47:0/1432138913 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59380d3bc0 msgr2=0x7f59381f5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:37.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.707+0000 7f59401c6640 1 --2- 172.21.15.47:0/1432138913 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59380d3bc0 0x7f59381f5380 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f592c0097a0 tx=0x7f592c0046c0 comp rx=0 tx=0).stop 2024-04-26T23:18:37.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.707+0000 7f59401c6640 1 -- 172.21.15.47:0/1432138913 shutdown_connections 2024-04-26T23:18:37.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.707+0000 7f59401c6640 1 --2- 172.21.15.47:0/1432138913 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f592406c8f0 0x7f592406edb0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:37.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.707+0000 7f59401c6640 1 --2- 172.21.15.47:0/1432138913 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5938162880 0x7f593815c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:37.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.707+0000 7f59401c6640 1 --2- 172.21.15.47:0/1432138913 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59380d3bc0 0x7f59381f5380 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:37.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.707+0000 7f59401c6640 1 --2- 172.21.15.47:0/1432138913 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59380b5f20 0x7f59380cbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:37.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.707+0000 7f59401c6640 1 -- 172.21.15.47:0/1432138913 >> 172.21.15.47:0/1432138913 conn(0x7f59380d4f60 msgr2=0x7f593806cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:37.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.707+0000 7f59401c6640 1 -- 172.21.15.47:0/1432138913 shutdown_connections 2024-04-26T23:18:37.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.707+0000 7f59401c6640 1 -- 172.21.15.47:0/1432138913 wait complete. 2024-04-26T23:18:37.719 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:18:37.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.820+0000 7fd70ad97640 1 Processor -- start 2024-04-26T23:18:37.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.820+0000 7fd70ad97640 1 -- start start 2024-04-26T23:18:37.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.821+0000 7fd70ad97640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd7040b4d40 0x7fd7040b6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:37.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.821+0000 7fd70ad97640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7040b6c50 0x7fd7040aed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:37.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.821+0000 7fd70ad97640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd704072fb0 con 0x7fd7040b0b70 2024-04-26T23:18:37.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.821+0000 7fd70ad97640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fd704073bc0 con 0x7fd7040b4d40 2024-04-26T23:18:37.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.821+0000 7fd70ad97640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd704073290 con 0x7fd7040b6c50 2024-04-26T23:18:37.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.821+0000 7fd708b33640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd7040b0b70 0x7fd7040b0f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35614/0 (socket says 172.21.15.47:35614) 2024-04-26T23:18:37.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.821+0000 7fd708b33640 1 -- 172.21.15.47:0/4094234487 learned_addr learned my addr 172.21.15.47:0/4094234487 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:18:37.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.821+0000 7fd703fff640 1 --2- 172.21.15.47:0/4094234487 >> v2:172.21.15.203:3300/0 conn(0x7fd7040b4d40 0x7fd7040b6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:37.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.821+0000 7fd709334640 1 --2- 172.21.15.47:0/4094234487 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7040b6c50 0x7fd7040aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:37.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.821+0000 7fd7037fe640 1 -- 172.21.15.47:0/4094234487 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1585389308 0 0) 0x7fd704072fb0 con 0x7fd7040b0b70 2024-04-26T23:18:37.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.822+0000 7fd7037fe640 1 -- 172.21.15.47:0/4094234487 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6ec003520 con 0x7fd7040b0b70 2024-04-26T23:18:37.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.822+0000 7fd7037fe640 1 -- 172.21.15.47:0/4094234487 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3829506502 0 0) 0x7fd6ec003520 con 0x7fd7040b0b70 2024-04-26T23:18:37.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.822+0000 7fd7037fe640 1 -- 172.21.15.47:0/4094234487 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7040b6c50 msgr2=0x7fd7040aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:37.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.822+0000 7fd7037fe640 1 --2- 172.21.15.47:0/4094234487 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7040b6c50 0x7fd7040aed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:37.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.822+0000 7fd7037fe640 1 -- 172.21.15.47:0/4094234487 >> v2:172.21.15.203:3300/0 conn(0x7fd7040b4d40 msgr2=0x7fd7040b6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:37.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.822+0000 7fd7037fe640 1 --2- 172.21.15.47:0/4094234487 >> v2:172.21.15.203:3300/0 conn(0x7fd7040b4d40 0x7fd7040b6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:37.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.822+0000 7fd7037fe640 1 -- 172.21.15.47:0/4094234487 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd704073570 con 0x7fd7040b0b70 2024-04-26T23:18:37.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.822+0000 7fd7037fe640 1 -- 172.21.15.47:0/4094234487 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7fd6f4001670 con 0x7fd7040b0b70 2024-04-26T23:18:37.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.822+0000 7fd703fff640 1 --2- 172.21.15.47:0/4094234487 >> v2:172.21.15.203:3300/0 conn(0x7fd7040b4d40 0x7fd7040b6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:18:37.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.822+0000 7fd7037fe640 1 -- 172.21.15.47:0/4094234487 >> v1:172.21.15.47:6789/0 conn(0x7fd7040b0b70 legacy=0x7fd7040b0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:37.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.822+0000 7fd7037fe640 1 --2- 172.21.15.47:0/4094234487 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7040b6c50 0x7fd6ec0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:37.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.822+0000 7fd7037fe640 1 --2- 172.21.15.47:0/4094234487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6ec003f30 0x7fd6ec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:37.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.822+0000 7fd7037fe640 1 --2- 172.21.15.47:0/4094234487 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6ec004d40 0x7fd6ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:37.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.822+0000 7fd7037fe640 1 -- 172.21.15.47:0/4094234487 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd704073bc0 con 0x7fd6ec003f30 2024-04-26T23:18:37.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.822+0000 7fd7037fe640 1 -- 172.21.15.47:0/4094234487 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd704073290 con 0x7fd6ec004d40 2024-04-26T23:18:37.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.822+0000 7fd7037fe640 1 -- 172.21.15.47:0/4094234487 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd6ec00b9c0 con 0x7fd7040b6c50 2024-04-26T23:18:37.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.822+0000 7fd703fff640 1 --2- 172.21.15.47:0/4094234487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6ec003f30 0x7fd6ec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:37.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.822+0000 7fd708b33640 1 --2- 172.21.15.47:0/4094234487 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7040b6c50 0x7fd6ec0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:37.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.822+0000 7fd709334640 1 --2- 172.21.15.47:0/4094234487 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6ec004d40 0x7fd6ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:37.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.822+0000 7fd703fff640 1 -- 172.21.15.47:0/4094234487 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7040b6c50 msgr2=0x7fd6ec0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:37.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.822+0000 7fd703fff640 1 --2- 172.21.15.47:0/4094234487 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7040b6c50 0x7fd6ec0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:37.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.822+0000 7fd703fff640 1 -- 172.21.15.47:0/4094234487 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6ec004d40 msgr2=0x7fd6ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:37.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.822+0000 7fd703fff640 1 --2- 172.21.15.47:0/4094234487 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6ec004d40 0x7fd6ec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:37.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.822+0000 7fd703fff640 1 -- 172.21.15.47:0/4094234487 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6ec003520 con 0x7fd6ec003f30 2024-04-26T23:18:37.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.823+0000 7fd703fff640 1 --2- 172.21.15.47:0/4094234487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6ec003f30 0x7fd6ec004800 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8009c60 tx=0x7fd6ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c35cd9de7c572eb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:37.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.823+0000 7fd7037fe640 1 -- 172.21.15.47:0/4094234487 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f8036070 con 0x7fd6ec003f30 2024-04-26T23:18:37.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.823+0000 7fd7037fe640 1 -- 172.21.15.47:0/4094234487 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6f8031400 con 0x7fd6ec003f30 2024-04-26T23:18:37.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.823+0000 7fd7037fe640 1 -- 172.21.15.47:0/4094234487 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f8009860 con 0x7fd6ec003f30 2024-04-26T23:18:37.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.824+0000 7fd70ad97640 1 -- 172.21.15.47:0/4094234487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6ec003f30 msgr2=0x7fd6ec004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:37.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.824+0000 7fd70ad97640 1 --2- 172.21.15.47:0/4094234487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6ec003f30 0x7fd6ec004800 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8009c60 tx=0x7fd6ec005830 comp rx=0 tx=0).stop 2024-04-26T23:18:37.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.824+0000 7fd709334640 1 -- 172.21.15.47:0/4094234487 reap_dead start 2024-04-26T23:18:37.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.824+0000 7fd70ad97640 1 -- 172.21.15.47:0/4094234487 shutdown_connections 2024-04-26T23:18:37.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.824+0000 7fd70ad97640 1 -- 172.21.15.47:0/4094234487 >> 172.21.15.47:0/4094234487 conn(0x7fd7040b39b0 msgr2=0x7fd7040b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:37.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.824+0000 7fd70ad97640 1 -- 172.21.15.47:0/4094234487 shutdown_connections 2024-04-26T23:18:37.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.824+0000 7fd70ad97640 1 -- 172.21.15.47:0/4094234487 wait complete. 2024-04-26T23:18:37.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.825+0000 7fd70ad97640 1 Processor -- start 2024-04-26T23:18:37.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.825+0000 7fd70ad97640 1 -- start start 2024-04-26T23:18:37.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.826+0000 7fd70ad97640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6ec003f30 0x7fd70415cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:37.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.826+0000 7fd70ad97640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7040ce520 0x7fd704145eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:37.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.826+0000 7fd70ad97640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7041f83e0 0x7fd7041f8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:37.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.826+0000 7fd70ad97640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd7041604e0 con 0x7fd6ec003f30 2024-04-26T23:18:37.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.826+0000 7fd70ad97640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd704160360 con 0x7fd7040ce520 2024-04-26T23:18:37.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.826+0000 7fd70ad97640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd7040aed90 con 0x7fd7041f83e0 2024-04-26T23:18:37.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.826+0000 7fd708b33640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6ec003f30 0x7fd70415cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:37.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.826+0000 7fd708b33640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6ec003f30 0x7fd70415cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41634/0 (socket says 172.21.15.47:41634) 2024-04-26T23:18:37.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.826+0000 7fd708b33640 1 -- 172.21.15.47:0/1069186014 learned_addr learned my addr 172.21.15.47:0/1069186014 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:37.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.826+0000 7fd703fff640 1 --2- 172.21.15.47:0/1069186014 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7040ce520 0x7fd704145eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:37.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.826+0000 7fd709334640 1 --2- 172.21.15.47:0/1069186014 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7041f83e0 0x7fd7041f8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:37.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.826+0000 7fd708b33640 1 -- 172.21.15.47:0/1069186014 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7041f83e0 msgr2=0x7fd7041f8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:37.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.826+0000 7fd708b33640 1 --2- 172.21.15.47:0/1069186014 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7041f83e0 0x7fd7041f8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:37.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.826+0000 7fd708b33640 1 -- 172.21.15.47:0/1069186014 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7040ce520 msgr2=0x7fd704145eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:37.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.826+0000 7fd708b33640 1 --2- 172.21.15.47:0/1069186014 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7040ce520 0x7fd704145eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:37.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.826+0000 7fd708b33640 1 -- 172.21.15.47:0/1069186014 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd70415d290 con 0x7fd6ec003f30 2024-04-26T23:18:37.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.826+0000 7fd708b33640 1 --2- 172.21.15.47:0/1069186014 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6ec003f30 0x7fd70415cb90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4007370 tx=0x7fd6f4007830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:37.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.827+0000 7fd7017fa640 1 -- 172.21.15.47:0/1069186014 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f400f960 con 0x7fd6ec003f30 2024-04-26T23:18:37.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.827+0000 7fd7017fa640 1 -- 172.21.15.47:0/1069186014 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6f4018070 con 0x7fd6ec003f30 2024-04-26T23:18:37.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.827+0000 7fd7017fa640 1 -- 172.21.15.47:0/1069186014 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f400fd40 con 0x7fd6ec003f30 2024-04-26T23:18:37.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.827+0000 7fd70ad97640 1 -- 172.21.15.47:0/1069186014 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd70415d570 con 0x7fd6ec003f30 2024-04-26T23:18:37.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.827+0000 7fd70ad97640 1 -- 172.21.15.47:0/1069186014 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd70415db30 con 0x7fd6ec003f30 2024-04-26T23:18:37.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.828+0000 7fd70ad97640 1 -- 172.21.15.47:0/1069186014 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7040694f0 con 0x7fd6ec003f30 2024-04-26T23:18:37.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.828+0000 7fd7017fa640 1 -- 172.21.15.47:0/1069186014 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd6f401d460 con 0x7fd6ec003f30 2024-04-26T23:18:37.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.829+0000 7fd7017fa640 1 --2- 172.21.15.47:0/1069186014 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd6f006c920 0x7fd6f006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:37.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.829+0000 7fd7017fa640 1 -- 172.21.15.47:0/1069186014 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd6f408bd90 con 0x7fd6ec003f30 2024-04-26T23:18:37.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.829+0000 7fd703fff640 1 --2- 172.21.15.47:0/1069186014 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd6f006c920 0x7fd6f006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:37.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.829+0000 7fd703fff640 1 --2- 172.21.15.47:0/1069186014 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd6f006c920 0x7fd6f006ede0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8033000 tx=0x7fd6f8008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:37.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:37.833+0000 7fd7017fa640 1 -- 172.21.15.47:0/1069186014 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd6f40598c0 con 0x7fd6ec003f30 2024-04-26T23:18:38.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.022+0000 7fd70ad97640 1 -- 172.21.15.47:0/1069186014 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd7040704a0 con 0x7fd6ec003f30 2024-04-26T23:18:38.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.023+0000 7fd7017fa640 1 -- 172.21.15.47:0/1069186014 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18478 (secure 0 0 0) 0x7fd6f405dbc0 con 0x7fd6ec003f30 2024-04-26T23:18:38.024 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:18:38.024 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":24,"fsid":"399006d3-4db1-43e6-ba81-a79a276eabfd","created":"2024-04-26T23:18:06.731211+0000","modified":"2024-04-26T23:18:36.402132+0000","last_up_change":"2024-04-26T23:18:17.808736+0000","last_in_change":"2024-04-26T23:18:08.945859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:18:16.937492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:18:29.033575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:18:30.318499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:18:34.014536+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ee091567-fea0-4a14-bf4e-0e16c20bb2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6819","nonce":326520779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6821","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6823","nonce":326520779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6829","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6831","nonce":326520779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6825","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6827","nonce":326520779}]},"public_addr":"172.21.15.47:6819/326520779","cluster_addr":"172.21.15.47:6823/326520779","heartbeat_back_addr":"172.21.15.47:6831/326520779","heartbeat_front_addr":"172.21.15.47:6827/326520779","state":["exists","up"]},{"osd":1,"uuid":"c14440a0-bba5-4931-a6db-b736c08e869b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6809","nonce":1912610931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6811","nonce":1912610931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6815","nonce":1912610931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6813","nonce":1912610931}]},"public_addr":"172.21.15.47:6809/1912610931","cluster_addr":"172.21.15.47:6811/1912610931","heartbeat_back_addr":"172.21.15.47:6815/1912610931","heartbeat_front_addr":"172.21.15.47:6813/1912610931","state":["exists","up"]},{"osd":2,"uuid":"3940f8de-c9b2-4cd3-a911-9c301729eb30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6801","nonce":2569869305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6803","nonce":2569869305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6807","nonce":2569869305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6805","nonce":2569869305}]},"public_addr":"172.21.15.47:6801/2569869305","cluster_addr":"172.21.15.47:6803/2569869305","heartbeat_back_addr":"172.21.15.47:6807/2569869305","heartbeat_front_addr":"172.21.15.47:6805/2569869305","state":["exists","up"]},{"osd":3,"uuid":"1286a67e-8958-426a-8c2e-bc85498c619c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6818","nonce":2791411683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6822","nonce":2791411683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6830","nonce":2791411683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6826","nonce":2791411683}]},"public_addr":"172.21.15.47:6818/2791411683","cluster_addr":"172.21.15.47:6822/2791411683","heartbeat_back_addr":"172.21.15.47:6830/2791411683","heartbeat_front_addr":"172.21.15.47:6826/2791411683","state":["exists","up"]},{"osd":4,"uuid":"d8ee2c20-e5b7-46c2-9999-b234070126cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6809","nonce":441313104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6811","nonce":441313104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6815","nonce":441313104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6813","nonce":441313104}]},"public_addr":"172.21.15.203:6809/441313104","cluster_addr":"172.21.15.203:6811/441313104","heartbeat_back_addr":"172.21.15.203:6815/441313104","heartbeat_front_addr":"172.21.15.203:6813/441313104","state":["exists","up"]},{"osd":5,"uuid":"b453a8f0-f324-4840-9ae0-d67bdadb9f8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6801","nonce":56838444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6803","nonce":56838444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6807","nonce":56838444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6805","nonce":56838444}]},"public_addr":"172.21.15.203:6801/56838444","cluster_addr":"172.21.15.203:6803/56838444","heartbeat_back_addr":"172.21.15.203:6807/56838444","heartbeat_front_addr":"172.21.15.203:6805/56838444","state":["exists","up"]},{"osd":6,"uuid":"a6d31951-2d4b-4e8d-a510-6283b01dc158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6825","nonce":2738517080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6827","nonce":2738517080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6831","nonce":2738517080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6829","nonce":2738517080}]},"public_addr":"172.21.15.203:6825/2738517080","cluster_addr":"172.21.15.203:6827/2738517080","heartbeat_back_addr":"172.21.15.203:6831/2738517080","heartbeat_front_addr":"172.21.15.203:6829/2738517080","state":["exists","up"]},{"osd":7,"uuid":"7119b8c0-fe96-402d-9e75-7b509d9d44d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6817","nonce":1315565058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6819","nonce":1315565058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6823","nonce":1315565058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6821","nonce":1315565058}]},"public_addr":"172.21.15.203:6817/1315565058","cluster_addr":"172.21.15.203:6819/1315565058","heartbeat_back_addr":"172.21.15.203:6823/1315565058","heartbeat_front_addr":"172.21.15.203:6821/1315565058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.121921+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.137494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.084814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.111593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.072430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.125066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.097867+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.057061+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:18:38.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.025+0000 7fd70ad97640 1 -- 172.21.15.47:0/1069186014 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd6f006c920 msgr2=0x7fd6f006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:38.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.025+0000 7fd70ad97640 1 --2- 172.21.15.47:0/1069186014 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd6f006c920 0x7fd6f006ede0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8033000 tx=0x7fd6f8008030 comp rx=0 tx=0).stop 2024-04-26T23:18:38.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.025+0000 7fd70ad97640 1 -- 172.21.15.47:0/1069186014 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6ec003f30 msgr2=0x7fd70415cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:38.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.025+0000 7fd70ad97640 1 --2- 172.21.15.47:0/1069186014 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6ec003f30 0x7fd70415cb90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4007370 tx=0x7fd6f4007830 comp rx=0 tx=0).stop 2024-04-26T23:18:38.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.025+0000 7fd70ad97640 1 -- 172.21.15.47:0/1069186014 shutdown_connections 2024-04-26T23:18:38.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.025+0000 7fd70ad97640 1 --2- 172.21.15.47:0/1069186014 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd6f006c920 0x7fd6f006ede0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:38.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.025+0000 7fd70ad97640 1 --2- 172.21.15.47:0/1069186014 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7041f83e0 0x7fd7041f8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:38.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.025+0000 7fd70ad97640 1 --2- 172.21.15.47:0/1069186014 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7040ce520 0x7fd704145eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:38.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.025+0000 7fd70ad97640 1 --2- 172.21.15.47:0/1069186014 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6ec003f30 0x7fd70415cb90 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:38.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.025+0000 7fd70ad97640 1 -- 172.21.15.47:0/1069186014 >> 172.21.15.47:0/1069186014 conn(0x7fd7040b39b0 msgr2=0x7fd704070780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:38.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.025+0000 7fd70ad97640 1 -- 172.21.15.47:0/1069186014 shutdown_connections 2024-04-26T23:18:38.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.025+0000 7fd70ad97640 1 -- 172.21.15.47:0/1069186014 wait complete. 2024-04-26T23:18:38.051 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:18:38.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.151+0000 7f07cf36a640 1 Processor -- start 2024-04-26T23:18:38.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.152+0000 7f07cf36a640 1 -- start start 2024-04-26T23:18:38.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.152+0000 7f07cf36a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f07c806e270 0x7f07c80d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:38.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.152+0000 7f07cf36a640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f07c806d560 0x7f07c80ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:38.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.152+0000 7f07cf36a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07c80d2420 con 0x7f07c80d0070 2024-04-26T23:18:38.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.152+0000 7f07cf36a640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f07c8162d00 con 0x7f07c806d560 2024-04-26T23:18:38.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.152+0000 7f07cf36a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f07c80d2d10 con 0x7f07c806e270 2024-04-26T23:18:38.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.152+0000 7f07cd907640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f07c806d560 0x7f07c80ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:38.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.153+0000 7f07cd907640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f07c806d560 0x7f07c80ce190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41644/0 (socket says 172.21.15.47:41644) 2024-04-26T23:18:38.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.153+0000 7f07cd907640 1 -- 172.21.15.47:0/1201234011 learned_addr learned my addr 172.21.15.47:0/1201234011 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:38.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.153+0000 7f07cc905640 1 --2- 172.21.15.47:0/1201234011 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f07c806e270 0x7f07c80d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:38.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.153+0000 7f07cd907640 1 -- 172.21.15.47:0/1201234011 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f07c806e270 msgr2=0x7f07c80d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:38.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.153+0000 7f07cd907640 1 --2- 172.21.15.47:0/1201234011 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f07c806e270 0x7f07c80d1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:38.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.153+0000 7f07cd907640 1 -- 172.21.15.47:0/1201234011 >> v1:172.21.15.203:6789/0 conn(0x7f07c80d0070 legacy=0x7f07c806db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:38.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.153+0000 7f07bffff640 1 -- 172.21.15.47:0/1201234011 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3698117071 0 0) 0x7f07c80d2420 con 0x7f07c80d0070 2024-04-26T23:18:38.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.153+0000 7f07cd907640 1 -- 172.21.15.47:0/1201234011 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07c80cd6b0 con 0x7f07c806d560 2024-04-26T23:18:38.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.153+0000 7f07cd907640 1 --2- 172.21.15.47:0/1201234011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07c806d560 0x7f07c80ce190 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f07c0009e50 tx=0x7f07c002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dae757c728cde733 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:38.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.154+0000 7f07bffff640 1 -- 172.21.15.47:0/1201234011 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c0036070 con 0x7f07c806d560 2024-04-26T23:18:38.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.154+0000 7f07bffff640 1 -- 172.21.15.47:0/1201234011 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07c0031600 con 0x7f07c806d560 2024-04-26T23:18:38.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.154+0000 7f07bffff640 1 -- 172.21.15.47:0/1201234011 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c00319d0 con 0x7f07c806d560 2024-04-26T23:18:38.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.154+0000 7f07cf36a640 1 -- 172.21.15.47:0/1201234011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07c806d560 msgr2=0x7f07c80ce190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:38.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.154+0000 7f07cf36a640 1 --2- 172.21.15.47:0/1201234011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07c806d560 0x7f07c80ce190 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f07c0009e50 tx=0x7f07c002a7a0 comp rx=0 tx=0).stop 2024-04-26T23:18:38.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.154+0000 7f07cf36a640 1 -- 172.21.15.47:0/1201234011 shutdown_connections 2024-04-26T23:18:38.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.154+0000 7f07cf36a640 1 --2- 172.21.15.47:0/1201234011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07c806d560 0x7f07c80ce190 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:38.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.154+0000 7f07cf36a640 1 --2- 172.21.15.47:0/1201234011 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f07c806e270 0x7f07c80d1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:38.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.154+0000 7f07cf36a640 1 -- 172.21.15.47:0/1201234011 >> 172.21.15.47:0/1201234011 conn(0x7f07c80d2ec0 msgr2=0x7f07c80d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:38.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.154+0000 7f07cf36a640 1 -- 172.21.15.47:0/1201234011 shutdown_connections 2024-04-26T23:18:38.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.154+0000 7f07cf36a640 1 -- 172.21.15.47:0/1201234011 wait complete. 2024-04-26T23:18:38.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.155+0000 7f07cf36a640 1 Processor -- start 2024-04-26T23:18:38.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.155+0000 7f07cf36a640 1 -- start start 2024-04-26T23:18:38.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.156+0000 7f07cf36a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07c806d560 0x7f07c815cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:38.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.156+0000 7f07cf36a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f07c806e270 0x7f07c81f4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:38.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.156+0000 7f07cf36a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07c80d0070 0x7f07c815af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:38.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.156+0000 7f07cf36a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f07c80ce6d0 con 0x7f07c806d560 2024-04-26T23:18:38.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.156+0000 7f07cf36a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f07c80cdc70 con 0x7f07c80d0070 2024-04-26T23:18:38.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.156+0000 7f07cf36a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f07c80ce850 con 0x7f07c806e270 2024-04-26T23:18:38.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.156+0000 7f07cd106640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07c806d560 0x7f07c815cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:38.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.156+0000 7f07cd106640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07c806d560 0x7f07c815cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41660/0 (socket says 172.21.15.47:41660) 2024-04-26T23:18:38.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.156+0000 7f07cd106640 1 -- 172.21.15.47:0/3391321759 learned_addr learned my addr 172.21.15.47:0/3391321759 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:38.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.156+0000 7f07cd907640 1 --2- 172.21.15.47:0/3391321759 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07c80d0070 0x7f07c815af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:38.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.156+0000 7f07cc905640 1 --2- 172.21.15.47:0/3391321759 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f07c806e270 0x7f07c81f4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:38.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.157+0000 7f07cd106640 1 -- 172.21.15.47:0/3391321759 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f07c806e270 msgr2=0x7f07c81f4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:38.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.157+0000 7f07cd106640 1 --2- 172.21.15.47:0/3391321759 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f07c806e270 0x7f07c81f4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:38.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.157+0000 7f07cd106640 1 -- 172.21.15.47:0/3391321759 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07c80d0070 msgr2=0x7f07c815af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:38.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.157+0000 7f07cd106640 1 --2- 172.21.15.47:0/3391321759 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07c80d0070 0x7f07c815af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:38.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.157+0000 7f07cd106640 1 -- 172.21.15.47:0/3391321759 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07c815d4f0 con 0x7f07c806d560 2024-04-26T23:18:38.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.157+0000 7f07cd907640 1 --2- 172.21.15.47:0/3391321759 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07c80d0070 0x7f07c815af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:18:38.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.157+0000 7f07cd106640 1 --2- 172.21.15.47:0/3391321759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07c806d560 0x7f07c815cf10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f07c400c410 tx=0x7f07c400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:38.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.157+0000 7f07bdffb640 1 -- 172.21.15.47:0/3391321759 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c4017070 con 0x7f07c806d560 2024-04-26T23:18:38.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.157+0000 7f07bdffb640 1 -- 172.21.15.47:0/3391321759 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07c40035f0 con 0x7f07c806d560 2024-04-26T23:18:38.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.157+0000 7f07cf36a640 1 -- 172.21.15.47:0/3391321759 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07c815d7d0 con 0x7f07c806d560 2024-04-26T23:18:38.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.157+0000 7f07bdffb640 1 -- 172.21.15.47:0/3391321759 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c40039e0 con 0x7f07c806d560 2024-04-26T23:18:38.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.157+0000 7f07cf36a640 1 -- 172.21.15.47:0/3391321759 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07c815dd90 con 0x7f07c806d560 2024-04-26T23:18:38.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.158+0000 7f07cf36a640 1 -- 172.21.15.47:0/3391321759 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07c80694f0 con 0x7f07c806d560 2024-04-26T23:18:38.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.158+0000 7f07bdffb640 1 -- 172.21.15.47:0/3391321759 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f07c401a070 con 0x7f07c806d560 2024-04-26T23:18:38.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.159+0000 7f07bdffb640 1 --2- 172.21.15.47:0/3391321759 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f07ac06c8f0 0x7f07ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:38.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.159+0000 7f07bdffb640 1 -- 172.21.15.47:0/3391321759 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f07c408b2a0 con 0x7f07c806d560 2024-04-26T23:18:38.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.159+0000 7f07cc905640 1 --2- 172.21.15.47:0/3391321759 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f07ac06c8f0 0x7f07ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:38.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.160+0000 7f07cc905640 1 --2- 172.21.15.47:0/3391321759 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f07ac06c8f0 0x7f07ac06edb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f07c8162480 tx=0x7f07b8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:38.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.163+0000 7f07bdffb640 1 -- 172.21.15.47:0/3391321759 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f07c4059390 con 0x7f07c806d560 2024-04-26T23:18:38.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.388+0000 7f07cf36a640 1 -- 172.21.15.47:0/3391321759 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f07c80d0450 con 0x7f07c806d560 2024-04-26T23:18:38.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.389+0000 7f07bdffb640 1 -- 172.21.15.47:0/3391321759 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5180 (secure 0 0 0) 0x7f07c4058c90 con 0x7f07c806d560 2024-04-26T23:18:38.390 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:18:38.390 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/1553696401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1553696401},{"type":"v1","addr":"172.21.15.47:6834","nonce":1553696401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/3792767564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3792767564},{"type":"v1","addr":"172.21.15.203:6836","nonce":3792767564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4088100766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":4088100766},{"type":"v1","addr":"172.21.15.47:6835","nonce":4088100766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:18:31.367292+0000","modified":"2024-04-26T23:18:36.402282+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/997115011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":997115011},{"type":"v1","addr":"172.21.15.203:6837","nonce":997115011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4282,"qdb_cluster":[4282]},"id":1}]} 2024-04-26T23:18:38.390 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 7 2024-04-26T23:18:38.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.391+0000 7f07cf36a640 1 -- 172.21.15.47:0/3391321759 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f07ac06c8f0 msgr2=0x7f07ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:38.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.391+0000 7f07cf36a640 1 --2- 172.21.15.47:0/3391321759 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f07ac06c8f0 0x7f07ac06edb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f07c8162480 tx=0x7f07b8008040 comp rx=0 tx=0).stop 2024-04-26T23:18:38.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.391+0000 7f07cf36a640 1 -- 172.21.15.47:0/3391321759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07c806d560 msgr2=0x7f07c815cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:38.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.391+0000 7f07cf36a640 1 --2- 172.21.15.47:0/3391321759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07c806d560 0x7f07c815cf10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f07c400c410 tx=0x7f07c400c8d0 comp rx=0 tx=0).stop 2024-04-26T23:18:38.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.391+0000 7f07cf36a640 1 -- 172.21.15.47:0/3391321759 shutdown_connections 2024-04-26T23:18:38.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.391+0000 7f07cf36a640 1 --2- 172.21.15.47:0/3391321759 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f07ac06c8f0 0x7f07ac06edb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:38.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.391+0000 7f07cf36a640 1 --2- 172.21.15.47:0/3391321759 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07c80d0070 0x7f07c815af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:38.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.391+0000 7f07cf36a640 1 --2- 172.21.15.47:0/3391321759 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f07c806e270 0x7f07c81f4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:38.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.391+0000 7f07cf36a640 1 --2- 172.21.15.47:0/3391321759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07c806d560 0x7f07c815cf10 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:38.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.391+0000 7f07cf36a640 1 -- 172.21.15.47:0/3391321759 >> 172.21.15.47:0/3391321759 conn(0x7f07c80d2ec0 msgr2=0x7f07c8071270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:38.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.391+0000 7f07cf36a640 1 -- 172.21.15.47:0/3391321759 shutdown_connections 2024-04-26T23:18:38.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.391+0000 7f07cf36a640 1 -- 172.21.15.47:0/3391321759 wait complete. 2024-04-26T23:18:38.402 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:18:38.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.501+0000 7f45d4904640 1 Processor -- start 2024-04-26T23:18:38.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.502+0000 7f45d4904640 1 -- start start 2024-04-26T23:18:38.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.502+0000 7f45d4904640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f45d0162880 0x7f45d00d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:38.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.502+0000 7f45d4904640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f45d00d3bc0 0x7f45d00b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:38.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.502+0000 7f45d4904640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45d00b7e50 con 0x7f45d00b5f20 2024-04-26T23:18:38.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.502+0000 7f45d4904640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f45d00d0580 con 0x7f45d00d3bc0 2024-04-26T23:18:38.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.502+0000 7f45d4904640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f45d00d1800 con 0x7f45d0162880 2024-04-26T23:18:38.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.503+0000 7f45ced9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f45d00d3bc0 0x7f45d00b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:38.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.503+0000 7f45cdd9b640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f45d00b5f20 0x7f45d00d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35616/0 (socket says 172.21.15.47:35616) 2024-04-26T23:18:38.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.503+0000 7f45cdd9b640 1 -- 172.21.15.47:0/2203838554 learned_addr learned my addr 172.21.15.47:0/2203838554 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:18:38.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.503+0000 7f45ce59c640 1 --2- 172.21.15.47:0/2203838554 >> v2:172.21.15.203:3300/0 conn(0x7f45d0162880 0x7f45d00d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:38.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.503+0000 7f45ced9d640 1 -- 172.21.15.47:0/2203838554 >> v2:172.21.15.203:3300/0 conn(0x7f45d0162880 msgr2=0x7f45d00d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:38.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.503+0000 7f45ced9d640 1 --2- 172.21.15.47:0/2203838554 >> v2:172.21.15.203:3300/0 conn(0x7f45d0162880 0x7f45d00d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:38.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.503+0000 7f45ced9d640 1 -- 172.21.15.47:0/2203838554 >> v1:172.21.15.47:6789/0 conn(0x7f45d00b5f20 legacy=0x7f45d00d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:38.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.503+0000 7f45ced9d640 1 -- 172.21.15.47:0/2203838554 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45d00d07d0 con 0x7f45d00d3bc0 2024-04-26T23:18:38.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.503+0000 7f45cd59a640 1 -- 172.21.15.47:0/2203838554 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3951946357 0 0) 0x7f45d00b7e50 con 0x7f45d00b5f20 2024-04-26T23:18:38.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.503+0000 7f45ce59c640 1 --2- 172.21.15.47:0/2203838554 >> v2:172.21.15.203:3300/0 conn(0x7f45d0162880 0x7f45d00d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:18:38.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.504+0000 7f45ced9d640 1 --2- 172.21.15.47:0/2203838554 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45d00d3bc0 0x7f45d00b8300 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f45d008d490 tx=0x7f45c002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c36bc64c23a38ee9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:38.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.504+0000 7f45cd59a640 1 -- 172.21.15.47:0/2203838554 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c0007740 con 0x7f45d00d3bc0 2024-04-26T23:18:38.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.504+0000 7f45cd59a640 1 -- 172.21.15.47:0/2203838554 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45c00078e0 con 0x7f45d00d3bc0 2024-04-26T23:18:38.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.504+0000 7f45cd59a640 1 -- 172.21.15.47:0/2203838554 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c0007cb0 con 0x7f45d00d3bc0 2024-04-26T23:18:38.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.504+0000 7f45d4904640 1 -- 172.21.15.47:0/2203838554 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45d00d3bc0 msgr2=0x7f45d00b8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:38.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.504+0000 7f45d4904640 1 --2- 172.21.15.47:0/2203838554 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45d00d3bc0 0x7f45d00b8300 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f45d008d490 tx=0x7f45c002a680 comp rx=0 tx=0).stop 2024-04-26T23:18:38.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.504+0000 7f45d4904640 1 -- 172.21.15.47:0/2203838554 shutdown_connections 2024-04-26T23:18:38.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.504+0000 7f45d4904640 1 --2- 172.21.15.47:0/2203838554 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45d00d3bc0 0x7f45d00b8300 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:38.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.504+0000 7f45d4904640 1 --2- 172.21.15.47:0/2203838554 >> v2:172.21.15.203:3300/0 conn(0x7f45d0162880 0x7f45d00d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:38.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.504+0000 7f45d4904640 1 -- 172.21.15.47:0/2203838554 >> 172.21.15.47:0/2203838554 conn(0x7f45d00d4f60 msgr2=0x7f45d00d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:38.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.504+0000 7f45d4904640 1 -- 172.21.15.47:0/2203838554 shutdown_connections 2024-04-26T23:18:38.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.504+0000 7f45d4904640 1 -- 172.21.15.47:0/2203838554 wait complete. 2024-04-26T23:18:38.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.506+0000 7f45d4904640 1 Processor -- start 2024-04-26T23:18:38.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.506+0000 7f45d4904640 1 -- start start 2024-04-26T23:18:38.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.506+0000 7f45d4904640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45d00b5f20 0x7f45d0150fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:38.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.506+0000 7f45d4904640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45d00d3bc0 0x7f45d00b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:38.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.506+0000 7f45d4904640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45d0162880 0x7f45d00ba160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:38.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.506+0000 7f45d4904640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f45d0162c60 con 0x7f45d00b5f20 2024-04-26T23:18:38.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.506+0000 7f45d4904640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f45d00d0d90 con 0x7f45d00d3bc0 2024-04-26T23:18:38.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.506+0000 7f45d4904640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f45d0162de0 con 0x7f45d0162880 2024-04-26T23:18:38.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.507+0000 7f45ce59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45d00b5f20 0x7f45d0150fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:38.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.507+0000 7f45ce59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45d00b5f20 0x7f45d0150fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41670/0 (socket says 172.21.15.47:41670) 2024-04-26T23:18:38.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.507+0000 7f45ce59c640 1 -- 172.21.15.47:0/702068890 learned_addr learned my addr 172.21.15.47:0/702068890 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:38.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.507+0000 7f45cdd9b640 1 --2- 172.21.15.47:0/702068890 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45d00d3bc0 0x7f45d00b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:38.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.507+0000 7f45ced9d640 1 --2- 172.21.15.47:0/702068890 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45d0162880 0x7f45d00ba160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:38.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.507+0000 7f45ce59c640 1 -- 172.21.15.47:0/702068890 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45d0162880 msgr2=0x7f45d00ba160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:38.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.507+0000 7f45ce59c640 1 --2- 172.21.15.47:0/702068890 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45d0162880 0x7f45d00ba160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:38.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.507+0000 7f45ce59c640 1 -- 172.21.15.47:0/702068890 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45d00d3bc0 msgr2=0x7f45d00b1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:38.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.507+0000 7f45ce59c640 1 --2- 172.21.15.47:0/702068890 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45d00d3bc0 0x7f45d00b1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:38.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.507+0000 7f45ce59c640 1 -- 172.21.15.47:0/702068890 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45d014e560 con 0x7f45d00b5f20 2024-04-26T23:18:38.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.507+0000 7f45cdd9b640 1 --2- 172.21.15.47:0/702068890 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45d00d3bc0 0x7f45d00b1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:18:38.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.507+0000 7f45ce59c640 1 --2- 172.21.15.47:0/702068890 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45d00b5f20 0x7f45d0150fc0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f45c800e7e0 tx=0x7f45c800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:38.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.507+0000 7f45beffd640 1 -- 172.21.15.47:0/702068890 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c800f040 con 0x7f45d00b5f20 2024-04-26T23:18:38.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.507+0000 7f45beffd640 1 -- 172.21.15.47:0/702068890 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45c8004030 con 0x7f45d00b5f20 2024-04-26T23:18:38.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.507+0000 7f45d4904640 1 -- 172.21.15.47:0/702068890 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45d0151500 con 0x7f45d00b5f20 2024-04-26T23:18:38.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.507+0000 7f45beffd640 1 -- 172.21.15.47:0/702068890 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c8004400 con 0x7f45d00b5f20 2024-04-26T23:18:38.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.507+0000 7f45d4904640 1 -- 172.21.15.47:0/702068890 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45d014eae0 con 0x7f45d00b5f20 2024-04-26T23:18:38.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.508+0000 7f45d4904640 1 -- 172.21.15.47:0/702068890 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45d00694f0 con 0x7f45d00b5f20 2024-04-26T23:18:38.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.509+0000 7f45beffd640 1 -- 172.21.15.47:0/702068890 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f45c800c6d0 con 0x7f45d00b5f20 2024-04-26T23:18:38.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.509+0000 7f45beffd640 1 --2- 172.21.15.47:0/702068890 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f45b006c8f0 0x7f45b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:38.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.509+0000 7f45beffd640 1 -- 172.21.15.47:0/702068890 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f45c808bc20 con 0x7f45d00b5f20 2024-04-26T23:18:38.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.509+0000 7f45cdd9b640 1 --2- 172.21.15.47:0/702068890 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f45b006c8f0 0x7f45b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:38.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.510+0000 7f45cdd9b640 1 --2- 172.21.15.47:0/702068890 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f45b006c8f0 0x7f45b006edb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f45d009ef40 tx=0x7f45c4007ca0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:38.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.513+0000 7f45beffd640 1 -- 172.21.15.47:0/702068890 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f45c8017020 con 0x7f45d00b5f20 2024-04-26T23:18:38.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.698+0000 7f45d4904640 1 -- 172.21.15.47:0/702068890 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f45d00cf390 con 0x7f45d00b5f20 2024-04-26T23:18:38.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.699+0000 7f45beffd640 1 -- 172.21.15.47:0/702068890 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18478 (secure 0 0 0) 0x7f45c8059d10 con 0x7f45d00b5f20 2024-04-26T23:18:38.700 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:18:38.700 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":24,"fsid":"399006d3-4db1-43e6-ba81-a79a276eabfd","created":"2024-04-26T23:18:06.731211+0000","modified":"2024-04-26T23:18:36.402132+0000","last_up_change":"2024-04-26T23:18:17.808736+0000","last_in_change":"2024-04-26T23:18:08.945859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:18:16.937492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:18:29.033575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:18:30.318499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:18:34.014536+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ee091567-fea0-4a14-bf4e-0e16c20bb2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6819","nonce":326520779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6821","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6823","nonce":326520779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6829","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6831","nonce":326520779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6825","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6827","nonce":326520779}]},"public_addr":"172.21.15.47:6819/326520779","cluster_addr":"172.21.15.47:6823/326520779","heartbeat_back_addr":"172.21.15.47:6831/326520779","heartbeat_front_addr":"172.21.15.47:6827/326520779","state":["exists","up"]},{"osd":1,"uuid":"c14440a0-bba5-4931-a6db-b736c08e869b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6809","nonce":1912610931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6811","nonce":1912610931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6815","nonce":1912610931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6813","nonce":1912610931}]},"public_addr":"172.21.15.47:6809/1912610931","cluster_addr":"172.21.15.47:6811/1912610931","heartbeat_back_addr":"172.21.15.47:6815/1912610931","heartbeat_front_addr":"172.21.15.47:6813/1912610931","state":["exists","up"]},{"osd":2,"uuid":"3940f8de-c9b2-4cd3-a911-9c301729eb30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6801","nonce":2569869305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6803","nonce":2569869305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6807","nonce":2569869305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6805","nonce":2569869305}]},"public_addr":"172.21.15.47:6801/2569869305","cluster_addr":"172.21.15.47:6803/2569869305","heartbeat_back_addr":"172.21.15.47:6807/2569869305","heartbeat_front_addr":"172.21.15.47:6805/2569869305","state":["exists","up"]},{"osd":3,"uuid":"1286a67e-8958-426a-8c2e-bc85498c619c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6818","nonce":2791411683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6822","nonce":2791411683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6830","nonce":2791411683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6826","nonce":2791411683}]},"public_addr":"172.21.15.47:6818/2791411683","cluster_addr":"172.21.15.47:6822/2791411683","heartbeat_back_addr":"172.21.15.47:6830/2791411683","heartbeat_front_addr":"172.21.15.47:6826/2791411683","state":["exists","up"]},{"osd":4,"uuid":"d8ee2c20-e5b7-46c2-9999-b234070126cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6809","nonce":441313104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6811","nonce":441313104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6815","nonce":441313104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6813","nonce":441313104}]},"public_addr":"172.21.15.203:6809/441313104","cluster_addr":"172.21.15.203:6811/441313104","heartbeat_back_addr":"172.21.15.203:6815/441313104","heartbeat_front_addr":"172.21.15.203:6813/441313104","state":["exists","up"]},{"osd":5,"uuid":"b453a8f0-f324-4840-9ae0-d67bdadb9f8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6801","nonce":56838444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6803","nonce":56838444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6807","nonce":56838444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6805","nonce":56838444}]},"public_addr":"172.21.15.203:6801/56838444","cluster_addr":"172.21.15.203:6803/56838444","heartbeat_back_addr":"172.21.15.203:6807/56838444","heartbeat_front_addr":"172.21.15.203:6805/56838444","state":["exists","up"]},{"osd":6,"uuid":"a6d31951-2d4b-4e8d-a510-6283b01dc158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6825","nonce":2738517080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6827","nonce":2738517080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6831","nonce":2738517080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6829","nonce":2738517080}]},"public_addr":"172.21.15.203:6825/2738517080","cluster_addr":"172.21.15.203:6827/2738517080","heartbeat_back_addr":"172.21.15.203:6831/2738517080","heartbeat_front_addr":"172.21.15.203:6829/2738517080","state":["exists","up"]},{"osd":7,"uuid":"7119b8c0-fe96-402d-9e75-7b509d9d44d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6817","nonce":1315565058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6819","nonce":1315565058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6823","nonce":1315565058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6821","nonce":1315565058}]},"public_addr":"172.21.15.203:6817/1315565058","cluster_addr":"172.21.15.203:6819/1315565058","heartbeat_back_addr":"172.21.15.203:6823/1315565058","heartbeat_front_addr":"172.21.15.203:6821/1315565058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.121921+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.137494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.084814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.111593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.072430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.125066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.097867+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.057061+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:18:38.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.701+0000 7f45d4904640 1 -- 172.21.15.47:0/702068890 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f45b006c8f0 msgr2=0x7f45b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:38.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.701+0000 7f45d4904640 1 --2- 172.21.15.47:0/702068890 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f45b006c8f0 0x7f45b006edb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f45d009ef40 tx=0x7f45c4007ca0 comp rx=0 tx=0).stop 2024-04-26T23:18:38.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.701+0000 7f45d4904640 1 -- 172.21.15.47:0/702068890 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45d00b5f20 msgr2=0x7f45d0150fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:38.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.701+0000 7f45d4904640 1 --2- 172.21.15.47:0/702068890 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45d00b5f20 0x7f45d0150fc0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f45c800e7e0 tx=0x7f45c800eca0 comp rx=0 tx=0).stop 2024-04-26T23:18:38.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.701+0000 7f45d4904640 1 -- 172.21.15.47:0/702068890 shutdown_connections 2024-04-26T23:18:38.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.701+0000 7f45d4904640 1 --2- 172.21.15.47:0/702068890 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f45b006c8f0 0x7f45b006edb0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:38.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.701+0000 7f45d4904640 1 --2- 172.21.15.47:0/702068890 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45d0162880 0x7f45d00ba160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:38.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.701+0000 7f45d4904640 1 --2- 172.21.15.47:0/702068890 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45d00d3bc0 0x7f45d00b1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:38.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.701+0000 7f45d4904640 1 --2- 172.21.15.47:0/702068890 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45d00b5f20 0x7f45d0150fc0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:38.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.701+0000 7f45d4904640 1 -- 172.21.15.47:0/702068890 >> 172.21.15.47:0/702068890 conn(0x7f45d00d4f60 msgr2=0x7f45d00d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:38.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.701+0000 7f45d4904640 1 -- 172.21.15.47:0/702068890 shutdown_connections 2024-04-26T23:18:38.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:38.702+0000 7f45d4904640 1 -- 172.21.15.47:0/702068890 wait complete. 2024-04-26T23:18:39.115 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T23:18:31.367292+0000', 'modified': '2024-04-26T23:18:36.402282+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4282}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4282': {'gid': 4282, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.203:6837/997115011', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6835', 'nonce': 997115011}, {'type': 'v1', 'addr': '172.21.15.203:6837', 'nonce': 997115011}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4282, 'qdb_cluster': [4282]} 2024-04-26T23:18:39.116 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T23:18:39.116 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-26T23:18:39.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.220+0000 7fa2d8f40640 1 Processor -- start 2024-04-26T23:18:39.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.220+0000 7fa2d8f40640 1 -- start start 2024-04-26T23:18:39.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.220+0000 7fa2d8f40640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa2d40b3860 0x7fa2d416a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:39.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.221+0000 7fa2d8f40640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2d40d0ba0 con 0x7fa2d40d06f0 2024-04-26T23:18:39.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.221+0000 7fa2d8f40640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2d40d2590 con 0x7fa2d415f3a0 2024-04-26T23:18:39.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.221+0000 7fa2d8f40640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fa2d40d0e20 con 0x7fa2d40b3860 2024-04-26T23:18:39.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.221+0000 7fa2d2d9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa2d40b3860 0x7fa2d416a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:39.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.221+0000 7fa2d2d9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa2d40b3860 0x7fa2d416a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41680/0 (socket says 172.21.15.47:41680) 2024-04-26T23:18:39.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.221+0000 7fa2d2d9d640 1 -- 172.21.15.47:0/2107238086 learned_addr learned my addr 172.21.15.47:0/2107238086 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:39.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.221+0000 7fa2cbfff640 1 --1- 172.21.15.47:0/2107238086 >> v1:172.21.15.47:6789/0 conn(0x7fa2d40d06f0 0x7fa2d40d1e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35622/0 (socket says 172.21.15.47:35622) 2024-04-26T23:18:39.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.221+0000 7fa2d2d9d640 1 -- 172.21.15.47:0/2107238086 >> v1:172.21.15.203:6789/0 conn(0x7fa2d415f3a0 legacy=0x7fa2d40d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:39.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.221+0000 7fa2d2d9d640 1 -- 172.21.15.47:0/2107238086 >> v1:172.21.15.47:6789/0 conn(0x7fa2d40d06f0 legacy=0x7fa2d40d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:39.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.221+0000 7fa2d1d9b640 1 -- 172.21.15.47:0/2107238086 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 320734279 0 0) 0x7fa2bc001110 con 0x7fa2d40d06f0 2024-04-26T23:18:39.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.221+0000 7fa2d2d9d640 1 -- 172.21.15.47:0/2107238086 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2d40b1d70 con 0x7fa2d40b3860 2024-04-26T23:18:39.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.222+0000 7fa2d2d9d640 1 --2- 172.21.15.47:0/2107238086 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2d40b3860 0x7fa2d416a000 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc009e50 tx=0x7fa2cc02a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f63f377387ae8e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:39.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.222+0000 7fa2d1d9b640 1 -- 172.21.15.47:0/2107238086 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc036070 con 0x7fa2d40b3860 2024-04-26T23:18:39.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.222+0000 7fa2d1d9b640 1 -- 172.21.15.47:0/2107238086 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2cc031600 con 0x7fa2d40b3860 2024-04-26T23:18:39.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.222+0000 7fa2d1d9b640 1 -- 172.21.15.47:0/2107238086 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc0319d0 con 0x7fa2d40b3860 2024-04-26T23:18:39.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.222+0000 7fa2d8f40640 1 -- 172.21.15.47:0/2107238086 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2d40b3860 msgr2=0x7fa2d416a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:39.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.222+0000 7fa2d8f40640 1 --2- 172.21.15.47:0/2107238086 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2d40b3860 0x7fa2d416a000 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc009e50 tx=0x7fa2cc02a6d0 comp rx=0 tx=0).stop 2024-04-26T23:18:39.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.222+0000 7fa2d8f40640 1 -- 172.21.15.47:0/2107238086 shutdown_connections 2024-04-26T23:18:39.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.222+0000 7fa2d8f40640 1 --2- 172.21.15.47:0/2107238086 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2d40b3860 0x7fa2d416a000 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:39.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.222+0000 7fa2d8f40640 1 -- 172.21.15.47:0/2107238086 >> 172.21.15.47:0/2107238086 conn(0x7fa2d40d4f90 msgr2=0x7fa2d40d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:39.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.222+0000 7fa2d8f40640 1 -- 172.21.15.47:0/2107238086 shutdown_connections 2024-04-26T23:18:39.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.222+0000 7fa2d8f40640 1 -- 172.21.15.47:0/2107238086 wait complete. 2024-04-26T23:18:39.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.224+0000 7fa2d8f40640 1 Processor -- start 2024-04-26T23:18:39.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.224+0000 7fa2d8f40640 1 -- start start 2024-04-26T23:18:39.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.224+0000 7fa2d8f40640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2d40b3860 0x7fa2d41f5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:39.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.224+0000 7fa2d8f40640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2d40d06f0 0x7fa2d40b93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:39.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.224+0000 7fa2d8f40640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2d415f3a0 0x7fa2d4158910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:39.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.224+0000 7fa2d8f40640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d415f780 con 0x7fa2d415f3a0 2024-04-26T23:18:39.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.224+0000 7fa2d8f40640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d40b2330 con 0x7fa2d40b3860 2024-04-26T23:18:39.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.224+0000 7fa2d8f40640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa2d415f900 con 0x7fa2d40d06f0 2024-04-26T23:18:39.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.225+0000 7fa2d2d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2d415f3a0 0x7fa2d4158910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:39.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.225+0000 7fa2d2d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2d415f3a0 0x7fa2d4158910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41688/0 (socket says 172.21.15.47:41688) 2024-04-26T23:18:39.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.225+0000 7fa2d2d9d640 1 -- 172.21.15.47:0/3011769566 learned_addr learned my addr 172.21.15.47:0/3011769566 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:39.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.225+0000 7fa2d259c640 1 --2- 172.21.15.47:0/3011769566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2d40b3860 0x7fa2d41f5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:39.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.225+0000 7fa2cbfff640 1 --2- 172.21.15.47:0/3011769566 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2d40d06f0 0x7fa2d40b93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:39.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.225+0000 7fa2d2d9d640 1 -- 172.21.15.47:0/3011769566 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2d40d06f0 msgr2=0x7fa2d40b93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:39.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.225+0000 7fa2d2d9d640 1 --2- 172.21.15.47:0/3011769566 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2d40d06f0 0x7fa2d40b93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:39.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.225+0000 7fa2d2d9d640 1 -- 172.21.15.47:0/3011769566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2d40b3860 msgr2=0x7fa2d41f5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:39.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.225+0000 7fa2d2d9d640 1 --2- 172.21.15.47:0/3011769566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2d40b3860 0x7fa2d41f5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:39.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.225+0000 7fa2d2d9d640 1 -- 172.21.15.47:0/3011769566 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2d41f56c0 con 0x7fa2d415f3a0 2024-04-26T23:18:39.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.225+0000 7fa2d2d9d640 1 --2- 172.21.15.47:0/3011769566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2d415f3a0 0x7fa2d4158910 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc034040 tx=0x7fa2cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:39.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.225+0000 7fa2caffd640 1 -- 172.21.15.47:0/3011769566 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc041020 con 0x7fa2d415f3a0 2024-04-26T23:18:39.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.225+0000 7fa2caffd640 1 -- 172.21.15.47:0/3011769566 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2cc0077a0 con 0x7fa2d415f3a0 2024-04-26T23:18:39.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.225+0000 7fa2d8f40640 1 -- 172.21.15.47:0/3011769566 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2d40b9e40 con 0x7fa2d415f3a0 2024-04-26T23:18:39.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.225+0000 7fa2caffd640 1 -- 172.21.15.47:0/3011769566 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc036040 con 0x7fa2d415f3a0 2024-04-26T23:18:39.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.225+0000 7fa2d8f40640 1 -- 172.21.15.47:0/3011769566 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2d40a0fb0 con 0x7fa2d415f3a0 2024-04-26T23:18:39.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.227+0000 7fa2caffd640 1 -- 172.21.15.47:0/3011769566 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa2cc044020 con 0x7fa2d415f3a0 2024-04-26T23:18:39.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.227+0000 7fa2caffd640 1 --2- 172.21.15.47:0/3011769566 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa2b406c8f0 0x7fa2b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:39.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.227+0000 7fa2caffd640 1 -- 172.21.15.47:0/3011769566 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fa2cc0acc40 con 0x7fa2d415f3a0 2024-04-26T23:18:39.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.227+0000 7fa2d259c640 1 --2- 172.21.15.47:0/3011769566 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa2b406c8f0 0x7fa2b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:39.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.228+0000 7fa2d259c640 1 --2- 172.21.15.47:0/3011769566 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa2b406c8f0 0x7fa2b406edb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4006fd0 tx=0x7fa2c4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:39.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.228+0000 7fa2dbc57740 1 Processor -- start 2024-04-26T23:18:39.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.229+0000 7fa2dbc57740 1 -- start start 2024-04-26T23:18:39.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.229+0000 7fa2dbc57740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c610e4f440 0x55c610e4f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:39.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.229+0000 7fa2dbc57740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c610e6cce0 0x55c610e6d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:39.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.229+0000 7fa2dbc57740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c610e25120 0x55c610e6b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:39.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.229+0000 7fa2dbc57740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55c610d89740 con 0x55c610e4f440 2024-04-26T23:18:39.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.229+0000 7fa2dbc57740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c610d253a0 con 0x55c610e25120 2024-04-26T23:18:39.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.229+0000 7fa2dbc57740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x55c610d85700 con 0x55c610e6cce0 2024-04-26T23:18:39.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.229+0000 7fa2d259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c610e4f440 0x55c610e4f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:39.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.229+0000 7fa2d259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c610e4f440 0x55c610e4f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41698/0 (socket says 172.21.15.47:41698) 2024-04-26T23:18:39.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.229+0000 7fa2d259c640 1 -- 172.21.15.47:0/1412888566 learned_addr learned my addr 172.21.15.47:0/1412888566 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:39.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.229+0000 7fa2cbfff640 1 --2- 172.21.15.47:0/1412888566 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c610e6cce0 0x55c610e6d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:39.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.229+0000 7fa2d2d9d640 1 --2- 172.21.15.47:0/1412888566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c610e25120 0x55c610e6b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:39.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.230+0000 7fa2d259c640 1 -- 172.21.15.47:0/1412888566 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c610e6cce0 msgr2=0x55c610e6d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:39.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.230+0000 7fa2d259c640 1 --2- 172.21.15.47:0/1412888566 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c610e6cce0 0x55c610e6d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:39.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.230+0000 7fa2d259c640 1 -- 172.21.15.47:0/1412888566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c610e25120 msgr2=0x55c610e6b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:39.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.230+0000 7fa2d259c640 1 --2- 172.21.15.47:0/1412888566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c610e25120 0x55c610e6b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:39.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.230+0000 7fa2d259c640 1 -- 172.21.15.47:0/1412888566 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c610d41750 con 0x55c610e4f440 2024-04-26T23:18:39.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.230+0000 7fa2d259c640 1 --2- 172.21.15.47:0/1412888566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c610e4f440 0x55c610e4f820 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa2c400d8b0 tx=0x7fa2c4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba2bacb02a0108ad server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:39.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.230+0000 7fa2b3fff640 1 -- 172.21.15.47:0/1412888566 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c401a040 con 0x55c610e4f440 2024-04-26T23:18:39.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.230+0000 7fa2b3fff640 1 -- 172.21.15.47:0/1412888566 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2c4023070 con 0x55c610e4f440 2024-04-26T23:18:39.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.230+0000 7fa2b3fff640 1 -- 172.21.15.47:0/1412888566 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c4014990 con 0x55c610e4f440 2024-04-26T23:18:39.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.231+0000 7fa2dbc57740 1 -- 172.21.15.47:0/1412888566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c610e4f440 msgr2=0x55c610e4f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:39.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.231+0000 7fa2dbc57740 1 --2- 172.21.15.47:0/1412888566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c610e4f440 0x55c610e4f820 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa2c400d8b0 tx=0x7fa2c4010000 comp rx=0 tx=0).stop 2024-04-26T23:18:39.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.231+0000 7fa2dbc57740 1 -- 172.21.15.47:0/1412888566 shutdown_connections 2024-04-26T23:18:39.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.231+0000 7fa2dbc57740 1 --2- 172.21.15.47:0/1412888566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c610e25120 0x55c610e6b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:39.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.231+0000 7fa2dbc57740 1 --2- 172.21.15.47:0/1412888566 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c610e6cce0 0x55c610e6d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:39.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.231+0000 7fa2dbc57740 1 --2- 172.21.15.47:0/1412888566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c610e4f440 0x55c610e4f820 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:39.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.231+0000 7fa2dbc57740 1 -- 172.21.15.47:0/1412888566 >> 172.21.15.47:0/1412888566 conn(0x55c610e21a50 msgr2=0x55c610e6a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:39.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.231+0000 7fa2dbc57740 1 -- 172.21.15.47:0/1412888566 shutdown_connections 2024-04-26T23:18:39.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.231+0000 7fa2dbc57740 1 -- 172.21.15.47:0/1412888566 wait complete. 2024-04-26T23:18:39.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.231+0000 7fa2dbc57740 1 -- start start 2024-04-26T23:18:39.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.231+0000 7fa2dbc57740 1 Processor -- start 2024-04-26T23:18:39.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.232+0000 7fa2dbc57740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c610e4f440 0x55c610e6aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:39.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.232+0000 7fa2dbc57740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c610e6cce0 0x55c610e62ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:39.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.232+0000 7fa2dbc57740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c610e63220 0x55c610e52c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:39.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.232+0000 7fa2dbc57740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55c610e25ec0 con 0x55c610e4f440 2024-04-26T23:18:39.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.232+0000 7fa2dbc57740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c610d10fe0 con 0x55c610e6cce0 2024-04-26T23:18:39.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.232+0000 7fa2dbc57740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x55c610e1f980 con 0x55c610e63220 2024-04-26T23:18:39.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.232+0000 7fa2dbc57740 10 client.0 _ll_register_callbacks cb 0x55c610d27150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:18:39.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.232+0000 7fa2d259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c610e4f440 0x55c610e6aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:39.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.232+0000 7fa2d259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c610e4f440 0x55c610e6aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41700/0 (socket says 172.21.15.47:41700) 2024-04-26T23:18:39.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.232+0000 7fa2d259c640 1 -- 172.21.15.47:0/2898438621 learned_addr learned my addr 172.21.15.47:0/2898438621 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:39.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.232+0000 7fa2cbfff640 1 --2- 172.21.15.47:0/2898438621 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c610e6cce0 0x55c610e62ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:39.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.232+0000 7fa2d2d9d640 1 --2- 172.21.15.47:0/2898438621 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c610e63220 0x55c610e52c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:39.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.233+0000 7fa2d259c640 1 -- 172.21.15.47:0/2898438621 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c610e63220 msgr2=0x55c610e52c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:39.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.233+0000 7fa2d259c640 1 --2- 172.21.15.47:0/2898438621 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c610e63220 0x55c610e52c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:39.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.233+0000 7fa2d259c640 1 -- 172.21.15.47:0/2898438621 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c610e6cce0 msgr2=0x55c610e62ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:39.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.233+0000 7fa2d259c640 1 --2- 172.21.15.47:0/2898438621 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c610e6cce0 0x55c610e62ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:39.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.233+0000 7fa2d259c640 1 -- 172.21.15.47:0/2898438621 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c610a9b2d0 con 0x55c610e4f440 2024-04-26T23:18:39.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.233+0000 7fa2d259c640 1 --2- 172.21.15.47:0/2898438621 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c610e4f440 0x55c610e6aae0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4014c70 tx=0x7fa2c4004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=962f28088b20518d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:39.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.233+0000 7fa2b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-26T23:18:39.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.233+0000 7fa2b17fa640 1 -- 172.21.15.47:0/2898438621 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c4004230 con 0x55c610e4f440 2024-04-26T23:18:39.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.233+0000 7fa2b17fa640 1 -- 172.21.15.47:0/2898438621 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fa2c40043d0 con 0x55c610e4f440 2024-04-26T23:18:39.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.233+0000 7fa2dbc57740 1 -- 172.21.15.47:0/2898438621 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2c4023070 con 0x55c610e4f440 2024-04-26T23:18:39.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.233+0000 7fa2b17fa640 1 -- 172.21.15.47:0/2898438621 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7fa2c4034050 con 0x55c610e4f440 2024-04-26T23:18:39.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.233+0000 7fa2dbc57740 10 client.4434 fetch_fsmap learned FSMap version 7 2024-04-26T23:18:39.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.233+0000 7fa2dbc57740 1 -- 172.21.15.47:0/2898438621 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55c610b656b0 con 0x55c610e4f440 2024-04-26T23:18:39.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.233+0000 7fa2b17fa640 1 -- 172.21.15.47:0/2898438621 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2c4023070 con 0x55c610e4f440 2024-04-26T23:18:39.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.233+0000 7fa2b17fa640 1 -- 172.21.15.47:0/2898438621 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7fa2c4037020 con 0x55c610e4f440 2024-04-26T23:18:39.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.233+0000 7fa2dbc57740 10 client.4434 fetch_fsmap finished waiting for FSMap version 7 2024-04-26T23:18:39.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.233+0000 7fa2dbc57740 10 client.4434 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T23:18:39.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.233+0000 7fa2dbc57740 20 client.4434 populate_metadata read hostname 'smithi047' 2024-04-26T23:18:39.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.233+0000 7fa2dbc57740 1 --2- 172.21.15.47:0/2898438621 >> [v2:172.21.15.203:6835/997115011,v1:172.21.15.203:6837/997115011] conn(0x55c610e50e30 0x55c610e61600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:39.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.233+0000 7fa2dbc57740 4 client.4434 mds_command: new command op to 4282 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:18:39.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.233+0000 7fa2dbc57740 1 -- 172.21.15.47:0/2898438621 --> [v2:172.21.15.203:6835/997115011,v1:172.21.15.203:6837/997115011] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c610d97350 con 0x55c610e50e30 2024-04-26T23:18:39.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.234+0000 7fa2cbfff640 1 --2- 172.21.15.47:0/2898438621 >> [v2:172.21.15.203:6835/997115011,v1:172.21.15.203:6837/997115011] conn(0x55c610e50e30 0x55c610e61600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:39.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.234+0000 7fa2cbfff640 1 --2- 172.21.15.47:0/2898438621 >> [v2:172.21.15.203:6835/997115011,v1:172.21.15.203:6837/997115011] conn(0x55c610e50e30 0x55c610e61600 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e477e3d3e440d21f server_cookie=8b061a244f26964 in_seq=0 out_seq=0 2024-04-26T23:18:39.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.234+0000 7fa2b17fa640 10 client.4434 ms_handle_connect on v2:172.21.15.203:6835/997115011 2024-04-26T23:18:39.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.235+0000 7fa2b17fa640 1 -- 172.21.15.47:0/2898438621 <== mds.0 v2:172.21.15.203:6835/997115011 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c610d97350 con 0x55c610e50e30 2024-04-26T23:18:39.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.235+0000 7fa2b17fa640 10 client.4434 handle_command_reply: tid=0 2024-04-26T23:18:39.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.235+0000 7fa2dbc57740 1 client.4434 shutdown 2024-04-26T23:18:39.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.235+0000 7fa2dbc57740 1 -- 172.21.15.47:0/2898438621 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c610e4f440 msgr2=0x55c610e6aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:39.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.235+0000 7fa2dbc57740 1 --2- 172.21.15.47:0/2898438621 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c610e4f440 0x55c610e6aae0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4014c70 tx=0x7fa2c4004680 comp rx=0 tx=0).stop 2024-04-26T23:18:39.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.235+0000 7fa2dbc57740 1 -- 172.21.15.47:0/2898438621 shutdown_connections 2024-04-26T23:18:39.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.235+0000 7fa2dbc57740 1 --2- 172.21.15.47:0/2898438621 >> [v2:172.21.15.203:6835/997115011,v1:172.21.15.203:6837/997115011] conn(0x55c610e50e30 0x55c610e61600 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:39.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.235+0000 7fa2dbc57740 1 --2- 172.21.15.47:0/2898438621 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c610e63220 0x55c610e52c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:39.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.235+0000 7fa2dbc57740 1 --2- 172.21.15.47:0/2898438621 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c610e6cce0 0x55c610e62ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:39.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.235+0000 7fa2dbc57740 1 --2- 172.21.15.47:0/2898438621 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c610e4f440 0x55c610e6aae0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:39.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.235+0000 7fa2b17fa640 0 client.4434 ms_handle_reset on v2:172.21.15.203:6835/997115011 2024-04-26T23:18:39.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.235+0000 7fa2dbc57740 1 -- 172.21.15.47:0/2898438621 >> 172.21.15.47:0/2898438621 conn(0x55c610e25120 msgr2=0x55c610e6e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:39.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.235+0000 7fa2dbc57740 1 -- 172.21.15.47:0/2898438621 shutdown_connections 2024-04-26T23:18:39.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.235+0000 7fa2dbc57740 1 -- 172.21.15.47:0/2898438621 wait complete. 2024-04-26T23:18:39.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.235+0000 7fa2dbc57740 20 client.4434 trim_cache size 0 max 16384 2024-04-26T23:18:39.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.253+0000 7fa2dbc57740 1 Processor -- start 2024-04-26T23:18:39.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.253+0000 7fa2dbc57740 1 -- start start 2024-04-26T23:18:39.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.254+0000 7fa2dbc57740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c610e4f440 0x55c610e61580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:39.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.254+0000 7fa2dbc57740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c610e63220 0x55c610e6bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:39.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.254+0000 7fa2dbc57740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c610e6cce0 0x55c610e25500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:39.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.254+0000 7fa2dbc57740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d4201db0 con 0x55c610e6cce0 2024-04-26T23:18:39.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.254+0000 7fa2dbc57740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d4201140 con 0x55c610e63220 2024-04-26T23:18:39.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.254+0000 7fa2dbc57740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa2d4201760 con 0x55c610e4f440 2024-04-26T23:18:39.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.254+0000 7fa2d2d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c610e6cce0 0x55c610e25500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:39.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.254+0000 7fa2d2d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c610e6cce0 0x55c610e25500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41714/0 (socket says 172.21.15.47:41714) 2024-04-26T23:18:39.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.254+0000 7fa2d2d9d640 1 -- 172.21.15.47:0/3375891874 learned_addr learned my addr 172.21.15.47:0/3375891874 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:39.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.254+0000 7fa2d259c640 1 --2- 172.21.15.47:0/3375891874 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c610e4f440 0x55c610e61580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:39.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.254+0000 7fa2d2d9d640 1 -- 172.21.15.47:0/3375891874 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c610e4f440 msgr2=0x55c610e61580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:39.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.254+0000 7fa2cbfff640 1 --2- 172.21.15.47:0/3375891874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c610e63220 0x55c610e6bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:39.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.254+0000 7fa2d2d9d640 1 --2- 172.21.15.47:0/3375891874 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c610e4f440 0x55c610e61580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:39.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.254+0000 7fa2d2d9d640 1 -- 172.21.15.47:0/3375891874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c610e63220 msgr2=0x55c610e6bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:39.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.254+0000 7fa2d2d9d640 1 --2- 172.21.15.47:0/3375891874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c610e63220 0x55c610e6bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:39.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.254+0000 7fa2d2d9d640 1 -- 172.21.15.47:0/3375891874 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c610e4ef40 con 0x55c610e6cce0 2024-04-26T23:18:39.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.255+0000 7fa2cbfff640 1 --2- 172.21.15.47:0/3375891874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c610e63220 0x55c610e6bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:18:39.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.255+0000 7fa2d2d9d640 1 --2- 172.21.15.47:0/3375891874 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c610e6cce0 0x55c610e25500 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc07a330 tx=0x7fa2cc048040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2b4289b38ddc1e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:39.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.255+0000 7fa2b37fe640 1 -- 172.21.15.47:0/3375891874 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc0a8070 con 0x55c610e6cce0 2024-04-26T23:18:39.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.255+0000 7fa2b37fe640 1 -- 172.21.15.47:0/3375891874 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2cc07a630 con 0x55c610e6cce0 2024-04-26T23:18:39.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.255+0000 7fa2b37fe640 1 -- 172.21.15.47:0/3375891874 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc07e9e0 con 0x55c610e6cce0 2024-04-26T23:18:39.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.255+0000 7fa2dbc57740 1 -- 172.21.15.47:0/3375891874 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c610e6cce0 msgr2=0x55c610e25500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:39.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.255+0000 7fa2dbc57740 1 --2- 172.21.15.47:0/3375891874 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c610e6cce0 0x55c610e25500 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc07a330 tx=0x7fa2cc048040 comp rx=0 tx=0).stop 2024-04-26T23:18:39.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.255+0000 7fa2dbc57740 1 -- 172.21.15.47:0/3375891874 shutdown_connections 2024-04-26T23:18:39.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.255+0000 7fa2dbc57740 1 --2- 172.21.15.47:0/3375891874 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c610e6cce0 0x55c610e25500 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:39.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.255+0000 7fa2dbc57740 1 --2- 172.21.15.47:0/3375891874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c610e63220 0x55c610e6bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:39.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.255+0000 7fa2dbc57740 1 --2- 172.21.15.47:0/3375891874 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c610e4f440 0x55c610e61580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:39.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.255+0000 7fa2dbc57740 1 -- 172.21.15.47:0/3375891874 >> 172.21.15.47:0/3375891874 conn(0x55c610e25120 msgr2=0x55c610e628b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:39.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.255+0000 7fa2dbc57740 1 -- 172.21.15.47:0/3375891874 shutdown_connections 2024-04-26T23:18:39.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.255+0000 7fa2dbc57740 1 -- 172.21.15.47:0/3375891874 wait complete. 2024-04-26T23:18:39.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.256+0000 7fa2dbc57740 1 -- start start 2024-04-26T23:18:39.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.256+0000 7fa2dbc57740 1 Processor -- start 2024-04-26T23:18:39.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.257+0000 7fa2dbc57740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c610e4f440 0x55c610e51f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:39.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.257+0000 7fa2dbc57740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c610e63220 0x55c610e25500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:39.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.257+0000 7fa2dbc57740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c610e6cce0 0x55c610e25a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:39.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.257+0000 7fa2dbc57740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55c610e7bbc0 con 0x55c610e4f440 2024-04-26T23:18:39.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.257+0000 7fa2dbc57740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c610e7b810 con 0x55c610e63220 2024-04-26T23:18:39.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.257+0000 7fa2dbc57740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x55c610e51270 con 0x55c610e6cce0 2024-04-26T23:18:39.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.257+0000 7fa2dbc57740 10 client.0 _ll_register_callbacks cb 0x7fa2cc009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:18:39.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.257+0000 7fa2d259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c610e4f440 0x55c610e51f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:39.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.257+0000 7fa2d259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c610e4f440 0x55c610e51f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41726/0 (socket says 172.21.15.47:41726) 2024-04-26T23:18:39.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.257+0000 7fa2d259c640 1 -- 172.21.15.47:0/1370425007 learned_addr learned my addr 172.21.15.47:0/1370425007 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:39.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.257+0000 7fa2cbfff640 1 --2- 172.21.15.47:0/1370425007 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c610e63220 0x55c610e25500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:39.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.257+0000 7fa2d2d9d640 1 --2- 172.21.15.47:0/1370425007 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c610e6cce0 0x55c610e25a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:39.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.257+0000 7fa2d259c640 1 -- 172.21.15.47:0/1370425007 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c610e6cce0 msgr2=0x55c610e25a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:39.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.257+0000 7fa2d259c640 1 --2- 172.21.15.47:0/1370425007 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c610e6cce0 0x55c610e25a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:39.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.257+0000 7fa2d259c640 1 -- 172.21.15.47:0/1370425007 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c610e63220 msgr2=0x55c610e25500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:39.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.257+0000 7fa2d259c640 1 --2- 172.21.15.47:0/1370425007 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c610e63220 0x55c610e25500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:39.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.257+0000 7fa2d259c640 1 -- 172.21.15.47:0/1370425007 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c610e52b90 con 0x55c610e4f440 2024-04-26T23:18:39.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.257+0000 7fa2d259c640 1 --2- 172.21.15.47:0/1370425007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c610e4f440 0x55c610e51f30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4006fa0 tx=0x7fa2c4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e1bdd06f62ae6bf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:39.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.257+0000 7fa2b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-26T23:18:39.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.258+0000 7fa2b17fa640 1 -- 172.21.15.47:0/1370425007 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c402f590 con 0x55c610e4f440 2024-04-26T23:18:39.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.258+0000 7fa2b17fa640 1 -- 172.21.15.47:0/1370425007 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fa2c402f730 con 0x55c610e4f440 2024-04-26T23:18:39.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.258+0000 7fa2dbc57740 1 -- 172.21.15.47:0/1370425007 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2cc07a630 con 0x55c610e4f440 2024-04-26T23:18:39.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.258+0000 7fa2b17fa640 1 -- 172.21.15.47:0/1370425007 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7fa2c4045050 con 0x55c610e4f440 2024-04-26T23:18:39.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.258+0000 7fa2dbc57740 10 client.4440 fetch_fsmap learned FSMap version 7 2024-04-26T23:18:39.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.258+0000 7fa2dbc57740 1 -- 172.21.15.47:0/1370425007 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55c610eccf40 con 0x55c610e4f440 2024-04-26T23:18:39.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.258+0000 7fa2b17fa640 1 -- 172.21.15.47:0/1370425007 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2cc07a630 con 0x55c610e4f440 2024-04-26T23:18:39.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.258+0000 7fa2b17fa640 1 -- 172.21.15.47:0/1370425007 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7fa2c4048020 con 0x55c610e4f440 2024-04-26T23:18:39.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.258+0000 7fa2dbc57740 10 client.4440 fetch_fsmap finished waiting for FSMap version 7 2024-04-26T23:18:39.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.258+0000 7fa2dbc57740 10 client.4440 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T23:18:39.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.258+0000 7fa2dbc57740 20 client.4440 populate_metadata read hostname 'smithi047' 2024-04-26T23:18:39.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.258+0000 7fa2dbc57740 1 --2- 172.21.15.47:0/1370425007 >> [v2:172.21.15.203:6835/997115011,v1:172.21.15.203:6837/997115011] conn(0x55c610e6a980 0x55c610e6ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:39.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.258+0000 7fa2dbc57740 4 client.4440 mds_command: new command op to 4282 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T23:18:39.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.258+0000 7fa2dbc57740 1 -- 172.21.15.47:0/1370425007 --> [v2:172.21.15.203:6835/997115011,v1:172.21.15.203:6837/997115011] -- command(tid 0: {"prefix": "status"}) -- 0x55c610d911c0 con 0x55c610e6a980 2024-04-26T23:18:39.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.258+0000 7fa2cbfff640 1 --2- 172.21.15.47:0/1370425007 >> [v2:172.21.15.203:6835/997115011,v1:172.21.15.203:6837/997115011] conn(0x55c610e6a980 0x55c610e6ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:39.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.259+0000 7fa2cbfff640 1 --2- 172.21.15.47:0/1370425007 >> [v2:172.21.15.203:6835/997115011,v1:172.21.15.203:6837/997115011] conn(0x55c610e6a980 0x55c610e6ad60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a376b1bbfd454935 server_cookie=e0d631d8996fead8 in_seq=0 out_seq=0 2024-04-26T23:18:39.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.259+0000 7fa2b17fa640 10 client.4440 ms_handle_connect on v2:172.21.15.203:6835/997115011 2024-04-26T23:18:39.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.259+0000 7fa2b17fa640 1 -- 172.21.15.47:0/1370425007 <== mds.0 v2:172.21.15.203:6835/997115011 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x55c610d911c0 con 0x55c610e6a980 2024-04-26T23:18:39.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.259+0000 7fa2b17fa640 10 client.4440 handle_command_reply: tid=0 2024-04-26T23:18:39.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.259+0000 7fa2dbc57740 1 client.4440 shutdown 2024-04-26T23:18:39.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.259+0000 7fa2dbc57740 1 -- 172.21.15.47:0/1370425007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c610e4f440 msgr2=0x55c610e51f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:39.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.259+0000 7fa2dbc57740 1 --2- 172.21.15.47:0/1370425007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c610e4f440 0x55c610e51f30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4006fa0 tx=0x7fa2c4014000 comp rx=0 tx=0).stop 2024-04-26T23:18:39.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.259+0000 7fa2dbc57740 1 -- 172.21.15.47:0/1370425007 shutdown_connections 2024-04-26T23:18:39.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.259+0000 7fa2dbc57740 1 --2- 172.21.15.47:0/1370425007 >> [v2:172.21.15.203:6835/997115011,v1:172.21.15.203:6837/997115011] conn(0x55c610e6a980 0x55c610e6ad60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:39.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.259+0000 7fa2dbc57740 1 --2- 172.21.15.47:0/1370425007 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c610e6cce0 0x55c610e25a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:39.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.259+0000 7fa2dbc57740 1 --2- 172.21.15.47:0/1370425007 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c610e63220 0x55c610e25500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:39.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.259+0000 7fa2dbc57740 1 --2- 172.21.15.47:0/1370425007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c610e4f440 0x55c610e51f30 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:39.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.259+0000 7fa2b17fa640 0 client.4440 ms_handle_reset on v2:172.21.15.203:6835/997115011 2024-04-26T23:18:39.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.259+0000 7fa2dbc57740 1 -- 172.21.15.47:0/1370425007 >> 172.21.15.47:0/1370425007 conn(0x55c610e25120 msgr2=0x55c610e24810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:39.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.259+0000 7fa2dbc57740 1 -- 172.21.15.47:0/1370425007 shutdown_connections 2024-04-26T23:18:39.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.259+0000 7fa2dbc57740 1 -- 172.21.15.47:0/1370425007 wait complete. 2024-04-26T23:18:39.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.259+0000 7fa2dbc57740 20 client.4440 trim_cache size 0 max 16384 2024-04-26T23:18:39.266 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-26T23:18:39.266 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "399006d3-4db1-43e6-ba81-a79a276eabfd", 2024-04-26T23:18:39.266 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 0, 2024-04-26T23:18:39.267 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 4282, 2024-04-26T23:18:39.267 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-04-26T23:18:39.267 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-04-26T23:18:39.267 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-04-26T23:18:39.267 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 7.2674581820000004, 2024-04-26T23:18:39.267 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 7, 2024-04-26T23:18:39.267 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 24, 2024-04-26T23:18:39.267 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 24, 2024-04-26T23:18:39.267 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 14.733676644999999 2024-04-26T23:18:39.267 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-26T23:18:39.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.260+0000 7fa2d8f40640 1 -- 172.21.15.47:0/3011769566 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa2b406c8f0 msgr2=0x7fa2b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:39.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.260+0000 7fa2d8f40640 1 --2- 172.21.15.47:0/3011769566 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa2b406c8f0 0x7fa2b406edb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4006fd0 tx=0x7fa2c4008040 comp rx=0 tx=0).stop 2024-04-26T23:18:39.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.260+0000 7fa2d8f40640 1 -- 172.21.15.47:0/3011769566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2d415f3a0 msgr2=0x7fa2d4158910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:39.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.260+0000 7fa2d8f40640 1 --2- 172.21.15.47:0/3011769566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2d415f3a0 0x7fa2d4158910 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc034040 tx=0x7fa2cc004a40 comp rx=0 tx=0).stop 2024-04-26T23:18:39.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.260+0000 7fa2d8f40640 1 -- 172.21.15.47:0/3011769566 shutdown_connections 2024-04-26T23:18:39.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.260+0000 7fa2d8f40640 1 --2- 172.21.15.47:0/3011769566 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa2b406c8f0 0x7fa2b406edb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:39.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.260+0000 7fa2d8f40640 1 --2- 172.21.15.47:0/3011769566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2d415f3a0 0x7fa2d4158910 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:39.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.260+0000 7fa2d8f40640 1 --2- 172.21.15.47:0/3011769566 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2d40d06f0 0x7fa2d40b93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:39.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.260+0000 7fa2d8f40640 1 --2- 172.21.15.47:0/3011769566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2d40b3860 0x7fa2d41f5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:39.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.260+0000 7fa2d8f40640 1 -- 172.21.15.47:0/3011769566 >> 172.21.15.47:0/3011769566 conn(0x7fa2d40d4f90 msgr2=0x7fa2d40d14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:39.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.260+0000 7fa2d8f40640 1 -- 172.21.15.47:0/3011769566 shutdown_connections 2024-04-26T23:18:39.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.260+0000 7fa2d8f40640 1 -- 172.21.15.47:0/3011769566 wait complete. 2024-04-26T23:18:39.271 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T23:18:39.271 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-26T23:18:39.271 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T23:18:39.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.371+0000 7f2ed02da640 1 Processor -- start 2024-04-26T23:18:39.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.371+0000 7f2ed02da640 1 -- start start 2024-04-26T23:18:39.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.372+0000 7f2ed02da640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ec8072fb0 0x7f2ec80b56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:39.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.372+0000 7f2ed02da640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2ec80b06f0 0x7f2ec806fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:39.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.372+0000 7f2ed02da640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ec80b5e30 con 0x7f2ec8073390 2024-04-26T23:18:39.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.372+0000 7f2ed02da640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f2ec80b0b80 con 0x7f2ec80b06f0 2024-04-26T23:18:39.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.372+0000 7f2ed02da640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f2ec80b5c00 con 0x7f2ec8072fb0 2024-04-26T23:18:39.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.372+0000 7f2ece877640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2ec80b06f0 0x7f2ec806fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:39.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.372+0000 7f2ecd875640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f2ec8073390 0x7f2ec806f740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35634/0 (socket says 172.21.15.47:35634) 2024-04-26T23:18:39.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.372+0000 7f2ecd875640 1 -- 172.21.15.47:0/1431304211 learned_addr learned my addr 172.21.15.47:0/1431304211 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:18:39.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.373+0000 7f2ece076640 1 --2- 172.21.15.47:0/1431304211 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ec8072fb0 0x7f2ec80b56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:39.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.373+0000 7f2ece877640 1 -- 172.21.15.47:0/1431304211 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ec8072fb0 msgr2=0x7f2ec80b56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:39.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.373+0000 7f2ece877640 1 --2- 172.21.15.47:0/1431304211 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ec8072fb0 0x7f2ec80b56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:39.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.373+0000 7f2ece877640 1 -- 172.21.15.47:0/1431304211 >> v1:172.21.15.47:6789/0 conn(0x7f2ec8073390 legacy=0x7f2ec806f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:39.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.373+0000 7f2ece877640 1 -- 172.21.15.47:0/1431304211 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ec80b6110 con 0x7f2ec80b06f0 2024-04-26T23:18:39.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.373+0000 7f2ecd074640 1 -- 172.21.15.47:0/1431304211 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1022406743 0 0) 0x7f2ec80b5e30 con 0x7f2ec8073390 2024-04-26T23:18:39.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.373+0000 7f2ece877640 1 --2- 172.21.15.47:0/1431304211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ec80b06f0 0x7f2ec806fe50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8009d90 tx=0x7f2eb802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=445c77bb9400db6b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:39.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.374+0000 7f2ecd074640 1 -- 172.21.15.47:0/1431304211 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb8007740 con 0x7f2ec80b06f0 2024-04-26T23:18:39.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.374+0000 7f2ecd074640 1 -- 172.21.15.47:0/1431304211 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2eb80078e0 con 0x7f2ec80b06f0 2024-04-26T23:18:39.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.374+0000 7f2ecd074640 1 -- 172.21.15.47:0/1431304211 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb8007cb0 con 0x7f2ec80b06f0 2024-04-26T23:18:39.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.374+0000 7f2ed02da640 1 -- 172.21.15.47:0/1431304211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ec80b06f0 msgr2=0x7f2ec806fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:39.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.374+0000 7f2ed02da640 1 --2- 172.21.15.47:0/1431304211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ec80b06f0 0x7f2ec806fe50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8009d90 tx=0x7f2eb802a660 comp rx=0 tx=0).stop 2024-04-26T23:18:39.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.374+0000 7f2ed02da640 1 -- 172.21.15.47:0/1431304211 shutdown_connections 2024-04-26T23:18:39.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.374+0000 7f2ed02da640 1 --2- 172.21.15.47:0/1431304211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ec80b06f0 0x7f2ec806fe50 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:39.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.374+0000 7f2ed02da640 1 --2- 172.21.15.47:0/1431304211 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ec8072fb0 0x7f2ec80b56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:39.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.374+0000 7f2ed02da640 1 -- 172.21.15.47:0/1431304211 >> 172.21.15.47:0/1431304211 conn(0x7f2ec80b39b0 msgr2=0x7f2ec80b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:39.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.374+0000 7f2ed02da640 1 -- 172.21.15.47:0/1431304211 shutdown_connections 2024-04-26T23:18:39.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.374+0000 7f2ed02da640 1 -- 172.21.15.47:0/1431304211 wait complete. 2024-04-26T23:18:39.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.375+0000 7f2ed02da640 1 Processor -- start 2024-04-26T23:18:39.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.376+0000 7f2ed02da640 1 -- start start 2024-04-26T23:18:39.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.376+0000 7f2ed02da640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ec8072fb0 0x7f2ec809ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:39.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.376+0000 7f2ed02da640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ec8073390 0x7f2ec815df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:39.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.376+0000 7f2ed02da640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ec80b06f0 0x7f2ec814b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:39.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.376+0000 7f2ed02da640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec80737f0 con 0x7f2ec80b06f0 2024-04-26T23:18:39.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.376+0000 7f2ed02da640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec806f2b0 con 0x7f2ec8072fb0 2024-04-26T23:18:39.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.376+0000 7f2ed02da640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f2ec815c160 con 0x7f2ec8073390 2024-04-26T23:18:39.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.376+0000 7f2ece877640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ec80b06f0 0x7f2ec814b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:39.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.376+0000 7f2ece877640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ec80b06f0 0x7f2ec814b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41736/0 (socket says 172.21.15.47:41736) 2024-04-26T23:18:39.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.376+0000 7f2ece877640 1 -- 172.21.15.47:0/2148669452 learned_addr learned my addr 172.21.15.47:0/2148669452 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:39.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.377+0000 7f2ece076640 1 --2- 172.21.15.47:0/2148669452 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ec8072fb0 0x7f2ec809ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:39.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.377+0000 7f2ecd875640 1 --2- 172.21.15.47:0/2148669452 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ec8073390 0x7f2ec815df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:39.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.377+0000 7f2ece877640 1 -- 172.21.15.47:0/2148669452 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ec8073390 msgr2=0x7f2ec815df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:39.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.377+0000 7f2ece877640 1 --2- 172.21.15.47:0/2148669452 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ec8073390 0x7f2ec815df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:39.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.377+0000 7f2ece877640 1 -- 172.21.15.47:0/2148669452 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ec8072fb0 msgr2=0x7f2ec809ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:39.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.377+0000 7f2ece877640 1 --2- 172.21.15.47:0/2148669452 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ec8072fb0 0x7f2ec809ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:39.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.377+0000 7f2ece877640 1 -- 172.21.15.47:0/2148669452 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ec81f5a20 con 0x7f2ec80b06f0 2024-04-26T23:18:39.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.377+0000 7f2ece076640 1 --2- 172.21.15.47:0/2148669452 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ec8072fb0 0x7f2ec809ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:18:39.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.377+0000 7f2ece877640 1 --2- 172.21.15.47:0/2148669452 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ec80b06f0 0x7f2ec814b780 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8009eb0 tx=0x7f2eb8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:39.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.377+0000 7f2ec6ffd640 1 -- 172.21.15.47:0/2148669452 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb803f070 con 0x7f2ec80b06f0 2024-04-26T23:18:39.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.377+0000 7f2ec6ffd640 1 -- 172.21.15.47:0/2148669452 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2eb8031bc0 con 0x7f2ec80b06f0 2024-04-26T23:18:39.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.377+0000 7f2ed02da640 1 -- 172.21.15.47:0/2148669452 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ec81f5cc0 con 0x7f2ec80b06f0 2024-04-26T23:18:39.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.377+0000 7f2ec6ffd640 1 -- 172.21.15.47:0/2148669452 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb8009450 con 0x7f2ec80b06f0 2024-04-26T23:18:39.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.377+0000 7f2ed02da640 1 -- 172.21.15.47:0/2148669452 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ec81f6280 con 0x7f2ec80b06f0 2024-04-26T23:18:39.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.378+0000 7f2ed02da640 1 -- 172.21.15.47:0/2148669452 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ec80694f0 con 0x7f2ec80b06f0 2024-04-26T23:18:39.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.379+0000 7f2ec6ffd640 1 -- 172.21.15.47:0/2148669452 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2eb803b410 con 0x7f2ec80b06f0 2024-04-26T23:18:39.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.379+0000 7f2ec6ffd640 1 --2- 172.21.15.47:0/2148669452 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f2eb406c8f0 0x7f2eb406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:39.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.379+0000 7f2ec6ffd640 1 -- 172.21.15.47:0/2148669452 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f2eb80ad630 con 0x7f2ec80b06f0 2024-04-26T23:18:39.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.379+0000 7f2ece076640 1 --2- 172.21.15.47:0/2148669452 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f2eb406c8f0 0x7f2eb406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:39.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.380+0000 7f2ece076640 1 --2- 172.21.15.47:0/2148669452 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f2eb406c8f0 0x7f2eb406edb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2ec0009b90 tx=0x7f2ec0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:39.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.384+0000 7f2ec6ffd640 1 -- 172.21.15.47:0/2148669452 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2eb807b160 con 0x7f2ec80b06f0 2024-04-26T23:18:39.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.610+0000 7f2ed02da640 1 -- 172.21.15.47:0/2148669452 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2ec809bf90 con 0x7f2ec80b06f0 2024-04-26T23:18:39.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.610+0000 7f2ec6ffd640 1 -- 172.21.15.47:0/2148669452 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+88 (secure 0 0 0) 0x7f2eb807f460 con 0x7f2ec80b06f0 2024-04-26T23:18:39.611 INFO:teuthology.orchestra.run.smithi047.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-26T23:18:39.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.612+0000 7f2ed02da640 1 -- 172.21.15.47:0/2148669452 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f2eb406c8f0 msgr2=0x7f2eb406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:39.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.612+0000 7f2ed02da640 1 --2- 172.21.15.47:0/2148669452 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f2eb406c8f0 0x7f2eb406edb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2ec0009b90 tx=0x7f2ec0008040 comp rx=0 tx=0).stop 2024-04-26T23:18:39.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.612+0000 7f2ed02da640 1 -- 172.21.15.47:0/2148669452 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ec80b06f0 msgr2=0x7f2ec814b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:39.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.612+0000 7f2ed02da640 1 --2- 172.21.15.47:0/2148669452 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ec80b06f0 0x7f2ec814b780 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8009eb0 tx=0x7f2eb8004a40 comp rx=0 tx=0).stop 2024-04-26T23:18:39.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.612+0000 7f2ed02da640 1 -- 172.21.15.47:0/2148669452 shutdown_connections 2024-04-26T23:18:39.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.612+0000 7f2ed02da640 1 --2- 172.21.15.47:0/2148669452 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f2eb406c8f0 0x7f2eb406edb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:39.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.612+0000 7f2ed02da640 1 --2- 172.21.15.47:0/2148669452 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ec80b06f0 0x7f2ec814b780 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:39.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.612+0000 7f2ed02da640 1 --2- 172.21.15.47:0/2148669452 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ec8073390 0x7f2ec815df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:39.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.612+0000 7f2ed02da640 1 --2- 172.21.15.47:0/2148669452 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ec8072fb0 0x7f2ec809ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:39.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.612+0000 7f2ed02da640 1 -- 172.21.15.47:0/2148669452 >> 172.21.15.47:0/2148669452 conn(0x7f2ec80b39b0 msgr2=0x7f2ec815c580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:39.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.612+0000 7f2ed02da640 1 -- 172.21.15.47:0/2148669452 shutdown_connections 2024-04-26T23:18:39.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39.612+0000 7f2ed02da640 1 -- 172.21.15.47:0/2148669452 wait complete. 2024-04-26T23:18:39.623 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T23:18:39.624 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T23:18:39.624 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi047.front.sepia.ceph.com 2024-04-26T23:18:39.624 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-26T23:18:39.624 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-26T23:18:39.624 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T23:18:39.624 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T23:18:39.624 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T23:18:39.624 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-26T23:18:39.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:18:39.624 DEBUG:teuthology.orchestra.run.smithi047:> ip addr 2024-04-26T23:18:39.640 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T23:18:39.640 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T23:18:39.640 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T23:18:39.640 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:18:39.640 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2024-04-26T23:18:39.640 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:18:39.640 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:18:39.640 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:18:39.640 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f0 2024-04-26T23:18:39.640 INFO:teuthology.orchestra.run.smithi047.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:18:39.640 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:18:39.640 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f1 2024-04-26T23:18:39.641 INFO:teuthology.orchestra.run.smithi047.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T23:18:39.641 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2024-04-26T23:18:39.641 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f0 2024-04-26T23:18:39.641 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-26T23:18:39.641 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42679sec preferred_lft 42679sec 2024-04-26T23:18:39.641 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link noprefixroute 2024-04-26T23:18:39.641 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:18:39.641 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:18:39.641 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2024-04-26T23:18:39.641 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f1 2024-04-26T23:18:39.642 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T23:18:39.642 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:18:39.642 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-04-26T23:18:39.642 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link add name ceph-brx type bridge 2024-04-26T23:18:39.642 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip addr flush dev ceph-brx 2024-04-26T23:18:39.642 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set ceph-brx up 2024-04-26T23:18:39.642 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T23:18:39.642 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-04-26T23:18:39.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:18:39.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:18:39.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:18:39.829 DEBUG:teuthology.orchestra.run.smithi047:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T23:18:39.854 INFO:teuthology.orchestra.run.smithi047.stdout:1 2024-04-26T23:18:39.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:18:39.855 DEBUG:teuthology.orchestra.run.smithi047:> route 2024-04-26T23:18:39.923 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2024-04-26T23:18:39.923 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T23:18:39.923 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T23:18:39.923 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T23:18:39.923 INFO:teuthology.orchestra.run.smithi047.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T23:18:39.923 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:18:39.924 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-04-26T23:18:39.924 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T23:18:39.924 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T23:18:39.924 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T23:18:39.924 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-04-26T23:18:40.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:18:40.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:18:40.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:18:40.188 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-04-26T23:18:40.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:18:40.204 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list-id 2024-04-26T23:18:40.260 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:18:40.260 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-04-26T23:18:40.260 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T23:18:40.260 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-26T23:18:40.260 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-04-26T23:18:40.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:18:40.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:18:40.380 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-26T23:18:40.381 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:18:40.381 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-04-26T23:18:40.381 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-26T23:18:40.381 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T23:18:40.381 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-26T23:18:40.381 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-26T23:18:40.381 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-26T23:18:40.381 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-04-26T23:18:40.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:18:40.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:18:40.517 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:18:40.517 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-04-26T23:18:40.518 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set brx.0 up 2024-04-26T23:18:40.518 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T23:18:40.518 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-04-26T23:18:40.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:18:40.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:18:40.597 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-26T23:18:40.597 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:18:40.597 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-26T23:18:40.621 INFO:teuthology.orchestra.run.smithi047.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-26T23:18:40.621 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:18:40.621 DEBUG:teuthology.orchestra.run.smithi047:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-26T23:18:40.678 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-04-26T23:18:40.744 DEBUG:teuthology.orchestra.run.smithi047:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T23:18:40.801 INFO:teuthology.orchestra.run.smithi047.stdout:/proc 2024-04-26T23:18:40.801 INFO:teuthology.orchestra.run.smithi047.stdout:/sys 2024-04-26T23:18:40.801 INFO:teuthology.orchestra.run.smithi047.stdout:/dev 2024-04-26T23:18:40.801 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/security 2024-04-26T23:18:40.801 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/shm 2024-04-26T23:18:40.801 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/pts 2024-04-26T23:18:40.801 INFO:teuthology.orchestra.run.smithi047.stdout:/run 2024-04-26T23:18:40.801 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/cgroup 2024-04-26T23:18:40.801 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/pstore 2024-04-26T23:18:40.801 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/bpf 2024-04-26T23:18:40.801 INFO:teuthology.orchestra.run.smithi047.stdout:/ 2024-04-26T23:18:40.802 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/selinux 2024-04-26T23:18:40.802 INFO:teuthology.orchestra.run.smithi047.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T23:18:40.802 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/hugepages 2024-04-26T23:18:40.802 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/mqueue 2024-04-26T23:18:40.802 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/debug 2024-04-26T23:18:40.802 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/tracing 2024-04-26T23:18:40.802 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/config 2024-04-26T23:18:40.802 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-sysusers.service 2024-04-26T23:18:40.802 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/fuse/connections 2024-04-26T23:18:40.802 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T23:18:40.802 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T23:18:40.802 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T23:18:40.802 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T23:18:40.802 INFO:teuthology.orchestra.run.smithi047.stdout:/run/user/1000 2024-04-26T23:18:40.803 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph 2024-04-26T23:18:40.803 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-0 2024-04-26T23:18:40.803 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-1 2024-04-26T23:18:40.803 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-2 2024-04-26T23:18:40.803 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-3 2024-04-26T23:18:40.803 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns 2024-04-26T23:18:40.803 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T23:18:40.803 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T23:18:40.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:18:40.803 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-04-26T23:18:40.856 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-26T23:18:40.856 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-26T23:18:40.898 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-04-26T23:18:40.923 DEBUG:teuthology.orchestra.run.smithi047:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T23:18:40.979 INFO:teuthology.orchestra.run.smithi047.stdout:/proc 2024-04-26T23:18:40.979 INFO:teuthology.orchestra.run.smithi047.stdout:/sys 2024-04-26T23:18:40.979 INFO:teuthology.orchestra.run.smithi047.stdout:/dev 2024-04-26T23:18:40.979 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/security 2024-04-26T23:18:40.979 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/shm 2024-04-26T23:18:40.979 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/pts 2024-04-26T23:18:40.979 INFO:teuthology.orchestra.run.smithi047.stdout:/run 2024-04-26T23:18:40.979 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/cgroup 2024-04-26T23:18:40.980 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/pstore 2024-04-26T23:18:40.980 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/bpf 2024-04-26T23:18:40.980 INFO:teuthology.orchestra.run.smithi047.stdout:/ 2024-04-26T23:18:40.980 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/selinux 2024-04-26T23:18:40.980 INFO:teuthology.orchestra.run.smithi047.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T23:18:40.980 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/hugepages 2024-04-26T23:18:40.980 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/mqueue 2024-04-26T23:18:40.980 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/debug 2024-04-26T23:18:40.980 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/tracing 2024-04-26T23:18:40.980 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/config 2024-04-26T23:18:40.980 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-sysusers.service 2024-04-26T23:18:40.980 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/fuse/connections 2024-04-26T23:18:40.980 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T23:18:40.980 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T23:18:40.980 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T23:18:40.981 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T23:18:40.981 INFO:teuthology.orchestra.run.smithi047.stdout:/run/user/1000 2024-04-26T23:18:40.981 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph 2024-04-26T23:18:40.981 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-0 2024-04-26T23:18:40.981 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-1 2024-04-26T23:18:40.981 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-2 2024-04-26T23:18:40.981 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-3 2024-04-26T23:18:40.981 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns 2024-04-26T23:18:40.981 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T23:18:40.981 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T23:18:40.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:18:40.981 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-04-26T23:18:41.011 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi047.stderr:2024-04-26T23:18:41.010+0000 7ff021e86480 -1 init, newargv = 0x560ba09a4990 newargc=15 2024-04-26T23:18:41.011 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi047.stderr:ceph-fuse[41290]: starting ceph client 2024-04-26T23:18:41.017 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi047.stderr:ceph-fuse[41290]: starting fuse 2024-04-26T23:18:41.026 INFO:teuthology.orchestra.run.smithi047.stdout:43 2024-04-26T23:18:41.026 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-04-26T23:18:41.026 DEBUG:teuthology.orchestra.run.smithi047:> sudo stdin-killer -- python3 -c ' 2024-04-26T23:18:41.026 DEBUG:teuthology.orchestra.run.smithi047:> import glob 2024-04-26T23:18:41.026 DEBUG:teuthology.orchestra.run.smithi047:> import re 2024-04-26T23:18:41.027 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-04-26T23:18:41.027 DEBUG:teuthology.orchestra.run.smithi047:> import subprocess 2024-04-26T23:18:41.027 DEBUG:teuthology.orchestra.run.smithi047:> 2024-04-26T23:18:41.027 DEBUG:teuthology.orchestra.run.smithi047:> def _find_admin_socket(client_name): 2024-04-26T23:18:41.027 DEBUG:teuthology.orchestra.run.smithi047:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-26T23:18:41.027 DEBUG:teuthology.orchestra.run.smithi047:> files = glob.glob(asok_path) 2024-04-26T23:18:41.027 DEBUG:teuthology.orchestra.run.smithi047:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-26T23:18:41.027 DEBUG:teuthology.orchestra.run.smithi047:> 2024-04-26T23:18:41.027 DEBUG:teuthology.orchestra.run.smithi047:> # Given a non-glob path, it better be there 2024-04-26T23:18:41.027 DEBUG:teuthology.orchestra.run.smithi047:> if "*" not in asok_path: 2024-04-26T23:18:41.027 DEBUG:teuthology.orchestra.run.smithi047:> assert(len(files) == 1) 2024-04-26T23:18:41.027 DEBUG:teuthology.orchestra.run.smithi047:> return files[0] 2024-04-26T23:18:41.027 DEBUG:teuthology.orchestra.run.smithi047:> 2024-04-26T23:18:41.027 DEBUG:teuthology.orchestra.run.smithi047:> for f in files: 2024-04-26T23:18:41.027 DEBUG:teuthology.orchestra.run.smithi047:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-26T23:18:41.027 DEBUG:teuthology.orchestra.run.smithi047:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-26T23:18:41.027 DEBUG:teuthology.orchestra.run.smithi047:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-26T23:18:41.028 DEBUG:teuthology.orchestra.run.smithi047:> contents = proc_f.read() 2024-04-26T23:18:41.028 DEBUG:teuthology.orchestra.run.smithi047:> if mountpoint in contents: 2024-04-26T23:18:41.028 DEBUG:teuthology.orchestra.run.smithi047:> return f 2024-04-26T23:18:41.028 DEBUG:teuthology.orchestra.run.smithi047:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-26T23:18:41.028 DEBUG:teuthology.orchestra.run.smithi047:> 2024-04-26T23:18:41.028 DEBUG:teuthology.orchestra.run.smithi047:> print(_find_admin_socket("client.admin")) 2024-04-26T23:18:41.028 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-04-26T23:18:41.145 INFO:teuthology.orchestra.run.smithi047.stdout:/var/run/ceph/ceph-client.admin.41290.asok 2024-04-26T23:18:41.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:18:41.156 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.41290.asok 2024-04-26T23:18:41.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:18:41.156 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.41290.asok status 2024-04-26T23:18:41.291 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-26T23:18:41.291 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": { 2024-04-26T23:18:41.292 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_sha1": "bc612ffb9c1d5bc27536bc917a705262a36d4387", 2024-04-26T23:18:41.292 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T23:18:41.292 INFO:teuthology.orchestra.run.smithi047.stdout: "entity_id": "admin", 2024-04-26T23:18:41.292 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": "smithi047", 2024-04-26T23:18:41.292 INFO:teuthology.orchestra.run.smithi047.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-26T23:18:41.292 INFO:teuthology.orchestra.run.smithi047.stdout: "pid": "41290", 2024-04-26T23:18:41.292 INFO:teuthology.orchestra.run.smithi047.stdout: "root": "/" 2024-04-26T23:18:41.292 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-26T23:18:41.292 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_count": 0, 2024-04-26T23:18:41.292 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_pinned_count": 0, 2024-04-26T23:18:41.292 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 4455, 2024-04-26T23:18:41.292 INFO:teuthology.orchestra.run.smithi047.stdout: "inst": { 2024-04-26T23:18:41.293 INFO:teuthology.orchestra.run.smithi047.stdout: "name": { 2024-04-26T23:18:41.293 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "client", 2024-04-26T23:18:41.293 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 4455 2024-04-26T23:18:41.293 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-26T23:18:41.293 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2024-04-26T23:18:41.293 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-26T23:18:41.293 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2024-04-26T23:18:41.293 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 4255024516 2024-04-26T23:18:41.293 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-26T23:18:41.293 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-26T23:18:41.293 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2024-04-26T23:18:41.294 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-26T23:18:41.294 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2024-04-26T23:18:41.294 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 4255024516 2024-04-26T23:18:41.294 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-26T23:18:41.294 INFO:teuthology.orchestra.run.smithi047.stdout: "inst_str": "client.4455 192.168.0.1:0/4255024516", 2024-04-26T23:18:41.294 INFO:teuthology.orchestra.run.smithi047.stdout: "addr_str": "192.168.0.1:0/4255024516", 2024-04-26T23:18:41.294 INFO:teuthology.orchestra.run.smithi047.stdout: "inode_count": 1, 2024-04-26T23:18:41.294 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_epoch": 7, 2024-04-26T23:18:41.294 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch": 24, 2024-04-26T23:18:41.294 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch_barrier": 0, 2024-04-26T23:18:41.294 INFO:teuthology.orchestra.run.smithi047.stdout: "blocklisted": false, 2024-04-26T23:18:41.295 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs" 2024-04-26T23:18:41.295 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-26T23:18:41.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:18:41.300 DEBUG:teuthology.orchestra.run.smithi047:> stat --file-system '--printf=%T 2024-04-26T23:18:41.300 DEBUG:teuthology.orchestra.run.smithi047:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-26T23:18:41.316 INFO:teuthology.orchestra.run.smithi047.stdout:fuseblk 2024-04-26T23:18:41.316 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-26T23:18:41.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:18:41.317 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-26T23:18:41.384 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-26T23:18:41.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:18:41.491 INFO:teuthology.orchestra.run.smithi047.stdout:# file: . 2024-04-26T23:18:41.491 INFO:teuthology.orchestra.run.smithi047.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-26T23:18:41.491 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:18:41.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:18:41.499 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:18:41.499 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:18:41.516 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:18:41.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.655+0000 7f1ac1c2a640 1 Processor -- start 2024-04-26T23:18:41.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.656+0000 7f1ac1c2a640 1 -- start start 2024-04-26T23:18:41.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.656+0000 7f1ac1c2a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1abc0d3bc0 0x7f1abc0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:41.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.656+0000 7f1ac1c2a640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1abc0b5f20 0x7f1abc0b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:41.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.656+0000 7f1ac1c2a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1abc0b7e50 con 0x7f1abc162880 2024-04-26T23:18:41.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.656+0000 7f1ac1c2a640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f1abc0d0580 con 0x7f1abc0b5f20 2024-04-26T23:18:41.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.656+0000 7f1ac1c2a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f1abc0d1800 con 0x7f1abc0d3bc0 2024-04-26T23:18:41.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.656+0000 7f1abbfff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1abc0b5f20 0x7f1abc0b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:41.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.656+0000 7f1abbfff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1abc0b5f20 0x7f1abc0b8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54202/0 (socket says 172.21.15.47:54202) 2024-04-26T23:18:41.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.657+0000 7f1abbfff640 1 -- 172.21.15.47:0/263071058 learned_addr learned my addr 172.21.15.47:0/263071058 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:41.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.657+0000 7f1abaffd640 1 --2- 172.21.15.47:0/263071058 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1abc0d3bc0 0x7f1abc0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:41.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.657+0000 7f1abbfff640 1 -- 172.21.15.47:0/263071058 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1abc0d3bc0 msgr2=0x7f1abc0d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:41.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.657+0000 7f1abbfff640 1 --2- 172.21.15.47:0/263071058 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1abc0d3bc0 0x7f1abc0d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:41.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.657+0000 7f1abbfff640 1 -- 172.21.15.47:0/263071058 >> v1:172.21.15.203:6789/0 conn(0x7f1abc162880 legacy=0x7f1abc0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:41.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.657+0000 7f1abbfff640 1 -- 172.21.15.47:0/263071058 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1abc0d07d0 con 0x7f1abc0b5f20 2024-04-26T23:18:41.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.657+0000 7f1abaffd640 1 --2- 172.21.15.47:0/263071058 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1abc0d3bc0 0x7f1abc0d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:18:41.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.657+0000 7f1abbfff640 1 --2- 172.21.15.47:0/263071058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1abc0b5f20 0x7f1abc0b8300 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4009e50 tx=0x7f1ab402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e42788442e511ae server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:41.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.658+0000 7f1aba7fc640 1 -- 172.21.15.47:0/263071058 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab4036070 con 0x7f1abc0b5f20 2024-04-26T23:18:41.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.658+0000 7f1aba7fc640 1 -- 172.21.15.47:0/263071058 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ab4031600 con 0x7f1abc0b5f20 2024-04-26T23:18:41.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.658+0000 7f1aba7fc640 1 -- 172.21.15.47:0/263071058 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab40319d0 con 0x7f1abc0b5f20 2024-04-26T23:18:41.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.658+0000 7f1ac1c2a640 1 -- 172.21.15.47:0/263071058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1abc0b5f20 msgr2=0x7f1abc0b8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:41.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.658+0000 7f1ac1c2a640 1 --2- 172.21.15.47:0/263071058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1abc0b5f20 0x7f1abc0b8300 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4009e50 tx=0x7f1ab402a6c0 comp rx=0 tx=0).stop 2024-04-26T23:18:41.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.658+0000 7f1ac1c2a640 1 -- 172.21.15.47:0/263071058 shutdown_connections 2024-04-26T23:18:41.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.658+0000 7f1ac1c2a640 1 --2- 172.21.15.47:0/263071058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1abc0b5f20 0x7f1abc0b8300 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:41.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.658+0000 7f1ac1c2a640 1 --2- 172.21.15.47:0/263071058 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1abc0d3bc0 0x7f1abc0d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:41.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.658+0000 7f1ac1c2a640 1 -- 172.21.15.47:0/263071058 >> 172.21.15.47:0/263071058 conn(0x7f1abc0d4f60 msgr2=0x7f1abc0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:41.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.658+0000 7f1ac1c2a640 1 -- 172.21.15.47:0/263071058 shutdown_connections 2024-04-26T23:18:41.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.658+0000 7f1ac1c2a640 1 -- 172.21.15.47:0/263071058 wait complete. 2024-04-26T23:18:41.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.659+0000 7f1ac1c2a640 1 Processor -- start 2024-04-26T23:18:41.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.660+0000 7f1ac1c2a640 1 -- start start 2024-04-26T23:18:41.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.660+0000 7f1ac1c2a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1abc0b5f20 0x7f1abc159440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:41.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.660+0000 7f1ac1c2a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1abc0d3bc0 0x7f1abc0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:41.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.660+0000 7f1ac1c2a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1abc162880 0x7f1abc06dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:41.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.660+0000 7f1ac1c2a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc162c60 con 0x7f1abc0b5f20 2024-04-26T23:18:41.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.660+0000 7f1ac1c2a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc0d0d90 con 0x7f1abc0d3bc0 2024-04-26T23:18:41.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.660+0000 7f1ac1c2a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f1abc162de0 con 0x7f1abc162880 2024-04-26T23:18:41.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.660+0000 7f1abb7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1abc0b5f20 0x7f1abc159440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:41.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.660+0000 7f1abb7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1abc0b5f20 0x7f1abc159440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54210/0 (socket says 172.21.15.47:54210) 2024-04-26T23:18:41.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.660+0000 7f1abb7fe640 1 -- 172.21.15.47:0/1540624293 learned_addr learned my addr 172.21.15.47:0/1540624293 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:41.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.661+0000 7f1abaffd640 1 --2- 172.21.15.47:0/1540624293 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1abc0d3bc0 0x7f1abc0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:41.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.661+0000 7f1abbfff640 1 --2- 172.21.15.47:0/1540624293 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1abc162880 0x7f1abc06dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:41.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.661+0000 7f1abb7fe640 1 -- 172.21.15.47:0/1540624293 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1abc162880 msgr2=0x7f1abc06dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:41.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.661+0000 7f1abb7fe640 1 --2- 172.21.15.47:0/1540624293 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1abc162880 0x7f1abc06dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:41.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.661+0000 7f1abb7fe640 1 -- 172.21.15.47:0/1540624293 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1abc0d3bc0 msgr2=0x7f1abc0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:41.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.661+0000 7f1abb7fe640 1 --2- 172.21.15.47:0/1540624293 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1abc0d3bc0 0x7f1abc0ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:41.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.661+0000 7f1abb7fe640 1 -- 172.21.15.47:0/1540624293 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1abc0cd8f0 con 0x7f1abc0b5f20 2024-04-26T23:18:41.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.661+0000 7f1abb7fe640 1 --2- 172.21.15.47:0/1540624293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1abc0b5f20 0x7f1abc159440 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1aac00c460 tx=0x7f1aac00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:41.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.661+0000 7f1aa3fff640 1 -- 172.21.15.47:0/1540624293 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac016020 con 0x7f1abc0b5f20 2024-04-26T23:18:41.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.661+0000 7f1aa3fff640 1 -- 172.21.15.47:0/1540624293 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1aac012870 con 0x7f1abc0b5f20 2024-04-26T23:18:41.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.661+0000 7f1aa3fff640 1 -- 172.21.15.47:0/1540624293 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac012c60 con 0x7f1abc0b5f20 2024-04-26T23:18:41.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.661+0000 7f1ac1c2a640 1 -- 172.21.15.47:0/1540624293 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1abc0cdbd0 con 0x7f1abc0b5f20 2024-04-26T23:18:41.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.661+0000 7f1ac1c2a640 1 -- 172.21.15.47:0/1540624293 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1abc0ce190 con 0x7f1abc0b5f20 2024-04-26T23:18:41.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.662+0000 7f1ac1c2a640 1 -- 172.21.15.47:0/1540624293 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1abc0694f0 con 0x7f1abc0b5f20 2024-04-26T23:18:41.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.663+0000 7f1aa3fff640 1 -- 172.21.15.47:0/1540624293 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1aac019020 con 0x7f1abc0b5f20 2024-04-26T23:18:41.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.663+0000 7f1aa3fff640 1 --2- 172.21.15.47:0/1540624293 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1aa806c8f0 0x7f1aa806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:41.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.663+0000 7f1aa3fff640 1 -- 172.21.15.47:0/1540624293 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1aac08bf30 con 0x7f1abc0b5f20 2024-04-26T23:18:41.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.663+0000 7f1abaffd640 1 --2- 172.21.15.47:0/1540624293 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1aa806c8f0 0x7f1aa806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:41.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.664+0000 7f1abaffd640 1 --2- 172.21.15.47:0/1540624293 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1aa806c8f0 0x7f1aa806edb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1abc0b5c60 tx=0x7f1ab0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:41.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.667+0000 7f1aa3fff640 1 -- 172.21.15.47:0/1540624293 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1aac020020 con 0x7f1abc0b5f20 2024-04-26T23:18:41.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.854+0000 7f1ac1c2a640 1 -- 172.21.15.47:0/1540624293 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1abc0d49f0 con 0x7f1abc0b5f20 2024-04-26T23:18:41.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.855+0000 7f1aa3fff640 1 -- 172.21.15.47:0/1540624293 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18478 (secure 0 0 0) 0x7f1aac0182c0 con 0x7f1abc0b5f20 2024-04-26T23:18:41.856 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:18:41.856 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":24,"fsid":"399006d3-4db1-43e6-ba81-a79a276eabfd","created":"2024-04-26T23:18:06.731211+0000","modified":"2024-04-26T23:18:36.402132+0000","last_up_change":"2024-04-26T23:18:17.808736+0000","last_in_change":"2024-04-26T23:18:08.945859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:18:16.937492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:18:29.033575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:18:30.318499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:18:34.014536+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ee091567-fea0-4a14-bf4e-0e16c20bb2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6819","nonce":326520779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6821","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6823","nonce":326520779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6829","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6831","nonce":326520779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6825","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6827","nonce":326520779}]},"public_addr":"172.21.15.47:6819/326520779","cluster_addr":"172.21.15.47:6823/326520779","heartbeat_back_addr":"172.21.15.47:6831/326520779","heartbeat_front_addr":"172.21.15.47:6827/326520779","state":["exists","up"]},{"osd":1,"uuid":"c14440a0-bba5-4931-a6db-b736c08e869b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6809","nonce":1912610931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6811","nonce":1912610931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6815","nonce":1912610931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6813","nonce":1912610931}]},"public_addr":"172.21.15.47:6809/1912610931","cluster_addr":"172.21.15.47:6811/1912610931","heartbeat_back_addr":"172.21.15.47:6815/1912610931","heartbeat_front_addr":"172.21.15.47:6813/1912610931","state":["exists","up"]},{"osd":2,"uuid":"3940f8de-c9b2-4cd3-a911-9c301729eb30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6801","nonce":2569869305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6803","nonce":2569869305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6807","nonce":2569869305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6805","nonce":2569869305}]},"public_addr":"172.21.15.47:6801/2569869305","cluster_addr":"172.21.15.47:6803/2569869305","heartbeat_back_addr":"172.21.15.47:6807/2569869305","heartbeat_front_addr":"172.21.15.47:6805/2569869305","state":["exists","up"]},{"osd":3,"uuid":"1286a67e-8958-426a-8c2e-bc85498c619c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6818","nonce":2791411683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6822","nonce":2791411683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6830","nonce":2791411683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6826","nonce":2791411683}]},"public_addr":"172.21.15.47:6818/2791411683","cluster_addr":"172.21.15.47:6822/2791411683","heartbeat_back_addr":"172.21.15.47:6830/2791411683","heartbeat_front_addr":"172.21.15.47:6826/2791411683","state":["exists","up"]},{"osd":4,"uuid":"d8ee2c20-e5b7-46c2-9999-b234070126cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6809","nonce":441313104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6811","nonce":441313104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6815","nonce":441313104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6813","nonce":441313104}]},"public_addr":"172.21.15.203:6809/441313104","cluster_addr":"172.21.15.203:6811/441313104","heartbeat_back_addr":"172.21.15.203:6815/441313104","heartbeat_front_addr":"172.21.15.203:6813/441313104","state":["exists","up"]},{"osd":5,"uuid":"b453a8f0-f324-4840-9ae0-d67bdadb9f8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6801","nonce":56838444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6803","nonce":56838444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6807","nonce":56838444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6805","nonce":56838444}]},"public_addr":"172.21.15.203:6801/56838444","cluster_addr":"172.21.15.203:6803/56838444","heartbeat_back_addr":"172.21.15.203:6807/56838444","heartbeat_front_addr":"172.21.15.203:6805/56838444","state":["exists","up"]},{"osd":6,"uuid":"a6d31951-2d4b-4e8d-a510-6283b01dc158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6825","nonce":2738517080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6827","nonce":2738517080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6831","nonce":2738517080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6829","nonce":2738517080}]},"public_addr":"172.21.15.203:6825/2738517080","cluster_addr":"172.21.15.203:6827/2738517080","heartbeat_back_addr":"172.21.15.203:6831/2738517080","heartbeat_front_addr":"172.21.15.203:6829/2738517080","state":["exists","up"]},{"osd":7,"uuid":"7119b8c0-fe96-402d-9e75-7b509d9d44d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6817","nonce":1315565058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6819","nonce":1315565058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6823","nonce":1315565058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6821","nonce":1315565058}]},"public_addr":"172.21.15.203:6817/1315565058","cluster_addr":"172.21.15.203:6819/1315565058","heartbeat_back_addr":"172.21.15.203:6823/1315565058","heartbeat_front_addr":"172.21.15.203:6821/1315565058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.121921+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.137494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.084814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.111593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.072430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.125066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.097867+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.057061+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:18:41.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.857+0000 7f1ac1c2a640 1 -- 172.21.15.47:0/1540624293 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1aa806c8f0 msgr2=0x7f1aa806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:41.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.857+0000 7f1ac1c2a640 1 --2- 172.21.15.47:0/1540624293 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1aa806c8f0 0x7f1aa806edb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1abc0b5c60 tx=0x7f1ab0008040 comp rx=0 tx=0).stop 2024-04-26T23:18:41.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.857+0000 7f1ac1c2a640 1 -- 172.21.15.47:0/1540624293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1abc0b5f20 msgr2=0x7f1abc159440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:41.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.857+0000 7f1ac1c2a640 1 --2- 172.21.15.47:0/1540624293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1abc0b5f20 0x7f1abc159440 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1aac00c460 tx=0x7f1aac00c920 comp rx=0 tx=0).stop 2024-04-26T23:18:41.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.857+0000 7f1ac1c2a640 1 -- 172.21.15.47:0/1540624293 shutdown_connections 2024-04-26T23:18:41.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.857+0000 7f1ac1c2a640 1 --2- 172.21.15.47:0/1540624293 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1aa806c8f0 0x7f1aa806edb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:41.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.857+0000 7f1ac1c2a640 1 --2- 172.21.15.47:0/1540624293 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1abc162880 0x7f1abc06dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:41.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.857+0000 7f1ac1c2a640 1 --2- 172.21.15.47:0/1540624293 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1abc0d3bc0 0x7f1abc0ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:41.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.857+0000 7f1ac1c2a640 1 --2- 172.21.15.47:0/1540624293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1abc0b5f20 0x7f1abc159440 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:41.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.857+0000 7f1ac1c2a640 1 -- 172.21.15.47:0/1540624293 >> 172.21.15.47:0/1540624293 conn(0x7f1abc0d4f60 msgr2=0x7f1abc0d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:41.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.857+0000 7f1ac1c2a640 1 -- 172.21.15.47:0/1540624293 shutdown_connections 2024-04-26T23:18:41.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.857+0000 7f1ac1c2a640 1 -- 172.21.15.47:0/1540624293 wait complete. 2024-04-26T23:18:41.869 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:18:16.937492+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:18:29.033575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:18:30.318499+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T23:18:34.014536+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T23:18:41.869 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:18:41.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.972+0000 7f55f8190640 1 Processor -- start 2024-04-26T23:18:41.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.973+0000 7f55f8190640 1 -- start start 2024-04-26T23:18:41.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.973+0000 7f55f8190640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55f015f1d0 0x7f55f006f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:41.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.973+0000 7f55f8190640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f55f006c240 0x7f55f006c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:41.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.973+0000 7f55f8190640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55f00b4480 con 0x7f55f00726e0 2024-04-26T23:18:41.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.973+0000 7f55f8190640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f55f006cbe0 con 0x7f55f006c240 2024-04-26T23:18:41.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.973+0000 7f55f8190640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f55f00ae8c0 con 0x7f55f015f1d0 2024-04-26T23:18:41.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.974+0000 7f55f5f2c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55f015f1d0 0x7f55f006f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:41.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.974+0000 7f55f572b640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f55f006c240 0x7f55f006c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:41.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.974+0000 7f55f5f2c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55f015f1d0 0x7f55f006f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.47:60362/0 (socket says 172.21.15.47:60362) 2024-04-26T23:18:41.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.974+0000 7f55f5f2c640 1 -- 172.21.15.47:0/738150839 learned_addr learned my addr 172.21.15.47:0/738150839 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:41.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.974+0000 7f55f572b640 1 -- 172.21.15.47:0/738150839 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55f015f1d0 msgr2=0x7f55f006f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:41.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.974+0000 7f55f572b640 1 --2- 172.21.15.47:0/738150839 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55f015f1d0 0x7f55f006f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:41.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.974+0000 7f55f572b640 1 -- 172.21.15.47:0/738150839 >> v1:172.21.15.203:6789/0 conn(0x7f55f00726e0 legacy=0x7f55f0072b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:41.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.974+0000 7f55f572b640 1 -- 172.21.15.47:0/738150839 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55f00b4720 con 0x7f55f006c240 2024-04-26T23:18:41.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.974+0000 7f55f5f2c640 1 --2- 172.21.15.47:0/738150839 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55f015f1d0 0x7f55f006f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:18:41.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.975+0000 7f55f572b640 1 --2- 172.21.15.47:0/738150839 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55f006c240 0x7f55f006c6a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f55e00097a0 tx=0x7f55e002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d436fe1d7c0532e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:41.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.975+0000 7f55f4f2a640 1 -- 172.21.15.47:0/738150839 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e0037070 con 0x7f55f006c240 2024-04-26T23:18:41.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.975+0000 7f55f4f2a640 1 -- 172.21.15.47:0/738150839 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55e0032440 con 0x7f55f006c240 2024-04-26T23:18:41.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.975+0000 7f55f4f2a640 1 -- 172.21.15.47:0/738150839 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e0032810 con 0x7f55f006c240 2024-04-26T23:18:41.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.976+0000 7f55f8190640 1 -- 172.21.15.47:0/738150839 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55f006c240 msgr2=0x7f55f006c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:41.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.976+0000 7f55f8190640 1 --2- 172.21.15.47:0/738150839 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55f006c240 0x7f55f006c6a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f55e00097a0 tx=0x7f55e002a180 comp rx=0 tx=0).stop 2024-04-26T23:18:41.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.976+0000 7f55f8190640 1 -- 172.21.15.47:0/738150839 shutdown_connections 2024-04-26T23:18:41.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.976+0000 7f55f8190640 1 --2- 172.21.15.47:0/738150839 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55f006c240 0x7f55f006c6a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:41.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.976+0000 7f55f8190640 1 --2- 172.21.15.47:0/738150839 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55f015f1d0 0x7f55f006f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:41.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.976+0000 7f55f8190640 1 -- 172.21.15.47:0/738150839 >> 172.21.15.47:0/738150839 conn(0x7f55f00b3180 msgr2=0x7f55f00b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:41.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.976+0000 7f55f8190640 1 -- 172.21.15.47:0/738150839 shutdown_connections 2024-04-26T23:18:41.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.976+0000 7f55f8190640 1 -- 172.21.15.47:0/738150839 wait complete. 2024-04-26T23:18:41.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.977+0000 7f55f8190640 1 Processor -- start 2024-04-26T23:18:41.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.977+0000 7f55f8190640 1 -- start start 2024-04-26T23:18:41.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.978+0000 7f55f8190640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55f006c240 0x7f55f01591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:41.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.978+0000 7f55f8190640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55f00726e0 0x7f55f015ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:41.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.978+0000 7f55f8190640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55f015f1d0 0x7f55f01f8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:41.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.978+0000 7f55f8190640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f55f00b4fc0 con 0x7f55f00726e0 2024-04-26T23:18:41.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.978+0000 7f55f8190640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f55f00b4ce0 con 0x7f55f015f1d0 2024-04-26T23:18:41.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.978+0000 7f55f8190640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f55f006f6b0 con 0x7f55f006c240 2024-04-26T23:18:41.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.978+0000 7f55f572b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55f00726e0 0x7f55f015ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:41.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.978+0000 7f55f572b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55f00726e0 0x7f55f015ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54214/0 (socket says 172.21.15.47:54214) 2024-04-26T23:18:41.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.978+0000 7f55f572b640 1 -- 172.21.15.47:0/1520976575 learned_addr learned my addr 172.21.15.47:0/1520976575 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:41.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.978+0000 7f55f5f2c640 1 --2- 172.21.15.47:0/1520976575 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55f006c240 0x7f55f01591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:41.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.978+0000 7f55f672d640 1 --2- 172.21.15.47:0/1520976575 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55f015f1d0 0x7f55f01f8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:41.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.978+0000 7f55f572b640 1 -- 172.21.15.47:0/1520976575 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55f006c240 msgr2=0x7f55f01591c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:41.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.978+0000 7f55f572b640 1 --2- 172.21.15.47:0/1520976575 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55f006c240 0x7f55f01591c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:41.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.978+0000 7f55f572b640 1 -- 172.21.15.47:0/1520976575 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55f015f1d0 msgr2=0x7f55f01f8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:41.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.978+0000 7f55f572b640 1 --2- 172.21.15.47:0/1520976575 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55f015f1d0 0x7f55f01f8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:41.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.978+0000 7f55f572b640 1 -- 172.21.15.47:0/1520976575 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55f01609a0 con 0x7f55f00726e0 2024-04-26T23:18:41.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.978+0000 7f55f572b640 1 --2- 172.21.15.47:0/1520976575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55f00726e0 0x7f55f015ffc0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f55e0035040 tx=0x7f55e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:41.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.979+0000 7f55e67fc640 1 -- 172.21.15.47:0/1520976575 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e0037040 con 0x7f55f00726e0 2024-04-26T23:18:41.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.979+0000 7f55e67fc640 1 -- 172.21.15.47:0/1520976575 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55e0030500 con 0x7f55f00726e0 2024-04-26T23:18:41.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.979+0000 7f55f8190640 1 -- 172.21.15.47:0/1520976575 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55f0160c80 con 0x7f55f00726e0 2024-04-26T23:18:41.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.979+0000 7f55e67fc640 1 -- 172.21.15.47:0/1520976575 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e0030910 con 0x7f55f00726e0 2024-04-26T23:18:41.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.979+0000 7f55f8190640 1 -- 172.21.15.47:0/1520976575 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55f00c6a00 con 0x7f55f00726e0 2024-04-26T23:18:41.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.980+0000 7f55f8190640 1 -- 172.21.15.47:0/1520976575 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55f0068fc0 con 0x7f55f00726e0 2024-04-26T23:18:41.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.980+0000 7f55e67fc640 1 -- 172.21.15.47:0/1520976575 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f55e0044020 con 0x7f55f00726e0 2024-04-26T23:18:41.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.980+0000 7f55e67fc640 1 --2- 172.21.15.47:0/1520976575 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f55d406c920 0x7f55d406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:41.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.981+0000 7f55e67fc640 1 -- 172.21.15.47:0/1520976575 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f55e00ac010 con 0x7f55f00726e0 2024-04-26T23:18:41.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.981+0000 7f55f5f2c640 1 --2- 172.21.15.47:0/1520976575 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f55d406c920 0x7f55d406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:41.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.981+0000 7f55f5f2c640 1 --2- 172.21.15.47:0/1520976575 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f55d406c920 0x7f55d406ede0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f55ec009b70 tx=0x7f55ec008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:41.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:41.985+0000 7f55e67fc640 1 -- 172.21.15.47:0/1520976575 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f55e007a190 con 0x7f55f00726e0 2024-04-26T23:18:42.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.170+0000 7f55f8190640 1 -- 172.21.15.47:0/1520976575 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f55f01629f0 con 0x7f55f00726e0 2024-04-26T23:18:42.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.171+0000 7f55e67fc640 1 -- 172.21.15.47:0/1520976575 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f55e0079a90 con 0x7f55f00726e0 2024-04-26T23:18:42.172 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-26T23:18:42.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.173+0000 7f55f8190640 1 -- 172.21.15.47:0/1520976575 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f55d406c920 msgr2=0x7f55d406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:42.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.173+0000 7f55f8190640 1 --2- 172.21.15.47:0/1520976575 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f55d406c920 0x7f55d406ede0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f55ec009b70 tx=0x7f55ec008040 comp rx=0 tx=0).stop 2024-04-26T23:18:42.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.173+0000 7f55f8190640 1 -- 172.21.15.47:0/1520976575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55f00726e0 msgr2=0x7f55f015ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:42.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.173+0000 7f55f8190640 1 --2- 172.21.15.47:0/1520976575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55f00726e0 0x7f55f015ffc0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f55e0035040 tx=0x7f55e00046c0 comp rx=0 tx=0).stop 2024-04-26T23:18:42.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.173+0000 7f55f8190640 1 -- 172.21.15.47:0/1520976575 shutdown_connections 2024-04-26T23:18:42.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.173+0000 7f55f8190640 1 --2- 172.21.15.47:0/1520976575 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f55d406c920 0x7f55d406ede0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:42.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.173+0000 7f55f8190640 1 --2- 172.21.15.47:0/1520976575 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55f015f1d0 0x7f55f01f8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:42.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.173+0000 7f55f8190640 1 --2- 172.21.15.47:0/1520976575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55f00726e0 0x7f55f015ffc0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:42.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.173+0000 7f55f8190640 1 --2- 172.21.15.47:0/1520976575 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55f006c240 0x7f55f01591c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:42.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.173+0000 7f55f8190640 1 -- 172.21.15.47:0/1520976575 >> 172.21.15.47:0/1520976575 conn(0x7f55f00b3180 msgr2=0x7f55f0072b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:42.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.173+0000 7f55f8190640 1 -- 172.21.15.47:0/1520976575 shutdown_connections 2024-04-26T23:18:42.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.173+0000 7f55f8190640 1 -- 172.21.15.47:0/1520976575 wait complete. 2024-04-26T23:18:42.184 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:18:42.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.288+0000 7f4128283640 1 Processor -- start 2024-04-26T23:18:42.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.288+0000 7f4128283640 1 -- start start 2024-04-26T23:18:42.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.289+0000 7f4128283640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f412015f170 0x7f41200d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:42.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.289+0000 7f4128283640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f412015f550 0x7f41200d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:42.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.289+0000 7f4128283640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41200d1040 con 0x7f41200d34b0 2024-04-26T23:18:42.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.289+0000 7f4128283640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f41200c9430 con 0x7f412015f550 2024-04-26T23:18:42.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.289+0000 7f4128283640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f41200c9680 con 0x7f412015f170 2024-04-26T23:18:42.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.289+0000 7f412581e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f412015f550 0x7f41200d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:42.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.289+0000 7f412581e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f412015f550 0x7f41200d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54230/0 (socket says 172.21.15.47:54230) 2024-04-26T23:18:42.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.289+0000 7f412581e640 1 -- 172.21.15.47:0/2250135564 learned_addr learned my addr 172.21.15.47:0/2250135564 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:42.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.289+0000 7f412601f640 1 --2- 172.21.15.47:0/2250135564 >> v2:172.21.15.203:3300/0 conn(0x7f412015f170 0x7f41200d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:42.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.289+0000 7f412581e640 1 -- 172.21.15.47:0/2250135564 >> v2:172.21.15.203:3300/0 conn(0x7f412015f170 msgr2=0x7f41200d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:42.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.289+0000 7f412581e640 1 --2- 172.21.15.47:0/2250135564 >> v2:172.21.15.203:3300/0 conn(0x7f412015f170 0x7f41200d5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:42.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.289+0000 7f412581e640 1 -- 172.21.15.47:0/2250135564 >> v1:172.21.15.203:6789/0 conn(0x7f41200d34b0 legacy=0x7f41200cfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:42.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.289+0000 7f412581e640 1 -- 172.21.15.47:0/2250135564 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41200c9960 con 0x7f412015f550 2024-04-26T23:18:42.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.289+0000 7f412601f640 1 --2- 172.21.15.47:0/2250135564 >> v2:172.21.15.203:3300/0 conn(0x7f412015f170 0x7f41200d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:18:42.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.290+0000 7f412581e640 1 --2- 172.21.15.47:0/2250135564 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f412015f550 0x7f41200d2cb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4110009870 tx=0x7f411002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2030c70a43f6dbc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:42.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.290+0000 7f412501d640 1 -- 172.21.15.47:0/2250135564 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4110037070 con 0x7f412015f550 2024-04-26T23:18:42.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.290+0000 7f412501d640 1 -- 172.21.15.47:0/2250135564 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4110032440 con 0x7f412015f550 2024-04-26T23:18:42.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.290+0000 7f412501d640 1 -- 172.21.15.47:0/2250135564 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4110032810 con 0x7f412015f550 2024-04-26T23:18:42.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.291+0000 7f4128283640 1 -- 172.21.15.47:0/2250135564 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f412015f550 msgr2=0x7f41200d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:42.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.291+0000 7f4128283640 1 --2- 172.21.15.47:0/2250135564 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f412015f550 0x7f41200d2cb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4110009870 tx=0x7f411002a2a0 comp rx=0 tx=0).stop 2024-04-26T23:18:42.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.291+0000 7f4128283640 1 -- 172.21.15.47:0/2250135564 shutdown_connections 2024-04-26T23:18:42.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.291+0000 7f4128283640 1 --2- 172.21.15.47:0/2250135564 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f412015f550 0x7f41200d2cb0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:42.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.291+0000 7f4128283640 1 --2- 172.21.15.47:0/2250135564 >> v2:172.21.15.203:3300/0 conn(0x7f412015f170 0x7f41200d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:42.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.291+0000 7f4128283640 1 -- 172.21.15.47:0/2250135564 >> 172.21.15.47:0/2250135564 conn(0x7f41200d4f60 msgr2=0x7f41200d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:42.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.291+0000 7f4128283640 1 -- 172.21.15.47:0/2250135564 shutdown_connections 2024-04-26T23:18:42.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.291+0000 7f4128283640 1 -- 172.21.15.47:0/2250135564 wait complete. 2024-04-26T23:18:42.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.292+0000 7f4128283640 1 Processor -- start 2024-04-26T23:18:42.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.292+0000 7f4128283640 1 -- start start 2024-04-26T23:18:42.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.293+0000 7f4128283640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41200d34b0 0x7f41200b8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:42.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.293+0000 7f4128283640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f412015f170 0x7f41200ba4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:42.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.293+0000 7f4128283640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41201586d0 0x7f412006d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:42.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.293+0000 7f4128283640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f41200cb5d0 con 0x7f41200d34b0 2024-04-26T23:18:42.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.293+0000 7f4128283640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f41200cb450 con 0x7f412015f170 2024-04-26T23:18:42.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.293+0000 7f4128283640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f41200cb750 con 0x7f41201586d0 2024-04-26T23:18:42.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.293+0000 7f412601f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41200d34b0 0x7f41200b8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:42.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.293+0000 7f412601f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41200d34b0 0x7f41200b8bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54244/0 (socket says 172.21.15.47:54244) 2024-04-26T23:18:42.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.293+0000 7f412601f640 1 -- 172.21.15.47:0/3284545217 learned_addr learned my addr 172.21.15.47:0/3284545217 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:42.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.293+0000 7f412581e640 1 --2- 172.21.15.47:0/3284545217 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f412015f170 0x7f41200ba4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:42.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.293+0000 7f412601f640 1 -- 172.21.15.47:0/3284545217 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41201586d0 msgr2=0x7f412006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:42.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.293+0000 7f4126820640 1 --2- 172.21.15.47:0/3284545217 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41201586d0 0x7f412006d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:42.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.293+0000 7f412601f640 1 --2- 172.21.15.47:0/3284545217 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41201586d0 0x7f412006d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:42.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.293+0000 7f412601f640 1 -- 172.21.15.47:0/3284545217 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f412015f170 msgr2=0x7f41200ba4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:42.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.293+0000 7f412601f640 1 --2- 172.21.15.47:0/3284545217 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f412015f170 0x7f41200ba4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:42.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.293+0000 7f412601f640 1 -- 172.21.15.47:0/3284545217 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41200b9400 con 0x7f41200d34b0 2024-04-26T23:18:42.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.293+0000 7f412581e640 1 --2- 172.21.15.47:0/3284545217 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f412015f170 0x7f41200ba4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:18:42.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.293+0000 7f4126820640 1 --2- 172.21.15.47:0/3284545217 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41201586d0 0x7f412006d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:18:42.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.293+0000 7f412601f640 1 --2- 172.21.15.47:0/3284545217 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41200d34b0 0x7f41200b8bd0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f411c00e810 tx=0x7f411c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:42.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.293+0000 7f4116ffd640 1 -- 172.21.15.47:0/3284545217 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411c00f040 con 0x7f41200d34b0 2024-04-26T23:18:42.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.293+0000 7f4116ffd640 1 -- 172.21.15.47:0/3284545217 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f411c004030 con 0x7f41200d34b0 2024-04-26T23:18:42.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.293+0000 7f4128283640 1 -- 172.21.15.47:0/3284545217 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41200b96e0 con 0x7f41200d34b0 2024-04-26T23:18:42.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.293+0000 7f4116ffd640 1 -- 172.21.15.47:0/3284545217 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411c004400 con 0x7f41200d34b0 2024-04-26T23:18:42.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.293+0000 7f4128283640 1 -- 172.21.15.47:0/3284545217 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41200b9ca0 con 0x7f41200d34b0 2024-04-26T23:18:42.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.295+0000 7f4128283640 1 -- 172.21.15.47:0/3284545217 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41200694f0 con 0x7f41200d34b0 2024-04-26T23:18:42.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.295+0000 7f4116ffd640 1 -- 172.21.15.47:0/3284545217 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f411c00c6d0 con 0x7f41200d34b0 2024-04-26T23:18:42.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.295+0000 7f4116ffd640 1 --2- 172.21.15.47:0/3284545217 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f410406c920 0x7f410406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:42.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.296+0000 7f4116ffd640 1 -- 172.21.15.47:0/3284545217 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f411c08bd60 con 0x7f41200d34b0 2024-04-26T23:18:42.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.296+0000 7f412581e640 1 --2- 172.21.15.47:0/3284545217 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f410406c920 0x7f410406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:42.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.296+0000 7f412581e640 1 --2- 172.21.15.47:0/3284545217 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f410406c920 0x7f410406ede0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4110034000 tx=0x7f411002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:42.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.300+0000 7f4116ffd640 1 -- 172.21.15.47:0/3284545217 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f411c017020 con 0x7f41200d34b0 2024-04-26T23:18:42.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.484+0000 7f4128283640 1 -- 172.21.15.47:0/3284545217 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f41200b9110 con 0x7f41200d34b0 2024-04-26T23:18:42.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.484+0000 7f4116ffd640 1 -- 172.21.15.47:0/3284545217 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f411c059d20 con 0x7f41200d34b0 2024-04-26T23:18:42.485 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-26T23:18:42.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.486+0000 7f4128283640 1 -- 172.21.15.47:0/3284545217 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f410406c920 msgr2=0x7f410406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:42.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.486+0000 7f4128283640 1 --2- 172.21.15.47:0/3284545217 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f410406c920 0x7f410406ede0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4110034000 tx=0x7f411002e000 comp rx=0 tx=0).stop 2024-04-26T23:18:42.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.486+0000 7f4128283640 1 -- 172.21.15.47:0/3284545217 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41200d34b0 msgr2=0x7f41200b8bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:42.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.486+0000 7f4128283640 1 --2- 172.21.15.47:0/3284545217 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41200d34b0 0x7f41200b8bd0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f411c00e810 tx=0x7f411c00ecd0 comp rx=0 tx=0).stop 2024-04-26T23:18:42.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.486+0000 7f4128283640 1 -- 172.21.15.47:0/3284545217 shutdown_connections 2024-04-26T23:18:42.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.486+0000 7f4128283640 1 --2- 172.21.15.47:0/3284545217 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f410406c920 0x7f410406ede0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:42.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.486+0000 7f4128283640 1 --2- 172.21.15.47:0/3284545217 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41201586d0 0x7f412006d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:42.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.486+0000 7f4128283640 1 --2- 172.21.15.47:0/3284545217 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f412015f170 0x7f41200ba4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:42.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.486+0000 7f4128283640 1 --2- 172.21.15.47:0/3284545217 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41200d34b0 0x7f41200b8bd0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:42.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.486+0000 7f4128283640 1 -- 172.21.15.47:0/3284545217 >> 172.21.15.47:0/3284545217 conn(0x7f41200d4f60 msgr2=0x7f41200d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:42.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.486+0000 7f4128283640 1 -- 172.21.15.47:0/3284545217 shutdown_connections 2024-04-26T23:18:42.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.486+0000 7f4128283640 1 -- 172.21.15.47:0/3284545217 wait complete. 2024-04-26T23:18:42.497 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:18:42.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.600+0000 7fec44420640 1 Processor -- start 2024-04-26T23:18:42.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.600+0000 7fec44420640 1 -- start start 2024-04-26T23:18:42.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.601+0000 7fec44420640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fec3c15f170 0x7fec3c0d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:42.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.601+0000 7fec44420640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec3c0d3bc0 0x7fec3c0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:42.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.601+0000 7fec44420640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec3c0d1040 con 0x7fec3c15f550 2024-04-26T23:18:42.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.601+0000 7fec44420640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fec3c0c9430 con 0x7fec3c15f170 2024-04-26T23:18:42.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.601+0000 7fec44420640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fec3c0c9680 con 0x7fec3c0d3bc0 2024-04-26T23:18:42.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.601+0000 7fec421bc640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fec3c15f170 0x7fec3c0d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:42.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.601+0000 7fec421bc640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fec3c15f170 0x7fec3c0d5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54246/0 (socket says 172.21.15.47:54246) 2024-04-26T23:18:42.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.601+0000 7fec421bc640 1 -- 172.21.15.47:0/1141044215 learned_addr learned my addr 172.21.15.47:0/1141044215 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:42.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.601+0000 7fec429bd640 1 --2- 172.21.15.47:0/1141044215 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec3c0d3bc0 0x7fec3c0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:42.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.601+0000 7fec421bc640 1 -- 172.21.15.47:0/1141044215 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec3c0d3bc0 msgr2=0x7fec3c0d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:42.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.601+0000 7fec421bc640 1 --2- 172.21.15.47:0/1141044215 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec3c0d3bc0 0x7fec3c0d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:42.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.601+0000 7fec421bc640 1 -- 172.21.15.47:0/1141044215 >> v1:172.21.15.203:6789/0 conn(0x7fec3c15f550 legacy=0x7fec3c0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:42.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.601+0000 7fec421bc640 1 -- 172.21.15.47:0/1141044215 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec3c0c9960 con 0x7fec3c15f170 2024-04-26T23:18:42.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.602+0000 7fec421bc640 1 --2- 172.21.15.47:0/1141044215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec3c15f170 0x7fec3c0d5a80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fec2c009e80 tx=0x7fec2c02a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e588fccd731421a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:42.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.602+0000 7fec411ba640 1 -- 172.21.15.47:0/1141044215 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2c036070 con 0x7fec3c15f170 2024-04-26T23:18:42.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.602+0000 7fec411ba640 1 -- 172.21.15.47:0/1141044215 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec2c031600 con 0x7fec3c15f170 2024-04-26T23:18:42.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.602+0000 7fec411ba640 1 -- 172.21.15.47:0/1141044215 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2c0319d0 con 0x7fec3c15f170 2024-04-26T23:18:42.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.603+0000 7fec44420640 1 -- 172.21.15.47:0/1141044215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec3c15f170 msgr2=0x7fec3c0d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:42.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.603+0000 7fec44420640 1 --2- 172.21.15.47:0/1141044215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec3c15f170 0x7fec3c0d5a80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fec2c009e80 tx=0x7fec2c02a750 comp rx=0 tx=0).stop 2024-04-26T23:18:42.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.603+0000 7fec44420640 1 -- 172.21.15.47:0/1141044215 shutdown_connections 2024-04-26T23:18:42.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.603+0000 7fec44420640 1 --2- 172.21.15.47:0/1141044215 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec3c0d3bc0 0x7fec3c0d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:42.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.603+0000 7fec44420640 1 --2- 172.21.15.47:0/1141044215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec3c15f170 0x7fec3c0d5a80 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:42.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.603+0000 7fec44420640 1 -- 172.21.15.47:0/1141044215 >> 172.21.15.47:0/1141044215 conn(0x7fec3c0d4f60 msgr2=0x7fec3c0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:42.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.603+0000 7fec44420640 1 -- 172.21.15.47:0/1141044215 shutdown_connections 2024-04-26T23:18:42.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.603+0000 7fec44420640 1 -- 172.21.15.47:0/1141044215 wait complete. 2024-04-26T23:18:42.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.604+0000 7fec44420640 1 Processor -- start 2024-04-26T23:18:42.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.604+0000 7fec44420640 1 -- start start 2024-04-26T23:18:42.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.605+0000 7fec44420640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec3c0d3bc0 0x7fec3c151ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:42.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.605+0000 7fec44420640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec3c15f170 0x7fec3c0ce170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:42.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.605+0000 7fec44420640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec3c0ce6b0 0x7fec3c154790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:42.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.605+0000 7fec44420640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fec3c0ca5c0 con 0x7fec3c15f170 2024-04-26T23:18:42.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.605+0000 7fec44420640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fec3c0ca440 con 0x7fec3c0d3bc0 2024-04-26T23:18:42.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.605+0000 7fec44420640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fec3c0ca740 con 0x7fec3c0ce6b0 2024-04-26T23:18:42.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.605+0000 7fec419bb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec3c15f170 0x7fec3c0ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:42.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.605+0000 7fec419bb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec3c15f170 0x7fec3c0ce170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54260/0 (socket says 172.21.15.47:54260) 2024-04-26T23:18:42.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.605+0000 7fec419bb640 1 -- 172.21.15.47:0/2534507096 learned_addr learned my addr 172.21.15.47:0/2534507096 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:42.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.605+0000 7fec421bc640 1 --2- 172.21.15.47:0/2534507096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec3c0d3bc0 0x7fec3c151ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:42.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.605+0000 7fec429bd640 1 --2- 172.21.15.47:0/2534507096 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec3c0ce6b0 0x7fec3c154790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:42.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.605+0000 7fec419bb640 1 -- 172.21.15.47:0/2534507096 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec3c0ce6b0 msgr2=0x7fec3c154790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:42.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.605+0000 7fec419bb640 1 --2- 172.21.15.47:0/2534507096 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec3c0ce6b0 0x7fec3c154790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:42.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.605+0000 7fec419bb640 1 -- 172.21.15.47:0/2534507096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec3c0d3bc0 msgr2=0x7fec3c151ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:42.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.605+0000 7fec419bb640 1 --2- 172.21.15.47:0/2534507096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec3c0d3bc0 0x7fec3c151ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:42.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.605+0000 7fec419bb640 1 -- 172.21.15.47:0/2534507096 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec3c09fc30 con 0x7fec3c15f170 2024-04-26T23:18:42.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.605+0000 7fec419bb640 1 --2- 172.21.15.47:0/2534507096 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec3c15f170 0x7fec3c0ce170 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fec3000c3c0 tx=0x7fec3000c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:42.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.606+0000 7fec3affd640 1 -- 172.21.15.47:0/2534507096 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec300052d0 con 0x7fec3c15f170 2024-04-26T23:18:42.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.606+0000 7fec3affd640 1 -- 172.21.15.47:0/2534507096 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec30005490 con 0x7fec3c15f170 2024-04-26T23:18:42.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.606+0000 7fec44420640 1 -- 172.21.15.47:0/2534507096 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec3c09ee70 con 0x7fec3c15f170 2024-04-26T23:18:42.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.606+0000 7fec3affd640 1 -- 172.21.15.47:0/2534507096 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec30007260 con 0x7fec3c15f170 2024-04-26T23:18:42.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.606+0000 7fec44420640 1 -- 172.21.15.47:0/2534507096 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec3c09f430 con 0x7fec3c15f170 2024-04-26T23:18:42.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.607+0000 7fec44420640 1 -- 172.21.15.47:0/2534507096 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec3c0694f0 con 0x7fec3c15f170 2024-04-26T23:18:42.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.607+0000 7fec3affd640 1 -- 172.21.15.47:0/2534507096 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fec30007400 con 0x7fec3c15f170 2024-04-26T23:18:42.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.607+0000 7fec3affd640 1 --2- 172.21.15.47:0/2534507096 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fec2806c8f0 0x7fec2806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:42.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.608+0000 7fec3affd640 1 -- 172.21.15.47:0/2534507096 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fec3008b380 con 0x7fec3c15f170 2024-04-26T23:18:42.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.608+0000 7fec421bc640 1 --2- 172.21.15.47:0/2534507096 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fec2806c8f0 0x7fec2806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:42.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.608+0000 7fec421bc640 1 --2- 172.21.15.47:0/2534507096 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fec2806c8f0 0x7fec2806edb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fec2c033000 tx=0x7fec2c009000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:42.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.612+0000 7fec3affd640 1 -- 172.21.15.47:0/2534507096 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fec3c0d1040 con 0x7fec3c15f170 2024-04-26T23:18:42.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.795+0000 7fec44420640 1 -- 172.21.15.47:0/2534507096 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fec3c15ea90 con 0x7fec3c15f170 2024-04-26T23:18:42.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.795+0000 7fec3affd640 1 -- 172.21.15.47:0/2534507096 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7fec30058da0 con 0x7fec3c15f170 2024-04-26T23:18:42.796 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-26T23:18:42.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.797+0000 7fec44420640 1 -- 172.21.15.47:0/2534507096 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fec2806c8f0 msgr2=0x7fec2806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:42.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.797+0000 7fec44420640 1 --2- 172.21.15.47:0/2534507096 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fec2806c8f0 0x7fec2806edb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fec2c033000 tx=0x7fec2c009000 comp rx=0 tx=0).stop 2024-04-26T23:18:42.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.797+0000 7fec44420640 1 -- 172.21.15.47:0/2534507096 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec3c15f170 msgr2=0x7fec3c0ce170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:42.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.797+0000 7fec44420640 1 --2- 172.21.15.47:0/2534507096 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec3c15f170 0x7fec3c0ce170 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fec3000c3c0 tx=0x7fec3000c7c0 comp rx=0 tx=0).stop 2024-04-26T23:18:42.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.797+0000 7fec44420640 1 -- 172.21.15.47:0/2534507096 shutdown_connections 2024-04-26T23:18:42.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.797+0000 7fec44420640 1 --2- 172.21.15.47:0/2534507096 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fec2806c8f0 0x7fec2806edb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:42.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.797+0000 7fec44420640 1 --2- 172.21.15.47:0/2534507096 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec3c0ce6b0 0x7fec3c154790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:42.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.797+0000 7fec44420640 1 --2- 172.21.15.47:0/2534507096 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec3c15f170 0x7fec3c0ce170 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:42.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.797+0000 7fec44420640 1 --2- 172.21.15.47:0/2534507096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec3c0d3bc0 0x7fec3c151ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:42.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.797+0000 7fec44420640 1 -- 172.21.15.47:0/2534507096 >> 172.21.15.47:0/2534507096 conn(0x7fec3c0d4f60 msgr2=0x7fec3c06cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:42.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.797+0000 7fec44420640 1 -- 172.21.15.47:0/2534507096 shutdown_connections 2024-04-26T23:18:42.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.797+0000 7fec44420640 1 -- 172.21.15.47:0/2534507096 wait complete. 2024-04-26T23:18:42.808 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T23:18:42.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.912+0000 7fc4ed997640 1 Processor -- start 2024-04-26T23:18:42.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.912+0000 7fc4ed997640 1 -- start start 2024-04-26T23:18:42.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.913+0000 7fc4ed997640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4e80ca360 0x7fc4e816a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:42.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.913+0000 7fc4ed997640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4e80c8bc0 con 0x7fc4e8162a20 2024-04-26T23:18:42.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.913+0000 7fc4ed997640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4e80d09f0 con 0x7fc4e80d3ee0 2024-04-26T23:18:42.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.913+0000 7fc4ed997640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc4e80d07a0 con 0x7fc4e80ca360 2024-04-26T23:18:42.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.913+0000 7fc4e6ffd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc4e8162a20 0x7fc4e80d1700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37404/0 (socket says 172.21.15.47:37404) 2024-04-26T23:18:42.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.913+0000 7fc4e6ffd640 1 -- 172.21.15.47:0/1593248921 learned_addr learned my addr 172.21.15.47:0/1593248921 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:18:42.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.913+0000 7fc4e77fe640 1 --2- 172.21.15.47:0/1593248921 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4e80ca360 0x7fc4e816a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:42.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.913+0000 7fc4e67fc640 1 -- 172.21.15.47:0/1593248921 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3930841990 0 0) 0x7fc4e80c8bc0 con 0x7fc4e8162a20 2024-04-26T23:18:42.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.913+0000 7fc4e67fc640 1 -- 172.21.15.47:0/1593248921 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc4d4003520 con 0x7fc4e8162a20 2024-04-26T23:18:42.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.913+0000 7fc4e67fc640 1 -- 172.21.15.47:0/1593248921 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 228846094 0 0) 0x7fc4d4003520 con 0x7fc4e8162a20 2024-04-26T23:18:42.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.913+0000 7fc4e67fc640 1 -- 172.21.15.47:0/1593248921 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4e80ca360 msgr2=0x7fc4e816a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:42.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.913+0000 7fc4e67fc640 1 --2- 172.21.15.47:0/1593248921 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4e80ca360 0x7fc4e816a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:42.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.913+0000 7fc4e67fc640 1 -- 172.21.15.47:0/1593248921 >> v1:172.21.15.203:6789/0 conn(0x7fc4e80d3ee0 legacy=0x7fc4e80d1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:42.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.913+0000 7fc4e67fc640 1 -- 172.21.15.47:0/1593248921 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4e80d0cd0 con 0x7fc4e8162a20 2024-04-26T23:18:42.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.913+0000 7fc4e67fc640 1 -- 172.21.15.47:0/1593248921 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7fc4d8001670 con 0x7fc4e8162a20 2024-04-26T23:18:42.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.913+0000 7fc4e67fc640 1 -- 172.21.15.47:0/1593248921 >> v1:172.21.15.47:6789/0 conn(0x7fc4e8162a20 legacy=0x7fc4e80d1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:42.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.913+0000 7fc4e67fc640 1 --2- 172.21.15.47:0/1593248921 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4d40039e0 0x7fc4d4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:42.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.913+0000 7fc4e67fc640 1 --2- 172.21.15.47:0/1593248921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4d4004390 0x7fc4d4004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:42.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.913+0000 7fc4e67fc640 1 --2- 172.21.15.47:0/1593248921 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4e80ca360 0x7fc4d4005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:42.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.913+0000 7fc4e67fc640 1 -- 172.21.15.47:0/1593248921 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e80d07a0 con 0x7fc4d40039e0 2024-04-26T23:18:42.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.913+0000 7fc4e67fc640 1 -- 172.21.15.47:0/1593248921 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d400baa0 con 0x7fc4d4004390 2024-04-26T23:18:42.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.913+0000 7fc4e67fc640 1 -- 172.21.15.47:0/1593248921 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc4d400bca0 con 0x7fc4e80ca360 2024-04-26T23:18:42.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.914+0000 7fc4e6ffd640 1 --2- 172.21.15.47:0/1593248921 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4d40039e0 0x7fc4d4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:42.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.914+0000 7fc4dffff640 1 --2- 172.21.15.47:0/1593248921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4d4004390 0x7fc4d4004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:42.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.914+0000 7fc4e77fe640 1 --2- 172.21.15.47:0/1593248921 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4e80ca360 0x7fc4d4005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:42.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.914+0000 7fc4e6ffd640 1 -- 172.21.15.47:0/1593248921 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4e80ca360 msgr2=0x7fc4d4005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:42.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.914+0000 7fc4e6ffd640 1 --2- 172.21.15.47:0/1593248921 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4e80ca360 0x7fc4d4005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:42.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.914+0000 7fc4e6ffd640 1 -- 172.21.15.47:0/1593248921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4d4004390 msgr2=0x7fc4d4004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:42.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.914+0000 7fc4e6ffd640 1 --2- 172.21.15.47:0/1593248921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4d4004390 0x7fc4d4004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:42.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.914+0000 7fc4e6ffd640 1 -- 172.21.15.47:0/1593248921 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4d4003520 con 0x7fc4d40039e0 2024-04-26T23:18:42.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.914+0000 7fc4e77fe640 1 --2- 172.21.15.47:0/1593248921 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4e80ca360 0x7fc4d4005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:18:42.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.915+0000 7fc4e6ffd640 1 --2- 172.21.15.47:0/1593248921 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4d40039e0 0x7fc4d4003dc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc4d800c290 tx=0x7fc4d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2c14037afbc4ecd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:42.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.915+0000 7fc4e67fc640 1 -- 172.21.15.47:0/1593248921 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d8007d50 con 0x7fc4d40039e0 2024-04-26T23:18:42.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.915+0000 7fc4e67fc640 1 -- 172.21.15.47:0/1593248921 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4d8005af0 con 0x7fc4d40039e0 2024-04-26T23:18:42.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.915+0000 7fc4e67fc640 1 -- 172.21.15.47:0/1593248921 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d8007070 con 0x7fc4d40039e0 2024-04-26T23:18:42.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.915+0000 7fc4ed997640 1 -- 172.21.15.47:0/1593248921 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4d40039e0 msgr2=0x7fc4d4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:42.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.915+0000 7fc4ed997640 1 --2- 172.21.15.47:0/1593248921 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4d40039e0 0x7fc4d4003dc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc4d800c290 tx=0x7fc4d4005830 comp rx=0 tx=0).stop 2024-04-26T23:18:42.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.915+0000 7fc4e77fe640 1 -- 172.21.15.47:0/1593248921 reap_dead start 2024-04-26T23:18:42.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.915+0000 7fc4ed997640 1 -- 172.21.15.47:0/1593248921 shutdown_connections 2024-04-26T23:18:42.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.915+0000 7fc4ed997640 1 -- 172.21.15.47:0/1593248921 >> 172.21.15.47:0/1593248921 conn(0x7fc4e80d4fd0 msgr2=0x7fc4e80d53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:42.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.915+0000 7fc4ed997640 1 -- 172.21.15.47:0/1593248921 shutdown_connections 2024-04-26T23:18:42.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.915+0000 7fc4ed997640 1 -- 172.21.15.47:0/1593248921 wait complete. 2024-04-26T23:18:42.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.917+0000 7fc4ed997640 1 Processor -- start 2024-04-26T23:18:42.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.917+0000 7fc4ed997640 1 -- start start 2024-04-26T23:18:42.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.917+0000 7fc4ed997640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4d40039e0 0x7fc4e814f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:42.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.917+0000 7fc4ed997640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4e814d070 0x7fc4e81f8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:42.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.917+0000 7fc4ed997640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4e80b9cc0 0x7fc4e80ba140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:42.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.917+0000 7fc4ed997640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e80d0fb0 con 0x7fc4e814d070 2024-04-26T23:18:42.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.917+0000 7fc4ed997640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e80d2520 con 0x7fc4d40039e0 2024-04-26T23:18:42.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.917+0000 7fc4ed997640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc4e80cbd90 con 0x7fc4e80b9cc0 2024-04-26T23:18:42.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.917+0000 7fc4dffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4e814d070 0x7fc4e81f8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:42.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.917+0000 7fc4dffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4e814d070 0x7fc4e81f8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54276/0 (socket says 172.21.15.47:54276) 2024-04-26T23:18:42.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.917+0000 7fc4dffff640 1 -- 172.21.15.47:0/3721676832 learned_addr learned my addr 172.21.15.47:0/3721676832 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:42.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.918+0000 7fc4e77fe640 1 --2- 172.21.15.47:0/3721676832 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4e80b9cc0 0x7fc4e80ba140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:42.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.918+0000 7fc4e6ffd640 1 --2- 172.21.15.47:0/3721676832 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4d40039e0 0x7fc4e814f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:42.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.918+0000 7fc4dffff640 1 -- 172.21.15.47:0/3721676832 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4e80b9cc0 msgr2=0x7fc4e80ba140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:42.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.918+0000 7fc4dffff640 1 --2- 172.21.15.47:0/3721676832 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4e80b9cc0 0x7fc4e80ba140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:42.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.918+0000 7fc4dffff640 1 -- 172.21.15.47:0/3721676832 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4d40039e0 msgr2=0x7fc4e814f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:42.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.918+0000 7fc4dffff640 1 --2- 172.21.15.47:0/3721676832 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4d40039e0 0x7fc4e814f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:42.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.918+0000 7fc4dffff640 1 -- 172.21.15.47:0/3721676832 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4e814d7c0 con 0x7fc4e814d070 2024-04-26T23:18:42.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.918+0000 7fc4e77fe640 1 --2- 172.21.15.47:0/3721676832 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4e80b9cc0 0x7fc4e80ba140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:18:42.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.918+0000 7fc4dffff640 1 --2- 172.21.15.47:0/3721676832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4e814d070 0x7fc4e81f8fb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc4d000f750 tx=0x7fc4d000fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:42.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.918+0000 7fc4e4ff9640 1 -- 172.21.15.47:0/3721676832 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d000fe60 con 0x7fc4e814d070 2024-04-26T23:18:42.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.918+0000 7fc4e4ff9640 1 -- 172.21.15.47:0/3721676832 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4d0018070 con 0x7fc4e814d070 2024-04-26T23:18:42.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.918+0000 7fc4ed997640 1 -- 172.21.15.47:0/3721676832 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4e814d4f0 con 0x7fc4e814d070 2024-04-26T23:18:42.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.918+0000 7fc4e4ff9640 1 -- 172.21.15.47:0/3721676832 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d000d730 con 0x7fc4e814d070 2024-04-26T23:18:42.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.918+0000 7fc4ed997640 1 -- 172.21.15.47:0/3721676832 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4e814fa00 con 0x7fc4e814d070 2024-04-26T23:18:42.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.919+0000 7fc4ed997640 1 -- 172.21.15.47:0/3721676832 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4e80694f0 con 0x7fc4e814d070 2024-04-26T23:18:42.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.920+0000 7fc4e4ff9640 1 -- 172.21.15.47:0/3721676832 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc4d000d8d0 con 0x7fc4e814d070 2024-04-26T23:18:42.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.920+0000 7fc4e4ff9640 1 --2- 172.21.15.47:0/3721676832 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fc4c806c920 0x7fc4c806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:42.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.920+0000 7fc4e4ff9640 1 -- 172.21.15.47:0/3721676832 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fc4d008bdb0 con 0x7fc4e814d070 2024-04-26T23:18:42.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.920+0000 7fc4e6ffd640 1 --2- 172.21.15.47:0/3721676832 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fc4c806c920 0x7fc4c806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:42.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.921+0000 7fc4e6ffd640 1 --2- 172.21.15.47:0/3721676832 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fc4c806c920 0x7fc4c806ede0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8033000 tx=0x7fc4d802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:42.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:42.925+0000 7fc4e4ff9640 1 -- 172.21.15.47:0/3721676832 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc4d001a310 con 0x7fc4e814d070 2024-04-26T23:18:43.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.111+0000 7fc4ed997640 1 -- 172.21.15.47:0/3721676832 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fc4e806d990 con 0x7fc4e814d070 2024-04-26T23:18:43.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.112+0000 7fc4e4ff9640 1 -- 172.21.15.47:0/3721676832 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7fc4d0059ea0 con 0x7fc4e814d070 2024-04-26T23:18:43.113 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-26T23:18:43.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.114+0000 7fc4ed997640 1 -- 172.21.15.47:0/3721676832 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fc4c806c920 msgr2=0x7fc4c806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:43.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.114+0000 7fc4ed997640 1 --2- 172.21.15.47:0/3721676832 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fc4c806c920 0x7fc4c806ede0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8033000 tx=0x7fc4d802e000 comp rx=0 tx=0).stop 2024-04-26T23:18:43.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.114+0000 7fc4ed997640 1 -- 172.21.15.47:0/3721676832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4e814d070 msgr2=0x7fc4e81f8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:43.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.114+0000 7fc4ed997640 1 --2- 172.21.15.47:0/3721676832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4e814d070 0x7fc4e81f8fb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc4d000f750 tx=0x7fc4d000fc10 comp rx=0 tx=0).stop 2024-04-26T23:18:43.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.114+0000 7fc4ed997640 1 -- 172.21.15.47:0/3721676832 shutdown_connections 2024-04-26T23:18:43.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.114+0000 7fc4ed997640 1 --2- 172.21.15.47:0/3721676832 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fc4c806c920 0x7fc4c806ede0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:43.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.114+0000 7fc4ed997640 1 --2- 172.21.15.47:0/3721676832 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4e80b9cc0 0x7fc4e80ba140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:43.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.114+0000 7fc4ed997640 1 --2- 172.21.15.47:0/3721676832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4e814d070 0x7fc4e81f8fb0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:43.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.114+0000 7fc4ed997640 1 --2- 172.21.15.47:0/3721676832 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4d40039e0 0x7fc4e814f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:43.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.114+0000 7fc4ed997640 1 -- 172.21.15.47:0/3721676832 >> 172.21.15.47:0/3721676832 conn(0x7fc4e80d4fd0 msgr2=0x7fc4e80cb270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:43.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.114+0000 7fc4ed997640 1 -- 172.21.15.47:0/3721676832 shutdown_connections 2024-04-26T23:18:43.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.114+0000 7fc4ed997640 1 -- 172.21.15.47:0/3721676832 wait complete. 2024-04-26T23:18:43.126 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T23:18:43.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.226+0000 7f96cc607640 1 Processor -- start 2024-04-26T23:18:43.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.226+0000 7f96cc607640 1 -- start start 2024-04-26T23:18:43.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.227+0000 7f96cc607640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f96c4070750 0x7f96c40b56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:43.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.227+0000 7f96cc607640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96c40af6e0 0x7f96c40afb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:43.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.227+0000 7f96cc607640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96c40b5e30 con 0x7f96c4072fb0 2024-04-26T23:18:43.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.227+0000 7f96cc607640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f96c40b5c00 con 0x7f96c4070750 2024-04-26T23:18:43.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.227+0000 7f96cc607640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f96c406f2b0 con 0x7f96c40af6e0 2024-04-26T23:18:43.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.228+0000 7f96c9ba2640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f96c4070750 0x7f96c40b56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:43.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.228+0000 7f96caba4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96c40af6e0 0x7f96c40afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:43.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.228+0000 7f96ca3a3640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f96c4072fb0 0x7f96c4073390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.47:33498/0 (socket says 172.21.15.47:33498) 2024-04-26T23:18:43.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.228+0000 7f96caba4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96c40af6e0 0x7f96c40afb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.47:60450/0 (socket says 172.21.15.47:60450) 2024-04-26T23:18:43.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.228+0000 7f96c9ba2640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f96c4070750 0x7f96c40b56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.47:49914/0 (socket says 172.21.15.47:49914) 2024-04-26T23:18:43.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.228+0000 7f96ca3a3640 1 -- 172.21.15.47:0/1720456425 learned_addr learned my addr 172.21.15.47:0/1720456425 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:18:43.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.228+0000 7f96caba4640 1 -- 172.21.15.47:0/1720456425 >> v2:172.21.15.203:3300/0 conn(0x7f96c4070750 msgr2=0x7f96c40b56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:43.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.228+0000 7f96caba4640 1 --2- 172.21.15.47:0/1720456425 >> v2:172.21.15.203:3300/0 conn(0x7f96c4070750 0x7f96c40b56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:43.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.228+0000 7f96caba4640 1 -- 172.21.15.47:0/1720456425 >> v1:172.21.15.203:6789/0 conn(0x7f96c4072fb0 legacy=0x7f96c4073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:43.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.228+0000 7f96caba4640 1 -- 172.21.15.47:0/1720456425 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96c40b6110 con 0x7f96c40af6e0 2024-04-26T23:18:43.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.228+0000 7f96c9ba2640 1 --2- 172.21.15.47:0/1720456425 >> v2:172.21.15.203:3300/0 conn(0x7f96c4070750 0x7f96c40b56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:18:43.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.229+0000 7f96caba4640 1 --2- 172.21.15.47:0/1720456425 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96c40af6e0 0x7f96c40afb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f96b4009d90 tx=0x7f96b402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ea91c3206506f9d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:43.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.229+0000 7f96c93a1640 1 -- 172.21.15.47:0/1720456425 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b4007740 con 0x7f96c40af6e0 2024-04-26T23:18:43.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.229+0000 7f96c93a1640 1 -- 172.21.15.47:0/1720456425 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96b40078e0 con 0x7f96c40af6e0 2024-04-26T23:18:43.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.229+0000 7f96c93a1640 1 -- 172.21.15.47:0/1720456425 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b4007cb0 con 0x7f96c40af6e0 2024-04-26T23:18:43.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.229+0000 7f96cc607640 1 -- 172.21.15.47:0/1720456425 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96c40af6e0 msgr2=0x7f96c40afb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:43.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.229+0000 7f96cc607640 1 --2- 172.21.15.47:0/1720456425 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96c40af6e0 0x7f96c40afb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f96b4009d90 tx=0x7f96b402a660 comp rx=0 tx=0).stop 2024-04-26T23:18:43.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.230+0000 7f96cc607640 1 -- 172.21.15.47:0/1720456425 shutdown_connections 2024-04-26T23:18:43.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.230+0000 7f96cc607640 1 --2- 172.21.15.47:0/1720456425 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96c40af6e0 0x7f96c40afb70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:43.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.230+0000 7f96cc607640 1 --2- 172.21.15.47:0/1720456425 >> v2:172.21.15.203:3300/0 conn(0x7f96c4070750 0x7f96c40b56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:43.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.230+0000 7f96cc607640 1 -- 172.21.15.47:0/1720456425 >> 172.21.15.47:0/1720456425 conn(0x7f96c40b39b0 msgr2=0x7f96c40b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:43.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.230+0000 7f96cc607640 1 -- 172.21.15.47:0/1720456425 shutdown_connections 2024-04-26T23:18:43.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.230+0000 7f96cc607640 1 -- 172.21.15.47:0/1720456425 wait complete. 2024-04-26T23:18:43.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.231+0000 7f96cc607640 1 Processor -- start 2024-04-26T23:18:43.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.231+0000 7f96cc607640 1 -- start start 2024-04-26T23:18:43.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.231+0000 7f96cc607640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96c4070750 0x7f96c415c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:43.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.231+0000 7f96cc607640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96c4072fb0 0x7f96c414e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:43.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.231+0000 7f96cc607640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96c40af6e0 0x7f96c415ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:43.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.231+0000 7f96cc607640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f96c415c2e0 con 0x7f96c40af6e0 2024-04-26T23:18:43.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.231+0000 7f96cc607640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f96c415c160 con 0x7f96c4072fb0 2024-04-26T23:18:43.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.231+0000 7f96cc607640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f96c40b00b0 con 0x7f96c4070750 2024-04-26T23:18:43.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.232+0000 7f96caba4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96c40af6e0 0x7f96c415ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:43.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.232+0000 7f96ca3a3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96c4070750 0x7f96c415c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:43.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.232+0000 7f96caba4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96c40af6e0 0x7f96c415ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54286/0 (socket says 172.21.15.47:54286) 2024-04-26T23:18:43.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.232+0000 7f96caba4640 1 -- 172.21.15.47:0/660554236 learned_addr learned my addr 172.21.15.47:0/660554236 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:43.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.232+0000 7f96c9ba2640 1 --2- 172.21.15.47:0/660554236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96c4072fb0 0x7f96c414e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:43.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.232+0000 7f96caba4640 1 -- 172.21.15.47:0/660554236 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96c4070750 msgr2=0x7f96c415c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:43.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.232+0000 7f96caba4640 1 --2- 172.21.15.47:0/660554236 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96c4070750 0x7f96c415c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:43.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.232+0000 7f96caba4640 1 -- 172.21.15.47:0/660554236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96c4072fb0 msgr2=0x7f96c414e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:43.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.232+0000 7f96caba4640 1 --2- 172.21.15.47:0/660554236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96c4072fb0 0x7f96c414e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:43.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.232+0000 7f96caba4640 1 -- 172.21.15.47:0/660554236 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96c415cb60 con 0x7f96c40af6e0 2024-04-26T23:18:43.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.232+0000 7f96caba4640 1 --2- 172.21.15.47:0/660554236 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96c40af6e0 0x7f96c415ebb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f96b4009eb0 tx=0x7f96b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:43.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.232+0000 7f96c2ffd640 1 -- 172.21.15.47:0/660554236 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b403f070 con 0x7f96c40af6e0 2024-04-26T23:18:43.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.232+0000 7f96c2ffd640 1 -- 172.21.15.47:0/660554236 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96b4031bc0 con 0x7f96c40af6e0 2024-04-26T23:18:43.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.232+0000 7f96cc607640 1 -- 172.21.15.47:0/660554236 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96c415ce40 con 0x7f96c40af6e0 2024-04-26T23:18:43.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.232+0000 7f96c2ffd640 1 -- 172.21.15.47:0/660554236 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b4009450 con 0x7f96c40af6e0 2024-04-26T23:18:43.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.232+0000 7f96cc607640 1 -- 172.21.15.47:0/660554236 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96c415d400 con 0x7f96c40af6e0 2024-04-26T23:18:43.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.233+0000 7f96cc607640 1 -- 172.21.15.47:0/660554236 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96c40694f0 con 0x7f96c40af6e0 2024-04-26T23:18:43.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.234+0000 7f96c2ffd640 1 -- 172.21.15.47:0/660554236 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f96b403b410 con 0x7f96c40af6e0 2024-04-26T23:18:43.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.234+0000 7f96c2ffd640 1 --2- 172.21.15.47:0/660554236 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f96b006c8f0 0x7f96b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:43.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.234+0000 7f96c2ffd640 1 -- 172.21.15.47:0/660554236 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f96b40ad620 con 0x7f96c40af6e0 2024-04-26T23:18:43.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.234+0000 7f96ca3a3640 1 --2- 172.21.15.47:0/660554236 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f96b006c8f0 0x7f96b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:43.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.235+0000 7f96ca3a3640 1 --2- 172.21.15.47:0/660554236 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f96b006c8f0 0x7f96b006edb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f96c4150df0 tx=0x7f96bc007e80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:43.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.238+0000 7f96c2ffd640 1 -- 172.21.15.47:0/660554236 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f96b407afa0 con 0x7f96c40af6e0 2024-04-26T23:18:43.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.435+0000 7f96cc607640 1 -- 172.21.15.47:0/660554236 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f96c4071470 con 0x7f96c40af6e0 2024-04-26T23:18:43.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.435+0000 7f96c2ffd640 1 -- 172.21.15.47:0/660554236 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7f96b407f2a0 con 0x7f96c40af6e0 2024-04-26T23:18:43.436 INFO:teuthology.orchestra.run.smithi047.stderr:listed 0 entries 2024-04-26T23:18:43.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.437+0000 7f96cc607640 1 -- 172.21.15.47:0/660554236 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f96b006c8f0 msgr2=0x7f96b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:43.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.437+0000 7f96cc607640 1 --2- 172.21.15.47:0/660554236 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f96b006c8f0 0x7f96b006edb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f96c4150df0 tx=0x7f96bc007e80 comp rx=0 tx=0).stop 2024-04-26T23:18:43.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.437+0000 7f96cc607640 1 -- 172.21.15.47:0/660554236 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96c40af6e0 msgr2=0x7f96c415ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:43.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.437+0000 7f96cc607640 1 --2- 172.21.15.47:0/660554236 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96c40af6e0 0x7f96c415ebb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f96b4009eb0 tx=0x7f96b4004a40 comp rx=0 tx=0).stop 2024-04-26T23:18:43.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.437+0000 7f96cc607640 1 -- 172.21.15.47:0/660554236 shutdown_connections 2024-04-26T23:18:43.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.437+0000 7f96cc607640 1 --2- 172.21.15.47:0/660554236 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f96b006c8f0 0x7f96b006edb0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:43.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.437+0000 7f96cc607640 1 --2- 172.21.15.47:0/660554236 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96c40af6e0 0x7f96c415ebb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:43.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.437+0000 7f96cc607640 1 --2- 172.21.15.47:0/660554236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96c4072fb0 0x7f96c414e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:43.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.437+0000 7f96cc607640 1 --2- 172.21.15.47:0/660554236 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96c4070750 0x7f96c415c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:43.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.437+0000 7f96cc607640 1 -- 172.21.15.47:0/660554236 >> 172.21.15.47:0/660554236 conn(0x7f96c40b39b0 msgr2=0x7f96c40b2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:43.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.437+0000 7f96cc607640 1 -- 172.21.15.47:0/660554236 shutdown_connections 2024-04-26T23:18:43.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.437+0000 7f96cc607640 1 -- 172.21.15.47:0/660554236 wait complete. 2024-04-26T23:18:43.449 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:18:43.449 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:18:43.464 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:18:43.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.604+0000 7f1bfb2c4640 1 Processor -- start 2024-04-26T23:18:43.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.604+0000 7f1bfb2c4640 1 -- start start 2024-04-26T23:18:43.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.605+0000 7f1bfb2c4640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1bf4162880 0x7f1bf40d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:43.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.605+0000 7f1bfb2c4640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bf40b7e50 con 0x7f1bf40d3bc0 2024-04-26T23:18:43.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.605+0000 7f1bfb2c4640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bf40d1800 con 0x7f1bf40b5f20 2024-04-26T23:18:43.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.605+0000 7f1bfb2c4640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f1bf40d0580 con 0x7f1bf4162880 2024-04-26T23:18:43.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.605+0000 7f1bf9060640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1bf4162880 0x7f1bf40d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:43.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.605+0000 7f1bf9861640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1bf40d3bc0 0x7f1bf40b8300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37410/0 (socket says 172.21.15.47:37410) 2024-04-26T23:18:43.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.605+0000 7f1bf9861640 1 -- 172.21.15.47:0/3202991988 learned_addr learned my addr 172.21.15.47:0/3202991988 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:18:43.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.605+0000 7f1bf9060640 1 -- 172.21.15.47:0/3202991988 >> v1:172.21.15.203:6789/0 conn(0x7f1bf40b5f20 legacy=0x7f1bf40d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:43.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.605+0000 7f1bf9060640 1 -- 172.21.15.47:0/3202991988 >> v1:172.21.15.47:6789/0 conn(0x7f1bf40d3bc0 legacy=0x7f1bf40b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:43.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.605+0000 7f1bf9060640 1 -- 172.21.15.47:0/3202991988 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bf40d07d0 con 0x7f1bf4162880 2024-04-26T23:18:43.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.606+0000 7f1bf9060640 1 --2- 172.21.15.47:0/3202991988 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1bf4162880 0x7f1bf40d2cb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1bec0097a0 tx=0x7f1bec02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d9ae2183079ffbb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:43.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.606+0000 7f1bf3fff640 1 -- 172.21.15.47:0/3202991988 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bec037070 con 0x7f1bf4162880 2024-04-26T23:18:43.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.606+0000 7f1bf3fff640 1 -- 172.21.15.47:0/3202991988 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bec032440 con 0x7f1bf4162880 2024-04-26T23:18:43.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.606+0000 7f1bf3fff640 1 -- 172.21.15.47:0/3202991988 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bec032810 con 0x7f1bf4162880 2024-04-26T23:18:43.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.607+0000 7f1bfb2c4640 1 -- 172.21.15.47:0/3202991988 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1bf4162880 msgr2=0x7f1bf40d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:43.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.607+0000 7f1bfb2c4640 1 --2- 172.21.15.47:0/3202991988 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1bf4162880 0x7f1bf40d2cb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1bec0097a0 tx=0x7f1bec02a1e0 comp rx=0 tx=0).stop 2024-04-26T23:18:43.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.607+0000 7f1bfb2c4640 1 -- 172.21.15.47:0/3202991988 shutdown_connections 2024-04-26T23:18:43.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.607+0000 7f1bfb2c4640 1 --2- 172.21.15.47:0/3202991988 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1bf4162880 0x7f1bf40d2cb0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:43.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.607+0000 7f1bfb2c4640 1 -- 172.21.15.47:0/3202991988 >> 172.21.15.47:0/3202991988 conn(0x7f1bf40d4f60 msgr2=0x7f1bf40d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:43.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.607+0000 7f1bfb2c4640 1 -- 172.21.15.47:0/3202991988 shutdown_connections 2024-04-26T23:18:43.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.607+0000 7f1bfb2c4640 1 -- 172.21.15.47:0/3202991988 wait complete. 2024-04-26T23:18:43.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.608+0000 7f1bfb2c4640 1 Processor -- start 2024-04-26T23:18:43.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.608+0000 7f1bfb2c4640 1 -- start start 2024-04-26T23:18:43.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.609+0000 7f1bfb2c4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1bf40b5f20 0x7f1bf414f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:43.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.609+0000 7f1bfb2c4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1bf40d3bc0 0x7f1bf40cda00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:43.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.609+0000 7f1bfb2c4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1bf4162880 0x7f1bf40b49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:43.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.609+0000 7f1bfb2c4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf4162c60 con 0x7f1bf4162880 2024-04-26T23:18:43.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.609+0000 7f1bfb2c4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf40d0d90 con 0x7f1bf40d3bc0 2024-04-26T23:18:43.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.609+0000 7f1bfb2c4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f1bf4162de0 con 0x7f1bf40b5f20 2024-04-26T23:18:43.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.609+0000 7f1bf9861640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1bf4162880 0x7f1bf40b49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:43.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.609+0000 7f1bf9861640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1bf4162880 0x7f1bf40b49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54304/0 (socket says 172.21.15.47:54304) 2024-04-26T23:18:43.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.609+0000 7f1bf9861640 1 -- 172.21.15.47:0/3751160373 learned_addr learned my addr 172.21.15.47:0/3751160373 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:43.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.609+0000 7f1bf9060640 1 --2- 172.21.15.47:0/3751160373 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1bf40b5f20 0x7f1bf414f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:43.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.609+0000 7f1bf885f640 1 --2- 172.21.15.47:0/3751160373 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1bf40d3bc0 0x7f1bf40cda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:43.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.609+0000 7f1bf9861640 1 -- 172.21.15.47:0/3751160373 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1bf40b5f20 msgr2=0x7f1bf414f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:43.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.609+0000 7f1bf9861640 1 --2- 172.21.15.47:0/3751160373 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1bf40b5f20 0x7f1bf414f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:43.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.609+0000 7f1bf9861640 1 -- 172.21.15.47:0/3751160373 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1bf40d3bc0 msgr2=0x7f1bf40cda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:43.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.609+0000 7f1bf9861640 1 --2- 172.21.15.47:0/3751160373 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1bf40d3bc0 0x7f1bf40cda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:43.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.609+0000 7f1bf9861640 1 -- 172.21.15.47:0/3751160373 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bf414f8e0 con 0x7f1bf4162880 2024-04-26T23:18:43.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.609+0000 7f1bf885f640 1 --2- 172.21.15.47:0/3751160373 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1bf40d3bc0 0x7f1bf40cda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:18:43.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.609+0000 7f1bf9060640 1 --2- 172.21.15.47:0/3751160373 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1bf40b5f20 0x7f1bf414f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:18:43.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.609+0000 7f1bf9861640 1 --2- 172.21.15.47:0/3751160373 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1bf4162880 0x7f1bf40b49a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1be400caf0 tx=0x7f1be400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:43.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.609+0000 7f1bf1ffb640 1 -- 172.21.15.47:0/3751160373 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be4017070 con 0x7f1bf4162880 2024-04-26T23:18:43.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.610+0000 7f1bf1ffb640 1 -- 172.21.15.47:0/3751160373 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1be400aca0 con 0x7f1bf4162880 2024-04-26T23:18:43.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.610+0000 7f1bfb2c4640 1 -- 172.21.15.47:0/3751160373 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bf40ce500 con 0x7f1bf4162880 2024-04-26T23:18:43.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.610+0000 7f1bf1ffb640 1 -- 172.21.15.47:0/3751160373 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be4012600 con 0x7f1bf4162880 2024-04-26T23:18:43.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.610+0000 7f1bfb2c4640 1 -- 172.21.15.47:0/3751160373 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bf4150040 con 0x7f1bf4162880 2024-04-26T23:18:43.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.611+0000 7f1bfb2c4640 1 -- 172.21.15.47:0/3751160373 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bf40694f0 con 0x7f1bf4162880 2024-04-26T23:18:43.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.611+0000 7f1bf1ffb640 1 -- 172.21.15.47:0/3751160373 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1be401a070 con 0x7f1bf4162880 2024-04-26T23:18:43.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.611+0000 7f1bf1ffb640 1 --2- 172.21.15.47:0/3751160373 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1be006c8f0 0x7f1be006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:43.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.612+0000 7f1bf1ffb640 1 -- 172.21.15.47:0/3751160373 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1be408b9f0 con 0x7f1bf4162880 2024-04-26T23:18:43.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.612+0000 7f1bf9060640 1 --2- 172.21.15.47:0/3751160373 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1be006c8f0 0x7f1be006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:43.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.612+0000 7f1bf9060640 1 --2- 172.21.15.47:0/3751160373 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1be006c8f0 0x7f1be006edb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1bec034000 tx=0x7f1bec02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:43.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.616+0000 7f1bf1ffb640 1 -- 172.21.15.47:0/3751160373 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1bf414f8e0 con 0x7f1bf4162880 2024-04-26T23:18:43.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.800+0000 7f1bfb2c4640 1 -- 172.21.15.47:0/3751160373 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1bf40cf370 con 0x7f1bf4162880 2024-04-26T23:18:43.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.801+0000 7f1bf1ffb640 1 -- 172.21.15.47:0/3751160373 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18478 (secure 0 0 0) 0x7f1be4059520 con 0x7f1bf4162880 2024-04-26T23:18:43.802 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:18:43.802 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":24,"fsid":"399006d3-4db1-43e6-ba81-a79a276eabfd","created":"2024-04-26T23:18:06.731211+0000","modified":"2024-04-26T23:18:36.402132+0000","last_up_change":"2024-04-26T23:18:17.808736+0000","last_in_change":"2024-04-26T23:18:08.945859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:18:16.937492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:18:29.033575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:18:30.318499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:18:34.014536+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ee091567-fea0-4a14-bf4e-0e16c20bb2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6819","nonce":326520779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6821","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6823","nonce":326520779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6829","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6831","nonce":326520779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6825","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6827","nonce":326520779}]},"public_addr":"172.21.15.47:6819/326520779","cluster_addr":"172.21.15.47:6823/326520779","heartbeat_back_addr":"172.21.15.47:6831/326520779","heartbeat_front_addr":"172.21.15.47:6827/326520779","state":["exists","up"]},{"osd":1,"uuid":"c14440a0-bba5-4931-a6db-b736c08e869b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6809","nonce":1912610931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6811","nonce":1912610931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6815","nonce":1912610931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6813","nonce":1912610931}]},"public_addr":"172.21.15.47:6809/1912610931","cluster_addr":"172.21.15.47:6811/1912610931","heartbeat_back_addr":"172.21.15.47:6815/1912610931","heartbeat_front_addr":"172.21.15.47:6813/1912610931","state":["exists","up"]},{"osd":2,"uuid":"3940f8de-c9b2-4cd3-a911-9c301729eb30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6801","nonce":2569869305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6803","nonce":2569869305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6807","nonce":2569869305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6805","nonce":2569869305}]},"public_addr":"172.21.15.47:6801/2569869305","cluster_addr":"172.21.15.47:6803/2569869305","heartbeat_back_addr":"172.21.15.47:6807/2569869305","heartbeat_front_addr":"172.21.15.47:6805/2569869305","state":["exists","up"]},{"osd":3,"uuid":"1286a67e-8958-426a-8c2e-bc85498c619c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6818","nonce":2791411683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6822","nonce":2791411683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6830","nonce":2791411683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6826","nonce":2791411683}]},"public_addr":"172.21.15.47:6818/2791411683","cluster_addr":"172.21.15.47:6822/2791411683","heartbeat_back_addr":"172.21.15.47:6830/2791411683","heartbeat_front_addr":"172.21.15.47:6826/2791411683","state":["exists","up"]},{"osd":4,"uuid":"d8ee2c20-e5b7-46c2-9999-b234070126cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6809","nonce":441313104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6811","nonce":441313104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6815","nonce":441313104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6813","nonce":441313104}]},"public_addr":"172.21.15.203:6809/441313104","cluster_addr":"172.21.15.203:6811/441313104","heartbeat_back_addr":"172.21.15.203:6815/441313104","heartbeat_front_addr":"172.21.15.203:6813/441313104","state":["exists","up"]},{"osd":5,"uuid":"b453a8f0-f324-4840-9ae0-d67bdadb9f8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6801","nonce":56838444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6803","nonce":56838444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6807","nonce":56838444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6805","nonce":56838444}]},"public_addr":"172.21.15.203:6801/56838444","cluster_addr":"172.21.15.203:6803/56838444","heartbeat_back_addr":"172.21.15.203:6807/56838444","heartbeat_front_addr":"172.21.15.203:6805/56838444","state":["exists","up"]},{"osd":6,"uuid":"a6d31951-2d4b-4e8d-a510-6283b01dc158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6825","nonce":2738517080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6827","nonce":2738517080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6831","nonce":2738517080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6829","nonce":2738517080}]},"public_addr":"172.21.15.203:6825/2738517080","cluster_addr":"172.21.15.203:6827/2738517080","heartbeat_back_addr":"172.21.15.203:6831/2738517080","heartbeat_front_addr":"172.21.15.203:6829/2738517080","state":["exists","up"]},{"osd":7,"uuid":"7119b8c0-fe96-402d-9e75-7b509d9d44d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6817","nonce":1315565058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6819","nonce":1315565058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6823","nonce":1315565058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6821","nonce":1315565058}]},"public_addr":"172.21.15.203:6817/1315565058","cluster_addr":"172.21.15.203:6819/1315565058","heartbeat_back_addr":"172.21.15.203:6823/1315565058","heartbeat_front_addr":"172.21.15.203:6821/1315565058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.121921+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.137494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.084814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.111593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.072430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.125066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.097867+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.057061+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:18:43.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.803+0000 7f1bfb2c4640 1 -- 172.21.15.47:0/3751160373 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1be006c8f0 msgr2=0x7f1be006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:43.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.803+0000 7f1bfb2c4640 1 --2- 172.21.15.47:0/3751160373 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1be006c8f0 0x7f1be006edb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1bec034000 tx=0x7f1bec02e000 comp rx=0 tx=0).stop 2024-04-26T23:18:43.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.803+0000 7f1bfb2c4640 1 -- 172.21.15.47:0/3751160373 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1bf4162880 msgr2=0x7f1bf40b49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:43.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.803+0000 7f1bfb2c4640 1 --2- 172.21.15.47:0/3751160373 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1bf4162880 0x7f1bf40b49a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1be400caf0 tx=0x7f1be400a060 comp rx=0 tx=0).stop 2024-04-26T23:18:43.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.803+0000 7f1bfb2c4640 1 -- 172.21.15.47:0/3751160373 shutdown_connections 2024-04-26T23:18:43.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.803+0000 7f1bfb2c4640 1 --2- 172.21.15.47:0/3751160373 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1be006c8f0 0x7f1be006edb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:43.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.804+0000 7f1bfb2c4640 1 --2- 172.21.15.47:0/3751160373 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1bf4162880 0x7f1bf40b49a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:43.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.804+0000 7f1bfb2c4640 1 --2- 172.21.15.47:0/3751160373 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1bf40d3bc0 0x7f1bf40cda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:43.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.804+0000 7f1bfb2c4640 1 --2- 172.21.15.47:0/3751160373 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1bf40b5f20 0x7f1bf414f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:43.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.804+0000 7f1bfb2c4640 1 -- 172.21.15.47:0/3751160373 >> 172.21.15.47:0/3751160373 conn(0x7f1bf40d4f60 msgr2=0x7f1bf415f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:43.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.804+0000 7f1bfb2c4640 1 -- 172.21.15.47:0/3751160373 shutdown_connections 2024-04-26T23:18:43.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.804+0000 7f1bfb2c4640 1 -- 172.21.15.47:0/3751160373 wait complete. 2024-04-26T23:18:43.815 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:18:16.937492+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:18:29.033575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:18:30.318499+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T23:18:34.014536+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T23:18:43.816 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:18:43.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.918+0000 7fa973ed1640 1 Processor -- start 2024-04-26T23:18:43.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.919+0000 7fa973ed1640 1 -- start start 2024-04-26T23:18:43.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.919+0000 7fa973ed1640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa96c0d1a00 0x7fa96c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:43.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.919+0000 7fa973ed1640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa96c0d0a10 con 0x7fa96c0b40d0 2024-04-26T23:18:43.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.919+0000 7fa973ed1640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa96c0d0cf0 con 0x7fa96c0d2320 2024-04-26T23:18:43.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.919+0000 7fa973ed1640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fa96c0d0fd0 con 0x7fa96c0d1a00 2024-04-26T23:18:43.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.919+0000 7fa971c6d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa96c0d1a00 0x7fa96c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:43.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.919+0000 7fa97246e640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa96c0b40d0 0x7fa96c0b4b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37422/0 (socket says 172.21.15.47:37422) 2024-04-26T23:18:43.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.919+0000 7fa97246e640 1 -- 172.21.15.47:0/1148212074 learned_addr learned my addr 172.21.15.47:0/1148212074 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:18:43.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.920+0000 7fa971c6d640 1 -- 172.21.15.47:0/1148212074 >> v1:172.21.15.203:6789/0 conn(0x7fa96c0d2320 legacy=0x7fa96c1625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:43.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.920+0000 7fa971c6d640 1 -- 172.21.15.47:0/1148212074 >> v1:172.21.15.47:6789/0 conn(0x7fa96c0b40d0 legacy=0x7fa96c0b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:43.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.920+0000 7fa971c6d640 1 -- 172.21.15.47:0/1148212074 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa96c0d40a0 con 0x7fa96c0d1a00 2024-04-26T23:18:43.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.920+0000 7fa970c6b640 1 -- 172.21.15.47:0/1148212074 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 640882876 0 0) 0x7fa96c0d0a10 con 0x7fa96c0b40d0 2024-04-26T23:18:43.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.920+0000 7fa971c6d640 1 --2- 172.21.15.47:0/1148212074 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa96c0d1a00 0x7fa96c0d1de0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa9640097a0 tx=0x7fa96402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1db0440dc7c73627 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:43.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.921+0000 7fa970c6b640 1 -- 172.21.15.47:0/1148212074 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa964037070 con 0x7fa96c0d1a00 2024-04-26T23:18:43.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.921+0000 7fa970c6b640 1 -- 172.21.15.47:0/1148212074 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa964032440 con 0x7fa96c0d1a00 2024-04-26T23:18:43.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.921+0000 7fa970c6b640 1 -- 172.21.15.47:0/1148212074 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96402ae60 con 0x7fa96c0d1a00 2024-04-26T23:18:43.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.921+0000 7fa973ed1640 1 -- 172.21.15.47:0/1148212074 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa96c0d1a00 msgr2=0x7fa96c0d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:43.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.921+0000 7fa973ed1640 1 --2- 172.21.15.47:0/1148212074 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa96c0d1a00 0x7fa96c0d1de0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa9640097a0 tx=0x7fa96402a110 comp rx=0 tx=0).stop 2024-04-26T23:18:43.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.921+0000 7fa973ed1640 1 -- 172.21.15.47:0/1148212074 shutdown_connections 2024-04-26T23:18:43.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.921+0000 7fa973ed1640 1 --2- 172.21.15.47:0/1148212074 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa96c0d1a00 0x7fa96c0d1de0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:43.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.921+0000 7fa973ed1640 1 -- 172.21.15.47:0/1148212074 >> 172.21.15.47:0/1148212074 conn(0x7fa96c0d4fd0 msgr2=0x7fa96c0d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:43.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.921+0000 7fa973ed1640 1 -- 172.21.15.47:0/1148212074 shutdown_connections 2024-04-26T23:18:43.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.921+0000 7fa973ed1640 1 -- 172.21.15.47:0/1148212074 wait complete. 2024-04-26T23:18:43.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.922+0000 7fa973ed1640 1 Processor -- start 2024-04-26T23:18:43.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.923+0000 7fa973ed1640 1 -- start start 2024-04-26T23:18:43.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.923+0000 7fa973ed1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa96c0b40d0 0x7fa96c1f6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:43.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.923+0000 7fa973ed1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa96c0d1a00 0x7fa96c0cb490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:43.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.923+0000 7fa973ed1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa96c0d2320 0x7fa96c0c8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:43.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.923+0000 7fa973ed1640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa96c0b5210 con 0x7fa96c0d1a00 2024-04-26T23:18:43.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.923+0000 7fa973ed1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa96c162f50 con 0x7fa96c0d2320 2024-04-26T23:18:43.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.923+0000 7fa973ed1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa96c0b5390 con 0x7fa96c0b40d0 2024-04-26T23:18:43.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.923+0000 7fa97146c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa96c0d1a00 0x7fa96c0cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:43.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.923+0000 7fa97146c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa96c0d1a00 0x7fa96c0cb490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54308/0 (socket says 172.21.15.47:54308) 2024-04-26T23:18:43.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.923+0000 7fa97146c640 1 -- 172.21.15.47:0/644520291 learned_addr learned my addr 172.21.15.47:0/644520291 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:43.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.924+0000 7fa971c6d640 1 --2- 172.21.15.47:0/644520291 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa96c0b40d0 0x7fa96c1f6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:43.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.924+0000 7fa97246e640 1 --2- 172.21.15.47:0/644520291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa96c0d2320 0x7fa96c0c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:43.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.924+0000 7fa97146c640 1 -- 172.21.15.47:0/644520291 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa96c0b40d0 msgr2=0x7fa96c1f6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:43.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.924+0000 7fa97146c640 1 --2- 172.21.15.47:0/644520291 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa96c0b40d0 0x7fa96c1f6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:43.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.924+0000 7fa97146c640 1 -- 172.21.15.47:0/644520291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa96c0d2320 msgr2=0x7fa96c0c8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:43.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.924+0000 7fa97146c640 1 --2- 172.21.15.47:0/644520291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa96c0d2320 0x7fa96c0c8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:43.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.924+0000 7fa97146c640 1 -- 172.21.15.47:0/644520291 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa96c0b8de0 con 0x7fa96c0d1a00 2024-04-26T23:18:43.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.924+0000 7fa97146c640 1 --2- 172.21.15.47:0/644520291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa96c0d1a00 0x7fa96c0cb490 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa96000c400 tx=0x7fa96000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:43.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.924+0000 7fa96a7fc640 1 -- 172.21.15.47:0/644520291 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa960017070 con 0x7fa96c0d1a00 2024-04-26T23:18:43.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.924+0000 7fa96a7fc640 1 -- 172.21.15.47:0/644520291 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9600052d0 con 0x7fa96c0d1a00 2024-04-26T23:18:43.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.924+0000 7fa973ed1640 1 -- 172.21.15.47:0/644520291 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa96c0b90c0 con 0x7fa96c0d1a00 2024-04-26T23:18:43.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.924+0000 7fa96a7fc640 1 -- 172.21.15.47:0/644520291 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa960003830 con 0x7fa96c0d1a00 2024-04-26T23:18:43.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.924+0000 7fa973ed1640 1 -- 172.21.15.47:0/644520291 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa96c0b9680 con 0x7fa96c0d1a00 2024-04-26T23:18:43.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.925+0000 7fa973ed1640 1 -- 172.21.15.47:0/644520291 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa96c0694f0 con 0x7fa96c0d1a00 2024-04-26T23:18:43.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.925+0000 7fa96a7fc640 1 -- 172.21.15.47:0/644520291 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa96001a070 con 0x7fa96c0d1a00 2024-04-26T23:18:43.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.926+0000 7fa96a7fc640 1 --2- 172.21.15.47:0/644520291 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa95806c8f0 0x7fa95806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:43.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.926+0000 7fa96a7fc640 1 -- 172.21.15.47:0/644520291 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fa96008b1f0 con 0x7fa96c0d1a00 2024-04-26T23:18:43.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.926+0000 7fa971c6d640 1 --2- 172.21.15.47:0/644520291 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa95806c8f0 0x7fa95806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:43.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.927+0000 7fa971c6d640 1 --2- 172.21.15.47:0/644520291 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa95806c8f0 0x7fa95806edb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa964034000 tx=0x7fa96402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:43.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:43.930+0000 7fa96a7fc640 1 -- 172.21.15.47:0/644520291 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa9600592e0 con 0x7fa96c0d1a00 2024-04-26T23:18:44.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.119+0000 7fa973ed1640 1 -- 172.21.15.47:0/644520291 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa96c06cca0 con 0x7fa96c0d1a00 2024-04-26T23:18:44.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.120+0000 7fa96a7fc640 1 -- 172.21.15.47:0/644520291 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7fa960058be0 con 0x7fa96c0d1a00 2024-04-26T23:18:44.121 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-26T23:18:44.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.122+0000 7fa973ed1640 1 -- 172.21.15.47:0/644520291 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa95806c8f0 msgr2=0x7fa95806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:44.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.122+0000 7fa973ed1640 1 --2- 172.21.15.47:0/644520291 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa95806c8f0 0x7fa95806edb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa964034000 tx=0x7fa96402e000 comp rx=0 tx=0).stop 2024-04-26T23:18:44.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.122+0000 7fa973ed1640 1 -- 172.21.15.47:0/644520291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa96c0d1a00 msgr2=0x7fa96c0cb490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:44.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.122+0000 7fa973ed1640 1 --2- 172.21.15.47:0/644520291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa96c0d1a00 0x7fa96c0cb490 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa96000c400 tx=0x7fa96000c8c0 comp rx=0 tx=0).stop 2024-04-26T23:18:44.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.122+0000 7fa973ed1640 1 -- 172.21.15.47:0/644520291 shutdown_connections 2024-04-26T23:18:44.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.122+0000 7fa973ed1640 1 --2- 172.21.15.47:0/644520291 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa95806c8f0 0x7fa95806edb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:44.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.122+0000 7fa973ed1640 1 --2- 172.21.15.47:0/644520291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa96c0d2320 0x7fa96c0c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:44.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.122+0000 7fa973ed1640 1 --2- 172.21.15.47:0/644520291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa96c0d1a00 0x7fa96c0cb490 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:44.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.122+0000 7fa973ed1640 1 --2- 172.21.15.47:0/644520291 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa96c0b40d0 0x7fa96c1f6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:44.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.122+0000 7fa973ed1640 1 -- 172.21.15.47:0/644520291 >> 172.21.15.47:0/644520291 conn(0x7fa96c0d4fd0 msgr2=0x7fa96c161640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:44.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.122+0000 7fa973ed1640 1 -- 172.21.15.47:0/644520291 shutdown_connections 2024-04-26T23:18:44.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.122+0000 7fa973ed1640 1 -- 172.21.15.47:0/644520291 wait complete. 2024-04-26T23:18:44.133 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:18:44.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.236+0000 7f08ae615640 1 Processor -- start 2024-04-26T23:18:44.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.236+0000 7f08ae615640 1 -- start start 2024-04-26T23:18:44.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.237+0000 7f08ae615640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f08a815f170 0x7f08a80d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:44.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.237+0000 7f08ae615640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f08a80d3bc0 0x7f08a80d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:44.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.237+0000 7f08ae615640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08a80d1040 con 0x7f08a815f550 2024-04-26T23:18:44.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.237+0000 7f08ae615640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f08a80c9430 con 0x7f08a80d3bc0 2024-04-26T23:18:44.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.237+0000 7f08ae615640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f08a80c9680 con 0x7f08a815f170 2024-04-26T23:18:44.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.237+0000 7f08acbb2640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f08a80d3bc0 0x7f08a80d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:44.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.237+0000 7f08acbb2640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f08a80d3bc0 0x7f08a80d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54322/0 (socket says 172.21.15.47:54322) 2024-04-26T23:18:44.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.237+0000 7f08acbb2640 1 -- 172.21.15.47:0/574052326 learned_addr learned my addr 172.21.15.47:0/574052326 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:44.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.237+0000 7f08a7fff640 1 --2- 172.21.15.47:0/574052326 >> v2:172.21.15.203:3300/0 conn(0x7f08a815f170 0x7f08a80d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:44.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.237+0000 7f08acbb2640 1 -- 172.21.15.47:0/574052326 >> v2:172.21.15.203:3300/0 conn(0x7f08a815f170 msgr2=0x7f08a80d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:44.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.237+0000 7f08acbb2640 1 --2- 172.21.15.47:0/574052326 >> v2:172.21.15.203:3300/0 conn(0x7f08a815f170 0x7f08a80d5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:44.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.237+0000 7f08acbb2640 1 -- 172.21.15.47:0/574052326 >> v1:172.21.15.203:6789/0 conn(0x7f08a815f550 legacy=0x7f08a80d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:44.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.238+0000 7f08acbb2640 1 -- 172.21.15.47:0/574052326 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08a80c9960 con 0x7f08a80d3bc0 2024-04-26T23:18:44.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.238+0000 7f08a7fff640 1 --2- 172.21.15.47:0/574052326 >> v2:172.21.15.203:3300/0 conn(0x7f08a815f170 0x7f08a80d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:18:44.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.238+0000 7f08acbb2640 1 --2- 172.21.15.47:0/574052326 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08a80d3bc0 0x7f08a80d2cb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f08a0009e50 tx=0x7f08a002a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58100481a6f341c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:44.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.238+0000 7f08a6ffd640 1 -- 172.21.15.47:0/574052326 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a0036070 con 0x7f08a80d3bc0 2024-04-26T23:18:44.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.238+0000 7f08a6ffd640 1 -- 172.21.15.47:0/574052326 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08a0031600 con 0x7f08a80d3bc0 2024-04-26T23:18:44.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.238+0000 7f08a6ffd640 1 -- 172.21.15.47:0/574052326 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a00319d0 con 0x7f08a80d3bc0 2024-04-26T23:18:44.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.239+0000 7f08ae615640 1 -- 172.21.15.47:0/574052326 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08a80d3bc0 msgr2=0x7f08a80d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:44.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.239+0000 7f08ae615640 1 --2- 172.21.15.47:0/574052326 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08a80d3bc0 0x7f08a80d2cb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f08a0009e50 tx=0x7f08a002a760 comp rx=0 tx=0).stop 2024-04-26T23:18:44.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.239+0000 7f08ae615640 1 -- 172.21.15.47:0/574052326 shutdown_connections 2024-04-26T23:18:44.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.239+0000 7f08ae615640 1 --2- 172.21.15.47:0/574052326 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08a80d3bc0 0x7f08a80d2cb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:44.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.239+0000 7f08ae615640 1 --2- 172.21.15.47:0/574052326 >> v2:172.21.15.203:3300/0 conn(0x7f08a815f170 0x7f08a80d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:44.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.239+0000 7f08ae615640 1 -- 172.21.15.47:0/574052326 >> 172.21.15.47:0/574052326 conn(0x7f08a80d4f60 msgr2=0x7f08a80d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:44.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.239+0000 7f08ae615640 1 -- 172.21.15.47:0/574052326 shutdown_connections 2024-04-26T23:18:44.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.239+0000 7f08ae615640 1 -- 172.21.15.47:0/574052326 wait complete. 2024-04-26T23:18:44.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.240+0000 7f08ae615640 1 Processor -- start 2024-04-26T23:18:44.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.240+0000 7f08ae615640 1 -- start start 2024-04-26T23:18:44.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.241+0000 7f08ae615640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08a80d3bc0 0x7f08a80a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:44.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.241+0000 7f08ae615640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08a815f170 0x7f08a8158e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:44.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.241+0000 7f08ae615640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08a80a0970 0x7f08a81593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:44.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.241+0000 7f08ae615640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f08a80ca5c0 con 0x7f08a80a0970 2024-04-26T23:18:44.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.241+0000 7f08ae615640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f08a80ca440 con 0x7f08a80d3bc0 2024-04-26T23:18:44.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.241+0000 7f08ae615640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f08a80ca740 con 0x7f08a815f170 2024-04-26T23:18:44.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.241+0000 7f08acbb2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08a80a0970 0x7f08a81593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:44.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.241+0000 7f08acbb2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08a80a0970 0x7f08a81593b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54324/0 (socket says 172.21.15.47:54324) 2024-04-26T23:18:44.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.241+0000 7f08acbb2640 1 -- 172.21.15.47:0/3997629727 learned_addr learned my addr 172.21.15.47:0/3997629727 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:44.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.241+0000 7f08a7fff640 1 --2- 172.21.15.47:0/3997629727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08a80d3bc0 0x7f08a80a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:44.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.241+0000 7f08a77fe640 1 --2- 172.21.15.47:0/3997629727 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08a815f170 0x7f08a8158e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:44.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.241+0000 7f08acbb2640 1 -- 172.21.15.47:0/3997629727 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08a815f170 msgr2=0x7f08a8158e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:44.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.241+0000 7f08acbb2640 1 --2- 172.21.15.47:0/3997629727 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08a815f170 0x7f08a8158e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:44.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.241+0000 7f08acbb2640 1 -- 172.21.15.47:0/3997629727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08a80d3bc0 msgr2=0x7f08a80a0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:44.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.241+0000 7f08acbb2640 1 --2- 172.21.15.47:0/3997629727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08a80d3bc0 0x7f08a80a0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:44.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.241+0000 7f08acbb2640 1 -- 172.21.15.47:0/3997629727 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08a80a0e80 con 0x7f08a80a0970 2024-04-26T23:18:44.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.242+0000 7f08acbb2640 1 --2- 172.21.15.47:0/3997629727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08a80a0970 0x7f08a81593b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f08a0034040 tx=0x7f08a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:44.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.242+0000 7f08a4ff9640 1 -- 172.21.15.47:0/3997629727 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a0036050 con 0x7f08a80a0970 2024-04-26T23:18:44.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.242+0000 7f08a4ff9640 1 -- 172.21.15.47:0/3997629727 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08a0042070 con 0x7f08a80a0970 2024-04-26T23:18:44.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.242+0000 7f08ae615640 1 -- 172.21.15.47:0/3997629727 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08a80a1160 con 0x7f08a80a0970 2024-04-26T23:18:44.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.242+0000 7f08a4ff9640 1 -- 172.21.15.47:0/3997629727 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a0009510 con 0x7f08a80a0970 2024-04-26T23:18:44.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.242+0000 7f08ae615640 1 -- 172.21.15.47:0/3997629727 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08a80a1720 con 0x7f08a80a0970 2024-04-26T23:18:44.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.243+0000 7f08ae615640 1 -- 172.21.15.47:0/3997629727 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08a80694f0 con 0x7f08a80a0970 2024-04-26T23:18:44.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.243+0000 7f08a4ff9640 1 -- 172.21.15.47:0/3997629727 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f08a0007bd0 con 0x7f08a80a0970 2024-04-26T23:18:44.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.244+0000 7f08a4ff9640 1 --2- 172.21.15.47:0/3997629727 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f089406c8f0 0x7f089406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:44.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.244+0000 7f08a4ff9640 1 -- 172.21.15.47:0/3997629727 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f08a00accb0 con 0x7f08a80a0970 2024-04-26T23:18:44.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.244+0000 7f08a7fff640 1 --2- 172.21.15.47:0/3997629727 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f089406c8f0 0x7f089406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:44.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.245+0000 7f08a7fff640 1 --2- 172.21.15.47:0/3997629727 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f089406c8f0 0x7f089406edb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f089c006fd0 tx=0x7f089c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:44.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.249+0000 7f08a4ff9640 1 -- 172.21.15.47:0/3997629727 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f08a007ada0 con 0x7f08a80a0970 2024-04-26T23:18:44.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.434+0000 7f08ae615640 1 -- 172.21.15.47:0/3997629727 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f08a815ea90 con 0x7f08a80a0970 2024-04-26T23:18:44.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.435+0000 7f08a4ff9640 1 -- 172.21.15.47:0/3997629727 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f08a007a6a0 con 0x7f08a80a0970 2024-04-26T23:18:44.437 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-26T23:18:44.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.437+0000 7f08ae615640 1 -- 172.21.15.47:0/3997629727 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f089406c8f0 msgr2=0x7f089406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:44.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.437+0000 7f08ae615640 1 --2- 172.21.15.47:0/3997629727 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f089406c8f0 0x7f089406edb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f089c006fd0 tx=0x7f089c008040 comp rx=0 tx=0).stop 2024-04-26T23:18:44.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.437+0000 7f08ae615640 1 -- 172.21.15.47:0/3997629727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08a80a0970 msgr2=0x7f08a81593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:44.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.437+0000 7f08ae615640 1 --2- 172.21.15.47:0/3997629727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08a80a0970 0x7f08a81593b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f08a0034040 tx=0x7f08a0004a40 comp rx=0 tx=0).stop 2024-04-26T23:18:44.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.437+0000 7f08ae615640 1 -- 172.21.15.47:0/3997629727 shutdown_connections 2024-04-26T23:18:44.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.437+0000 7f08ae615640 1 --2- 172.21.15.47:0/3997629727 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f089406c8f0 0x7f089406edb0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:44.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.437+0000 7f08ae615640 1 --2- 172.21.15.47:0/3997629727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08a80a0970 0x7f08a81593b0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:44.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.437+0000 7f08ae615640 1 --2- 172.21.15.47:0/3997629727 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08a815f170 0x7f08a8158e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:44.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.437+0000 7f08ae615640 1 --2- 172.21.15.47:0/3997629727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08a80d3bc0 0x7f08a80a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:44.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.437+0000 7f08ae615640 1 -- 172.21.15.47:0/3997629727 >> 172.21.15.47:0/3997629727 conn(0x7f08a80d4f60 msgr2=0x7f08a806d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:44.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.437+0000 7f08ae615640 1 -- 172.21.15.47:0/3997629727 shutdown_connections 2024-04-26T23:18:44.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.437+0000 7f08ae615640 1 -- 172.21.15.47:0/3997629727 wait complete. 2024-04-26T23:18:44.449 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:18:44.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.552+0000 7f8ab99f0640 1 Processor -- start 2024-04-26T23:18:44.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.552+0000 7f8ab99f0640 1 -- start start 2024-04-26T23:18:44.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.552+0000 7f8ab99f0640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8ab415f550 0x7f8ab40d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:44.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.552+0000 7f8ab99f0640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ab40d1040 con 0x7f8ab40d3bc0 2024-04-26T23:18:44.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.552+0000 7f8ab99f0640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ab40c9680 con 0x7f8ab415f170 2024-04-26T23:18:44.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.552+0000 7f8ab99f0640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f8ab40c9430 con 0x7f8ab415f550 2024-04-26T23:18:44.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.553+0000 7f8ab37fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f8ab40d3bc0 0x7f8ab40ca440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37436/0 (socket says 172.21.15.47:37436) 2024-04-26T23:18:44.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.553+0000 7f8ab37fe640 1 -- 172.21.15.47:0/1304517070 learned_addr learned my addr 172.21.15.47:0/1304517070 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:18:44.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.553+0000 7f8aabfff640 1 --2- 172.21.15.47:0/1304517070 >> v2:172.21.15.203:3300/0 conn(0x7f8ab415f550 0x7f8ab40d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:44.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.553+0000 7f8ab27fc640 1 -- 172.21.15.47:0/1304517070 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3389305400 0 0) 0x7f8aac001490 con 0x7f8ab40d3bc0 2024-04-26T23:18:44.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.553+0000 7f8ab27fc640 1 -- 172.21.15.47:0/1304517070 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8aa0003520 con 0x7f8ab40d3bc0 2024-04-26T23:18:44.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.553+0000 7f8ab27fc640 1 -- 172.21.15.47:0/1304517070 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3073743090 0 0) 0x7f8ab40c9680 con 0x7f8ab415f170 2024-04-26T23:18:44.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.553+0000 7f8ab27fc640 1 -- 172.21.15.47:0/1304517070 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8aac001490 con 0x7f8ab415f170 2024-04-26T23:18:44.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.553+0000 7f8ab27fc640 1 -- 172.21.15.47:0/1304517070 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3063074349 0 0) 0x7f8aa0003520 con 0x7f8ab40d3bc0 2024-04-26T23:18:44.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.553+0000 7f8ab27fc640 1 -- 172.21.15.47:0/1304517070 >> v2:172.21.15.203:3300/0 conn(0x7f8ab415f550 msgr2=0x7f8ab40d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:44.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.553+0000 7f8ab27fc640 1 --2- 172.21.15.47:0/1304517070 >> v2:172.21.15.203:3300/0 conn(0x7f8ab415f550 0x7f8ab40d5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:44.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.553+0000 7f8ab27fc640 1 -- 172.21.15.47:0/1304517070 >> v1:172.21.15.203:6789/0 conn(0x7f8ab415f170 legacy=0x7f8ab40d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:44.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.553+0000 7f8ab27fc640 1 -- 172.21.15.47:0/1304517070 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ab40c9960 con 0x7f8ab40d3bc0 2024-04-26T23:18:44.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.553+0000 7f8ab27fc640 1 -- 172.21.15.47:0/1304517070 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7f8aac001bc0 con 0x7f8ab40d3bc0 2024-04-26T23:18:44.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.553+0000 7f8ab27fc640 1 -- 172.21.15.47:0/1304517070 >> v1:172.21.15.47:6789/0 conn(0x7f8ab40d3bc0 legacy=0x7f8ab40ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:44.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.553+0000 7f8ab27fc640 1 --2- 172.21.15.47:0/1304517070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8aa0003b70 0x7f8aa0003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:44.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.553+0000 7f8ab27fc640 1 --2- 172.21.15.47:0/1304517070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8aa0004800 0x7f8aa0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:44.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.553+0000 7f8ab27fc640 1 --2- 172.21.15.47:0/1304517070 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8aa00051c0 0x7f8aa000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:44.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.553+0000 7f8ab27fc640 1 -- 172.21.15.47:0/1304517070 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab40c9430 con 0x7f8aa0004800 2024-04-26T23:18:44.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.553+0000 7f8ab27fc640 1 -- 172.21.15.47:0/1304517070 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa000c090 con 0x7f8aa0003b70 2024-04-26T23:18:44.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.553+0000 7f8ab27fc640 1 -- 172.21.15.47:0/1304517070 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f8aa000c290 con 0x7f8aa00051c0 2024-04-26T23:18:44.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.553+0000 7f8aabfff640 1 --2- 172.21.15.47:0/1304517070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8aa0004800 0x7f8aa0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:44.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.553+0000 7f8ab2ffd640 1 --2- 172.21.15.47:0/1304517070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8aa0003b70 0x7f8aa0003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:44.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.554+0000 7f8ab37fe640 1 --2- 172.21.15.47:0/1304517070 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8aa00051c0 0x7f8aa000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:44.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.554+0000 7f8aabfff640 1 -- 172.21.15.47:0/1304517070 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8aa00051c0 msgr2=0x7f8aa000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:44.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.554+0000 7f8aabfff640 1 --2- 172.21.15.47:0/1304517070 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8aa00051c0 0x7f8aa000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:44.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.554+0000 7f8aabfff640 1 -- 172.21.15.47:0/1304517070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8aa0003b70 msgr2=0x7f8aa0003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:44.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.554+0000 7f8aabfff640 1 --2- 172.21.15.47:0/1304517070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8aa0003b70 0x7f8aa0003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:44.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.554+0000 7f8aabfff640 1 -- 172.21.15.47:0/1304517070 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aa0003520 con 0x7f8aa0004800 2024-04-26T23:18:44.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.554+0000 7f8ab37fe640 1 -- 172.21.15.47:0/1304517070 reap_dead start 2024-04-26T23:18:44.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.554+0000 7f8aabfff640 1 --2- 172.21.15.47:0/1304517070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8aa0004800 0x7f8aa0004c80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8ab40c8990 tx=0x7f8aa0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=169683888cd137c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:44.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.555+0000 7f8ab27fc640 1 -- 172.21.15.47:0/1304517070 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c036070 con 0x7f8aa0004800 2024-04-26T23:18:44.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.555+0000 7f8ab27fc640 1 -- 172.21.15.47:0/1304517070 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a9c031400 con 0x7f8aa0004800 2024-04-26T23:18:44.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.555+0000 7f8ab27fc640 1 -- 172.21.15.47:0/1304517070 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c009690 con 0x7f8aa0004800 2024-04-26T23:18:44.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.555+0000 7f8ab99f0640 1 -- 172.21.15.47:0/1304517070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8aa0004800 msgr2=0x7f8aa0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:44.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.555+0000 7f8ab99f0640 1 --2- 172.21.15.47:0/1304517070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8aa0004800 0x7f8aa0004c80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8ab40c8990 tx=0x7f8aa0005830 comp rx=0 tx=0).stop 2024-04-26T23:18:44.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.555+0000 7f8ab99f0640 1 -- 172.21.15.47:0/1304517070 shutdown_connections 2024-04-26T23:18:44.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.555+0000 7f8ab99f0640 1 --2- 172.21.15.47:0/1304517070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8aa0004800 0x7f8aa0004c80 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:44.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.555+0000 7f8ab99f0640 1 -- 172.21.15.47:0/1304517070 >> 172.21.15.47:0/1304517070 conn(0x7f8ab40d4f60 msgr2=0x7f8ab40d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:44.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.555+0000 7f8ab99f0640 1 -- 172.21.15.47:0/1304517070 shutdown_connections 2024-04-26T23:18:44.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.555+0000 7f8ab99f0640 1 -- 172.21.15.47:0/1304517070 wait complete. 2024-04-26T23:18:44.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.556+0000 7f8ab99f0640 1 Processor -- start 2024-04-26T23:18:44.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.556+0000 7f8ab99f0640 1 -- start start 2024-04-26T23:18:44.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.557+0000 7f8ab99f0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8aa0004800 0x7f8ab409ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:44.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.557+0000 7f8ab99f0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ab40a1be0 0x7f8ab40b9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:44.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.557+0000 7f8ab99f0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ab40a2040 0x7f8ab40ba320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:44.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.557+0000 7f8ab99f0640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab40d5c00 con 0x7f8ab40a2040 2024-04-26T23:18:44.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.557+0000 7f8ab99f0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab40d5a80 con 0x7f8ab40a1be0 2024-04-26T23:18:44.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.557+0000 7f8ab99f0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f8ab40d5d80 con 0x7f8aa0004800 2024-04-26T23:18:44.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.557+0000 7f8ab37fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ab40a2040 0x7f8ab40ba320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:44.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.557+0000 7f8ab37fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ab40a2040 0x7f8ab40ba320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54340/0 (socket says 172.21.15.47:54340) 2024-04-26T23:18:44.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.557+0000 7f8ab37fe640 1 -- 172.21.15.47:0/2605423800 learned_addr learned my addr 172.21.15.47:0/2605423800 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:44.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.557+0000 7f8ab2ffd640 1 --2- 172.21.15.47:0/2605423800 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8aa0004800 0x7f8ab409ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:44.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.557+0000 7f8aabfff640 1 --2- 172.21.15.47:0/2605423800 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ab40a1be0 0x7f8ab40b9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:44.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.557+0000 7f8ab37fe640 1 -- 172.21.15.47:0/2605423800 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8aa0004800 msgr2=0x7f8ab409ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:44.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.557+0000 7f8ab37fe640 1 --2- 172.21.15.47:0/2605423800 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8aa0004800 0x7f8ab409ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:44.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.557+0000 7f8ab37fe640 1 -- 172.21.15.47:0/2605423800 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ab40a1be0 msgr2=0x7f8ab40b9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:44.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.557+0000 7f8ab37fe640 1 --2- 172.21.15.47:0/2605423800 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ab40a1be0 0x7f8ab40b9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:44.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.558+0000 7f8ab37fe640 1 -- 172.21.15.47:0/2605423800 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ab409f490 con 0x7f8ab40a2040 2024-04-26T23:18:44.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.558+0000 7f8ab2ffd640 1 --2- 172.21.15.47:0/2605423800 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8aa0004800 0x7f8ab409ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:18:44.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.558+0000 7f8aabfff640 1 --2- 172.21.15.47:0/2605423800 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ab40a1be0 0x7f8ab40b9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:18:44.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.558+0000 7f8ab37fe640 1 --2- 172.21.15.47:0/2605423800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ab40a2040 0x7f8ab40ba320 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8ab40d4ba0 tx=0x7f8aac007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:44.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.558+0000 7f8ab0ff9640 1 -- 172.21.15.47:0/2605423800 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aac00ae60 con 0x7f8ab40a2040 2024-04-26T23:18:44.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.558+0000 7f8ab0ff9640 1 -- 172.21.15.47:0/2605423800 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8aac018070 con 0x7f8ab40a2040 2024-04-26T23:18:44.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.558+0000 7f8ab99f0640 1 -- 172.21.15.47:0/2605423800 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ab4151010 con 0x7f8ab40a2040 2024-04-26T23:18:44.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.558+0000 7f8ab0ff9640 1 -- 172.21.15.47:0/2605423800 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aac013630 con 0x7f8ab40a2040 2024-04-26T23:18:44.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.558+0000 7f8ab99f0640 1 -- 172.21.15.47:0/2605423800 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ab41515b0 con 0x7f8ab40a2040 2024-04-26T23:18:44.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.559+0000 7f8ab99f0640 1 -- 172.21.15.47:0/2605423800 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ab40694f0 con 0x7f8ab40a2040 2024-04-26T23:18:44.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.559+0000 7f8ab0ff9640 1 -- 172.21.15.47:0/2605423800 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8aac0137d0 con 0x7f8ab40a2040 2024-04-26T23:18:44.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.560+0000 7f8ab0ff9640 1 --2- 172.21.15.47:0/2605423800 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f8a9406c8f0 0x7f8a9406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:44.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.560+0000 7f8ab0ff9640 1 -- 172.21.15.47:0/2605423800 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f8aac08c660 con 0x7f8ab40a2040 2024-04-26T23:18:44.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.560+0000 7f8ab2ffd640 1 --2- 172.21.15.47:0/2605423800 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f8a9406c8f0 0x7f8a9406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:44.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.560+0000 7f8ab2ffd640 1 --2- 172.21.15.47:0/2605423800 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f8a9406c8f0 0x7f8a9406edb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8aa4000c00 tx=0x7f8aa4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:44.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.564+0000 7f8ab0ff9640 1 -- 172.21.15.47:0/2605423800 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8ab40d1040 con 0x7f8ab40a2040 2024-04-26T23:18:44.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.753+0000 7f8ab99f0640 1 -- 172.21.15.47:0/2605423800 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8ab4074590 con 0x7f8ab40a2040 2024-04-26T23:18:44.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.753+0000 7f8ab0ff9640 1 -- 172.21.15.47:0/2605423800 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7f8aac05a620 con 0x7f8ab40a2040 2024-04-26T23:18:44.754 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-26T23:18:44.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.756+0000 7f8ab99f0640 1 -- 172.21.15.47:0/2605423800 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f8a9406c8f0 msgr2=0x7f8a9406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:44.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.756+0000 7f8ab99f0640 1 --2- 172.21.15.47:0/2605423800 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f8a9406c8f0 0x7f8a9406edb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8aa4000c00 tx=0x7f8aa4009040 comp rx=0 tx=0).stop 2024-04-26T23:18:44.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.756+0000 7f8ab99f0640 1 -- 172.21.15.47:0/2605423800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ab40a2040 msgr2=0x7f8ab40ba320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:44.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.756+0000 7f8ab99f0640 1 --2- 172.21.15.47:0/2605423800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ab40a2040 0x7f8ab40ba320 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8ab40d4ba0 tx=0x7f8aac007430 comp rx=0 tx=0).stop 2024-04-26T23:18:44.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.756+0000 7f8ab99f0640 1 -- 172.21.15.47:0/2605423800 shutdown_connections 2024-04-26T23:18:44.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.756+0000 7f8ab99f0640 1 --2- 172.21.15.47:0/2605423800 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f8a9406c8f0 0x7f8a9406edb0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:44.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.756+0000 7f8ab99f0640 1 --2- 172.21.15.47:0/2605423800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ab40a2040 0x7f8ab40ba320 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:44.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.756+0000 7f8ab99f0640 1 --2- 172.21.15.47:0/2605423800 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ab40a1be0 0x7f8ab40b9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:44.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.756+0000 7f8ab99f0640 1 --2- 172.21.15.47:0/2605423800 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8aa0004800 0x7f8ab409ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:44.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.756+0000 7f8ab99f0640 1 -- 172.21.15.47:0/2605423800 >> 172.21.15.47:0/2605423800 conn(0x7f8ab40d4f60 msgr2=0x7f8ab40cb1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:44.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.756+0000 7f8ab99f0640 1 -- 172.21.15.47:0/2605423800 shutdown_connections 2024-04-26T23:18:44.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.756+0000 7f8ab99f0640 1 -- 172.21.15.47:0/2605423800 wait complete. 2024-04-26T23:18:44.769 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T23:18:44.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.872+0000 7f00b5ccc640 1 Processor -- start 2024-04-26T23:18:44.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.872+0000 7f00b5ccc640 1 -- start start 2024-04-26T23:18:44.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.873+0000 7f00b5ccc640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f00b015f550 0x7f00b00d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:44.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.873+0000 7f00b5ccc640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00b00d1040 con 0x7f00b015f170 2024-04-26T23:18:44.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.873+0000 7f00b5ccc640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00b00c9680 con 0x7f00b00d3bc0 2024-04-26T23:18:44.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.873+0000 7f00b5ccc640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f00b00c9430 con 0x7f00b015f550 2024-04-26T23:18:44.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.873+0000 7f00af7fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f00b015f170 0x7f00b00d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37446/0 (socket says 172.21.15.47:37446) 2024-04-26T23:18:44.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.873+0000 7f00af7fe640 1 -- 172.21.15.47:0/1622357620 learned_addr learned my addr 172.21.15.47:0/1622357620 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:18:44.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.873+0000 7f00ae7fc640 1 -- 172.21.15.47:0/1622357620 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3417480 0 0) 0x7f00a0001110 con 0x7f00b015f170 2024-04-26T23:18:44.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.873+0000 7f00ae7fc640 1 -- 172.21.15.47:0/1622357620 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0098003520 con 0x7f00b015f170 2024-04-26T23:18:44.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.873+0000 7f00aeffd640 1 --2- 172.21.15.47:0/1622357620 >> v2:172.21.15.203:3300/0 conn(0x7f00b015f550 0x7f00b00d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:44.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.874+0000 7f00ae7fc640 1 -- 172.21.15.47:0/1622357620 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 36443239 0 0) 0x7f0098003520 con 0x7f00b015f170 2024-04-26T23:18:44.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.874+0000 7f00ae7fc640 1 -- 172.21.15.47:0/1622357620 >> v2:172.21.15.203:3300/0 conn(0x7f00b015f550 msgr2=0x7f00b00d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:44.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.874+0000 7f00ae7fc640 1 --2- 172.21.15.47:0/1622357620 >> v2:172.21.15.203:3300/0 conn(0x7f00b015f550 0x7f00b00d5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:44.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.874+0000 7f00ae7fc640 1 -- 172.21.15.47:0/1622357620 >> v1:172.21.15.203:6789/0 conn(0x7f00b00d3bc0 legacy=0x7f00b00ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:44.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.874+0000 7f00ae7fc640 1 -- 172.21.15.47:0/1622357620 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00b00c9960 con 0x7f00b015f170 2024-04-26T23:18:44.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.874+0000 7f00ae7fc640 1 -- 172.21.15.47:0/1622357620 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7f00a0001840 con 0x7f00b015f170 2024-04-26T23:18:44.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.874+0000 7f00aeffd640 1 --2- 172.21.15.47:0/1622357620 >> v2:172.21.15.203:3300/0 conn(0x7f00b015f550 0x7f00b00d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:18:44.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.874+0000 7f00ae7fc640 1 -- 172.21.15.47:0/1622357620 >> v1:172.21.15.47:6789/0 conn(0x7f00b015f170 legacy=0x7f00b00d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:44.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.874+0000 7f00ae7fc640 1 --2- 172.21.15.47:0/1622357620 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00980039e0 0x7f0098003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:44.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.874+0000 7f00ae7fc640 1 --2- 172.21.15.47:0/1622357620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0098004800 0x7f0098004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:44.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.874+0000 7f00ae7fc640 1 --2- 172.21.15.47:0/1622357620 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0098005210 0x7f009800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:44.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.874+0000 7f00ae7fc640 1 -- 172.21.15.47:0/1622357620 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f00b00c9430 con 0x7f0098004800 2024-04-26T23:18:44.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.874+0000 7f00ae7fc640 1 -- 172.21.15.47:0/1622357620 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f009800c160 con 0x7f0098005210 2024-04-26T23:18:44.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.874+0000 7f00ae7fc640 1 -- 172.21.15.47:0/1622357620 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f009800c360 con 0x7f00980039e0 2024-04-26T23:18:44.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.874+0000 7f00aeffd640 1 --2- 172.21.15.47:0/1622357620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0098004800 0x7f0098004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:44.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.874+0000 7f00affff640 1 --2- 172.21.15.47:0/1622357620 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0098005210 0x7f009800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:44.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.874+0000 7f00af7fe640 1 --2- 172.21.15.47:0/1622357620 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00980039e0 0x7f0098003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:44.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.874+0000 7f00aeffd640 1 -- 172.21.15.47:0/1622357620 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00980039e0 msgr2=0x7f0098003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:44.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.874+0000 7f00aeffd640 1 --2- 172.21.15.47:0/1622357620 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00980039e0 0x7f0098003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:44.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.874+0000 7f00aeffd640 1 -- 172.21.15.47:0/1622357620 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0098005210 msgr2=0x7f009800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:44.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.874+0000 7f00aeffd640 1 --2- 172.21.15.47:0/1622357620 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0098005210 0x7f009800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:44.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.874+0000 7f00affff640 1 -- 172.21.15.47:0/1622357620 reap_dead start 2024-04-26T23:18:44.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.874+0000 7f00aeffd640 1 -- 172.21.15.47:0/1622357620 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0098003520 con 0x7f0098004800 2024-04-26T23:18:44.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.875+0000 7f00aeffd640 1 --2- 172.21.15.47:0/1622357620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0098004800 0x7f0098004c80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f00b008e440 tx=0x7f0098005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eef6b7635347e34 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:44.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.875+0000 7f00ae7fc640 1 -- 172.21.15.47:0/1622357620 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a4037070 con 0x7f0098004800 2024-04-26T23:18:44.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.875+0000 7f00ae7fc640 1 -- 172.21.15.47:0/1622357620 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00a402ac70 con 0x7f0098004800 2024-04-26T23:18:44.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.875+0000 7f00ae7fc640 1 -- 172.21.15.47:0/1622357620 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a400f040 con 0x7f0098004800 2024-04-26T23:18:44.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.876+0000 7f00b5ccc640 1 -- 172.21.15.47:0/1622357620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0098004800 msgr2=0x7f0098004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:44.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.876+0000 7f00b5ccc640 1 --2- 172.21.15.47:0/1622357620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0098004800 0x7f0098004c80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f00b008e440 tx=0x7f0098005830 comp rx=0 tx=0).stop 2024-04-26T23:18:44.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.876+0000 7f00b5ccc640 1 -- 172.21.15.47:0/1622357620 shutdown_connections 2024-04-26T23:18:44.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.876+0000 7f00b5ccc640 1 --2- 172.21.15.47:0/1622357620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0098004800 0x7f0098004c80 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:44.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.876+0000 7f00b5ccc640 1 -- 172.21.15.47:0/1622357620 >> 172.21.15.47:0/1622357620 conn(0x7f00b00d4f60 msgr2=0x7f00b00d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:44.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.876+0000 7f00b5ccc640 1 -- 172.21.15.47:0/1622357620 shutdown_connections 2024-04-26T23:18:44.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.876+0000 7f00b5ccc640 1 -- 172.21.15.47:0/1622357620 wait complete. 2024-04-26T23:18:44.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.877+0000 7f00b5ccc640 1 Processor -- start 2024-04-26T23:18:44.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.877+0000 7f00b5ccc640 1 -- start start 2024-04-26T23:18:44.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.878+0000 7f00b5ccc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0098004800 0x7f00b00ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:44.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.878+0000 7f00b5ccc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00b00a0260 0x7f00b00a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:44.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.878+0000 7f00b5ccc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00b01f83e0 0x7f00b01f8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:44.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.878+0000 7f00b5ccc640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f00b00d5c00 con 0x7f0098004800 2024-04-26T23:18:44.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.878+0000 7f00b5ccc640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f00b00d5a80 con 0x7f00b01f83e0 2024-04-26T23:18:44.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.878+0000 7f00b5ccc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f00b00d5d80 con 0x7f00b00a0260 2024-04-26T23:18:44.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.878+0000 7f00af7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0098004800 0x7f00b00ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:44.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.878+0000 7f00af7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0098004800 0x7f00b00ce0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54344/0 (socket says 172.21.15.47:54344) 2024-04-26T23:18:44.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.878+0000 7f00af7fe640 1 -- 172.21.15.47:0/1282461718 learned_addr learned my addr 172.21.15.47:0/1282461718 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:44.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.878+0000 7f00aeffd640 1 --2- 172.21.15.47:0/1282461718 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00b00a0260 0x7f00b00a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:44.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.878+0000 7f00affff640 1 --2- 172.21.15.47:0/1282461718 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00b01f83e0 0x7f00b01f8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:44.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.878+0000 7f00af7fe640 1 -- 172.21.15.47:0/1282461718 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00b00a0260 msgr2=0x7f00b00a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:44.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.878+0000 7f00af7fe640 1 --2- 172.21.15.47:0/1282461718 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00b00a0260 0x7f00b00a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:44.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.878+0000 7f00af7fe640 1 -- 172.21.15.47:0/1282461718 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00b01f83e0 msgr2=0x7f00b01f8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:44.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.878+0000 7f00af7fe640 1 --2- 172.21.15.47:0/1282461718 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00b01f83e0 0x7f00b01f8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:44.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.878+0000 7f00af7fe640 1 -- 172.21.15.47:0/1282461718 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00b00ce610 con 0x7f0098004800 2024-04-26T23:18:44.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.878+0000 7f00af7fe640 1 --2- 172.21.15.47:0/1282461718 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0098004800 0x7f00b00ce0d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f00a0007290 tx=0x7f00a0007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:44.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.878+0000 7f0097fff640 1 -- 172.21.15.47:0/1282461718 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a000f9c0 con 0x7f0098004800 2024-04-26T23:18:44.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.879+0000 7f0097fff640 1 -- 172.21.15.47:0/1282461718 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00a0018070 con 0x7f0098004800 2024-04-26T23:18:44.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.879+0000 7f00b5ccc640 1 -- 172.21.15.47:0/1282461718 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00b009efb0 con 0x7f0098004800 2024-04-26T23:18:44.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.879+0000 7f0097fff640 1 -- 172.21.15.47:0/1282461718 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a000fdc0 con 0x7f0098004800 2024-04-26T23:18:44.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.879+0000 7f00b5ccc640 1 -- 172.21.15.47:0/1282461718 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00b009f570 con 0x7f0098004800 2024-04-26T23:18:44.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.880+0000 7f00b5ccc640 1 -- 172.21.15.47:0/1282461718 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00b00694f0 con 0x7f0098004800 2024-04-26T23:18:44.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.880+0000 7f0097fff640 1 -- 172.21.15.47:0/1282461718 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f00a001d460 con 0x7f0098004800 2024-04-26T23:18:44.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.880+0000 7f0097fff640 1 --2- 172.21.15.47:0/1282461718 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f009c06c8f0 0x7f009c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:44.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.880+0000 7f0097fff640 1 -- 172.21.15.47:0/1282461718 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f00a008bf80 con 0x7f0098004800 2024-04-26T23:18:44.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.881+0000 7f00aeffd640 1 --2- 172.21.15.47:0/1282461718 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f009c06c8f0 0x7f009c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:44.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.881+0000 7f00aeffd640 1 --2- 172.21.15.47:0/1282461718 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f009c06c8f0 0x7f009c06edb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f00a4034000 tx=0x7f00a402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:44.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:44.885+0000 7f0097fff640 1 -- 172.21.15.47:0/1282461718 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f00b00d1040 con 0x7f0098004800 2024-04-26T23:18:45.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.069+0000 7f00b5ccc640 1 -- 172.21.15.47:0/1282461718 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f00b008d070 con 0x7f0098004800 2024-04-26T23:18:45.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.070+0000 7f0097fff640 1 -- 172.21.15.47:0/1282461718 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7f00a005a070 con 0x7f0098004800 2024-04-26T23:18:45.070 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-26T23:18:45.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.072+0000 7f00b5ccc640 1 -- 172.21.15.47:0/1282461718 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f009c06c8f0 msgr2=0x7f009c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:45.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.072+0000 7f00b5ccc640 1 --2- 172.21.15.47:0/1282461718 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f009c06c8f0 0x7f009c06edb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f00a4034000 tx=0x7f00a402e000 comp rx=0 tx=0).stop 2024-04-26T23:18:45.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.072+0000 7f00b5ccc640 1 -- 172.21.15.47:0/1282461718 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0098004800 msgr2=0x7f00b00ce0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:45.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.072+0000 7f00b5ccc640 1 --2- 172.21.15.47:0/1282461718 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0098004800 0x7f00b00ce0d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f00a0007290 tx=0x7f00a0007750 comp rx=0 tx=0).stop 2024-04-26T23:18:45.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.072+0000 7f00b5ccc640 1 -- 172.21.15.47:0/1282461718 shutdown_connections 2024-04-26T23:18:45.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.072+0000 7f00b5ccc640 1 --2- 172.21.15.47:0/1282461718 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f009c06c8f0 0x7f009c06edb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:45.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.072+0000 7f00b5ccc640 1 --2- 172.21.15.47:0/1282461718 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00b01f83e0 0x7f00b01f8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:45.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.072+0000 7f00b5ccc640 1 --2- 172.21.15.47:0/1282461718 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00b00a0260 0x7f00b00a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:45.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.072+0000 7f00b5ccc640 1 --2- 172.21.15.47:0/1282461718 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0098004800 0x7f00b00ce0d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:45.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.072+0000 7f00b5ccc640 1 -- 172.21.15.47:0/1282461718 >> 172.21.15.47:0/1282461718 conn(0x7f00b00d4f60 msgr2=0x7f00b00d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:45.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.072+0000 7f00b5ccc640 1 -- 172.21.15.47:0/1282461718 shutdown_connections 2024-04-26T23:18:45.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.072+0000 7f00b5ccc640 1 -- 172.21.15.47:0/1282461718 wait complete. 2024-04-26T23:18:45.083 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T23:18:45.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.184+0000 7fee034d9640 1 Processor -- start 2024-04-26T23:18:45.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.185+0000 7fee034d9640 1 -- start start 2024-04-26T23:18:45.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.185+0000 7fee034d9640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fedfc15d100 0x7fedfc160ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:45.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.185+0000 7fee034d9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedfc06c950 0x7fedfc0aed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:45.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.185+0000 7fee034d9640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedfc0b0990 con 0x7fedfc070930 2024-04-26T23:18:45.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.185+0000 7fee034d9640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fedfc0af2d0 con 0x7fedfc15d100 2024-04-26T23:18:45.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.185+0000 7fee034d9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fedfc0b0c70 con 0x7fedfc06c950 2024-04-26T23:18:45.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.186+0000 7fee01275640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fedfc15d100 0x7fedfc160ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:45.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.186+0000 7fee01275640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fedfc15d100 0x7fedfc160ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54360/0 (socket says 172.21.15.47:54360) 2024-04-26T23:18:45.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.186+0000 7fee01275640 1 -- 172.21.15.47:0/4239779833 learned_addr learned my addr 172.21.15.47:0/4239779833 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:45.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.186+0000 7fee00a74640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fedfc070930 0x7fedfc06c240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37450/0 (socket says 172.21.15.47:37450) 2024-04-26T23:18:45.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.186+0000 7fee01a76640 1 --2- 172.21.15.47:0/4239779833 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedfc06c950 0x7fedfc0aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:45.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.186+0000 7fee01275640 1 -- 172.21.15.47:0/4239779833 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedfc06c950 msgr2=0x7fedfc0aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:45.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.186+0000 7fee01275640 1 --2- 172.21.15.47:0/4239779833 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedfc06c950 0x7fedfc0aed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:45.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.186+0000 7fee01275640 1 -- 172.21.15.47:0/4239779833 >> v1:172.21.15.47:6789/0 conn(0x7fedfc070930 legacy=0x7fedfc06c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:45.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.186+0000 7fee01275640 1 -- 172.21.15.47:0/4239779833 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedfc0b0f50 con 0x7fedfc15d100 2024-04-26T23:18:45.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.186+0000 7fedf3fff640 1 -- 172.21.15.47:0/4239779833 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1726441309 0 0) 0x7fedfc0b0990 con 0x7fedfc070930 2024-04-26T23:18:45.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.187+0000 7fee01275640 1 --2- 172.21.15.47:0/4239779833 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fedfc15d100 0x7fedfc160ba0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fedf8009870 tx=0x7fedf802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edcf22aebca31802 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:45.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.187+0000 7fedf3fff640 1 -- 172.21.15.47:0/4239779833 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf8037070 con 0x7fedfc15d100 2024-04-26T23:18:45.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.187+0000 7fedf3fff640 1 -- 172.21.15.47:0/4239779833 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedf8032440 con 0x7fedfc15d100 2024-04-26T23:18:45.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.187+0000 7fedf3fff640 1 -- 172.21.15.47:0/4239779833 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf8032810 con 0x7fedfc15d100 2024-04-26T23:18:45.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.187+0000 7fee034d9640 1 -- 172.21.15.47:0/4239779833 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fedfc15d100 msgr2=0x7fedfc160ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:45.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.187+0000 7fee034d9640 1 --2- 172.21.15.47:0/4239779833 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fedfc15d100 0x7fedfc160ba0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fedf8009870 tx=0x7fedf802a270 comp rx=0 tx=0).stop 2024-04-26T23:18:45.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.187+0000 7fee034d9640 1 -- 172.21.15.47:0/4239779833 shutdown_connections 2024-04-26T23:18:45.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.187+0000 7fee034d9640 1 --2- 172.21.15.47:0/4239779833 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedfc06c950 0x7fedfc0aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:45.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.187+0000 7fee034d9640 1 --2- 172.21.15.47:0/4239779833 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fedfc15d100 0x7fedfc160ba0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:45.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.187+0000 7fee034d9640 1 -- 172.21.15.47:0/4239779833 >> 172.21.15.47:0/4239779833 conn(0x7fedfc0b39b0 msgr2=0x7fedfc0b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:45.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.187+0000 7fee034d9640 1 -- 172.21.15.47:0/4239779833 shutdown_connections 2024-04-26T23:18:45.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.187+0000 7fee034d9640 1 -- 172.21.15.47:0/4239779833 wait complete. 2024-04-26T23:18:45.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.188+0000 7fee034d9640 1 Processor -- start 2024-04-26T23:18:45.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.189+0000 7fee034d9640 1 -- start start 2024-04-26T23:18:45.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.189+0000 7fee034d9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedfc06c950 0x7fedfc148950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:45.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.189+0000 7fee034d9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fedfc070930 0x7fedfc0b6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:45.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.189+0000 7fee034d9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fedfc15d100 0x7fedfc073740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:45.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.189+0000 7fee034d9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fedfc073160 con 0x7fedfc15d100 2024-04-26T23:18:45.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.189+0000 7fee034d9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fedfc0b1510 con 0x7fedfc070930 2024-04-26T23:18:45.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.189+0000 7fee034d9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fedfc0732e0 con 0x7fedfc06c950 2024-04-26T23:18:45.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.189+0000 7fee01a76640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fedfc15d100 0x7fedfc073740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:45.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.189+0000 7fee01a76640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fedfc15d100 0x7fedfc073740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54366/0 (socket says 172.21.15.47:54366) 2024-04-26T23:18:45.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.189+0000 7fee01a76640 1 -- 172.21.15.47:0/1909595321 learned_addr learned my addr 172.21.15.47:0/1909595321 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:45.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.190+0000 7fee01275640 1 --2- 172.21.15.47:0/1909595321 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedfc06c950 0x7fedfc148950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:45.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.190+0000 7fee00a74640 1 --2- 172.21.15.47:0/1909595321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fedfc070930 0x7fedfc0b6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:45.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.190+0000 7fee01a76640 1 -- 172.21.15.47:0/1909595321 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedfc06c950 msgr2=0x7fedfc148950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:45.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.190+0000 7fee01a76640 1 --2- 172.21.15.47:0/1909595321 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedfc06c950 0x7fedfc148950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:45.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.190+0000 7fee01a76640 1 -- 172.21.15.47:0/1909595321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fedfc070930 msgr2=0x7fedfc0b6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:45.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.190+0000 7fee01a76640 1 --2- 172.21.15.47:0/1909595321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fedfc070930 0x7fedfc0b6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:45.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.190+0000 7fee01a76640 1 -- 172.21.15.47:0/1909595321 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedfc0b6ba0 con 0x7fedfc15d100 2024-04-26T23:18:45.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.190+0000 7fee00a74640 1 --2- 172.21.15.47:0/1909595321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fedfc070930 0x7fedfc0b6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:18:45.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.190+0000 7fee01a76640 1 --2- 172.21.15.47:0/1909595321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fedfc15d100 0x7fedfc073740 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fedf4007e40 tx=0x7fedf400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:45.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.190+0000 7fedf1ffb640 1 -- 172.21.15.47:0/1909595321 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf4014620 con 0x7fedfc15d100 2024-04-26T23:18:45.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.190+0000 7fedf1ffb640 1 -- 172.21.15.47:0/1909595321 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedf4019070 con 0x7fedfc15d100 2024-04-26T23:18:45.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.190+0000 7fee034d9640 1 -- 172.21.15.47:0/1909595321 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedfc158ff0 con 0x7fedfc15d100 2024-04-26T23:18:45.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.190+0000 7fedf1ffb640 1 -- 172.21.15.47:0/1909595321 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf4014a20 con 0x7fedfc15d100 2024-04-26T23:18:45.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.190+0000 7fee034d9640 1 -- 172.21.15.47:0/1909595321 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedfc1595b0 con 0x7fedfc15d100 2024-04-26T23:18:45.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.191+0000 7fee034d9640 1 -- 172.21.15.47:0/1909595321 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedfc1597b0 con 0x7fedfc15d100 2024-04-26T23:18:45.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.191+0000 7fedf1ffb640 1 -- 172.21.15.47:0/1909595321 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fedf4014bc0 con 0x7fedfc15d100 2024-04-26T23:18:45.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.192+0000 7fedf1ffb640 1 --2- 172.21.15.47:0/1909595321 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fede006c8f0 0x7fede006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:45.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.192+0000 7fedf1ffb640 1 -- 172.21.15.47:0/1909595321 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fedf408c6d0 con 0x7fedfc15d100 2024-04-26T23:18:45.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.192+0000 7fee01275640 1 --2- 172.21.15.47:0/1909595321 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fede006c8f0 0x7fede006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:45.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.193+0000 7fee01275640 1 --2- 172.21.15.47:0/1909595321 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fede006c8f0 0x7fede006edb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fedf8034000 tx=0x7fedf802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:45.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.197+0000 7fedf1ffb640 1 -- 172.21.15.47:0/1909595321 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fedfc1597b0 con 0x7fedfc15d100 2024-04-26T23:18:45.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.395+0000 7fee034d9640 1 -- 172.21.15.47:0/1909595321 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fedfc0694f0 con 0x7fedfc15d100 2024-04-26T23:18:45.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.395+0000 7fedf1ffb640 1 -- 172.21.15.47:0/1909595321 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7fedf405a850 con 0x7fedfc15d100 2024-04-26T23:18:45.396 INFO:teuthology.orchestra.run.smithi047.stderr:listed 0 entries 2024-04-26T23:18:45.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.397+0000 7fee034d9640 1 -- 172.21.15.47:0/1909595321 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fede006c8f0 msgr2=0x7fede006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:45.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.397+0000 7fee034d9640 1 --2- 172.21.15.47:0/1909595321 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fede006c8f0 0x7fede006edb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fedf8034000 tx=0x7fedf802e000 comp rx=0 tx=0).stop 2024-04-26T23:18:45.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.397+0000 7fee034d9640 1 -- 172.21.15.47:0/1909595321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fedfc15d100 msgr2=0x7fedfc073740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:45.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.397+0000 7fee034d9640 1 --2- 172.21.15.47:0/1909595321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fedfc15d100 0x7fedfc073740 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fedf4007e40 tx=0x7fedf400e4b0 comp rx=0 tx=0).stop 2024-04-26T23:18:45.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.397+0000 7fee034d9640 1 -- 172.21.15.47:0/1909595321 shutdown_connections 2024-04-26T23:18:45.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.397+0000 7fee034d9640 1 --2- 172.21.15.47:0/1909595321 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fede006c8f0 0x7fede006edb0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:45.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.397+0000 7fee034d9640 1 --2- 172.21.15.47:0/1909595321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fedfc15d100 0x7fedfc073740 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:45.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.397+0000 7fee034d9640 1 --2- 172.21.15.47:0/1909595321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fedfc070930 0x7fedfc0b6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:45.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.397+0000 7fee034d9640 1 --2- 172.21.15.47:0/1909595321 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedfc06c950 0x7fedfc148950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:45.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.397+0000 7fee034d9640 1 -- 172.21.15.47:0/1909595321 >> 172.21.15.47:0/1909595321 conn(0x7fedfc0b39b0 msgr2=0x7fedfc06fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:45.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.397+0000 7fee034d9640 1 -- 172.21.15.47:0/1909595321 shutdown_connections 2024-04-26T23:18:45.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:45.397+0000 7fee034d9640 1 -- 172.21.15.47:0/1909595321 wait complete. 2024-04-26T23:18:45.408 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi047.front.sepia.ceph.com... 2024-04-26T23:18:45.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:18:45.408 DEBUG:teuthology.orchestra.run.smithi047:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-26T23:18:45.431 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi047.stderr:ceph-fuse[41290]: fuse finished with error 0 and tester_r 0 2024-04-26T23:18:45.433 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:18:51.534 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi047.front.sepia.ceph.com 2024-04-26T23:18:51.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:18:51.535 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-26T23:18:51.559 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:18:51.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.707+0000 7f94f0fb4640 1 Processor -- start 2024-04-26T23:18:51.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.707+0000 7f94f0fb4640 1 -- start start 2024-04-26T23:18:51.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.708+0000 7f94f0fb4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94ec0d3bc0 0x7f94ec0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:51.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.708+0000 7f94f0fb4640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94ec0b7e50 con 0x7f94ec162880 2024-04-26T23:18:51.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.708+0000 7f94f0fb4640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94ec0d1800 con 0x7f94ec0b5f20 2024-04-26T23:18:51.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.708+0000 7f94f0fb4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f94ec0d0580 con 0x7f94ec0d3bc0 2024-04-26T23:18:51.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.708+0000 7f94ea59c640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f94ec162880 0x7f94ec0d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:54466/0 (socket says 172.21.15.47:54466) 2024-04-26T23:18:51.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.708+0000 7f94ea59c640 1 -- 172.21.15.47:0/3014629387 learned_addr learned my addr 172.21.15.47:0/3014629387 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:18:51.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.708+0000 7f94e3fff640 1 --2- 172.21.15.47:0/3014629387 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94ec0d3bc0 0x7f94ec0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:51.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.708+0000 7f94e9d9b640 1 -- 172.21.15.47:0/3014629387 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3758326346 0 0) 0x7f94ec0b7e50 con 0x7f94ec162880 2024-04-26T23:18:51.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.708+0000 7f94e9d9b640 1 -- 172.21.15.47:0/3014629387 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f94d8003520 con 0x7f94ec162880 2024-04-26T23:18:51.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.708+0000 7f94e9d9b640 1 -- 172.21.15.47:0/3014629387 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1125963725 0 0) 0x7f94d8003520 con 0x7f94ec162880 2024-04-26T23:18:51.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.708+0000 7f94e9d9b640 1 -- 172.21.15.47:0/3014629387 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94ec0d3bc0 msgr2=0x7f94ec0d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:51.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.708+0000 7f94e9d9b640 1 --2- 172.21.15.47:0/3014629387 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94ec0d3bc0 0x7f94ec0d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:51.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.708+0000 7f94e9d9b640 1 -- 172.21.15.47:0/3014629387 >> v1:172.21.15.203:6789/0 conn(0x7f94ec0b5f20 legacy=0x7f94ec0b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:51.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.709+0000 7f94e9d9b640 1 -- 172.21.15.47:0/3014629387 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94ec0d07d0 con 0x7f94ec162880 2024-04-26T23:18:51.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.709+0000 7f94e9d9b640 1 -- 172.21.15.47:0/3014629387 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7f94dc001670 con 0x7f94ec162880 2024-04-26T23:18:51.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.709+0000 7f94e9d9b640 1 -- 172.21.15.47:0/3014629387 >> v1:172.21.15.47:6789/0 conn(0x7f94ec162880 legacy=0x7f94ec0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:51.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.709+0000 7f94e9d9b640 1 --2- 172.21.15.47:0/3014629387 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94d80039e0 0x7f94d8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:51.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.709+0000 7f94e9d9b640 1 --2- 172.21.15.47:0/3014629387 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94d8004800 0x7f94d8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:51.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.709+0000 7f94e9d9b640 1 --2- 172.21.15.47:0/3014629387 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94ec0d3bc0 0x7f94d80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:51.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.709+0000 7f94e9d9b640 1 -- 172.21.15.47:0/3014629387 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f94ec0d0580 con 0x7f94d80039e0 2024-04-26T23:18:51.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.709+0000 7f94e9d9b640 1 -- 172.21.15.47:0/3014629387 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f94d800ba70 con 0x7f94d8004800 2024-04-26T23:18:51.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.709+0000 7f94e9d9b640 1 -- 172.21.15.47:0/3014629387 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f94d800bc70 con 0x7f94ec0d3bc0 2024-04-26T23:18:51.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.709+0000 7f94ea59c640 1 --2- 172.21.15.47:0/3014629387 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94d80039e0 0x7f94d8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:51.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.709+0000 7f94e3fff640 1 --2- 172.21.15.47:0/3014629387 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94d8004800 0x7f94d8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:51.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.709+0000 7f94ead9d640 1 --2- 172.21.15.47:0/3014629387 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94ec0d3bc0 0x7f94d80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:51.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.709+0000 7f94ea59c640 1 -- 172.21.15.47:0/3014629387 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94ec0d3bc0 msgr2=0x7f94d80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:51.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.709+0000 7f94ea59c640 1 --2- 172.21.15.47:0/3014629387 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94ec0d3bc0 0x7f94d80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:51.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.709+0000 7f94ea59c640 1 -- 172.21.15.47:0/3014629387 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94d8004800 msgr2=0x7f94d8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:51.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.709+0000 7f94ea59c640 1 --2- 172.21.15.47:0/3014629387 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94d8004800 0x7f94d8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:51.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.709+0000 7f94ea59c640 1 -- 172.21.15.47:0/3014629387 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94d8003520 con 0x7f94d80039e0 2024-04-26T23:18:51.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.709+0000 7f94ead9d640 1 --2- 172.21.15.47:0/3014629387 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94ec0d3bc0 0x7f94d80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:18:51.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.710+0000 7f94ea59c640 1 --2- 172.21.15.47:0/3014629387 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94d80039e0 0x7f94d8003e00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f94dc00c210 tx=0x7f94d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b269cd43b6a80d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:51.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.710+0000 7f94e9d9b640 1 -- 172.21.15.47:0/3014629387 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94dc007d10 con 0x7f94d80039e0 2024-04-26T23:18:51.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.710+0000 7f94e9d9b640 1 -- 172.21.15.47:0/3014629387 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94dc005ad0 con 0x7f94d80039e0 2024-04-26T23:18:51.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.710+0000 7f94e9d9b640 1 -- 172.21.15.47:0/3014629387 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94dc007070 con 0x7f94d80039e0 2024-04-26T23:18:51.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.710+0000 7f94f0fb4640 1 -- 172.21.15.47:0/3014629387 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94d80039e0 msgr2=0x7f94d8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:51.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.710+0000 7f94f0fb4640 1 --2- 172.21.15.47:0/3014629387 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94d80039e0 0x7f94d8003e00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f94dc00c210 tx=0x7f94d8005830 comp rx=0 tx=0).stop 2024-04-26T23:18:51.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.710+0000 7f94ead9d640 1 -- 172.21.15.47:0/3014629387 reap_dead start 2024-04-26T23:18:51.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.710+0000 7f94f0fb4640 1 -- 172.21.15.47:0/3014629387 shutdown_connections 2024-04-26T23:18:51.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.710+0000 7f94f0fb4640 1 -- 172.21.15.47:0/3014629387 >> 172.21.15.47:0/3014629387 conn(0x7f94ec0d4f60 msgr2=0x7f94ec0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:51.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.710+0000 7f94f0fb4640 1 -- 172.21.15.47:0/3014629387 shutdown_connections 2024-04-26T23:18:51.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.711+0000 7f94f0fb4640 1 -- 172.21.15.47:0/3014629387 wait complete. 2024-04-26T23:18:51.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.712+0000 7f94f0fb4640 1 Processor -- start 2024-04-26T23:18:51.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.712+0000 7f94f0fb4640 1 -- start start 2024-04-26T23:18:51.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.713+0000 7f94f0fb4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94d80039e0 0x7f94ec15e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:51.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.713+0000 7f94f0fb4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94ec1f8d70 0x7f94ec0a0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:51.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.713+0000 7f94f0fb4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94ec0a0780 0x7f94ec0b9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:51.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.713+0000 7f94f0fb4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f94ec162c60 con 0x7f94d80039e0 2024-04-26T23:18:51.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.713+0000 7f94f0fb4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f94ec0d0d90 con 0x7f94ec1f8d70 2024-04-26T23:18:51.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.713+0000 7f94f0fb4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f94ec162de0 con 0x7f94ec0a0780 2024-04-26T23:18:51.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.713+0000 7f94ea59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94d80039e0 0x7f94ec15e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:51.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.713+0000 7f94ea59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94d80039e0 0x7f94ec15e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58434/0 (socket says 172.21.15.47:58434) 2024-04-26T23:18:51.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.713+0000 7f94ea59c640 1 -- 172.21.15.47:0/1469471534 learned_addr learned my addr 172.21.15.47:0/1469471534 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:51.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.713+0000 7f94e3fff640 1 --2- 172.21.15.47:0/1469471534 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94ec1f8d70 0x7f94ec0a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:51.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.713+0000 7f94ead9d640 1 --2- 172.21.15.47:0/1469471534 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94ec0a0780 0x7f94ec0b9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:51.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.713+0000 7f94ea59c640 1 -- 172.21.15.47:0/1469471534 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94ec0a0780 msgr2=0x7f94ec0b9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:51.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.713+0000 7f94ea59c640 1 --2- 172.21.15.47:0/1469471534 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94ec0a0780 0x7f94ec0b9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:51.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.713+0000 7f94ea59c640 1 -- 172.21.15.47:0/1469471534 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94ec1f8d70 msgr2=0x7f94ec0a0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:51.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.713+0000 7f94ea59c640 1 --2- 172.21.15.47:0/1469471534 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94ec1f8d70 0x7f94ec0a0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:51.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.713+0000 7f94ea59c640 1 -- 172.21.15.47:0/1469471534 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94ec0b8250 con 0x7f94d80039e0 2024-04-26T23:18:51.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.713+0000 7f94ea59c640 1 --2- 172.21.15.47:0/1469471534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94d80039e0 0x7f94ec15e4e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f94dc00c330 tx=0x7f94ec0b6830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:51.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.713+0000 7f94e2ffd640 1 -- 172.21.15.47:0/1469471534 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94dc041020 con 0x7f94d80039e0 2024-04-26T23:18:51.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.713+0000 7f94e2ffd640 1 -- 172.21.15.47:0/1469471534 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94dc001430 con 0x7f94d80039e0 2024-04-26T23:18:51.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.713+0000 7f94e2ffd640 1 -- 172.21.15.47:0/1469471534 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94dc007d10 con 0x7f94d80039e0 2024-04-26T23:18:51.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.713+0000 7f94f0fb4640 1 -- 172.21.15.47:0/1469471534 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94ec0b8530 con 0x7f94d80039e0 2024-04-26T23:18:51.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.714+0000 7f94f0fb4640 1 -- 172.21.15.47:0/1469471534 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94ec0b8af0 con 0x7f94d80039e0 2024-04-26T23:18:51.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.715+0000 7f94f0fb4640 1 -- 172.21.15.47:0/1469471534 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94ec0694f0 con 0x7f94d80039e0 2024-04-26T23:18:51.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.715+0000 7f94e2ffd640 1 -- 172.21.15.47:0/1469471534 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f94dc044020 con 0x7f94d80039e0 2024-04-26T23:18:51.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.716+0000 7f94e2ffd640 1 --2- 172.21.15.47:0/1469471534 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f94cc06c8f0 0x7f94cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:51.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.716+0000 7f94e2ffd640 1 -- 172.21.15.47:0/1469471534 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f94dc0ac170 con 0x7f94d80039e0 2024-04-26T23:18:51.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.716+0000 7f94e3fff640 1 --2- 172.21.15.47:0/1469471534 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f94cc06c8f0 0x7f94cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:51.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.716+0000 7f94e3fff640 1 --2- 172.21.15.47:0/1469471534 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f94cc06c8f0 0x7f94cc06edb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f94ec0b2870 tx=0x7f94d4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:51.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.720+0000 7f94e2ffd640 1 -- 172.21.15.47:0/1469471534 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f94dc005aa0 con 0x7f94d80039e0 2024-04-26T23:18:51.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.906+0000 7f94f0fb4640 1 -- 172.21.15.47:0/1469471534 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f94ec0d2e30 con 0x7f94d80039e0 2024-04-26T23:18:51.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.907+0000 7f94e2ffd640 1 -- 172.21.15.47:0/1469471534 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18478 (secure 0 0 0) 0x7f94dc07a2f0 con 0x7f94d80039e0 2024-04-26T23:18:51.908 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:18:51.909 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":24,"fsid":"399006d3-4db1-43e6-ba81-a79a276eabfd","created":"2024-04-26T23:18:06.731211+0000","modified":"2024-04-26T23:18:36.402132+0000","last_up_change":"2024-04-26T23:18:17.808736+0000","last_in_change":"2024-04-26T23:18:08.945859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:18:16.937492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:18:29.033575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:18:30.318499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:18:34.014536+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ee091567-fea0-4a14-bf4e-0e16c20bb2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6819","nonce":326520779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6821","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6823","nonce":326520779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6829","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6831","nonce":326520779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6825","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6827","nonce":326520779}]},"public_addr":"172.21.15.47:6819/326520779","cluster_addr":"172.21.15.47:6823/326520779","heartbeat_back_addr":"172.21.15.47:6831/326520779","heartbeat_front_addr":"172.21.15.47:6827/326520779","state":["exists","up"]},{"osd":1,"uuid":"c14440a0-bba5-4931-a6db-b736c08e869b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6809","nonce":1912610931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6811","nonce":1912610931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6815","nonce":1912610931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6813","nonce":1912610931}]},"public_addr":"172.21.15.47:6809/1912610931","cluster_addr":"172.21.15.47:6811/1912610931","heartbeat_back_addr":"172.21.15.47:6815/1912610931","heartbeat_front_addr":"172.21.15.47:6813/1912610931","state":["exists","up"]},{"osd":2,"uuid":"3940f8de-c9b2-4cd3-a911-9c301729eb30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6801","nonce":2569869305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6803","nonce":2569869305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6807","nonce":2569869305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6805","nonce":2569869305}]},"public_addr":"172.21.15.47:6801/2569869305","cluster_addr":"172.21.15.47:6803/2569869305","heartbeat_back_addr":"172.21.15.47:6807/2569869305","heartbeat_front_addr":"172.21.15.47:6805/2569869305","state":["exists","up"]},{"osd":3,"uuid":"1286a67e-8958-426a-8c2e-bc85498c619c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6818","nonce":2791411683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6822","nonce":2791411683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6830","nonce":2791411683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6826","nonce":2791411683}]},"public_addr":"172.21.15.47:6818/2791411683","cluster_addr":"172.21.15.47:6822/2791411683","heartbeat_back_addr":"172.21.15.47:6830/2791411683","heartbeat_front_addr":"172.21.15.47:6826/2791411683","state":["exists","up"]},{"osd":4,"uuid":"d8ee2c20-e5b7-46c2-9999-b234070126cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6809","nonce":441313104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6811","nonce":441313104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6815","nonce":441313104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6813","nonce":441313104}]},"public_addr":"172.21.15.203:6809/441313104","cluster_addr":"172.21.15.203:6811/441313104","heartbeat_back_addr":"172.21.15.203:6815/441313104","heartbeat_front_addr":"172.21.15.203:6813/441313104","state":["exists","up"]},{"osd":5,"uuid":"b453a8f0-f324-4840-9ae0-d67bdadb9f8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6801","nonce":56838444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6803","nonce":56838444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6807","nonce":56838444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6805","nonce":56838444}]},"public_addr":"172.21.15.203:6801/56838444","cluster_addr":"172.21.15.203:6803/56838444","heartbeat_back_addr":"172.21.15.203:6807/56838444","heartbeat_front_addr":"172.21.15.203:6805/56838444","state":["exists","up"]},{"osd":6,"uuid":"a6d31951-2d4b-4e8d-a510-6283b01dc158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6825","nonce":2738517080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6827","nonce":2738517080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6831","nonce":2738517080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6829","nonce":2738517080}]},"public_addr":"172.21.15.203:6825/2738517080","cluster_addr":"172.21.15.203:6827/2738517080","heartbeat_back_addr":"172.21.15.203:6831/2738517080","heartbeat_front_addr":"172.21.15.203:6829/2738517080","state":["exists","up"]},{"osd":7,"uuid":"7119b8c0-fe96-402d-9e75-7b509d9d44d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6817","nonce":1315565058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6819","nonce":1315565058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6823","nonce":1315565058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6821","nonce":1315565058}]},"public_addr":"172.21.15.203:6817/1315565058","cluster_addr":"172.21.15.203:6819/1315565058","heartbeat_back_addr":"172.21.15.203:6823/1315565058","heartbeat_front_addr":"172.21.15.203:6821/1315565058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.121921+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.137494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.084814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.111593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.072430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.125066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.097867+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.057061+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:18:51.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.909+0000 7f94f0fb4640 1 -- 172.21.15.47:0/1469471534 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f94cc06c8f0 msgr2=0x7f94cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:51.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.909+0000 7f94f0fb4640 1 --2- 172.21.15.47:0/1469471534 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f94cc06c8f0 0x7f94cc06edb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f94ec0b2870 tx=0x7f94d4008040 comp rx=0 tx=0).stop 2024-04-26T23:18:51.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.909+0000 7f94f0fb4640 1 -- 172.21.15.47:0/1469471534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94d80039e0 msgr2=0x7f94ec15e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:51.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.909+0000 7f94f0fb4640 1 --2- 172.21.15.47:0/1469471534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94d80039e0 0x7f94ec15e4e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f94dc00c330 tx=0x7f94ec0b6830 comp rx=0 tx=0).stop 2024-04-26T23:18:51.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.909+0000 7f94f0fb4640 1 -- 172.21.15.47:0/1469471534 shutdown_connections 2024-04-26T23:18:51.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.909+0000 7f94f0fb4640 1 --2- 172.21.15.47:0/1469471534 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f94cc06c8f0 0x7f94cc06edb0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:51.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.909+0000 7f94f0fb4640 1 --2- 172.21.15.47:0/1469471534 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94ec0a0780 0x7f94ec0b9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:51.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.909+0000 7f94f0fb4640 1 --2- 172.21.15.47:0/1469471534 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94ec1f8d70 0x7f94ec0a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:51.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.909+0000 7f94f0fb4640 1 --2- 172.21.15.47:0/1469471534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94d80039e0 0x7f94ec15e4e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:51.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.910+0000 7f94f0fb4640 1 -- 172.21.15.47:0/1469471534 >> 172.21.15.47:0/1469471534 conn(0x7f94ec0d4f60 msgr2=0x7f94ec06cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:51.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.910+0000 7f94f0fb4640 1 -- 172.21.15.47:0/1469471534 shutdown_connections 2024-04-26T23:18:51.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:51.910+0000 7f94f0fb4640 1 -- 172.21.15.47:0/1469471534 wait complete. 2024-04-26T23:18:51.922 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:18:52.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.021+0000 7fc7bc339640 1 Processor -- start 2024-04-26T23:18:52.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.022+0000 7fc7bc339640 1 -- start start 2024-04-26T23:18:52.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.022+0000 7fc7bc339640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc7b4162810 0x7fc7b4070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:52.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.022+0000 7fc7bc339640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7b40d10d0 con 0x7fc7b40b2340 2024-04-26T23:18:52.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.022+0000 7fc7bc339640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7b40d1370 con 0x7fc7b40d1b30 2024-04-26T23:18:52.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.022+0000 7fc7bc339640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fc7b40d0750 con 0x7fc7b4162810 2024-04-26T23:18:52.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.023+0000 7fc7ba0d5640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc7b4162810 0x7fc7b4070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:52.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.023+0000 7fc7ba0d5640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc7b4162810 0x7fc7b4070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58444/0 (socket says 172.21.15.47:58444) 2024-04-26T23:18:52.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.023+0000 7fc7b98d4640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc7b40b2340 0x7fc7b40cd3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:54478/0 (socket says 172.21.15.47:54478) 2024-04-26T23:18:52.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.023+0000 7fc7ba0d5640 1 -- 172.21.15.47:0/93502408 learned_addr learned my addr 172.21.15.47:0/93502408 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:52.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.023+0000 7fc7ba0d5640 1 -- 172.21.15.47:0/93502408 >> v1:172.21.15.203:6789/0 conn(0x7fc7b40d1b30 legacy=0x7fc7b4070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:52.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.023+0000 7fc7ba0d5640 1 -- 172.21.15.47:0/93502408 >> v1:172.21.15.47:6789/0 conn(0x7fc7b40b2340 legacy=0x7fc7b40cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:52.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.023+0000 7fc7ba0d5640 1 -- 172.21.15.47:0/93502408 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7b40d01b0 con 0x7fc7b4162810 2024-04-26T23:18:52.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.024+0000 7fc7ba0d5640 1 --2- 172.21.15.47:0/93502408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7b4162810 0x7fc7b4070350 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0009870 tx=0x7fc7b002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deaab4b8b7d65d5e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:52.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.024+0000 7fc7b90d3640 1 -- 172.21.15.47:0/93502408 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b0037070 con 0x7fc7b4162810 2024-04-26T23:18:52.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.024+0000 7fc7b90d3640 1 -- 172.21.15.47:0/93502408 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7b0032440 con 0x7fc7b4162810 2024-04-26T23:18:52.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.024+0000 7fc7b90d3640 1 -- 172.21.15.47:0/93502408 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b0032810 con 0x7fc7b4162810 2024-04-26T23:18:52.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.024+0000 7fc7bc339640 1 -- 172.21.15.47:0/93502408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7b4162810 msgr2=0x7fc7b4070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:52.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.024+0000 7fc7bc339640 1 --2- 172.21.15.47:0/93502408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7b4162810 0x7fc7b4070350 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0009870 tx=0x7fc7b002a2a0 comp rx=0 tx=0).stop 2024-04-26T23:18:52.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.024+0000 7fc7bc339640 1 -- 172.21.15.47:0/93502408 shutdown_connections 2024-04-26T23:18:52.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.024+0000 7fc7bc339640 1 --2- 172.21.15.47:0/93502408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7b4162810 0x7fc7b4070350 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:52.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.024+0000 7fc7bc339640 1 -- 172.21.15.47:0/93502408 >> 172.21.15.47:0/93502408 conn(0x7fc7b40d2da0 msgr2=0x7fc7b40d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:52.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.025+0000 7fc7bc339640 1 -- 172.21.15.47:0/93502408 shutdown_connections 2024-04-26T23:18:52.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.025+0000 7fc7bc339640 1 -- 172.21.15.47:0/93502408 wait complete. 2024-04-26T23:18:52.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.026+0000 7fc7bc339640 1 Processor -- start 2024-04-26T23:18:52.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.026+0000 7fc7bc339640 1 -- start start 2024-04-26T23:18:52.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.026+0000 7fc7bc339640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7b40b2340 0x7fc7b415d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:52.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.026+0000 7fc7bc339640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc7b40d1b30 0x7fc7b415de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:52.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.026+0000 7fc7bc339640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7b4162810 0x7fc7b40c69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:52.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.026+0000 7fc7bc339640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b4070890 con 0x7fc7b4162810 2024-04-26T23:18:52.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.026+0000 7fc7bc339640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b40d16a0 con 0x7fc7b40b2340 2024-04-26T23:18:52.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.026+0000 7fc7bc339640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc7b40cdb00 con 0x7fc7b40d1b30 2024-04-26T23:18:52.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.027+0000 7fc7ba8d6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7b4162810 0x7fc7b40c69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:52.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.027+0000 7fc7ba8d6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7b4162810 0x7fc7b40c69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58458/0 (socket says 172.21.15.47:58458) 2024-04-26T23:18:52.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.027+0000 7fc7ba8d6640 1 -- 172.21.15.47:0/2791690663 learned_addr learned my addr 172.21.15.47:0/2791690663 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:52.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.027+0000 7fc7ba0d5640 1 --2- 172.21.15.47:0/2791690663 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7b40b2340 0x7fc7b415d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:52.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.027+0000 7fc7b98d4640 1 --2- 172.21.15.47:0/2791690663 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc7b40d1b30 0x7fc7b415de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:52.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.027+0000 7fc7ba8d6640 1 -- 172.21.15.47:0/2791690663 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc7b40d1b30 msgr2=0x7fc7b415de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:52.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.027+0000 7fc7ba8d6640 1 --2- 172.21.15.47:0/2791690663 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc7b40d1b30 0x7fc7b415de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:52.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.027+0000 7fc7ba8d6640 1 -- 172.21.15.47:0/2791690663 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7b40b2340 msgr2=0x7fc7b415d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:52.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.027+0000 7fc7ba8d6640 1 --2- 172.21.15.47:0/2791690663 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7b40b2340 0x7fc7b415d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:52.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.027+0000 7fc7ba8d6640 1 -- 172.21.15.47:0/2791690663 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7b40c7750 con 0x7fc7b4162810 2024-04-26T23:18:52.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.027+0000 7fc7ba8d6640 1 --2- 172.21.15.47:0/2791690663 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7b4162810 0x7fc7b40c69d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac00c7b0 tx=0x7fc7ac00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:52.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.027+0000 7fc7aaffd640 1 -- 172.21.15.47:0/2791690663 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac004b60 con 0x7fc7b4162810 2024-04-26T23:18:52.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.027+0000 7fc7aaffd640 1 -- 172.21.15.47:0/2791690663 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7ac004d00 con 0x7fc7b4162810 2024-04-26T23:18:52.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.027+0000 7fc7bc339640 1 -- 172.21.15.47:0/2791690663 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7b40ca0c0 con 0x7fc7b4162810 2024-04-26T23:18:52.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.027+0000 7fc7aaffd640 1 -- 172.21.15.47:0/2791690663 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac00aa90 con 0x7fc7b4162810 2024-04-26T23:18:52.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.027+0000 7fc7bc339640 1 -- 172.21.15.47:0/2791690663 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7b4158850 con 0x7fc7b4162810 2024-04-26T23:18:52.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.028+0000 7fc7bc339640 1 -- 172.21.15.47:0/2791690663 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7b40694f0 con 0x7fc7b4162810 2024-04-26T23:18:52.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.029+0000 7fc7aaffd640 1 -- 172.21.15.47:0/2791690663 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc7ac00ac30 con 0x7fc7b4162810 2024-04-26T23:18:52.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.029+0000 7fc7aaffd640 1 --2- 172.21.15.47:0/2791690663 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fc79806c8f0 0x7fc79806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:52.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.030+0000 7fc7aaffd640 1 -- 172.21.15.47:0/2791690663 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fc7ac08c080 con 0x7fc7b4162810 2024-04-26T23:18:52.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.030+0000 7fc7ba0d5640 1 --2- 172.21.15.47:0/2791690663 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fc79806c8f0 0x7fc79806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:52.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.030+0000 7fc7ba0d5640 1 --2- 172.21.15.47:0/2791690663 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fc79806c8f0 0x7fc79806edb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0034000 tx=0x7fc7b002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:52.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.034+0000 7fc7aaffd640 1 -- 172.21.15.47:0/2791690663 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc7ac020070 con 0x7fc7b4162810 2024-04-26T23:18:52.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.229+0000 7fc7bc339640 1 -- 172.21.15.47:0/2791690663 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc7b4069b20 con 0x7fc7b4162810 2024-04-26T23:18:52.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.230+0000 7fc7aaffd640 1 -- 172.21.15.47:0/2791690663 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18478 (secure 0 0 0) 0x7fc7ac05a200 con 0x7fc7b4162810 2024-04-26T23:18:52.231 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:18:52.231 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":24,"fsid":"399006d3-4db1-43e6-ba81-a79a276eabfd","created":"2024-04-26T23:18:06.731211+0000","modified":"2024-04-26T23:18:36.402132+0000","last_up_change":"2024-04-26T23:18:17.808736+0000","last_in_change":"2024-04-26T23:18:08.945859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:18:16.937492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:18:29.033575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:18:30.318499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:18:34.014536+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ee091567-fea0-4a14-bf4e-0e16c20bb2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6819","nonce":326520779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6821","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6823","nonce":326520779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6829","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6831","nonce":326520779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6825","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6827","nonce":326520779}]},"public_addr":"172.21.15.47:6819/326520779","cluster_addr":"172.21.15.47:6823/326520779","heartbeat_back_addr":"172.21.15.47:6831/326520779","heartbeat_front_addr":"172.21.15.47:6827/326520779","state":["exists","up"]},{"osd":1,"uuid":"c14440a0-bba5-4931-a6db-b736c08e869b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6809","nonce":1912610931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6811","nonce":1912610931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6815","nonce":1912610931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6813","nonce":1912610931}]},"public_addr":"172.21.15.47:6809/1912610931","cluster_addr":"172.21.15.47:6811/1912610931","heartbeat_back_addr":"172.21.15.47:6815/1912610931","heartbeat_front_addr":"172.21.15.47:6813/1912610931","state":["exists","up"]},{"osd":2,"uuid":"3940f8de-c9b2-4cd3-a911-9c301729eb30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6801","nonce":2569869305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6803","nonce":2569869305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6807","nonce":2569869305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6805","nonce":2569869305}]},"public_addr":"172.21.15.47:6801/2569869305","cluster_addr":"172.21.15.47:6803/2569869305","heartbeat_back_addr":"172.21.15.47:6807/2569869305","heartbeat_front_addr":"172.21.15.47:6805/2569869305","state":["exists","up"]},{"osd":3,"uuid":"1286a67e-8958-426a-8c2e-bc85498c619c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6818","nonce":2791411683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6822","nonce":2791411683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6830","nonce":2791411683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6826","nonce":2791411683}]},"public_addr":"172.21.15.47:6818/2791411683","cluster_addr":"172.21.15.47:6822/2791411683","heartbeat_back_addr":"172.21.15.47:6830/2791411683","heartbeat_front_addr":"172.21.15.47:6826/2791411683","state":["exists","up"]},{"osd":4,"uuid":"d8ee2c20-e5b7-46c2-9999-b234070126cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6809","nonce":441313104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6811","nonce":441313104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6815","nonce":441313104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6813","nonce":441313104}]},"public_addr":"172.21.15.203:6809/441313104","cluster_addr":"172.21.15.203:6811/441313104","heartbeat_back_addr":"172.21.15.203:6815/441313104","heartbeat_front_addr":"172.21.15.203:6813/441313104","state":["exists","up"]},{"osd":5,"uuid":"b453a8f0-f324-4840-9ae0-d67bdadb9f8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6801","nonce":56838444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6803","nonce":56838444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6807","nonce":56838444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6805","nonce":56838444}]},"public_addr":"172.21.15.203:6801/56838444","cluster_addr":"172.21.15.203:6803/56838444","heartbeat_back_addr":"172.21.15.203:6807/56838444","heartbeat_front_addr":"172.21.15.203:6805/56838444","state":["exists","up"]},{"osd":6,"uuid":"a6d31951-2d4b-4e8d-a510-6283b01dc158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6825","nonce":2738517080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6827","nonce":2738517080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6831","nonce":2738517080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6829","nonce":2738517080}]},"public_addr":"172.21.15.203:6825/2738517080","cluster_addr":"172.21.15.203:6827/2738517080","heartbeat_back_addr":"172.21.15.203:6831/2738517080","heartbeat_front_addr":"172.21.15.203:6829/2738517080","state":["exists","up"]},{"osd":7,"uuid":"7119b8c0-fe96-402d-9e75-7b509d9d44d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6817","nonce":1315565058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6819","nonce":1315565058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6823","nonce":1315565058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6821","nonce":1315565058}]},"public_addr":"172.21.15.203:6817/1315565058","cluster_addr":"172.21.15.203:6819/1315565058","heartbeat_back_addr":"172.21.15.203:6823/1315565058","heartbeat_front_addr":"172.21.15.203:6821/1315565058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.121921+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.137494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.084814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.111593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.072430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.125066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.097867+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.057061+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:18:52.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.232+0000 7fc7bc339640 1 -- 172.21.15.47:0/2791690663 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fc79806c8f0 msgr2=0x7fc79806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:52.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.232+0000 7fc7bc339640 1 --2- 172.21.15.47:0/2791690663 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fc79806c8f0 0x7fc79806edb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0034000 tx=0x7fc7b002e000 comp rx=0 tx=0).stop 2024-04-26T23:18:52.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.232+0000 7fc7bc339640 1 -- 172.21.15.47:0/2791690663 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7b4162810 msgr2=0x7fc7b40c69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:52.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.232+0000 7fc7bc339640 1 --2- 172.21.15.47:0/2791690663 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7b4162810 0x7fc7b40c69d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac00c7b0 tx=0x7fc7ac00cc70 comp rx=0 tx=0).stop 2024-04-26T23:18:52.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.232+0000 7fc7bc339640 1 -- 172.21.15.47:0/2791690663 shutdown_connections 2024-04-26T23:18:52.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.232+0000 7fc7bc339640 1 --2- 172.21.15.47:0/2791690663 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fc79806c8f0 0x7fc79806edb0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:52.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.232+0000 7fc7bc339640 1 --2- 172.21.15.47:0/2791690663 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7b4162810 0x7fc7b40c69d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:52.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.232+0000 7fc7bc339640 1 --2- 172.21.15.47:0/2791690663 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc7b40d1b30 0x7fc7b415de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:52.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.232+0000 7fc7bc339640 1 --2- 172.21.15.47:0/2791690663 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7b40b2340 0x7fc7b415d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:52.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.232+0000 7fc7bc339640 1 -- 172.21.15.47:0/2791690663 >> 172.21.15.47:0/2791690663 conn(0x7fc7b40d2da0 msgr2=0x7fc7b40b3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:52.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.232+0000 7fc7bc339640 1 -- 172.21.15.47:0/2791690663 shutdown_connections 2024-04-26T23:18:52.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.232+0000 7fc7bc339640 1 -- 172.21.15.47:0/2791690663 wait complete. 2024-04-26T23:18:52.244 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T23:18:52.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.347+0000 7feead208640 1 Processor -- start 2024-04-26T23:18:52.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.347+0000 7feead208640 1 -- start start 2024-04-26T23:18:52.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.348+0000 7feead208640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7feea80b40d0 0x7feea81625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:52.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.348+0000 7feead208640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feea80d0a10 con 0x7feea8162b30 2024-04-26T23:18:52.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.348+0000 7feead208640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feea80d0cf0 con 0x7feea80d1a00 2024-04-26T23:18:52.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.348+0000 7feead208640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7feea80d0fd0 con 0x7feea80b40d0 2024-04-26T23:18:52.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.348+0000 7feea759e640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7feea8162b30 0x7feea80b4b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:54488/0 (socket says 172.21.15.47:54488) 2024-04-26T23:18:52.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.348+0000 7feea759e640 1 -- 172.21.15.47:0/3820980865 learned_addr learned my addr 172.21.15.47:0/3820980865 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:18:52.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.348+0000 7feea659c640 1 --2- 172.21.15.47:0/3820980865 >> v2:172.21.15.203:3300/0 conn(0x7feea80b40d0 0x7feea81625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:52.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.348+0000 7feea5d9b640 1 -- 172.21.15.47:0/3820980865 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3701853301 0 0) 0x7feea80d0a10 con 0x7feea8162b30 2024-04-26T23:18:52.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.348+0000 7feea5d9b640 1 -- 172.21.15.47:0/3820980865 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee94003520 con 0x7feea8162b30 2024-04-26T23:18:52.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.349+0000 7feea5d9b640 1 -- 172.21.15.47:0/3820980865 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4244741979 0 0) 0x7fee94003520 con 0x7feea8162b30 2024-04-26T23:18:52.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.349+0000 7feea5d9b640 1 -- 172.21.15.47:0/3820980865 >> v2:172.21.15.203:3300/0 conn(0x7feea80b40d0 msgr2=0x7feea81625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:52.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.349+0000 7feea5d9b640 1 --2- 172.21.15.47:0/3820980865 >> v2:172.21.15.203:3300/0 conn(0x7feea80b40d0 0x7feea81625f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:52.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.349+0000 7feea5d9b640 1 -- 172.21.15.47:0/3820980865 >> v1:172.21.15.203:6789/0 conn(0x7feea80d1a00 legacy=0x7feea80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:52.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.349+0000 7feea5d9b640 1 -- 172.21.15.47:0/3820980865 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feea8162f80 con 0x7feea8162b30 2024-04-26T23:18:52.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.349+0000 7feea5d9b640 1 -- 172.21.15.47:0/3820980865 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7feea0001670 con 0x7feea8162b30 2024-04-26T23:18:52.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.349+0000 7feea5d9b640 1 -- 172.21.15.47:0/3820980865 >> v1:172.21.15.47:6789/0 conn(0x7feea8162b30 legacy=0x7feea80b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:52.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.349+0000 7feea5d9b640 1 --2- 172.21.15.47:0/3820980865 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee940039b0 0x7fee94003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:52.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.349+0000 7feea5d9b640 1 --2- 172.21.15.47:0/3820980865 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee94004800 0x7fee94004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:52.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.349+0000 7feea5d9b640 1 --2- 172.21.15.47:0/3820980865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee940051c0 0x7fee9400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:52.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.349+0000 7feea5d9b640 1 -- 172.21.15.47:0/3820980865 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7feea80d0fd0 con 0x7fee940051c0 2024-04-26T23:18:52.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.349+0000 7feea5d9b640 1 -- 172.21.15.47:0/3820980865 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fee9400c160 con 0x7fee94004800 2024-04-26T23:18:52.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.349+0000 7feea5d9b640 1 -- 172.21.15.47:0/3820980865 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fee9400c360 con 0x7fee940039b0 2024-04-26T23:18:52.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.349+0000 7feea759e640 1 --2- 172.21.15.47:0/3820980865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee940051c0 0x7fee9400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:52.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.349+0000 7feea659c640 1 --2- 172.21.15.47:0/3820980865 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee94004800 0x7fee94004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:52.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.349+0000 7feea6d9d640 1 --2- 172.21.15.47:0/3820980865 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee940039b0 0x7fee94003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:52.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.349+0000 7feea759e640 1 -- 172.21.15.47:0/3820980865 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee940039b0 msgr2=0x7fee94003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:52.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.349+0000 7feea759e640 1 --2- 172.21.15.47:0/3820980865 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee940039b0 0x7fee94003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:52.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.349+0000 7feea759e640 1 -- 172.21.15.47:0/3820980865 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee94004800 msgr2=0x7fee94004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:52.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.349+0000 7feea759e640 1 --2- 172.21.15.47:0/3820980865 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee94004800 0x7fee94004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:52.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.349+0000 7feea759e640 1 -- 172.21.15.47:0/3820980865 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee94003520 con 0x7fee940051c0 2024-04-26T23:18:52.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.350+0000 7feea759e640 1 -- 172.21.15.47:0/3820980865 reap_dead start 2024-04-26T23:18:52.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.350+0000 7feea759e640 1 --2- 172.21.15.47:0/3820980865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee940051c0 0x7fee9400b9c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7feea000c210 tx=0x7fee94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4fd3e49e816a63b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:52.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.350+0000 7feea5d9b640 1 -- 172.21.15.47:0/3820980865 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea0007d10 con 0x7fee940051c0 2024-04-26T23:18:52.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.350+0000 7feea5d9b640 1 -- 172.21.15.47:0/3820980865 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feea00058d0 con 0x7fee940051c0 2024-04-26T23:18:52.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.350+0000 7feea5d9b640 1 -- 172.21.15.47:0/3820980865 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea0005ce0 con 0x7fee940051c0 2024-04-26T23:18:52.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.351+0000 7feead208640 1 -- 172.21.15.47:0/3820980865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee940051c0 msgr2=0x7fee9400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:52.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.351+0000 7feead208640 1 --2- 172.21.15.47:0/3820980865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee940051c0 0x7fee9400b9c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7feea000c210 tx=0x7fee94005830 comp rx=0 tx=0).stop 2024-04-26T23:18:52.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.351+0000 7feead208640 1 -- 172.21.15.47:0/3820980865 shutdown_connections 2024-04-26T23:18:52.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.351+0000 7feead208640 1 --2- 172.21.15.47:0/3820980865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee940051c0 0x7fee9400b9c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:52.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.351+0000 7feead208640 1 -- 172.21.15.47:0/3820980865 >> 172.21.15.47:0/3820980865 conn(0x7feea80d4fd0 msgr2=0x7feea80d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:52.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.351+0000 7feead208640 1 -- 172.21.15.47:0/3820980865 shutdown_connections 2024-04-26T23:18:52.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.351+0000 7feead208640 1 -- 172.21.15.47:0/3820980865 wait complete. 2024-04-26T23:18:52.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.352+0000 7feead208640 1 Processor -- start 2024-04-26T23:18:52.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.352+0000 7feead208640 1 -- start start 2024-04-26T23:18:52.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.353+0000 7feead208640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee940051c0 0x7feea80ba8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:52.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.353+0000 7feead208640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feea80c86f0 0x7feea80c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:52.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.353+0000 7feead208640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feea8151350 0x7feea81517d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:52.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.353+0000 7feead208640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7feea8162770 con 0x7feea80c86f0 2024-04-26T23:18:52.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.353+0000 7feead208640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7feea81625f0 con 0x7fee940051c0 2024-04-26T23:18:52.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.353+0000 7feead208640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7feea81628f0 con 0x7feea8151350 2024-04-26T23:18:52.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.353+0000 7feea659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feea80c86f0 0x7feea80c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:52.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.353+0000 7feea659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feea80c86f0 0x7feea80c8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58470/0 (socket says 172.21.15.47:58470) 2024-04-26T23:18:52.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.353+0000 7feea659c640 1 -- 172.21.15.47:0/141196679 learned_addr learned my addr 172.21.15.47:0/141196679 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:52.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.353+0000 7feea6d9d640 1 --2- 172.21.15.47:0/141196679 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee940051c0 0x7feea80ba8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:52.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.353+0000 7feea759e640 1 --2- 172.21.15.47:0/141196679 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feea8151350 0x7feea81517d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:52.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.353+0000 7feea659c640 1 -- 172.21.15.47:0/141196679 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feea8151350 msgr2=0x7feea81517d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:52.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.353+0000 7feea659c640 1 --2- 172.21.15.47:0/141196679 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feea8151350 0x7feea81517d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:52.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.353+0000 7feea659c640 1 -- 172.21.15.47:0/141196679 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee940051c0 msgr2=0x7feea80ba8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:52.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.353+0000 7feea659c640 1 --2- 172.21.15.47:0/141196679 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee940051c0 0x7feea80ba8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:52.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.353+0000 7feea659c640 1 -- 172.21.15.47:0/141196679 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feea80cafc0 con 0x7feea80c86f0 2024-04-26T23:18:52.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.353+0000 7feea6d9d640 1 --2- 172.21.15.47:0/141196679 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee940051c0 0x7feea80ba8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:18:52.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.353+0000 7feea759e640 1 --2- 172.21.15.47:0/141196679 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feea8151350 0x7feea81517d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:18:52.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.353+0000 7feea659c640 1 --2- 172.21.15.47:0/141196679 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feea80c86f0 0x7feea80c8b70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fee900118e0 tx=0x7fee90011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:52.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.354+0000 7fee9f7fe640 1 -- 172.21.15.47:0/141196679 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9000faf0 con 0x7feea80c86f0 2024-04-26T23:18:52.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.354+0000 7fee9f7fe640 1 -- 172.21.15.47:0/141196679 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee9000fc90 con 0x7feea80c86f0 2024-04-26T23:18:52.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.354+0000 7fee9f7fe640 1 -- 172.21.15.47:0/141196679 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee90014440 con 0x7feea80c86f0 2024-04-26T23:18:52.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.354+0000 7feead208640 1 -- 172.21.15.47:0/141196679 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feea80cb2a0 con 0x7feea80c86f0 2024-04-26T23:18:52.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.354+0000 7feead208640 1 -- 172.21.15.47:0/141196679 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feea80cb860 con 0x7feea80c86f0 2024-04-26T23:18:52.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.355+0000 7feead208640 1 -- 172.21.15.47:0/141196679 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feea80d1de0 con 0x7feea80c86f0 2024-04-26T23:18:52.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.355+0000 7fee9f7fe640 1 -- 172.21.15.47:0/141196679 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fee90012040 con 0x7feea80c86f0 2024-04-26T23:18:52.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.356+0000 7fee9f7fe640 1 --2- 172.21.15.47:0/141196679 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fee8806c8f0 0x7fee8806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:52.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.356+0000 7fee9f7fe640 1 -- 172.21.15.47:0/141196679 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fee9008c5d0 con 0x7feea80c86f0 2024-04-26T23:18:52.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.356+0000 7feea6d9d640 1 --2- 172.21.15.47:0/141196679 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fee8806c8f0 0x7fee8806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:52.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.356+0000 7feea6d9d640 1 --2- 172.21.15.47:0/141196679 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fee8806c8f0 0x7fee8806edb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fee98000f80 tx=0x7fee98009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:52.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.360+0000 7fee9f7fe640 1 -- 172.21.15.47:0/141196679 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7feea80d1de0 con 0x7feea80c86f0 2024-04-26T23:18:52.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:52.585+0000 7feead208640 1 -- 172.21.15.47:0/141196679 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7feea80694f0 con 0x7feea80c86f0 2024-04-26T23:18:53.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.065+0000 7fee9f7fe640 1 -- 172.21.15.47:0/141196679 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) ==== 118+0+0 (secure 0 0 0) 0x7fee90019030 con 0x7feea80c86f0 2024-04-26T23:18:53.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.067+0000 7feead208640 1 -- 172.21.15.47:0/141196679 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fee8806c8f0 msgr2=0x7fee8806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:53.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.067+0000 7feead208640 1 --2- 172.21.15.47:0/141196679 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fee8806c8f0 0x7fee8806edb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fee98000f80 tx=0x7fee98009040 comp rx=0 tx=0).stop 2024-04-26T23:18:53.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.067+0000 7feead208640 1 -- 172.21.15.47:0/141196679 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feea80c86f0 msgr2=0x7feea80c8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:53.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.067+0000 7feead208640 1 --2- 172.21.15.47:0/141196679 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feea80c86f0 0x7feea80c8b70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fee900118e0 tx=0x7fee90011da0 comp rx=0 tx=0).stop 2024-04-26T23:18:53.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.067+0000 7feead208640 1 -- 172.21.15.47:0/141196679 shutdown_connections 2024-04-26T23:18:53.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.067+0000 7feead208640 1 --2- 172.21.15.47:0/141196679 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fee8806c8f0 0x7fee8806edb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:53.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.067+0000 7feead208640 1 --2- 172.21.15.47:0/141196679 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feea8151350 0x7feea81517d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:53.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.067+0000 7feead208640 1 --2- 172.21.15.47:0/141196679 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feea80c86f0 0x7feea80c8b70 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:53.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.067+0000 7feead208640 1 --2- 172.21.15.47:0/141196679 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee940051c0 0x7feea80ba8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:53.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.067+0000 7feead208640 1 -- 172.21.15.47:0/141196679 >> 172.21.15.47:0/141196679 conn(0x7feea80d4fd0 msgr2=0x7feea806cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:53.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.067+0000 7feead208640 1 -- 172.21.15.47:0/141196679 shutdown_connections 2024-04-26T23:18:53.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.068+0000 7feead208640 1 -- 172.21.15.47:0/141196679 wait complete. 2024-04-26T23:18:53.080 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-04-26T23:18:53.081 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T23:18:53.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.194+0000 7f9984074640 1 Processor -- start 2024-04-26T23:18:53.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.194+0000 7f9984074640 1 -- start start 2024-04-26T23:18:53.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.194+0000 7f9984074640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f997c0d1a00 0x7f997c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:53.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.195+0000 7f9984074640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f997c0b40d0 0x7f997c0b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:53.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.195+0000 7f9984074640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f997c0d0a10 con 0x7f997c0d2320 2024-04-26T23:18:53.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.195+0000 7f9984074640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f997c0d0cf0 con 0x7f997c0d1a00 2024-04-26T23:18:53.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.195+0000 7f9984074640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f997c0d0fd0 con 0x7f997c0b40d0 2024-04-26T23:18:53.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.195+0000 7f9981e10640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f997c0d1a00 0x7f997c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:53.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.195+0000 7f998160f640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f997c0d2320 0x7f997c1625f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:54496/0 (socket says 172.21.15.47:54496) 2024-04-26T23:18:53.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.195+0000 7f9981e10640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f997c0d1a00 0x7f997c0d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58482/0 (socket says 172.21.15.47:58482) 2024-04-26T23:18:53.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.195+0000 7f998160f640 1 -- 172.21.15.47:0/1101337511 learned_addr learned my addr 172.21.15.47:0/1101337511 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:18:53.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.195+0000 7f9982611640 1 --2- 172.21.15.47:0/1101337511 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f997c0b40d0 0x7f997c0b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:53.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.195+0000 7f9981e10640 1 -- 172.21.15.47:0/1101337511 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f997c0b40d0 msgr2=0x7f997c0b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:53.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.195+0000 7f9981e10640 1 --2- 172.21.15.47:0/1101337511 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f997c0b40d0 0x7f997c0b4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:53.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.195+0000 7f9981e10640 1 -- 172.21.15.47:0/1101337511 >> v1:172.21.15.47:6789/0 conn(0x7f997c0d2320 legacy=0x7f997c1625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:53.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.195+0000 7f9981e10640 1 -- 172.21.15.47:0/1101337511 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f997c0d40a0 con 0x7f997c0d1a00 2024-04-26T23:18:53.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.195+0000 7f9980e0e640 1 -- 172.21.15.47:0/1101337511 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 49280774 0 0) 0x7f997c0d0a10 con 0x7f997c0d2320 2024-04-26T23:18:53.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.196+0000 7f9981e10640 1 --2- 172.21.15.47:0/1101337511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f997c0d1a00 0x7f997c0d1de0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f99780097a0 tx=0x7f997802a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=701b494a5e660a46 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:53.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.196+0000 7f9980e0e640 1 -- 172.21.15.47:0/1101337511 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9978037070 con 0x7f997c0d1a00 2024-04-26T23:18:53.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.196+0000 7f9980e0e640 1 -- 172.21.15.47:0/1101337511 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f997802adb0 con 0x7f997c0d1a00 2024-04-26T23:18:53.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.196+0000 7f9980e0e640 1 -- 172.21.15.47:0/1101337511 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9978032660 con 0x7f997c0d1a00 2024-04-26T23:18:53.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.196+0000 7f9984074640 1 -- 172.21.15.47:0/1101337511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f997c0d1a00 msgr2=0x7f997c0d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:53.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.196+0000 7f9984074640 1 --2- 172.21.15.47:0/1101337511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f997c0d1a00 0x7f997c0d1de0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f99780097a0 tx=0x7f997802a060 comp rx=0 tx=0).stop 2024-04-26T23:18:53.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.196+0000 7f9984074640 1 -- 172.21.15.47:0/1101337511 shutdown_connections 2024-04-26T23:18:53.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.196+0000 7f9984074640 1 --2- 172.21.15.47:0/1101337511 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f997c0b40d0 0x7f997c0b4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:53.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.196+0000 7f9984074640 1 --2- 172.21.15.47:0/1101337511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f997c0d1a00 0x7f997c0d1de0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:53.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.196+0000 7f9984074640 1 -- 172.21.15.47:0/1101337511 >> 172.21.15.47:0/1101337511 conn(0x7f997c0d4fd0 msgr2=0x7f997c0d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:53.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.197+0000 7f9984074640 1 -- 172.21.15.47:0/1101337511 shutdown_connections 2024-04-26T23:18:53.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.197+0000 7f9984074640 1 -- 172.21.15.47:0/1101337511 wait complete. 2024-04-26T23:18:53.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.198+0000 7f9984074640 1 Processor -- start 2024-04-26T23:18:53.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.198+0000 7f9984074640 1 -- start start 2024-04-26T23:18:53.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.198+0000 7f9984074640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f997c0b40d0 0x7f997c0cc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:53.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.198+0000 7f9984074640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f997c0d1a00 0x7f997c0ceb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:53.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.198+0000 7f9984074640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f997c0d2320 0x7f997c0b3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:53.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.198+0000 7f9984074640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f997c0b5040 con 0x7f997c0b40d0 2024-04-26T23:18:53.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.198+0000 7f9984074640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f997c162f50 con 0x7f997c0d2320 2024-04-26T23:18:53.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.198+0000 7f9984074640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f997c0b51c0 con 0x7f997c0d1a00 2024-04-26T23:18:53.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.199+0000 7f9981e10640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f997c0b40d0 0x7f997c0cc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:53.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.199+0000 7f9981e10640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f997c0b40d0 0x7f997c0cc0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58486/0 (socket says 172.21.15.47:58486) 2024-04-26T23:18:53.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.199+0000 7f9981e10640 1 -- 172.21.15.47:0/1578665080 learned_addr learned my addr 172.21.15.47:0/1578665080 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:53.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.199+0000 7f998160f640 1 --2- 172.21.15.47:0/1578665080 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f997c0d1a00 0x7f997c0ceb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:53.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.199+0000 7f9982611640 1 --2- 172.21.15.47:0/1578665080 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f997c0d2320 0x7f997c0b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:53.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.199+0000 7f9981e10640 1 -- 172.21.15.47:0/1578665080 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f997c0d1a00 msgr2=0x7f997c0ceb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:53.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.199+0000 7f9981e10640 1 --2- 172.21.15.47:0/1578665080 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f997c0d1a00 0x7f997c0ceb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:53.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.199+0000 7f9981e10640 1 -- 172.21.15.47:0/1578665080 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f997c0d2320 msgr2=0x7f997c0b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:53.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.199+0000 7f9981e10640 1 --2- 172.21.15.47:0/1578665080 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f997c0d2320 0x7f997c0b3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:53.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.199+0000 7f9981e10640 1 -- 172.21.15.47:0/1578665080 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f997c1f95d0 con 0x7f997c0b40d0 2024-04-26T23:18:53.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.199+0000 7f9982611640 1 --2- 172.21.15.47:0/1578665080 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f997c0d2320 0x7f997c0b3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:18:53.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.199+0000 7f9981e10640 1 --2- 172.21.15.47:0/1578665080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f997c0b40d0 0x7f997c0cc0a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9978035040 tx=0x7f99780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:53.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.199+0000 7f99727fc640 1 -- 172.21.15.47:0/1578665080 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9978041070 con 0x7f997c0b40d0 2024-04-26T23:18:53.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.199+0000 7f99727fc640 1 -- 172.21.15.47:0/1578665080 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f997803c670 con 0x7f997c0b40d0 2024-04-26T23:18:53.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.199+0000 7f9984074640 1 -- 172.21.15.47:0/1578665080 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f997c0ca470 con 0x7f997c0b40d0 2024-04-26T23:18:53.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.199+0000 7f99727fc640 1 -- 172.21.15.47:0/1578665080 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9978037040 con 0x7f997c0b40d0 2024-04-26T23:18:53.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.199+0000 7f9984074640 1 -- 172.21.15.47:0/1578665080 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f997c0caa30 con 0x7f997c0b40d0 2024-04-26T23:18:53.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.200+0000 7f9984074640 1 -- 172.21.15.47:0/1578665080 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f997c0694f0 con 0x7f997c0b40d0 2024-04-26T23:18:53.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.201+0000 7f99727fc640 1 -- 172.21.15.47:0/1578665080 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f997803a070 con 0x7f997c0b40d0 2024-04-26T23:18:53.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.201+0000 7f99727fc640 1 --2- 172.21.15.47:0/1578665080 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f996006c8f0 0x7f996006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:53.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.201+0000 7f99727fc640 1 -- 172.21.15.47:0/1578665080 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f99780ac230 con 0x7f997c0b40d0 2024-04-26T23:18:53.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.201+0000 7f998160f640 1 --2- 172.21.15.47:0/1578665080 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f996006c8f0 0x7f996006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:53.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.202+0000 7f998160f640 1 --2- 172.21.15.47:0/1578665080 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f996006c8f0 0x7f996006edb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f997c0b9d10 tx=0x7f996c007c00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:53.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.205+0000 7f99727fc640 1 -- 172.21.15.47:0/1578665080 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f997807a3b0 con 0x7f997c0b40d0 2024-04-26T23:18:53.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:53.429+0000 7f9984074640 1 -- 172.21.15.47:0/1578665080 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f997c162080 con 0x7f997c0b40d0 2024-04-26T23:18:54.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.067+0000 7f99727fc640 1 -- 172.21.15.47:0/1578665080 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) ==== 122+0+0 (secure 0 0 0) 0x7f9978079cb0 con 0x7f997c0b40d0 2024-04-26T23:18:54.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.069+0000 7f9984074640 1 -- 172.21.15.47:0/1578665080 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f996006c8f0 msgr2=0x7f996006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:54.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.069+0000 7f9984074640 1 --2- 172.21.15.47:0/1578665080 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f996006c8f0 0x7f996006edb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f997c0b9d10 tx=0x7f996c007c00 comp rx=0 tx=0).stop 2024-04-26T23:18:54.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.069+0000 7f9984074640 1 -- 172.21.15.47:0/1578665080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f997c0b40d0 msgr2=0x7f997c0cc0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:54.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.069+0000 7f9984074640 1 --2- 172.21.15.47:0/1578665080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f997c0b40d0 0x7f997c0cc0a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9978035040 tx=0x7f99780046c0 comp rx=0 tx=0).stop 2024-04-26T23:18:54.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.069+0000 7f9984074640 1 -- 172.21.15.47:0/1578665080 shutdown_connections 2024-04-26T23:18:54.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.069+0000 7f9984074640 1 --2- 172.21.15.47:0/1578665080 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f996006c8f0 0x7f996006edb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:54.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.069+0000 7f9984074640 1 --2- 172.21.15.47:0/1578665080 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f997c0d2320 0x7f997c0b3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:54.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.069+0000 7f9984074640 1 --2- 172.21.15.47:0/1578665080 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f997c0d1a00 0x7f997c0ceb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:54.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.069+0000 7f9984074640 1 --2- 172.21.15.47:0/1578665080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f997c0b40d0 0x7f997c0cc0a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:54.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.069+0000 7f9984074640 1 -- 172.21.15.47:0/1578665080 >> 172.21.15.47:0/1578665080 conn(0x7f997c0d4fd0 msgr2=0x7f997c06dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:54.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.069+0000 7f9984074640 1 -- 172.21.15.47:0/1578665080 shutdown_connections 2024-04-26T23:18:54.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.069+0000 7f9984074640 1 -- 172.21.15.47:0/1578665080 wait complete. 2024-04-26T23:18:54.083 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T23:18:54.083 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:18:54.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.189+0000 7faa88d4e640 1 Processor -- start 2024-04-26T23:18:54.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.189+0000 7faa88d4e640 1 -- start start 2024-04-26T23:18:54.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.190+0000 7faa88d4e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7faa8415f3a0 0x7faa840d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:54.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.190+0000 7faa88d4e640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa840d0e20 con 0x7faa840d2330 2024-04-26T23:18:54.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.190+0000 7faa88d4e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa840b1d70 con 0x7faa840d1770 2024-04-26T23:18:54.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.190+0000 7faa88d4e640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7faa840b2050 con 0x7faa8415f3a0 2024-04-26T23:18:54.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.190+0000 7faa8259c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7faa8415f3a0 0x7faa840d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:54.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.190+0000 7faa82d9d640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7faa840d2330 0x7faa840b2850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:54512/0 (socket says 172.21.15.47:54512) 2024-04-26T23:18:54.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.190+0000 7faa8259c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7faa8415f3a0 0x7faa840d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58488/0 (socket says 172.21.15.47:58488) 2024-04-26T23:18:54.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.190+0000 7faa82d9d640 1 -- 172.21.15.47:0/1422489813 learned_addr learned my addr 172.21.15.47:0/1422489813 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:18:54.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.191+0000 7faa8259c640 1 -- 172.21.15.47:0/1422489813 >> v1:172.21.15.203:6789/0 conn(0x7faa840d1770 legacy=0x7faa840d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:54.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.191+0000 7faa8259c640 1 -- 172.21.15.47:0/1422489813 >> v1:172.21.15.47:6789/0 conn(0x7faa840d2330 legacy=0x7faa840b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:54.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.191+0000 7faa8259c640 1 -- 172.21.15.47:0/1422489813 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa840b2330 con 0x7faa8415f3a0 2024-04-26T23:18:54.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.191+0000 7faa8159a640 1 -- 172.21.15.47:0/1422489813 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2767023141 0 0) 0x7faa840d0e20 con 0x7faa840d2330 2024-04-26T23:18:54.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.191+0000 7faa8259c640 1 --2- 172.21.15.47:0/1422489813 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa8415f3a0 0x7faa840d06f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7faa7c0097a0 tx=0x7faa7c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36df49c09f498eba server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:54.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.192+0000 7faa8159a640 1 -- 172.21.15.47:0/1422489813 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa7c037070 con 0x7faa8415f3a0 2024-04-26T23:18:54.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.192+0000 7faa8159a640 1 -- 172.21.15.47:0/1422489813 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa7c032440 con 0x7faa8415f3a0 2024-04-26T23:18:54.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.192+0000 7faa8159a640 1 -- 172.21.15.47:0/1422489813 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa7c032810 con 0x7faa8415f3a0 2024-04-26T23:18:54.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.192+0000 7faa88d4e640 1 -- 172.21.15.47:0/1422489813 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa8415f3a0 msgr2=0x7faa840d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:54.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.192+0000 7faa88d4e640 1 --2- 172.21.15.47:0/1422489813 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa8415f3a0 0x7faa840d06f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7faa7c0097a0 tx=0x7faa7c02a1e0 comp rx=0 tx=0).stop 2024-04-26T23:18:54.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.192+0000 7faa88d4e640 1 -- 172.21.15.47:0/1422489813 shutdown_connections 2024-04-26T23:18:54.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.192+0000 7faa88d4e640 1 --2- 172.21.15.47:0/1422489813 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa8415f3a0 0x7faa840d06f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:54.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.192+0000 7faa88d4e640 1 -- 172.21.15.47:0/1422489813 >> 172.21.15.47:0/1422489813 conn(0x7faa840d4f90 msgr2=0x7faa840d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:54.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.192+0000 7faa88d4e640 1 -- 172.21.15.47:0/1422489813 shutdown_connections 2024-04-26T23:18:54.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.192+0000 7faa88d4e640 1 -- 172.21.15.47:0/1422489813 wait complete. 2024-04-26T23:18:54.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.194+0000 7faa88d4e640 1 Processor -- start 2024-04-26T23:18:54.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.194+0000 7faa88d4e640 1 -- start start 2024-04-26T23:18:54.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.194+0000 7faa88d4e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa840d1770 0x7faa840a26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:54.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.194+0000 7faa88d4e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa840d2330 0x7faa841f51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:54.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.194+0000 7faa88d4e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa8415f3a0 0x7faa841f5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:54.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.194+0000 7faa88d4e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7faa840b30e0 con 0x7faa840d1770 2024-04-26T23:18:54.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.194+0000 7faa88d4e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7faa840b2f60 con 0x7faa840d2330 2024-04-26T23:18:54.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.194+0000 7faa88d4e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7faa840b3260 con 0x7faa8415f3a0 2024-04-26T23:18:54.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.194+0000 7faa8259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa840d1770 0x7faa840a26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:54.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.194+0000 7faa8259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa840d1770 0x7faa840a26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58496/0 (socket says 172.21.15.47:58496) 2024-04-26T23:18:54.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.194+0000 7faa8259c640 1 -- 172.21.15.47:0/1551255805 learned_addr learned my addr 172.21.15.47:0/1551255805 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:54.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.195+0000 7faa82d9d640 1 --2- 172.21.15.47:0/1551255805 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa8415f3a0 0x7faa841f5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:54.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.195+0000 7faa8259c640 1 -- 172.21.15.47:0/1551255805 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa8415f3a0 msgr2=0x7faa841f5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:54.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.195+0000 7faa81d9b640 1 --2- 172.21.15.47:0/1551255805 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa840d2330 0x7faa841f51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:54.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.195+0000 7faa8259c640 1 --2- 172.21.15.47:0/1551255805 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa8415f3a0 0x7faa841f5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:54.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.195+0000 7faa8259c640 1 -- 172.21.15.47:0/1551255805 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa840d2330 msgr2=0x7faa841f51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:54.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.195+0000 7faa8259c640 1 --2- 172.21.15.47:0/1551255805 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa840d2330 0x7faa841f51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:54.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.195+0000 7faa8259c640 1 -- 172.21.15.47:0/1551255805 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa840a2be0 con 0x7faa840d1770 2024-04-26T23:18:54.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.195+0000 7faa81d9b640 1 --2- 172.21.15.47:0/1551255805 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa840d2330 0x7faa841f51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:18:54.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.195+0000 7faa82d9d640 1 --2- 172.21.15.47:0/1551255805 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa8415f3a0 0x7faa841f5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:18:54.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.195+0000 7faa8259c640 1 --2- 172.21.15.47:0/1551255805 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa840d1770 0x7faa840a26a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7faa7c0097a0 tx=0x7faa7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:54.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.195+0000 7faa7affd640 1 -- 172.21.15.47:0/1551255805 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa7c041070 con 0x7faa840d1770 2024-04-26T23:18:54.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.195+0000 7faa7affd640 1 -- 172.21.15.47:0/1551255805 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa7c03c870 con 0x7faa840d1770 2024-04-26T23:18:54.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.195+0000 7faa88d4e640 1 -- 172.21.15.47:0/1551255805 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa840b9820 con 0x7faa840d1770 2024-04-26T23:18:54.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.195+0000 7faa7affd640 1 -- 172.21.15.47:0/1551255805 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa7c037040 con 0x7faa840d1770 2024-04-26T23:18:54.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.195+0000 7faa88d4e640 1 -- 172.21.15.47:0/1551255805 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa840b9de0 con 0x7faa840d1770 2024-04-26T23:18:54.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.196+0000 7faa88d4e640 1 -- 172.21.15.47:0/1551255805 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa840b3fb0 con 0x7faa840d1770 2024-04-26T23:18:54.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.197+0000 7faa7affd640 1 -- 172.21.15.47:0/1551255805 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faa7c044070 con 0x7faa840d1770 2024-04-26T23:18:54.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.197+0000 7faa7affd640 1 --2- 172.21.15.47:0/1551255805 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7faa6406c8f0 0x7faa6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:54.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.197+0000 7faa7affd640 1 -- 172.21.15.47:0/1551255805 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7faa7c0ac9d0 con 0x7faa840d1770 2024-04-26T23:18:54.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.197+0000 7faa81d9b640 1 --2- 172.21.15.47:0/1551255805 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7faa6406c8f0 0x7faa6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:54.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.198+0000 7faa81d9b640 1 --2- 172.21.15.47:0/1551255805 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7faa6406c8f0 0x7faa6406edb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7faa841f62f0 tx=0x7faa6c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:54.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.201+0000 7faa7affd640 1 -- 172.21.15.47:0/1551255805 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faa840b3fb0 con 0x7faa840d1770 2024-04-26T23:18:54.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.423+0000 7faa88d4e640 1 -- 172.21.15.47:0/1551255805 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faa840d0330 con 0x7faa840d1770 2024-04-26T23:18:54.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.424+0000 7faa7affd640 1 -- 172.21.15.47:0/1551255805 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5180 (secure 0 0 0) 0x7faa840d0330 con 0x7faa840d1770 2024-04-26T23:18:54.424 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:18:54.425 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/1553696401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1553696401},{"type":"v1","addr":"172.21.15.47:6834","nonce":1553696401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/3792767564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3792767564},{"type":"v1","addr":"172.21.15.203:6836","nonce":3792767564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4088100766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":4088100766},{"type":"v1","addr":"172.21.15.47:6835","nonce":4088100766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:18:31.367292+0000","modified":"2024-04-26T23:18:53.430267+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/997115011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":997115011},{"type":"v1","addr":"172.21.15.203:6837","nonce":997115011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4282,"qdb_cluster":[4282]},"id":1}]} 2024-04-26T23:18:54.425 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 9 2024-04-26T23:18:54.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.425+0000 7faa88d4e640 1 -- 172.21.15.47:0/1551255805 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7faa6406c8f0 msgr2=0x7faa6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:54.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.425+0000 7faa88d4e640 1 --2- 172.21.15.47:0/1551255805 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7faa6406c8f0 0x7faa6406edb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7faa841f62f0 tx=0x7faa6c009040 comp rx=0 tx=0).stop 2024-04-26T23:18:54.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.426+0000 7faa88d4e640 1 -- 172.21.15.47:0/1551255805 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa840d1770 msgr2=0x7faa840a26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:54.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.426+0000 7faa88d4e640 1 --2- 172.21.15.47:0/1551255805 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa840d1770 0x7faa840a26a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7faa7c0097a0 tx=0x7faa7c0046c0 comp rx=0 tx=0).stop 2024-04-26T23:18:54.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.426+0000 7faa88d4e640 1 -- 172.21.15.47:0/1551255805 shutdown_connections 2024-04-26T23:18:54.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.426+0000 7faa88d4e640 1 --2- 172.21.15.47:0/1551255805 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7faa6406c8f0 0x7faa6406edb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:54.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.426+0000 7faa88d4e640 1 --2- 172.21.15.47:0/1551255805 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa8415f3a0 0x7faa841f5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:54.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.426+0000 7faa88d4e640 1 --2- 172.21.15.47:0/1551255805 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa840d2330 0x7faa841f51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:54.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.426+0000 7faa88d4e640 1 --2- 172.21.15.47:0/1551255805 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa840d1770 0x7faa840a26a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:54.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.426+0000 7faa88d4e640 1 -- 172.21.15.47:0/1551255805 >> 172.21.15.47:0/1551255805 conn(0x7faa840d4f90 msgr2=0x7faa840b2850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:54.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.426+0000 7faa88d4e640 1 -- 172.21.15.47:0/1551255805 shutdown_connections 2024-04-26T23:18:54.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.426+0000 7faa88d4e640 1 -- 172.21.15.47:0/1551255805 wait complete. 2024-04-26T23:18:54.437 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:18:54.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.537+0000 7f8aea062640 1 Processor -- start 2024-04-26T23:18:54.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.538+0000 7f8aea062640 1 -- start start 2024-04-26T23:18:54.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.538+0000 7f8aea062640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8ae4156460 0x7f8ae40cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:54.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.538+0000 7f8aea062640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8ae406fea0 0x7f8ae4072c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:54.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.538+0000 7f8aea062640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ae4156840 con 0x7f8ae40cba10 2024-04-26T23:18:54.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.538+0000 7f8aea062640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f8ae40cbf20 con 0x7f8ae4156460 2024-04-26T23:18:54.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.538+0000 7f8aea062640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f8ae40cc200 con 0x7f8ae406fea0 2024-04-26T23:18:54.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.539+0000 7f8ae37fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8ae4156460 0x7f8ae40cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:54.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.539+0000 7f8ae37fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8ae4156460 0x7f8ae40cb4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58504/0 (socket says 172.21.15.47:58504) 2024-04-26T23:18:54.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.539+0000 7f8ae37fe640 1 -- 172.21.15.47:0/2851814942 learned_addr learned my addr 172.21.15.47:0/2851814942 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:54.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.539+0000 7f8ae3fff640 1 --2- 172.21.15.47:0/2851814942 >> v2:172.21.15.203:3300/0 conn(0x7f8ae406fea0 0x7f8ae4072c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:54.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.539+0000 7f8ae37fe640 1 -- 172.21.15.47:0/2851814942 >> v2:172.21.15.203:3300/0 conn(0x7f8ae406fea0 msgr2=0x7f8ae4072c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:54.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.539+0000 7f8ae37fe640 1 --2- 172.21.15.47:0/2851814942 >> v2:172.21.15.203:3300/0 conn(0x7f8ae406fea0 0x7f8ae4072c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:54.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.539+0000 7f8ae37fe640 1 -- 172.21.15.47:0/2851814942 >> v1:172.21.15.203:6789/0 conn(0x7f8ae40cba10 legacy=0x7f8ae406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:54.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.539+0000 7f8ae37fe640 1 -- 172.21.15.47:0/2851814942 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ae40cc4e0 con 0x7f8ae4156460 2024-04-26T23:18:54.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.539+0000 7f8ae3fff640 1 --2- 172.21.15.47:0/2851814942 >> v2:172.21.15.203:3300/0 conn(0x7f8ae406fea0 0x7f8ae4072c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:18:54.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.540+0000 7f8ae37fe640 1 --2- 172.21.15.47:0/2851814942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ae4156460 0x7f8ae40cb4d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8ae40c7c40 tx=0x7f8ad402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be368f64e7286f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:54.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.540+0000 7f8ae27fc640 1 -- 172.21.15.47:0/2851814942 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4037070 con 0x7f8ae4156460 2024-04-26T23:18:54.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.540+0000 7f8ae27fc640 1 -- 172.21.15.47:0/2851814942 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ad4032440 con 0x7f8ae4156460 2024-04-26T23:18:54.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.540+0000 7f8ae27fc640 1 -- 172.21.15.47:0/2851814942 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4032810 con 0x7f8ae4156460 2024-04-26T23:18:54.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.540+0000 7f8aea062640 1 -- 172.21.15.47:0/2851814942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ae4156460 msgr2=0x7f8ae40cb4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:54.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.540+0000 7f8aea062640 1 --2- 172.21.15.47:0/2851814942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ae4156460 0x7f8ae40cb4d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8ae40c7c40 tx=0x7f8ad402a270 comp rx=0 tx=0).stop 2024-04-26T23:18:54.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.540+0000 7f8aea062640 1 -- 172.21.15.47:0/2851814942 shutdown_connections 2024-04-26T23:18:54.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.540+0000 7f8aea062640 1 --2- 172.21.15.47:0/2851814942 >> v2:172.21.15.203:3300/0 conn(0x7f8ae406fea0 0x7f8ae4072c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:54.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.540+0000 7f8aea062640 1 --2- 172.21.15.47:0/2851814942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ae4156460 0x7f8ae40cb4d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:54.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.540+0000 7f8aea062640 1 -- 172.21.15.47:0/2851814942 >> 172.21.15.47:0/2851814942 conn(0x7f8ae40d0be0 msgr2=0x7f8ae40d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:54.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.540+0000 7f8aea062640 1 -- 172.21.15.47:0/2851814942 shutdown_connections 2024-04-26T23:18:54.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.540+0000 7f8aea062640 1 -- 172.21.15.47:0/2851814942 wait complete. 2024-04-26T23:18:54.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.541+0000 7f8aea062640 1 Processor -- start 2024-04-26T23:18:54.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.542+0000 7f8aea062640 1 -- start start 2024-04-26T23:18:54.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.542+0000 7f8aea062640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ae406fea0 0x7f8ae414dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:54.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.542+0000 7f8aea062640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ae40cba10 0x7f8ae41543c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:54.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.542+0000 7f8aea062640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ae4156460 0x7f8ae4154900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:54.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.542+0000 7f8aea062640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae406e260 con 0x7f8ae40cba10 2024-04-26T23:18:54.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.542+0000 7f8aea062640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae4072700 con 0x7f8ae4156460 2024-04-26T23:18:54.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.542+0000 7f8aea062640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f8ae406e3e0 con 0x7f8ae406fea0 2024-04-26T23:18:54.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.542+0000 7f8ae2ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ae40cba10 0x7f8ae41543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:54.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.543+0000 7f8ae2ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ae40cba10 0x7f8ae41543c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58508/0 (socket says 172.21.15.47:58508) 2024-04-26T23:18:54.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.543+0000 7f8ae2ffd640 1 -- 172.21.15.47:0/2197425806 learned_addr learned my addr 172.21.15.47:0/2197425806 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:54.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.543+0000 7f8ae37fe640 1 --2- 172.21.15.47:0/2197425806 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ae406fea0 0x7f8ae414dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:54.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.543+0000 7f8ae3fff640 1 --2- 172.21.15.47:0/2197425806 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ae4156460 0x7f8ae4154900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:54.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.543+0000 7f8ae2ffd640 1 -- 172.21.15.47:0/2197425806 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ae406fea0 msgr2=0x7f8ae414dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:54.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.543+0000 7f8ae2ffd640 1 --2- 172.21.15.47:0/2197425806 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ae406fea0 0x7f8ae414dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:54.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.543+0000 7f8ae2ffd640 1 -- 172.21.15.47:0/2197425806 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ae4156460 msgr2=0x7f8ae4154900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:54.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.543+0000 7f8ae2ffd640 1 --2- 172.21.15.47:0/2197425806 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ae4156460 0x7f8ae4154900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:54.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.543+0000 7f8ae2ffd640 1 -- 172.21.15.47:0/2197425806 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ae41535f0 con 0x7f8ae40cba10 2024-04-26T23:18:54.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.543+0000 7f8ae3fff640 1 --2- 172.21.15.47:0/2197425806 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ae4156460 0x7f8ae4154900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:18:54.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.543+0000 7f8ae2ffd640 1 --2- 172.21.15.47:0/2197425806 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ae40cba10 0x7f8ae41543c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8ad800c400 tx=0x7f8ad800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:54.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.543+0000 7f8acbfff640 1 -- 172.21.15.47:0/2197425806 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad80052d0 con 0x7f8ae40cba10 2024-04-26T23:18:54.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.543+0000 7f8acbfff640 1 -- 172.21.15.47:0/2197425806 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ad8005490 con 0x7f8ae40cba10 2024-04-26T23:18:54.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.543+0000 7f8aea062640 1 -- 172.21.15.47:0/2197425806 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ae41538d0 con 0x7f8ae40cba10 2024-04-26T23:18:54.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.543+0000 7f8acbfff640 1 -- 172.21.15.47:0/2197425806 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad8007260 con 0x7f8ae40cba10 2024-04-26T23:18:54.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.543+0000 7f8aea062640 1 -- 172.21.15.47:0/2197425806 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ae40b3fb0 con 0x7f8ae40cba10 2024-04-26T23:18:54.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.544+0000 7f8aea062640 1 -- 172.21.15.47:0/2197425806 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ae40694f0 con 0x7f8ae40cba10 2024-04-26T23:18:54.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.545+0000 7f8acbfff640 1 -- 172.21.15.47:0/2197425806 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8ad8007400 con 0x7f8ae40cba10 2024-04-26T23:18:54.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.545+0000 7f8acbfff640 1 --2- 172.21.15.47:0/2197425806 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f8ad006c8f0 0x7f8ad006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:54.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.545+0000 7f8acbfff640 1 -- 172.21.15.47:0/2197425806 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f8ad808b410 con 0x7f8ae40cba10 2024-04-26T23:18:54.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.545+0000 7f8ae37fe640 1 --2- 172.21.15.47:0/2197425806 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f8ad006c8f0 0x7f8ad006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:54.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.546+0000 7f8ae37fe640 1 --2- 172.21.15.47:0/2197425806 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f8ad006c8f0 0x7f8ad006edb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4034000 tx=0x7f8ad402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:54.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.549+0000 7f8acbfff640 1 -- 172.21.15.47:0/2197425806 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8ad8059500 con 0x7f8ae40cba10 2024-04-26T23:18:54.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.735+0000 7f8aea062640 1 -- 172.21.15.47:0/2197425806 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8ae406de80 con 0x7f8ae40cba10 2024-04-26T23:18:54.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.736+0000 7f8acbfff640 1 -- 172.21.15.47:0/2197425806 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18478 (secure 0 0 0) 0x7f8ad8058e00 con 0x7f8ae40cba10 2024-04-26T23:18:54.737 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:18:54.737 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":24,"fsid":"399006d3-4db1-43e6-ba81-a79a276eabfd","created":"2024-04-26T23:18:06.731211+0000","modified":"2024-04-26T23:18:36.402132+0000","last_up_change":"2024-04-26T23:18:17.808736+0000","last_in_change":"2024-04-26T23:18:08.945859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:18:16.937492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:18:29.033575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:18:30.318499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:18:34.014536+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ee091567-fea0-4a14-bf4e-0e16c20bb2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6819","nonce":326520779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6821","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6823","nonce":326520779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6829","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6831","nonce":326520779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6825","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6827","nonce":326520779}]},"public_addr":"172.21.15.47:6819/326520779","cluster_addr":"172.21.15.47:6823/326520779","heartbeat_back_addr":"172.21.15.47:6831/326520779","heartbeat_front_addr":"172.21.15.47:6827/326520779","state":["exists","up"]},{"osd":1,"uuid":"c14440a0-bba5-4931-a6db-b736c08e869b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6809","nonce":1912610931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6811","nonce":1912610931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6815","nonce":1912610931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6813","nonce":1912610931}]},"public_addr":"172.21.15.47:6809/1912610931","cluster_addr":"172.21.15.47:6811/1912610931","heartbeat_back_addr":"172.21.15.47:6815/1912610931","heartbeat_front_addr":"172.21.15.47:6813/1912610931","state":["exists","up"]},{"osd":2,"uuid":"3940f8de-c9b2-4cd3-a911-9c301729eb30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6801","nonce":2569869305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6803","nonce":2569869305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6807","nonce":2569869305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6805","nonce":2569869305}]},"public_addr":"172.21.15.47:6801/2569869305","cluster_addr":"172.21.15.47:6803/2569869305","heartbeat_back_addr":"172.21.15.47:6807/2569869305","heartbeat_front_addr":"172.21.15.47:6805/2569869305","state":["exists","up"]},{"osd":3,"uuid":"1286a67e-8958-426a-8c2e-bc85498c619c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6818","nonce":2791411683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6822","nonce":2791411683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6830","nonce":2791411683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6826","nonce":2791411683}]},"public_addr":"172.21.15.47:6818/2791411683","cluster_addr":"172.21.15.47:6822/2791411683","heartbeat_back_addr":"172.21.15.47:6830/2791411683","heartbeat_front_addr":"172.21.15.47:6826/2791411683","state":["exists","up"]},{"osd":4,"uuid":"d8ee2c20-e5b7-46c2-9999-b234070126cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6809","nonce":441313104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6811","nonce":441313104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6815","nonce":441313104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6813","nonce":441313104}]},"public_addr":"172.21.15.203:6809/441313104","cluster_addr":"172.21.15.203:6811/441313104","heartbeat_back_addr":"172.21.15.203:6815/441313104","heartbeat_front_addr":"172.21.15.203:6813/441313104","state":["exists","up"]},{"osd":5,"uuid":"b453a8f0-f324-4840-9ae0-d67bdadb9f8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6801","nonce":56838444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6803","nonce":56838444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6807","nonce":56838444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6805","nonce":56838444}]},"public_addr":"172.21.15.203:6801/56838444","cluster_addr":"172.21.15.203:6803/56838444","heartbeat_back_addr":"172.21.15.203:6807/56838444","heartbeat_front_addr":"172.21.15.203:6805/56838444","state":["exists","up"]},{"osd":6,"uuid":"a6d31951-2d4b-4e8d-a510-6283b01dc158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6825","nonce":2738517080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6827","nonce":2738517080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6831","nonce":2738517080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6829","nonce":2738517080}]},"public_addr":"172.21.15.203:6825/2738517080","cluster_addr":"172.21.15.203:6827/2738517080","heartbeat_back_addr":"172.21.15.203:6831/2738517080","heartbeat_front_addr":"172.21.15.203:6829/2738517080","state":["exists","up"]},{"osd":7,"uuid":"7119b8c0-fe96-402d-9e75-7b509d9d44d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6817","nonce":1315565058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6819","nonce":1315565058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6823","nonce":1315565058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6821","nonce":1315565058}]},"public_addr":"172.21.15.203:6817/1315565058","cluster_addr":"172.21.15.203:6819/1315565058","heartbeat_back_addr":"172.21.15.203:6823/1315565058","heartbeat_front_addr":"172.21.15.203:6821/1315565058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.121921+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.137494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.084814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.111593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.072430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.125066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.097867+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.057061+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:18:54.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.738+0000 7f8aea062640 1 -- 172.21.15.47:0/2197425806 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f8ad006c8f0 msgr2=0x7f8ad006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:54.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.738+0000 7f8aea062640 1 --2- 172.21.15.47:0/2197425806 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f8ad006c8f0 0x7f8ad006edb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4034000 tx=0x7f8ad402e000 comp rx=0 tx=0).stop 2024-04-26T23:18:54.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.738+0000 7f8aea062640 1 -- 172.21.15.47:0/2197425806 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ae40cba10 msgr2=0x7f8ae41543c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:54.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.738+0000 7f8aea062640 1 --2- 172.21.15.47:0/2197425806 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ae40cba10 0x7f8ae41543c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8ad800c400 tx=0x7f8ad800c8c0 comp rx=0 tx=0).stop 2024-04-26T23:18:54.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.739+0000 7f8aea062640 1 -- 172.21.15.47:0/2197425806 shutdown_connections 2024-04-26T23:18:54.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.739+0000 7f8aea062640 1 --2- 172.21.15.47:0/2197425806 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f8ad006c8f0 0x7f8ad006edb0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:54.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.739+0000 7f8aea062640 1 --2- 172.21.15.47:0/2197425806 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ae4156460 0x7f8ae4154900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:54.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.739+0000 7f8aea062640 1 --2- 172.21.15.47:0/2197425806 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ae40cba10 0x7f8ae41543c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:54.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.739+0000 7f8aea062640 1 --2- 172.21.15.47:0/2197425806 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ae406fea0 0x7f8ae414dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:54.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.739+0000 7f8aea062640 1 -- 172.21.15.47:0/2197425806 >> 172.21.15.47:0/2197425806 conn(0x7f8ae40d0be0 msgr2=0x7f8ae40b0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:54.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.739+0000 7f8aea062640 1 -- 172.21.15.47:0/2197425806 shutdown_connections 2024-04-26T23:18:54.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.739+0000 7f8aea062640 1 -- 172.21.15.47:0/2197425806 wait complete. 2024-04-26T23:18:54.751 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T23:18:54.751 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T23:18:54.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.850+0000 7f1182de4640 1 Processor -- start 2024-04-26T23:18:54.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.851+0000 7f1182de4640 1 -- start start 2024-04-26T23:18:54.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.851+0000 7f1182de4640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f117c0d0070 0x7f117c0d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:54.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.851+0000 7f1182de4640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f117c0d2420 con 0x7f117c06d560 2024-04-26T23:18:54.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.851+0000 7f1182de4640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f117c0cd6b0 con 0x7f117c06e270 2024-04-26T23:18:54.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.851+0000 7f1182de4640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f117c162d00 con 0x7f117c0d0070 2024-04-26T23:18:54.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.851+0000 7f117bfff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f117c06d560 0x7f117c06db60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:54514/0 (socket says 172.21.15.47:54514) 2024-04-26T23:18:54.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.852+0000 7f117bfff640 1 -- 172.21.15.47:0/1496953687 learned_addr learned my addr 172.21.15.47:0/1496953687 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:18:54.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.852+0000 7f1180b80640 1 --2- 172.21.15.47:0/1496953687 >> v2:172.21.15.203:3300/0 conn(0x7f117c0d0070 0x7f117c0d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:54.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.852+0000 7f117b7fe640 1 -- 172.21.15.47:0/1496953687 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1743822398 0 0) 0x7f117c0d2420 con 0x7f117c06d560 2024-04-26T23:18:54.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.852+0000 7f117b7fe640 1 -- 172.21.15.47:0/1496953687 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f116c003520 con 0x7f117c06d560 2024-04-26T23:18:54.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.852+0000 7f117b7fe640 1 -- 172.21.15.47:0/1496953687 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3693681984 0 0) 0x7f116c003520 con 0x7f117c06d560 2024-04-26T23:18:54.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.852+0000 7f117b7fe640 1 -- 172.21.15.47:0/1496953687 >> v2:172.21.15.203:3300/0 conn(0x7f117c0d0070 msgr2=0x7f117c0d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:54.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.852+0000 7f117b7fe640 1 --2- 172.21.15.47:0/1496953687 >> v2:172.21.15.203:3300/0 conn(0x7f117c0d0070 0x7f117c0d1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:54.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.852+0000 7f117b7fe640 1 -- 172.21.15.47:0/1496953687 >> v1:172.21.15.203:6789/0 conn(0x7f117c06e270 legacy=0x7f117c0ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:54.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.852+0000 7f117b7fe640 1 -- 172.21.15.47:0/1496953687 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f117c0cd990 con 0x7f117c06d560 2024-04-26T23:18:54.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.852+0000 7f117b7fe640 1 -- 172.21.15.47:0/1496953687 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7f1168001670 con 0x7f117c06d560 2024-04-26T23:18:54.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.852+0000 7f117b7fe640 1 -- 172.21.15.47:0/1496953687 >> v1:172.21.15.47:6789/0 conn(0x7f117c06d560 legacy=0x7f117c06db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:54.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.852+0000 7f117b7fe640 1 --2- 172.21.15.47:0/1496953687 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f116c0039b0 0x7f116c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:54.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.852+0000 7f117b7fe640 1 --2- 172.21.15.47:0/1496953687 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f116c004800 0x7f116c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:54.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.852+0000 7f117b7fe640 1 --2- 172.21.15.47:0/1496953687 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f116c0051c0 0x7f116c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:54.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.852+0000 7f117b7fe640 1 -- 172.21.15.47:0/1496953687 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f117c162d00 con 0x7f116c0039b0 2024-04-26T23:18:54.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.852+0000 7f117b7fe640 1 -- 172.21.15.47:0/1496953687 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f116c00c060 con 0x7f116c004800 2024-04-26T23:18:54.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.852+0000 7f117b7fe640 1 -- 172.21.15.47:0/1496953687 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f116c00c260 con 0x7f116c0051c0 2024-04-26T23:18:54.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.852+0000 7f1180b80640 1 --2- 172.21.15.47:0/1496953687 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f116c0039b0 0x7f116c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:54.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.852+0000 7f1181381640 1 --2- 172.21.15.47:0/1496953687 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f116c0051c0 0x7f116c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:54.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.852+0000 7f117bfff640 1 --2- 172.21.15.47:0/1496953687 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f116c004800 0x7f116c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:54.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.853+0000 7f1180b80640 1 -- 172.21.15.47:0/1496953687 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f116c0051c0 msgr2=0x7f116c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:54.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.853+0000 7f1180b80640 1 --2- 172.21.15.47:0/1496953687 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f116c0051c0 0x7f116c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:54.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.853+0000 7f1180b80640 1 -- 172.21.15.47:0/1496953687 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f116c004800 msgr2=0x7f116c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:54.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.853+0000 7f1180b80640 1 --2- 172.21.15.47:0/1496953687 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f116c004800 0x7f116c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:54.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.853+0000 7f1180b80640 1 -- 172.21.15.47:0/1496953687 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f116c003520 con 0x7f116c0039b0 2024-04-26T23:18:54.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.853+0000 7f1181381640 1 -- 172.21.15.47:0/1496953687 reap_dead start 2024-04-26T23:18:54.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.853+0000 7f1180b80640 1 --2- 172.21.15.47:0/1496953687 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f116c0039b0 0x7f116c003dd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1170009ac0 tx=0x7f116c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac91f18f10cebe8a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:54.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.853+0000 7f117b7fe640 1 -- 172.21.15.47:0/1496953687 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1170036070 con 0x7f116c0039b0 2024-04-26T23:18:54.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.853+0000 7f117b7fe640 1 -- 172.21.15.47:0/1496953687 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1170031400 con 0x7f116c0039b0 2024-04-26T23:18:54.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.853+0000 7f117b7fe640 1 -- 172.21.15.47:0/1496953687 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1170009690 con 0x7f116c0039b0 2024-04-26T23:18:54.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.854+0000 7f1182de4640 1 -- 172.21.15.47:0/1496953687 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f116c0039b0 msgr2=0x7f116c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:54.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.854+0000 7f1182de4640 1 --2- 172.21.15.47:0/1496953687 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f116c0039b0 0x7f116c003dd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1170009ac0 tx=0x7f116c005830 comp rx=0 tx=0).stop 2024-04-26T23:18:54.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.854+0000 7f1182de4640 1 -- 172.21.15.47:0/1496953687 shutdown_connections 2024-04-26T23:18:54.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.854+0000 7f1182de4640 1 --2- 172.21.15.47:0/1496953687 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f116c0039b0 0x7f116c003dd0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:54.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.854+0000 7f1182de4640 1 -- 172.21.15.47:0/1496953687 >> 172.21.15.47:0/1496953687 conn(0x7f117c0d2ec0 msgr2=0x7f117c0d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:54.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.854+0000 7f1182de4640 1 -- 172.21.15.47:0/1496953687 shutdown_connections 2024-04-26T23:18:54.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.854+0000 7f1182de4640 1 -- 172.21.15.47:0/1496953687 wait complete. 2024-04-26T23:18:54.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.855+0000 7f1182de4640 1 Processor -- start 2024-04-26T23:18:54.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.855+0000 7f1182de4640 1 -- start start 2024-04-26T23:18:54.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.856+0000 7f1182de4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f116c0039b0 0x7f117c1f8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:54.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.856+0000 7f1182de4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f117c159a40 0x7f117c0c8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:54.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.856+0000 7f1182de4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f117c15c430 0x7f117c15c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:54.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.856+0000 7f1182de4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f117c0d1d40 con 0x7f116c0039b0 2024-04-26T23:18:54.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.856+0000 7f1182de4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f117c0d2d10 con 0x7f117c159a40 2024-04-26T23:18:54.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.856+0000 7f1182de4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f117c0d1ec0 con 0x7f117c15c430 2024-04-26T23:18:54.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.856+0000 7f1180b80640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f116c0039b0 0x7f117c1f8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:54.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.856+0000 7f1180b80640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f116c0039b0 0x7f117c1f8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58530/0 (socket says 172.21.15.47:58530) 2024-04-26T23:18:54.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.856+0000 7f1180b80640 1 -- 172.21.15.47:0/2477785391 learned_addr learned my addr 172.21.15.47:0/2477785391 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:54.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.856+0000 7f117bfff640 1 --2- 172.21.15.47:0/2477785391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f117c159a40 0x7f117c0c8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:54.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.856+0000 7f1181381640 1 --2- 172.21.15.47:0/2477785391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f117c15c430 0x7f117c15c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:54.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.857+0000 7f1180b80640 1 -- 172.21.15.47:0/2477785391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f117c15c430 msgr2=0x7f117c15c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:54.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.857+0000 7f1180b80640 1 --2- 172.21.15.47:0/2477785391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f117c15c430 0x7f117c15c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:54.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.857+0000 7f1180b80640 1 -- 172.21.15.47:0/2477785391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f117c159a40 msgr2=0x7f117c0c8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:54.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.857+0000 7f1180b80640 1 --2- 172.21.15.47:0/2477785391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f117c159a40 0x7f117c0c8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:54.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.857+0000 7f1180b80640 1 -- 172.21.15.47:0/2477785391 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f117c1f93d0 con 0x7f116c0039b0 2024-04-26T23:18:54.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.857+0000 7f1181381640 1 --2- 172.21.15.47:0/2477785391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f117c15c430 0x7f117c15c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:18:54.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.857+0000 7f117bfff640 1 --2- 172.21.15.47:0/2477785391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f117c159a40 0x7f117c0c8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:18:54.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.857+0000 7f1180b80640 1 --2- 172.21.15.47:0/2477785391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f116c0039b0 0x7f117c1f8e90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1170034040 tx=0x7f1170004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:54.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.857+0000 7f11797fa640 1 -- 172.21.15.47:0/2477785391 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f117003f020 con 0x7f116c0039b0 2024-04-26T23:18:54.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.857+0000 7f11797fa640 1 -- 172.21.15.47:0/2477785391 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1170037040 con 0x7f116c0039b0 2024-04-26T23:18:54.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.857+0000 7f1182de4640 1 -- 172.21.15.47:0/2477785391 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f117c15e630 con 0x7f116c0039b0 2024-04-26T23:18:54.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.857+0000 7f11797fa640 1 -- 172.21.15.47:0/2477785391 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1170036070 con 0x7f116c0039b0 2024-04-26T23:18:54.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.857+0000 7f1182de4640 1 -- 172.21.15.47:0/2477785391 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f117c15ebf0 con 0x7f116c0039b0 2024-04-26T23:18:54.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.858+0000 7f1182de4640 1 -- 172.21.15.47:0/2477785391 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f117c0694f0 con 0x7f116c0039b0 2024-04-26T23:18:54.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.859+0000 7f11797fa640 1 -- 172.21.15.47:0/2477785391 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1170042020 con 0x7f116c0039b0 2024-04-26T23:18:54.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.859+0000 7f11797fa640 1 --2- 172.21.15.47:0/2477785391 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f116006c920 0x7f116006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:54.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.859+0000 7f11797fa640 1 -- 172.21.15.47:0/2477785391 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f11700abe10 con 0x7f116c0039b0 2024-04-26T23:18:54.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.859+0000 7f117bfff640 1 --2- 172.21.15.47:0/2477785391 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f116006c920 0x7f116006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:54.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.860+0000 7f117bfff640 1 --2- 172.21.15.47:0/2477785391 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f116006c920 0x7f116006ede0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f117c0a0210 tx=0x7f1168009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:54.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:54.863+0000 7f11797fa640 1 -- 172.21.15.47:0/2477785391 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1170049030 con 0x7f116c0039b0 2024-04-26T23:18:55.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.048+0000 7f1182de4640 1 -- 172.21.15.47:0/2477785391 --> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f117c0c9ce0 con 0x7f116006c920 2024-04-26T23:18:55.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.057+0000 7f11797fa640 1 -- 172.21.15.47:0/2477785391 <== mgr.4099 v2:172.21.15.203:6832/992162732 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332530 (secure 0 0 0) 0x7f117c0c9ce0 con 0x7f116006c920 2024-04-26T23:18:55.062 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:18:55.064 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-26T23:18:55.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.061+0000 7f1182de4640 1 -- 172.21.15.47:0/2477785391 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f116006c920 msgr2=0x7f116006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:55.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.061+0000 7f1182de4640 1 --2- 172.21.15.47:0/2477785391 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f116006c920 0x7f116006ede0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f117c0a0210 tx=0x7f1168009040 comp rx=0 tx=0).stop 2024-04-26T23:18:55.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.061+0000 7f1182de4640 1 -- 172.21.15.47:0/2477785391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f116c0039b0 msgr2=0x7f117c1f8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:55.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.061+0000 7f1182de4640 1 --2- 172.21.15.47:0/2477785391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f116c0039b0 0x7f117c1f8e90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1170034040 tx=0x7f1170004730 comp rx=0 tx=0).stop 2024-04-26T23:18:55.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.061+0000 7f1182de4640 1 -- 172.21.15.47:0/2477785391 shutdown_connections 2024-04-26T23:18:55.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.061+0000 7f1182de4640 1 --2- 172.21.15.47:0/2477785391 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f116006c920 0x7f116006ede0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:55.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.061+0000 7f1182de4640 1 --2- 172.21.15.47:0/2477785391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f117c15c430 0x7f117c15c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:55.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.061+0000 7f1182de4640 1 --2- 172.21.15.47:0/2477785391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f117c159a40 0x7f117c0c8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:55.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.061+0000 7f1182de4640 1 --2- 172.21.15.47:0/2477785391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f116c0039b0 0x7f117c1f8e90 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:55.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.061+0000 7f1182de4640 1 -- 172.21.15.47:0/2477785391 >> 172.21.15.47:0/2477785391 conn(0x7f117c0d2ec0 msgr2=0x7f117c0cacf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:55.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.061+0000 7f1182de4640 1 -- 172.21.15.47:0/2477785391 shutdown_connections 2024-04-26T23:18:55.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.061+0000 7f1182de4640 1 -- 172.21.15.47:0/2477785391 wait complete. 2024-04-26T23:18:55.072 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-04-26T23:18:54.929134+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":231832,"kb_used_data":3176,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754742888,"statfs":{"total":773094113280,"available":772856717312,"internally_reserved":0,"allocated":3252224,"data_stored":1983104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":234016304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003517"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044475+0000","last_change":"2024-04-26T23:18:31.011895+0000","last_active":"2024-04-26T23:18:37.044475+0000","last_peered":"2024-04-26T23:18:37.044475+0000","last_clean":"2024-04-26T23:18:37.044475+0000","last_became_active":"2024-04-26T23:18:31.010976+0000","last_became_peered":"2024-04-26T23:18:31.010976+0000","last_unstale":"2024-04-26T23:18:37.044475+0000","last_undegraded":"2024-04-26T23:18:37.044475+0000","last_fullsized":"2024-04-26T23:18:37.044475+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:57:14.165871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030065+0000","last_change":"2024-04-26T23:18:32.020612+0000","last_active":"2024-04-26T23:18:37.030065+0000","last_peered":"2024-04-26T23:18:37.030065+0000","last_clean":"2024-04-26T23:18:37.030065+0000","last_became_active":"2024-04-26T23:18:32.018903+0000","last_became_peered":"2024-04-26T23:18:32.018903+0000","last_unstale":"2024-04-26T23:18:37.030065+0000","last_undegraded":"2024-04-26T23:18:37.030065+0000","last_fullsized":"2024-04-26T23:18:37.030065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:55:55.562630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029701+0000","last_change":"2024-04-26T23:18:36.051662+0000","last_active":"2024-04-26T23:18:37.029701+0000","last_peered":"2024-04-26T23:18:37.029701+0000","last_clean":"2024-04-26T23:18:37.029701+0000","last_became_active":"2024-04-26T23:18:36.051204+0000","last_became_peered":"2024-04-26T23:18:36.051204+0000","last_unstale":"2024-04-26T23:18:37.029701+0000","last_undegraded":"2024-04-26T23:18:37.029701+0000","last_fullsized":"2024-04-26T23:18:37.029701+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:24:24.209562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030279+0000","last_change":"2024-04-26T23:18:31.008954+0000","last_active":"2024-04-26T23:18:37.030279+0000","last_peered":"2024-04-26T23:18:37.030279+0000","last_clean":"2024-04-26T23:18:37.030279+0000","last_became_active":"2024-04-26T23:18:31.008597+0000","last_became_peered":"2024-04-26T23:18:31.008597+0000","last_unstale":"2024-04-26T23:18:37.030279+0000","last_undegraded":"2024-04-26T23:18:37.030279+0000","last_fullsized":"2024-04-26T23:18:37.030279+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:16:07.026710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044341+0000","last_change":"2024-04-26T23:18:32.019526+0000","last_active":"2024-04-26T23:18:37.044341+0000","last_peered":"2024-04-26T23:18:37.044341+0000","last_clean":"2024-04-26T23:18:37.044341+0000","last_became_active":"2024-04-26T23:18:32.019264+0000","last_became_peered":"2024-04-26T23:18:32.019264+0000","last_unstale":"2024-04-26T23:18:37.044341+0000","last_undegraded":"2024-04-26T23:18:37.044341+0000","last_fullsized":"2024-04-26T23:18:37.044341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:37:21.828417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044512+0000","last_change":"2024-04-26T23:18:36.041188+0000","last_active":"2024-04-26T23:18:37.044512+0000","last_peered":"2024-04-26T23:18:37.044512+0000","last_clean":"2024-04-26T23:18:37.044512+0000","last_became_active":"2024-04-26T23:18:36.040788+0000","last_became_peered":"2024-04-26T23:18:36.040788+0000","last_unstale":"2024-04-26T23:18:37.044512+0000","last_undegraded":"2024-04-26T23:18:37.044512+0000","last_fullsized":"2024-04-26T23:18:37.044512+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:49:51.239332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.047074+0000","last_change":"2024-04-26T23:18:31.011094+0000","last_active":"2024-04-26T23:18:37.047074+0000","last_peered":"2024-04-26T23:18:37.047074+0000","last_clean":"2024-04-26T23:18:37.047074+0000","last_became_active":"2024-04-26T23:18:31.010756+0000","last_became_peered":"2024-04-26T23:18:31.010756+0000","last_unstale":"2024-04-26T23:18:37.047074+0000","last_undegraded":"2024-04-26T23:18:37.047074+0000","last_fullsized":"2024-04-26T23:18:37.047074+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:28.986242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044870+0000","last_change":"2024-04-26T23:18:32.019790+0000","last_active":"2024-04-26T23:18:37.044870+0000","last_peered":"2024-04-26T23:18:37.044870+0000","last_clean":"2024-04-26T23:18:37.044870+0000","last_became_active":"2024-04-26T23:18:32.019535+0000","last_became_peered":"2024-04-26T23:18:32.019535+0000","last_unstale":"2024-04-26T23:18:37.044870+0000","last_undegraded":"2024-04-26T23:18:37.044870+0000","last_fullsized":"2024-04-26T23:18:37.044870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:10.370646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045799+0000","last_change":"2024-04-26T23:18:36.046954+0000","last_active":"2024-04-26T23:18:37.045799+0000","last_peered":"2024-04-26T23:18:37.045799+0000","last_clean":"2024-04-26T23:18:37.045799+0000","last_became_active":"2024-04-26T23:18:36.046588+0000","last_became_peered":"2024-04-26T23:18:36.046588+0000","last_unstale":"2024-04-26T23:18:37.045799+0000","last_undegraded":"2024-04-26T23:18:37.045799+0000","last_fullsized":"2024-04-26T23:18:37.045799+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:48:46.371866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046759+0000","last_change":"2024-04-26T23:18:31.011351+0000","last_active":"2024-04-26T23:18:37.046759+0000","last_peered":"2024-04-26T23:18:37.046759+0000","last_clean":"2024-04-26T23:18:37.046759+0000","last_became_active":"2024-04-26T23:18:31.010990+0000","last_became_peered":"2024-04-26T23:18:31.010990+0000","last_unstale":"2024-04-26T23:18:37.046759+0000","last_undegraded":"2024-04-26T23:18:37.046759+0000","last_fullsized":"2024-04-26T23:18:37.046759+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:08.243628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046200+0000","last_change":"2024-04-26T23:18:32.018999+0000","last_active":"2024-04-26T23:18:37.046200+0000","last_peered":"2024-04-26T23:18:37.046200+0000","last_clean":"2024-04-26T23:18:37.046200+0000","last_became_active":"2024-04-26T23:18:32.018713+0000","last_became_peered":"2024-04-26T23:18:32.018713+0000","last_unstale":"2024-04-26T23:18:37.046200+0000","last_undegraded":"2024-04-26T23:18:37.046200+0000","last_fullsized":"2024-04-26T23:18:37.046200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:39:01.950309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045585+0000","last_change":"2024-04-26T23:18:36.042244+0000","last_active":"2024-04-26T23:18:37.045585+0000","last_peered":"2024-04-26T23:18:37.045585+0000","last_clean":"2024-04-26T23:18:37.045585+0000","last_became_active":"2024-04-26T23:18:36.041833+0000","last_became_peered":"2024-04-26T23:18:36.041833+0000","last_unstale":"2024-04-26T23:18:37.045585+0000","last_undegraded":"2024-04-26T23:18:37.045585+0000","last_fullsized":"2024-04-26T23:18:37.045585+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:39:05.974905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045365+0000","last_change":"2024-04-26T23:18:32.017614+0000","last_active":"2024-04-26T23:18:37.045365+0000","last_peered":"2024-04-26T23:18:37.045365+0000","last_clean":"2024-04-26T23:18:37.045365+0000","last_became_active":"2024-04-26T23:18:32.017251+0000","last_became_peered":"2024-04-26T23:18:32.017251+0000","last_unstale":"2024-04-26T23:18:37.045365+0000","last_undegraded":"2024-04-26T23:18:37.045365+0000","last_fullsized":"2024-04-26T23:18:37.045365+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:37:21.828417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.047093+0000","last_change":"2024-04-26T23:18:31.011768+0000","last_active":"2024-04-26T23:18:37.047093+0000","last_peered":"2024-04-26T23:18:37.047093+0000","last_clean":"2024-04-26T23:18:37.047093+0000","last_became_active":"2024-04-26T23:18:31.011495+0000","last_became_peered":"2024-04-26T23:18:31.011495+0000","last_unstale":"2024-04-26T23:18:37.047093+0000","last_undegraded":"2024-04-26T23:18:37.047093+0000","last_fullsized":"2024-04-26T23:18:37.047093+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:25:38.515775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045600+0000","last_change":"2024-04-26T23:18:36.039079+0000","last_active":"2024-04-26T23:18:37.045600+0000","last_peered":"2024-04-26T23:18:37.045600+0000","last_clean":"2024-04-26T23:18:37.045600+0000","last_became_active":"2024-04-26T23:18:36.038665+0000","last_became_peered":"2024-04-26T23:18:36.038665+0000","last_unstale":"2024-04-26T23:18:37.045600+0000","last_undegraded":"2024-04-26T23:18:37.045600+0000","last_fullsized":"2024-04-26T23:18:37.045600+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:45:26.386750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029982+0000","last_change":"2024-04-26T23:18:32.012705+0000","last_active":"2024-04-26T23:18:37.029982+0000","last_peered":"2024-04-26T23:18:37.029982+0000","last_clean":"2024-04-26T23:18:37.029982+0000","last_became_active":"2024-04-26T23:18:32.012421+0000","last_became_peered":"2024-04-26T23:18:32.012421+0000","last_unstale":"2024-04-26T23:18:37.029982+0000","last_undegraded":"2024-04-26T23:18:37.029982+0000","last_fullsized":"2024-04-26T23:18:37.029982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:49:47.214737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030444+0000","last_change":"2024-04-26T23:18:31.011273+0000","last_active":"2024-04-26T23:18:37.030444+0000","last_peered":"2024-04-26T23:18:37.030444+0000","last_clean":"2024-04-26T23:18:37.030444+0000","last_became_active":"2024-04-26T23:18:31.010978+0000","last_became_peered":"2024-04-26T23:18:31.010978+0000","last_unstale":"2024-04-26T23:18:37.030444+0000","last_undegraded":"2024-04-26T23:18:37.030444+0000","last_fullsized":"2024-04-26T23:18:37.030444+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:13.713713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045147+0000","last_change":"2024-04-26T23:18:36.040606+0000","last_active":"2024-04-26T23:18:37.045147+0000","last_peered":"2024-04-26T23:18:37.045147+0000","last_clean":"2024-04-26T23:18:37.045147+0000","last_became_active":"2024-04-26T23:18:36.040058+0000","last_became_peered":"2024-04-26T23:18:36.040058+0000","last_unstale":"2024-04-26T23:18:37.045147+0000","last_undegraded":"2024-04-26T23:18:37.045147+0000","last_fullsized":"2024-04-26T23:18:37.045147+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:48:55.902259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030935+0000","last_change":"2024-04-26T23:18:32.015298+0000","last_active":"2024-04-26T23:18:37.030935+0000","last_peered":"2024-04-26T23:18:37.030935+0000","last_clean":"2024-04-26T23:18:37.030935+0000","last_became_active":"2024-04-26T23:18:32.014508+0000","last_became_peered":"2024-04-26T23:18:32.014508+0000","last_unstale":"2024-04-26T23:18:37.030935+0000","last_undegraded":"2024-04-26T23:18:37.030935+0000","last_fullsized":"2024-04-26T23:18:37.030935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:57:02.596299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029055+0000","last_change":"2024-04-26T23:18:31.008495+0000","last_active":"2024-04-26T23:18:37.029055+0000","last_peered":"2024-04-26T23:18:37.029055+0000","last_clean":"2024-04-26T23:18:37.029055+0000","last_became_active":"2024-04-26T23:18:31.008116+0000","last_became_peered":"2024-04-26T23:18:31.008116+0000","last_unstale":"2024-04-26T23:18:37.029055+0000","last_undegraded":"2024-04-26T23:18:37.029055+0000","last_fullsized":"2024-04-26T23:18:37.029055+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:34:02.747044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030423+0000","last_change":"2024-04-26T23:18:36.045871+0000","last_active":"2024-04-26T23:18:37.030423+0000","last_peered":"2024-04-26T23:18:37.030423+0000","last_clean":"2024-04-26T23:18:37.030423+0000","last_became_active":"2024-04-26T23:18:36.044888+0000","last_became_peered":"2024-04-26T23:18:36.044888+0000","last_unstale":"2024-04-26T23:18:37.030423+0000","last_undegraded":"2024-04-26T23:18:37.030423+0000","last_fullsized":"2024-04-26T23:18:37.030423+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:08:51.442254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045209+0000","last_change":"2024-04-26T23:18:32.015241+0000","last_active":"2024-04-26T23:18:37.045209+0000","last_peered":"2024-04-26T23:18:37.045209+0000","last_clean":"2024-04-26T23:18:37.045209+0000","last_became_active":"2024-04-26T23:18:32.013246+0000","last_became_peered":"2024-04-26T23:18:32.013246+0000","last_unstale":"2024-04-26T23:18:37.045209+0000","last_undegraded":"2024-04-26T23:18:37.045209+0000","last_fullsized":"2024-04-26T23:18:37.045209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:21.586522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045696+0000","last_change":"2024-04-26T23:18:31.015485+0000","last_active":"2024-04-26T23:18:37.045696+0000","last_peered":"2024-04-26T23:18:37.045696+0000","last_clean":"2024-04-26T23:18:37.045696+0000","last_became_active":"2024-04-26T23:18:31.015187+0000","last_became_peered":"2024-04-26T23:18:31.015187+0000","last_unstale":"2024-04-26T23:18:37.045696+0000","last_undegraded":"2024-04-26T23:18:37.045696+0000","last_fullsized":"2024-04-26T23:18:37.045696+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:53:15.812510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044493+0000","last_change":"2024-04-26T23:18:36.040493+0000","last_active":"2024-04-26T23:18:37.044493+0000","last_peered":"2024-04-26T23:18:37.044493+0000","last_clean":"2024-04-26T23:18:37.044493+0000","last_became_active":"2024-04-26T23:18:36.040040+0000","last_became_peered":"2024-04-26T23:18:36.040040+0000","last_unstale":"2024-04-26T23:18:37.044493+0000","last_undegraded":"2024-04-26T23:18:37.044493+0000","last_fullsized":"2024-04-26T23:18:37.044493+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:37:25.853012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044514+0000","last_change":"2024-04-26T23:18:32.001467+0000","last_active":"2024-04-26T23:18:37.044514+0000","last_peered":"2024-04-26T23:18:37.044514+0000","last_clean":"2024-04-26T23:18:37.044514+0000","last_became_active":"2024-04-26T23:18:32.001170+0000","last_became_peered":"2024-04-26T23:18:32.001170+0000","last_unstale":"2024-04-26T23:18:37.044514+0000","last_undegraded":"2024-04-26T23:18:37.044514+0000","last_fullsized":"2024-04-26T23:18:37.044514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:17:04.527690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046567+0000","last_change":"2024-04-26T23:18:31.014468+0000","last_active":"2024-04-26T23:18:37.046567+0000","last_peered":"2024-04-26T23:18:37.046567+0000","last_clean":"2024-04-26T23:18:37.046567+0000","last_became_active":"2024-04-26T23:18:31.014192+0000","last_became_peered":"2024-04-26T23:18:31.014192+0000","last_unstale":"2024-04-26T23:18:37.046567+0000","last_undegraded":"2024-04-26T23:18:37.046567+0000","last_fullsized":"2024-04-26T23:18:37.046567+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:20.581173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044988+0000","last_change":"2024-04-26T23:18:36.044805+0000","last_active":"2024-04-26T23:18:37.044988+0000","last_peered":"2024-04-26T23:18:37.044988+0000","last_clean":"2024-04-26T23:18:37.044988+0000","last_became_active":"2024-04-26T23:18:36.044409+0000","last_became_peered":"2024-04-26T23:18:36.044409+0000","last_unstale":"2024-04-26T23:18:37.044988+0000","last_undegraded":"2024-04-26T23:18:37.044988+0000","last_fullsized":"2024-04-26T23:18:37.044988+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:04:17.836184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030862+0000","last_change":"2024-04-26T23:18:32.018289+0000","last_active":"2024-04-26T23:18:37.030862+0000","last_peered":"2024-04-26T23:18:37.030862+0000","last_clean":"2024-04-26T23:18:37.030862+0000","last_became_active":"2024-04-26T23:18:32.017788+0000","last_became_peered":"2024-04-26T23:18:32.017788+0000","last_unstale":"2024-04-26T23:18:37.030862+0000","last_undegraded":"2024-04-26T23:18:37.030862+0000","last_fullsized":"2024-04-26T23:18:37.030862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:55.832329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030196+0000","last_change":"2024-04-26T23:18:31.009677+0000","last_active":"2024-04-26T23:18:37.030196+0000","last_peered":"2024-04-26T23:18:37.030196+0000","last_clean":"2024-04-26T23:18:37.030196+0000","last_became_active":"2024-04-26T23:18:31.009368+0000","last_became_peered":"2024-04-26T23:18:31.009368+0000","last_unstale":"2024-04-26T23:18:37.030196+0000","last_undegraded":"2024-04-26T23:18:37.030196+0000","last_fullsized":"2024-04-26T23:18:37.030196+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:28.986242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029507+0000","last_change":"2024-04-26T23:18:36.050766+0000","last_active":"2024-04-26T23:18:37.029507+0000","last_peered":"2024-04-26T23:18:37.029507+0000","last_clean":"2024-04-26T23:18:37.029507+0000","last_became_active":"2024-04-26T23:18:36.050091+0000","last_became_peered":"2024-04-26T23:18:36.050091+0000","last_unstale":"2024-04-26T23:18:37.029507+0000","last_undegraded":"2024-04-26T23:18:37.029507+0000","last_fullsized":"2024-04-26T23:18:37.029507+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:16:16.544972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029722+0000","last_change":"2024-04-26T23:18:32.007502+0000","last_active":"2024-04-26T23:18:37.029722+0000","last_peered":"2024-04-26T23:18:37.029722+0000","last_clean":"2024-04-26T23:18:37.029722+0000","last_became_active":"2024-04-26T23:18:32.006451+0000","last_became_peered":"2024-04-26T23:18:32.006451+0000","last_unstale":"2024-04-26T23:18:37.029722+0000","last_undegraded":"2024-04-26T23:18:37.029722+0000","last_fullsized":"2024-04-26T23:18:37.029722+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:55:55.562630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.047238+0000","last_change":"2024-04-26T23:18:31.012702+0000","last_active":"2024-04-26T23:18:37.047238+0000","last_peered":"2024-04-26T23:18:37.047238+0000","last_clean":"2024-04-26T23:18:37.047238+0000","last_became_active":"2024-04-26T23:18:31.012425+0000","last_became_peered":"2024-04-26T23:18:31.012425+0000","last_unstale":"2024-04-26T23:18:37.047238+0000","last_undegraded":"2024-04-26T23:18:37.047238+0000","last_fullsized":"2024-04-26T23:18:37.047238+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:08:31.793547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044060+0000","last_change":"2024-04-26T23:18:36.049119+0000","last_active":"2024-04-26T23:18:37.044060+0000","last_peered":"2024-04-26T23:18:37.044060+0000","last_clean":"2024-04-26T23:18:37.044060+0000","last_became_active":"2024-04-26T23:18:36.047621+0000","last_became_peered":"2024-04-26T23:18:36.047621+0000","last_unstale":"2024-04-26T23:18:37.044060+0000","last_undegraded":"2024-04-26T23:18:37.044060+0000","last_fullsized":"2024-04-26T23:18:37.044060+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:48:46.371866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046280+0000","last_change":"2024-04-26T23:18:32.021370+0000","last_active":"2024-04-26T23:18:37.046280+0000","last_peered":"2024-04-26T23:18:37.046280+0000","last_clean":"2024-04-26T23:18:37.046280+0000","last_became_active":"2024-04-26T23:18:32.020816+0000","last_became_peered":"2024-04-26T23:18:32.020816+0000","last_unstale":"2024-04-26T23:18:37.046280+0000","last_undegraded":"2024-04-26T23:18:37.046280+0000","last_fullsized":"2024-04-26T23:18:37.046280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:45:22.362155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029882+0000","last_change":"2024-04-26T23:18:31.007687+0000","last_active":"2024-04-26T23:18:37.029882+0000","last_peered":"2024-04-26T23:18:37.029882+0000","last_clean":"2024-04-26T23:18:37.029882+0000","last_became_active":"2024-04-26T23:18:31.006664+0000","last_became_peered":"2024-04-26T23:18:31.006664+0000","last_unstale":"2024-04-26T23:18:37.029882+0000","last_undegraded":"2024-04-26T23:18:37.029882+0000","last_fullsized":"2024-04-26T23:18:37.029882+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:44.844711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029232+0000","last_change":"2024-04-26T23:18:36.047590+0000","last_active":"2024-04-26T23:18:37.029232+0000","last_peered":"2024-04-26T23:18:37.029232+0000","last_clean":"2024-04-26T23:18:37.029232+0000","last_became_active":"2024-04-26T23:18:36.046796+0000","last_became_peered":"2024-04-26T23:18:36.046796+0000","last_unstale":"2024-04-26T23:18:37.029232+0000","last_undegraded":"2024-04-26T23:18:37.029232+0000","last_fullsized":"2024-04-26T23:18:37.029232+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:09.320477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046041+0000","last_change":"2024-04-26T23:18:32.002280+0000","last_active":"2024-04-26T23:18:37.046041+0000","last_peered":"2024-04-26T23:18:37.046041+0000","last_clean":"2024-04-26T23:18:37.046041+0000","last_became_active":"2024-04-26T23:18:32.002051+0000","last_became_peered":"2024-04-26T23:18:32.002051+0000","last_unstale":"2024-04-26T23:18:37.046041+0000","last_undegraded":"2024-04-26T23:18:37.046041+0000","last_fullsized":"2024-04-26T23:18:37.046041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:09.248977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044993+0000","last_change":"2024-04-26T23:18:31.011488+0000","last_active":"2024-04-26T23:18:37.044993+0000","last_peered":"2024-04-26T23:18:37.044993+0000","last_clean":"2024-04-26T23:18:37.044993+0000","last_became_active":"2024-04-26T23:18:31.010229+0000","last_became_peered":"2024-04-26T23:18:31.010229+0000","last_unstale":"2024-04-26T23:18:37.044993+0000","last_undegraded":"2024-04-26T23:18:37.044993+0000","last_fullsized":"2024-04-26T23:18:37.044993+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:25:38.515775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029825+0000","last_change":"2024-04-26T23:18:36.042997+0000","last_active":"2024-04-26T23:18:37.029825+0000","last_peered":"2024-04-26T23:18:37.029825+0000","last_clean":"2024-04-26T23:18:37.029825+0000","last_became_active":"2024-04-26T23:18:36.042510+0000","last_became_peered":"2024-04-26T23:18:36.042510+0000","last_unstale":"2024-04-26T23:18:37.029825+0000","last_undegraded":"2024-04-26T23:18:37.029825+0000","last_fullsized":"2024-04-26T23:18:37.029825+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:30.558000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.028944+0000","last_change":"2024-04-26T23:18:32.008615+0000","last_active":"2024-04-26T23:18:37.028944+0000","last_peered":"2024-04-26T23:18:37.028944+0000","last_clean":"2024-04-26T23:18:37.028944+0000","last_became_active":"2024-04-26T23:18:32.006670+0000","last_became_peered":"2024-04-26T23:18:32.006670+0000","last_unstale":"2024-04-26T23:18:37.028944+0000","last_undegraded":"2024-04-26T23:18:37.028944+0000","last_fullsized":"2024-04-26T23:18:37.028944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:54:25.468799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.027917+0000","last_change":"2024-04-26T23:18:31.005153+0000","last_active":"2024-04-26T23:18:37.027917+0000","last_peered":"2024-04-26T23:18:37.027917+0000","last_clean":"2024-04-26T23:18:37.027917+0000","last_became_active":"2024-04-26T23:18:31.004670+0000","last_became_peered":"2024-04-26T23:18:31.004670+0000","last_unstale":"2024-04-26T23:18:37.027917+0000","last_undegraded":"2024-04-26T23:18:37.027917+0000","last_fullsized":"2024-04-26T23:18:37.027917+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:34:02.747044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030347+0000","last_change":"2024-04-26T23:18:36.048181+0000","last_active":"2024-04-26T23:18:37.030347+0000","last_peered":"2024-04-26T23:18:37.030347+0000","last_clean":"2024-04-26T23:18:37.030347+0000","last_became_active":"2024-04-26T23:18:36.047762+0000","last_became_peered":"2024-04-26T23:18:36.047762+0000","last_unstale":"2024-04-26T23:18:37.030347+0000","last_undegraded":"2024-04-26T23:18:37.030347+0000","last_fullsized":"2024-04-26T23:18:37.030347+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:39:32.810976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.043710+0000","last_change":"2024-04-26T23:18:32.005125+0000","last_active":"2024-04-26T23:18:37.043710+0000","last_peered":"2024-04-26T23:18:37.043710+0000","last_clean":"2024-04-26T23:18:37.043710+0000","last_became_active":"2024-04-26T23:18:32.004826+0000","last_became_peered":"2024-04-26T23:18:32.004826+0000","last_unstale":"2024-04-26T23:18:37.043710+0000","last_undegraded":"2024-04-26T23:18:37.043710+0000","last_fullsized":"2024-04-26T23:18:37.043710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:53:16.817858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045815+0000","last_change":"2024-04-26T23:18:31.007674+0000","last_active":"2024-04-26T23:18:37.045815+0000","last_peered":"2024-04-26T23:18:37.045815+0000","last_clean":"2024-04-26T23:18:37.045815+0000","last_became_active":"2024-04-26T23:18:31.006787+0000","last_became_peered":"2024-04-26T23:18:31.006787+0000","last_unstale":"2024-04-26T23:18:37.045815+0000","last_undegraded":"2024-04-26T23:18:37.045815+0000","last_fullsized":"2024-04-26T23:18:37.045815+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:59:17.263468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044484+0000","last_change":"2024-04-26T23:18:36.046197+0000","last_active":"2024-04-26T23:18:37.044484+0000","last_peered":"2024-04-26T23:18:37.044484+0000","last_clean":"2024-04-26T23:18:37.044484+0000","last_became_active":"2024-04-26T23:18:36.045205+0000","last_became_peered":"2024-04-26T23:18:36.045205+0000","last_unstale":"2024-04-26T23:18:37.044484+0000","last_undegraded":"2024-04-26T23:18:37.044484+0000","last_fullsized":"2024-04-26T23:18:37.044484+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:39:32.810976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030371+0000","last_change":"2024-04-26T23:18:32.025774+0000","last_active":"2024-04-26T23:18:37.030371+0000","last_peered":"2024-04-26T23:18:37.030371+0000","last_clean":"2024-04-26T23:18:37.030371+0000","last_became_active":"2024-04-26T23:18:32.025487+0000","last_became_peered":"2024-04-26T23:18:32.025487+0000","last_unstale":"2024-04-26T23:18:37.030371+0000","last_undegraded":"2024-04-26T23:18:37.030371+0000","last_fullsized":"2024-04-26T23:18:37.030371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:04:13.811589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044628+0000","last_change":"2024-04-26T23:18:31.008744+0000","last_active":"2024-04-26T23:18:37.044628+0000","last_peered":"2024-04-26T23:18:37.044628+0000","last_clean":"2024-04-26T23:18:37.044628+0000","last_became_active":"2024-04-26T23:18:31.008449+0000","last_became_peered":"2024-04-26T23:18:31.008449+0000","last_unstale":"2024-04-26T23:18:37.044628+0000","last_undegraded":"2024-04-26T23:18:37.044628+0000","last_fullsized":"2024-04-26T23:18:37.044628+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:08.243628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045855+0000","last_change":"2024-04-26T23:18:18.862631+0000","last_active":"2024-04-26T23:18:37.045855+0000","last_peered":"2024-04-26T23:18:37.045855+0000","last_clean":"2024-04-26T23:18:37.045855+0000","last_became_active":"2024-04-26T23:18:18.861536+0000","last_became_peered":"2024-04-26T23:18:18.861536+0000","last_unstale":"2024-04-26T23:18:37.045855+0000","last_undegraded":"2024-04-26T23:18:37.045855+0000","last_fullsized":"2024-04-26T23:18:37.045855+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:17.808736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:17.808736+0000","last_clean_scrub_stamp":"2024-04-26T23:18:17.808736+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:59:26.359282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045887+0000","last_change":"2024-04-26T23:18:36.045014+0000","last_active":"2024-04-26T23:18:37.045887+0000","last_peered":"2024-04-26T23:18:37.045887+0000","last_clean":"2024-04-26T23:18:37.045887+0000","last_became_active":"2024-04-26T23:18:36.044467+0000","last_became_peered":"2024-04-26T23:18:36.044467+0000","last_unstale":"2024-04-26T23:18:37.045887+0000","last_undegraded":"2024-04-26T23:18:37.045887+0000","last_fullsized":"2024-04-26T23:18:37.045887+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:48:46.371866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046652+0000","last_change":"2024-04-26T23:18:32.030669+0000","last_active":"2024-04-26T23:18:37.046652+0000","last_peered":"2024-04-26T23:18:37.046652+0000","last_clean":"2024-04-26T23:18:37.046652+0000","last_became_active":"2024-04-26T23:18:32.030359+0000","last_became_peered":"2024-04-26T23:18:32.030359+0000","last_unstale":"2024-04-26T23:18:37.046652+0000","last_undegraded":"2024-04-26T23:18:37.046652+0000","last_fullsized":"2024-04-26T23:18:37.046652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:05.295882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.028346+0000","last_change":"2024-04-26T23:18:31.011343+0000","last_active":"2024-04-26T23:18:37.028346+0000","last_peered":"2024-04-26T23:18:37.028346+0000","last_clean":"2024-04-26T23:18:37.028346+0000","last_became_active":"2024-04-26T23:18:31.011011+0000","last_became_peered":"2024-04-26T23:18:31.011011+0000","last_unstale":"2024-04-26T23:18:37.028346+0000","last_undegraded":"2024-04-26T23:18:37.028346+0000","last_fullsized":"2024-04-26T23:18:37.028346+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:22.746946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045303+0000","last_change":"2024-04-26T23:18:36.045541+0000","last_active":"2024-04-26T23:18:37.045303+0000","last_peered":"2024-04-26T23:18:37.045303+0000","last_clean":"2024-04-26T23:18:37.045303+0000","last_became_active":"2024-04-26T23:18:36.045133+0000","last_became_peered":"2024-04-26T23:18:36.045133+0000","last_unstale":"2024-04-26T23:18:37.045303+0000","last_undegraded":"2024-04-26T23:18:37.045303+0000","last_fullsized":"2024-04-26T23:18:37.045303+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:45:26.386750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045276+0000","last_change":"2024-04-26T23:18:32.018614+0000","last_active":"2024-04-26T23:18:37.045276+0000","last_peered":"2024-04-26T23:18:37.045276+0000","last_clean":"2024-04-26T23:18:37.045276+0000","last_became_active":"2024-04-26T23:18:32.018088+0000","last_became_peered":"2024-04-26T23:18:32.018088+0000","last_unstale":"2024-04-26T23:18:37.045276+0000","last_undegraded":"2024-04-26T23:18:37.045276+0000","last_fullsized":"2024-04-26T23:18:37.045276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:26.533404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030223+0000","last_change":"2024-04-26T23:18:31.010377+0000","last_active":"2024-04-26T23:18:37.030223+0000","last_peered":"2024-04-26T23:18:37.030223+0000","last_clean":"2024-04-26T23:18:37.030223+0000","last_became_active":"2024-04-26T23:18:31.010084+0000","last_became_peered":"2024-04-26T23:18:31.010084+0000","last_unstale":"2024-04-26T23:18:37.030223+0000","last_undegraded":"2024-04-26T23:18:37.030223+0000","last_fullsized":"2024-04-26T23:18:37.030223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:08.243628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044284+0000","last_change":"2024-04-26T23:18:36.047754+0000","last_active":"2024-04-26T23:18:37.044284+0000","last_peered":"2024-04-26T23:18:37.044284+0000","last_clean":"2024-04-26T23:18:37.044284+0000","last_became_active":"2024-04-26T23:18:36.047105+0000","last_became_peered":"2024-04-26T23:18:36.047105+0000","last_unstale":"2024-04-26T23:18:37.044284+0000","last_undegraded":"2024-04-26T23:18:37.044284+0000","last_fullsized":"2024-04-26T23:18:37.044284+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:14.395241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.047247+0000","last_change":"2024-04-26T23:18:32.026649+0000","last_active":"2024-04-26T23:18:37.047247+0000","last_peered":"2024-04-26T23:18:37.047247+0000","last_clean":"2024-04-26T23:18:37.047247+0000","last_became_active":"2024-04-26T23:18:32.026014+0000","last_became_peered":"2024-04-26T23:18:32.026014+0000","last_unstale":"2024-04-26T23:18:37.047247+0000","last_undegraded":"2024-04-26T23:18:37.047247+0000","last_fullsized":"2024-04-26T23:18:37.047247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:08:47.417659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.028103+0000","last_change":"2024-04-26T23:18:31.006215+0000","last_active":"2024-04-26T23:18:37.028103+0000","last_peered":"2024-04-26T23:18:37.028103+0000","last_clean":"2024-04-26T23:18:37.028103+0000","last_became_active":"2024-04-26T23:18:31.005847+0000","last_became_peered":"2024-04-26T23:18:31.005847+0000","last_unstale":"2024-04-26T23:18:37.028103+0000","last_undegraded":"2024-04-26T23:18:37.028103+0000","last_fullsized":"2024-04-26T23:18:37.028103+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:44.844711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044109+0000","last_change":"2024-04-26T23:18:36.045812+0000","last_active":"2024-04-26T23:18:37.044109+0000","last_peered":"2024-04-26T23:18:37.044109+0000","last_clean":"2024-04-26T23:18:37.044109+0000","last_became_active":"2024-04-26T23:18:36.045445+0000","last_became_peered":"2024-04-26T23:18:36.045445+0000","last_unstale":"2024-04-26T23:18:37.044109+0000","last_undegraded":"2024-04-26T23:18:37.044109+0000","last_fullsized":"2024-04-26T23:18:37.044109+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:16:16.544972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046893+0000","last_change":"2024-04-26T23:18:32.025072+0000","last_active":"2024-04-26T23:18:37.046893+0000","last_peered":"2024-04-26T23:18:37.046893+0000","last_clean":"2024-04-26T23:18:37.046893+0000","last_became_active":"2024-04-26T23:18:32.024754+0000","last_became_peered":"2024-04-26T23:18:32.024754+0000","last_unstale":"2024-04-26T23:18:37.046893+0000","last_undegraded":"2024-04-26T23:18:37.046893+0000","last_fullsized":"2024-04-26T23:18:37.046893+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:45:22.362155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044833+0000","last_change":"2024-04-26T23:18:31.013340+0000","last_active":"2024-04-26T23:18:37.044833+0000","last_peered":"2024-04-26T23:18:37.044833+0000","last_clean":"2024-04-26T23:18:37.044833+0000","last_became_active":"2024-04-26T23:18:31.012824+0000","last_became_peered":"2024-04-26T23:18:31.012824+0000","last_unstale":"2024-04-26T23:18:37.044833+0000","last_undegraded":"2024-04-26T23:18:37.044833+0000","last_fullsized":"2024-04-26T23:18:37.044833+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:17.988227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044762+0000","last_change":"2024-04-26T23:18:36.041004+0000","last_active":"2024-04-26T23:18:37.044762+0000","last_peered":"2024-04-26T23:18:37.044762+0000","last_clean":"2024-04-26T23:18:37.044762+0000","last_became_active":"2024-04-26T23:18:36.040155+0000","last_became_peered":"2024-04-26T23:18:36.040155+0000","last_unstale":"2024-04-26T23:18:37.044762+0000","last_undegraded":"2024-04-26T23:18:37.044762+0000","last_fullsized":"2024-04-26T23:18:37.044762+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:59.856924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046692+0000","last_change":"2024-04-26T23:18:32.025214+0000","last_active":"2024-04-26T23:18:37.046692+0000","last_peered":"2024-04-26T23:18:37.046692+0000","last_clean":"2024-04-26T23:18:37.046692+0000","last_became_active":"2024-04-26T23:18:32.024952+0000","last_became_peered":"2024-04-26T23:18:32.024952+0000","last_unstale":"2024-04-26T23:18:37.046692+0000","last_undegraded":"2024-04-26T23:18:37.046692+0000","last_fullsized":"2024-04-26T23:18:37.046692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:39:01.950309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.027616+0000","last_change":"2024-04-26T23:18:31.001581+0000","last_active":"2024-04-26T23:18:37.027616+0000","last_peered":"2024-04-26T23:18:37.027616+0000","last_clean":"2024-04-26T23:18:37.027616+0000","last_became_active":"2024-04-26T23:18:31.001256+0000","last_became_peered":"2024-04-26T23:18:31.001256+0000","last_unstale":"2024-04-26T23:18:37.027616+0000","last_undegraded":"2024-04-26T23:18:37.027616+0000","last_fullsized":"2024-04-26T23:18:37.027616+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:13.713713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044283+0000","last_change":"2024-04-26T23:18:36.040673+0000","last_active":"2024-04-26T23:18:37.044283+0000","last_peered":"2024-04-26T23:18:37.044283+0000","last_clean":"2024-04-26T23:18:37.044283+0000","last_became_active":"2024-04-26T23:18:36.040246+0000","last_became_peered":"2024-04-26T23:18:36.040246+0000","last_unstale":"2024-04-26T23:18:37.044283+0000","last_undegraded":"2024-04-26T23:18:37.044283+0000","last_fullsized":"2024-04-26T23:18:37.044283+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:30.558000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044353+0000","last_change":"2024-04-26T23:18:32.013824+0000","last_active":"2024-04-26T23:18:37.044353+0000","last_peered":"2024-04-26T23:18:37.044353+0000","last_clean":"2024-04-26T23:18:37.044353+0000","last_became_active":"2024-04-26T23:18:32.012613+0000","last_became_peered":"2024-04-26T23:18:32.012613+0000","last_unstale":"2024-04-26T23:18:37.044353+0000","last_undegraded":"2024-04-26T23:18:37.044353+0000","last_fullsized":"2024-04-26T23:18:37.044353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:49:47.214737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044914+0000","last_change":"2024-04-26T23:18:31.011248+0000","last_active":"2024-04-26T23:18:37.044914+0000","last_peered":"2024-04-26T23:18:37.044914+0000","last_clean":"2024-04-26T23:18:37.044914+0000","last_became_active":"2024-04-26T23:18:31.010965+0000","last_became_peered":"2024-04-26T23:18:31.010965+0000","last_unstale":"2024-04-26T23:18:37.044914+0000","last_undegraded":"2024-04-26T23:18:37.044914+0000","last_fullsized":"2024-04-26T23:18:37.044914+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:13.713713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030962+0000","last_change":"2024-04-26T23:18:36.045330+0000","last_active":"2024-04-26T23:18:37.030962+0000","last_peered":"2024-04-26T23:18:37.030962+0000","last_clean":"2024-04-26T23:18:37.030962+0000","last_became_active":"2024-04-26T23:18:36.044903+0000","last_became_peered":"2024-04-26T23:18:36.044903+0000","last_unstale":"2024-04-26T23:18:37.030962+0000","last_undegraded":"2024-04-26T23:18:37.030962+0000","last_fullsized":"2024-04-26T23:18:37.030962+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:53:15.682467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.047083+0000","last_change":"2024-04-26T23:18:32.026417+0000","last_active":"2024-04-26T23:18:37.047083+0000","last_peered":"2024-04-26T23:18:37.047083+0000","last_clean":"2024-04-26T23:18:37.047083+0000","last_became_active":"2024-04-26T23:18:32.025839+0000","last_became_peered":"2024-04-26T23:18:32.025839+0000","last_unstale":"2024-04-26T23:18:37.047083+0000","last_undegraded":"2024-04-26T23:18:37.047083+0000","last_fullsized":"2024-04-26T23:18:37.047083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:39:28.786381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044878+0000","last_change":"2024-04-26T23:18:31.011956+0000","last_active":"2024-04-26T23:18:37.044878+0000","last_peered":"2024-04-26T23:18:37.044878+0000","last_clean":"2024-04-26T23:18:37.044878+0000","last_became_active":"2024-04-26T23:18:31.010493+0000","last_became_peered":"2024-04-26T23:18:31.010493+0000","last_unstale":"2024-04-26T23:18:37.044878+0000","last_undegraded":"2024-04-26T23:18:37.044878+0000","last_fullsized":"2024-04-26T23:18:37.044878+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:59:17.263468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030567+0000","last_change":"2024-04-26T23:18:36.047105+0000","last_active":"2024-04-26T23:18:37.030567+0000","last_peered":"2024-04-26T23:18:37.030567+0000","last_clean":"2024-04-26T23:18:37.030567+0000","last_became_active":"2024-04-26T23:18:36.046724+0000","last_became_peered":"2024-04-26T23:18:36.046724+0000","last_unstale":"2024-04-26T23:18:37.030567+0000","last_undegraded":"2024-04-26T23:18:37.030567+0000","last_fullsized":"2024-04-26T23:18:37.030567+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:46:07.146408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.043961+0000","last_change":"2024-04-26T23:18:32.015408+0000","last_active":"2024-04-26T23:18:37.043961+0000","last_peered":"2024-04-26T23:18:37.043961+0000","last_clean":"2024-04-26T23:18:37.043961+0000","last_became_active":"2024-04-26T23:18:32.015037+0000","last_became_peered":"2024-04-26T23:18:32.015037+0000","last_unstale":"2024-04-26T23:18:37.043961+0000","last_undegraded":"2024-04-26T23:18:37.043961+0000","last_fullsized":"2024-04-26T23:18:37.043961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:49.567077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044345+0000","last_change":"2024-04-26T23:18:31.010413+0000","last_active":"2024-04-26T23:18:37.044345+0000","last_peered":"2024-04-26T23:18:37.044345+0000","last_clean":"2024-04-26T23:18:37.044345+0000","last_became_active":"2024-04-26T23:18:31.010138+0000","last_became_peered":"2024-04-26T23:18:31.010138+0000","last_unstale":"2024-04-26T23:18:37.044345+0000","last_undegraded":"2024-04-26T23:18:37.044345+0000","last_fullsized":"2024-04-26T23:18:37.044345+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:15:53.470195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046231+0000","last_change":"2024-04-26T23:18:36.044782+0000","last_active":"2024-04-26T23:18:37.046231+0000","last_peered":"2024-04-26T23:18:37.046231+0000","last_clean":"2024-04-26T23:18:37.046231+0000","last_became_active":"2024-04-26T23:18:36.044157+0000","last_became_peered":"2024-04-26T23:18:36.044157+0000","last_unstale":"2024-04-26T23:18:37.046231+0000","last_undegraded":"2024-04-26T23:18:37.046231+0000","last_fullsized":"2024-04-26T23:18:37.046231+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:24:24.209562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046631+0000","last_change":"2024-04-26T23:18:32.023453+0000","last_active":"2024-04-26T23:18:37.046631+0000","last_peered":"2024-04-26T23:18:37.046631+0000","last_clean":"2024-04-26T23:18:37.046631+0000","last_became_active":"2024-04-26T23:18:32.023170+0000","last_became_peered":"2024-04-26T23:18:32.023170+0000","last_unstale":"2024-04-26T23:18:37.046631+0000","last_undegraded":"2024-04-26T23:18:37.046631+0000","last_fullsized":"2024-04-26T23:18:37.046631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:48:42.347270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:41.489012+0000","last_change":"2024-04-26T23:18:31.013165+0000","last_active":"2024-04-26T23:18:41.489012+0000","last_peered":"2024-04-26T23:18:41.489012+0000","last_clean":"2024-04-26T23:18:41.489012+0000","last_became_active":"2024-04-26T23:18:31.012790+0000","last_became_peered":"2024-04-26T23:18:31.012790+0000","last_unstale":"2024-04-26T23:18:41.489012+0000","last_undegraded":"2024-04-26T23:18:41.489012+0000","last_fullsized":"2024-04-26T23:18:41.489012+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:44.844711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029334+0000","last_change":"2024-04-26T23:18:36.047231+0000","last_active":"2024-04-26T23:18:37.029334+0000","last_peered":"2024-04-26T23:18:37.029334+0000","last_clean":"2024-04-26T23:18:37.029334+0000","last_became_active":"2024-04-26T23:18:36.046579+0000","last_became_peered":"2024-04-26T23:18:36.046579+0000","last_unstale":"2024-04-26T23:18:37.029334+0000","last_undegraded":"2024-04-26T23:18:37.029334+0000","last_fullsized":"2024-04-26T23:18:37.029334+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:14.395241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045390+0000","last_change":"2024-04-26T23:18:32.022720+0000","last_active":"2024-04-26T23:18:37.045390+0000","last_peered":"2024-04-26T23:18:37.045390+0000","last_clean":"2024-04-26T23:18:37.045390+0000","last_became_active":"2024-04-26T23:18:32.022450+0000","last_became_peered":"2024-04-26T23:18:32.022450+0000","last_unstale":"2024-04-26T23:18:37.045390+0000","last_undegraded":"2024-04-26T23:18:37.045390+0000","last_fullsized":"2024-04-26T23:18:37.045390+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:25:39.521124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029950+0000","last_change":"2024-04-26T23:18:31.008154+0000","last_active":"2024-04-26T23:18:37.029950+0000","last_peered":"2024-04-26T23:18:37.029950+0000","last_clean":"2024-04-26T23:18:37.029950+0000","last_became_active":"2024-04-26T23:18:31.007881+0000","last_became_peered":"2024-04-26T23:18:31.007881+0000","last_unstale":"2024-04-26T23:18:37.029950+0000","last_undegraded":"2024-04-26T23:18:37.029950+0000","last_fullsized":"2024-04-26T23:18:37.029950+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:20.581173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046058+0000","last_change":"2024-04-26T23:18:36.047010+0000","last_active":"2024-04-26T23:18:37.046058+0000","last_peered":"2024-04-26T23:18:37.046058+0000","last_clean":"2024-04-26T23:18:37.046058+0000","last_became_active":"2024-04-26T23:18:36.045988+0000","last_became_peered":"2024-04-26T23:18:36.045988+0000","last_unstale":"2024-04-26T23:18:37.046058+0000","last_undegraded":"2024-04-26T23:18:37.046058+0000","last_fullsized":"2024-04-26T23:18:37.046058+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:09.320477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044476+0000","last_change":"2024-04-26T23:18:32.019460+0000","last_active":"2024-04-26T23:18:37.044476+0000","last_peered":"2024-04-26T23:18:37.044476+0000","last_clean":"2024-04-26T23:18:37.044476+0000","last_became_active":"2024-04-26T23:18:32.019146+0000","last_became_peered":"2024-04-26T23:18:32.019146+0000","last_unstale":"2024-04-26T23:18:37.044476+0000","last_undegraded":"2024-04-26T23:18:37.044476+0000","last_fullsized":"2024-04-26T23:18:37.044476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:55:55.562630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045137+0000","last_change":"2024-04-26T23:18:31.010513+0000","last_active":"2024-04-26T23:18:37.045137+0000","last_peered":"2024-04-26T23:18:37.045137+0000","last_clean":"2024-04-26T23:18:37.045137+0000","last_became_active":"2024-04-26T23:18:31.010206+0000","last_became_peered":"2024-04-26T23:18:31.010206+0000","last_unstale":"2024-04-26T23:18:37.045137+0000","last_undegraded":"2024-04-26T23:18:37.045137+0000","last_fullsized":"2024-04-26T23:18:37.045137+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:17:03.522342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030875+0000","last_change":"2024-04-26T23:18:36.039670+0000","last_active":"2024-04-26T23:18:37.030875+0000","last_peered":"2024-04-26T23:18:37.030875+0000","last_clean":"2024-04-26T23:18:37.030875+0000","last_became_active":"2024-04-26T23:18:36.039247+0000","last_became_peered":"2024-04-26T23:18:36.039247+0000","last_unstale":"2024-04-26T23:18:37.030875+0000","last_undegraded":"2024-04-26T23:18:37.030875+0000","last_fullsized":"2024-04-26T23:18:37.030875+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:45:37.196730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045040+0000","last_change":"2024-04-26T23:18:32.029408+0000","last_active":"2024-04-26T23:18:37.045040+0000","last_peered":"2024-04-26T23:18:37.045040+0000","last_clean":"2024-04-26T23:18:37.045040+0000","last_became_active":"2024-04-26T23:18:32.029150+0000","last_became_peered":"2024-04-26T23:18:32.029150+0000","last_unstale":"2024-04-26T23:18:37.045040+0000","last_undegraded":"2024-04-26T23:18:37.045040+0000","last_fullsized":"2024-04-26T23:18:37.045040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:48:42.347270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046379+0000","last_change":"2024-04-26T23:18:31.013885+0000","last_active":"2024-04-26T23:18:37.046379+0000","last_peered":"2024-04-26T23:18:37.046379+0000","last_clean":"2024-04-26T23:18:37.046379+0000","last_became_active":"2024-04-26T23:18:31.013617+0000","last_became_peered":"2024-04-26T23:18:31.013617+0000","last_unstale":"2024-04-26T23:18:37.046379+0000","last_undegraded":"2024-04-26T23:18:37.046379+0000","last_fullsized":"2024-04-26T23:18:37.046379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:08.243628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029721+0000","last_change":"2024-04-26T23:18:36.049794+0000","last_active":"2024-04-26T23:18:37.029721+0000","last_peered":"2024-04-26T23:18:37.029721+0000","last_clean":"2024-04-26T23:18:37.029721+0000","last_became_active":"2024-04-26T23:18:36.044164+0000","last_became_peered":"2024-04-26T23:18:36.044164+0000","last_unstale":"2024-04-26T23:18:37.029721+0000","last_undegraded":"2024-04-26T23:18:37.029721+0000","last_fullsized":"2024-04-26T23:18:37.029721+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:41:38.190204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044157+0000","last_change":"2024-04-26T23:18:36.046931+0000","last_active":"2024-04-26T23:18:37.044157+0000","last_peered":"2024-04-26T23:18:37.044157+0000","last_clean":"2024-04-26T23:18:37.044157+0000","last_became_active":"2024-04-26T23:18:36.045786+0000","last_became_peered":"2024-04-26T23:18:36.045786+0000","last_unstale":"2024-04-26T23:18:37.044157+0000","last_undegraded":"2024-04-26T23:18:37.044157+0000","last_fullsized":"2024-04-26T23:18:37.044157+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:09.320477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.028930+0000","last_change":"2024-04-26T23:18:32.011300+0000","last_active":"2024-04-26T23:18:37.028930+0000","last_peered":"2024-04-26T23:18:37.028930+0000","last_clean":"2024-04-26T23:18:37.028930+0000","last_became_active":"2024-04-26T23:18:32.010998+0000","last_became_peered":"2024-04-26T23:18:32.010998+0000","last_unstale":"2024-04-26T23:18:37.028930+0000","last_undegraded":"2024-04-26T23:18:37.028930+0000","last_fullsized":"2024-04-26T23:18:37.028930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:57:02.596299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044365+0000","last_change":"2024-04-26T23:18:31.011198+0000","last_active":"2024-04-26T23:18:37.044365+0000","last_peered":"2024-04-26T23:18:37.044365+0000","last_clean":"2024-04-26T23:18:37.044365+0000","last_became_active":"2024-04-26T23:18:31.010908+0000","last_became_peered":"2024-04-26T23:18:31.010908+0000","last_unstale":"2024-04-26T23:18:37.044365+0000","last_undegraded":"2024-04-26T23:18:37.044365+0000","last_fullsized":"2024-04-26T23:18:37.044365+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:22.746946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046251+0000","last_change":"2024-04-26T23:18:36.042049+0000","last_active":"2024-04-26T23:18:37.046251+0000","last_peered":"2024-04-26T23:18:37.046251+0000","last_clean":"2024-04-26T23:18:37.046251+0000","last_became_active":"2024-04-26T23:18:36.041661+0000","last_became_peered":"2024-04-26T23:18:36.041661+0000","last_unstale":"2024-04-26T23:18:37.046251+0000","last_undegraded":"2024-04-26T23:18:37.046251+0000","last_fullsized":"2024-04-26T23:18:37.046251+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:09.320477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030801+0000","last_change":"2024-04-26T23:18:32.013285+0000","last_active":"2024-04-26T23:18:37.030801+0000","last_peered":"2024-04-26T23:18:37.030801+0000","last_clean":"2024-04-26T23:18:37.030801+0000","last_became_active":"2024-04-26T23:18:32.013013+0000","last_became_peered":"2024-04-26T23:18:32.013013+0000","last_unstale":"2024-04-26T23:18:37.030801+0000","last_undegraded":"2024-04-26T23:18:37.030801+0000","last_fullsized":"2024-04-26T23:18:37.030801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:54:25.468799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029792+0000","last_change":"2024-04-26T23:18:31.011691+0000","last_active":"2024-04-26T23:18:37.029792+0000","last_peered":"2024-04-26T23:18:37.029792+0000","last_clean":"2024-04-26T23:18:37.029792+0000","last_became_active":"2024-04-26T23:18:31.009521+0000","last_became_peered":"2024-04-26T23:18:31.009521+0000","last_unstale":"2024-04-26T23:18:37.029792+0000","last_undegraded":"2024-04-26T23:18:37.029792+0000","last_fullsized":"2024-04-26T23:18:37.029792+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:22.746946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045573+0000","last_change":"2024-04-26T23:18:31.010360+0000","last_active":"2024-04-26T23:18:37.045573+0000","last_peered":"2024-04-26T23:18:37.045573+0000","last_clean":"2024-04-26T23:18:37.045573+0000","last_became_active":"2024-04-26T23:18:31.010083+0000","last_became_peered":"2024-04-26T23:18:31.010083+0000","last_unstale":"2024-04-26T23:18:37.045573+0000","last_undegraded":"2024-04-26T23:18:37.045573+0000","last_fullsized":"2024-04-26T23:18:37.045573+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:34:02.747044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044258+0000","last_change":"2024-04-26T23:18:32.000871+0000","last_active":"2024-04-26T23:18:37.044258+0000","last_peered":"2024-04-26T23:18:37.044258+0000","last_clean":"2024-04-26T23:18:37.044258+0000","last_became_active":"2024-04-26T23:18:31.999865+0000","last_became_peered":"2024-04-26T23:18:31.999865+0000","last_unstale":"2024-04-26T23:18:37.044258+0000","last_undegraded":"2024-04-26T23:18:37.044258+0000","last_fullsized":"2024-04-26T23:18:37.044258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:57:02.596299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.028327+0000","last_change":"2024-04-26T23:18:36.040907+0000","last_active":"2024-04-26T23:18:37.028327+0000","last_peered":"2024-04-26T23:18:37.028327+0000","last_clean":"2024-04-26T23:18:37.028327+0000","last_became_active":"2024-04-26T23:18:36.040494+0000","last_became_peered":"2024-04-26T23:18:36.040494+0000","last_unstale":"2024-04-26T23:18:37.028327+0000","last_undegraded":"2024-04-26T23:18:37.028327+0000","last_fullsized":"2024-04-26T23:18:37.028327+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:17:08.552285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030078+0000","last_change":"2024-04-26T23:18:31.012855+0000","last_active":"2024-04-26T23:18:37.030078+0000","last_peered":"2024-04-26T23:18:37.030078+0000","last_clean":"2024-04-26T23:18:37.030078+0000","last_became_active":"2024-04-26T23:18:31.012510+0000","last_became_peered":"2024-04-26T23:18:31.012510+0000","last_unstale":"2024-04-26T23:18:37.030078+0000","last_undegraded":"2024-04-26T23:18:37.030078+0000","last_fullsized":"2024-04-26T23:18:37.030078+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:15:53.470195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045692+0000","last_change":"2024-04-26T23:18:32.002285+0000","last_active":"2024-04-26T23:18:37.045692+0000","last_peered":"2024-04-26T23:18:37.045692+0000","last_clean":"2024-04-26T23:18:37.045692+0000","last_became_active":"2024-04-26T23:18:32.001991+0000","last_became_peered":"2024-04-26T23:18:32.001991+0000","last_unstale":"2024-04-26T23:18:37.045692+0000","last_undegraded":"2024-04-26T23:18:37.045692+0000","last_fullsized":"2024-04-26T23:18:37.045692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:45.850060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030744+0000","last_change":"2024-04-26T23:18:36.038108+0000","last_active":"2024-04-26T23:18:37.030744+0000","last_peered":"2024-04-26T23:18:37.030744+0000","last_clean":"2024-04-26T23:18:37.030744+0000","last_became_active":"2024-04-26T23:18:36.037655+0000","last_became_peered":"2024-04-26T23:18:36.037655+0000","last_unstale":"2024-04-26T23:18:37.030744+0000","last_undegraded":"2024-04-26T23:18:37.030744+0000","last_fullsized":"2024-04-26T23:18:37.030744+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:02.473474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044156+0000","last_change":"2024-04-26T23:18:32.014058+0000","last_active":"2024-04-26T23:18:37.044156+0000","last_peered":"2024-04-26T23:18:37.044156+0000","last_clean":"2024-04-26T23:18:37.044156+0000","last_became_active":"2024-04-26T23:18:32.013500+0000","last_became_peered":"2024-04-26T23:18:32.013500+0000","last_unstale":"2024-04-26T23:18:37.044156+0000","last_undegraded":"2024-04-26T23:18:37.044156+0000","last_fullsized":"2024-04-26T23:18:37.044156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:54:25.468799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.027802+0000","last_change":"2024-04-26T23:18:32.018280+0000","last_active":"2024-04-26T23:18:37.027802+0000","last_peered":"2024-04-26T23:18:37.027802+0000","last_clean":"2024-04-26T23:18:37.027802+0000","last_became_active":"2024-04-26T23:18:32.017993+0000","last_became_peered":"2024-04-26T23:18:32.017993+0000","last_unstale":"2024-04-26T23:18:37.027802+0000","last_undegraded":"2024-04-26T23:18:37.027802+0000","last_fullsized":"2024-04-26T23:18:37.027802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:08:32.798896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044791+0000","last_change":"2024-04-26T23:18:32.022457+0000","last_active":"2024-04-26T23:18:37.044791+0000","last_peered":"2024-04-26T23:18:37.044791+0000","last_clean":"2024-04-26T23:18:37.044791+0000","last_became_active":"2024-04-26T23:18:32.021766+0000","last_became_peered":"2024-04-26T23:18:32.021766+0000","last_unstale":"2024-04-26T23:18:37.044791+0000","last_undegraded":"2024-04-26T23:18:37.044791+0000","last_fullsized":"2024-04-26T23:18:37.044791+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:48:51.877664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044133+0000","last_change":"2024-04-26T23:18:32.011541+0000","last_active":"2024-04-26T23:18:37.044133+0000","last_peered":"2024-04-26T23:18:37.044133+0000","last_clean":"2024-04-26T23:18:37.044133+0000","last_became_active":"2024-04-26T23:18:32.009252+0000","last_became_peered":"2024-04-26T23:18:32.009252+0000","last_unstale":"2024-04-26T23:18:37.044133+0000","last_undegraded":"2024-04-26T23:18:37.044133+0000","last_fullsized":"2024-04-26T23:18:37.044133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:54:25.468799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030659+0000","last_change":"2024-04-26T23:18:32.018905+0000","last_active":"2024-04-26T23:18:37.030659+0000","last_peered":"2024-04-26T23:18:37.030659+0000","last_clean":"2024-04-26T23:18:37.030659+0000","last_became_active":"2024-04-26T23:18:32.018586+0000","last_became_peered":"2024-04-26T23:18:32.018586+0000","last_unstale":"2024-04-26T23:18:37.030659+0000","last_undegraded":"2024-04-26T23:18:37.030659+0000","last_fullsized":"2024-04-26T23:18:37.030659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:37:21.828417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029798+0000","last_change":"2024-04-26T23:18:32.011775+0000","last_active":"2024-04-26T23:18:37.029798+0000","last_peered":"2024-04-26T23:18:37.029798+0000","last_clean":"2024-04-26T23:18:37.029798+0000","last_became_active":"2024-04-26T23:18:32.011443+0000","last_became_peered":"2024-04-26T23:18:32.011443+0000","last_unstale":"2024-04-26T23:18:37.029798+0000","last_undegraded":"2024-04-26T23:18:37.029798+0000","last_fullsized":"2024-04-26T23:18:37.029798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:49.567077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046666+0000","last_change":"2024-04-26T23:18:32.026884+0000","last_active":"2024-04-26T23:18:37.046666+0000","last_peered":"2024-04-26T23:18:37.046666+0000","last_clean":"2024-04-26T23:18:37.046666+0000","last_became_active":"2024-04-26T23:18:32.026579+0000","last_became_peered":"2024-04-26T23:18:32.026579+0000","last_unstale":"2024-04-26T23:18:37.046666+0000","last_undegraded":"2024-04-26T23:18:37.046666+0000","last_fullsized":"2024-04-26T23:18:37.046666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:55:55.562630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030747+0000","last_change":"2024-04-26T23:18:32.018063+0000","last_active":"2024-04-26T23:18:37.030747+0000","last_peered":"2024-04-26T23:18:37.030747+0000","last_clean":"2024-04-26T23:18:37.030747+0000","last_became_active":"2024-04-26T23:18:32.017430+0000","last_became_peered":"2024-04-26T23:18:32.017430+0000","last_unstale":"2024-04-26T23:18:37.030747+0000","last_undegraded":"2024-04-26T23:18:37.030747+0000","last_fullsized":"2024-04-26T23:18:37.030747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:48:51.877664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044241+0000","last_change":"2024-04-26T23:18:32.018383+0000","last_active":"2024-04-26T23:18:37.044241+0000","last_peered":"2024-04-26T23:18:37.044241+0000","last_clean":"2024-04-26T23:18:37.044241+0000","last_became_active":"2024-04-26T23:18:32.018101+0000","last_became_peered":"2024-04-26T23:18:32.018101+0000","last_unstale":"2024-04-26T23:18:37.044241+0000","last_undegraded":"2024-04-26T23:18:37.044241+0000","last_fullsized":"2024-04-26T23:18:37.044241+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:49:47.214737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029898+0000","last_change":"2024-04-26T23:18:32.020368+0000","last_active":"2024-04-26T23:18:37.029898+0000","last_peered":"2024-04-26T23:18:37.029898+0000","last_clean":"2024-04-26T23:18:37.029898+0000","last_became_active":"2024-04-26T23:18:32.019278+0000","last_became_peered":"2024-04-26T23:18:32.019278+0000","last_unstale":"2024-04-26T23:18:37.029898+0000","last_undegraded":"2024-04-26T23:18:37.029898+0000","last_fullsized":"2024-04-26T23:18:37.029898+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:51:42.870329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044697+0000","last_change":"2024-04-26T23:18:32.029085+0000","last_active":"2024-04-26T23:18:37.044697+0000","last_peered":"2024-04-26T23:18:37.044697+0000","last_clean":"2024-04-26T23:18:37.044697+0000","last_became_active":"2024-04-26T23:18:32.028769+0000","last_became_peered":"2024-04-26T23:18:32.028769+0000","last_unstale":"2024-04-26T23:18:37.044697+0000","last_undegraded":"2024-04-26T23:18:37.044697+0000","last_fullsized":"2024-04-26T23:18:37.044697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:55.832329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.043961+0000","last_change":"2024-04-26T23:18:32.014926+0000","last_active":"2024-04-26T23:18:37.043961+0000","last_peered":"2024-04-26T23:18:37.043961+0000","last_clean":"2024-04-26T23:18:37.043961+0000","last_became_active":"2024-04-26T23:18:32.014414+0000","last_became_peered":"2024-04-26T23:18:32.014414+0000","last_unstale":"2024-04-26T23:18:37.043961+0000","last_undegraded":"2024-04-26T23:18:37.043961+0000","last_fullsized":"2024-04-26T23:18:37.043961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:58:44.698418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045548+0000","last_change":"2024-04-26T23:18:32.019959+0000","last_active":"2024-04-26T23:18:37.045548+0000","last_peered":"2024-04-26T23:18:37.045548+0000","last_clean":"2024-04-26T23:18:37.045548+0000","last_became_active":"2024-04-26T23:18:32.019690+0000","last_became_peered":"2024-04-26T23:18:32.019690+0000","last_unstale":"2024-04-26T23:18:37.045548+0000","last_undegraded":"2024-04-26T23:18:37.045548+0000","last_fullsized":"2024-04-26T23:18:37.045548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:49.567077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.047090+0000","last_change":"2024-04-26T23:18:32.023444+0000","last_active":"2024-04-26T23:18:37.047090+0000","last_peered":"2024-04-26T23:18:37.047090+0000","last_clean":"2024-04-26T23:18:37.047090+0000","last_became_active":"2024-04-26T23:18:32.023211+0000","last_became_peered":"2024-04-26T23:18:32.023211+0000","last_unstale":"2024-04-26T23:18:37.047090+0000","last_undegraded":"2024-04-26T23:18:37.047090+0000","last_fullsized":"2024-04-26T23:18:37.047090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:57:19.300614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.027980+0000","last_change":"2024-04-26T23:18:32.011378+0000","last_active":"2024-04-26T23:18:37.027980+0000","last_peered":"2024-04-26T23:18:37.027980+0000","last_clean":"2024-04-26T23:18:37.027980+0000","last_became_active":"2024-04-26T23:18:32.011101+0000","last_became_peered":"2024-04-26T23:18:32.011101+0000","last_unstale":"2024-04-26T23:18:37.027980+0000","last_undegraded":"2024-04-26T23:18:37.027980+0000","last_fullsized":"2024-04-26T23:18:37.027980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:53:16.817858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045910+0000","last_change":"2024-04-26T23:18:32.028996+0000","last_active":"2024-04-26T23:18:37.045910+0000","last_peered":"2024-04-26T23:18:37.045910+0000","last_clean":"2024-04-26T23:18:37.045910+0000","last_became_active":"2024-04-26T23:18:32.028697+0000","last_became_peered":"2024-04-26T23:18:32.028697+0000","last_unstale":"2024-04-26T23:18:37.045910+0000","last_undegraded":"2024-04-26T23:18:37.045910+0000","last_fullsized":"2024-04-26T23:18:37.045910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:04:13.811589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044240+0000","last_change":"2024-04-26T23:18:32.018149+0000","last_active":"2024-04-26T23:18:37.044240+0000","last_peered":"2024-04-26T23:18:37.044240+0000","last_clean":"2024-04-26T23:18:37.044240+0000","last_became_active":"2024-04-26T23:18:32.017657+0000","last_became_peered":"2024-04-26T23:18:32.017657+0000","last_unstale":"2024-04-26T23:18:37.044240+0000","last_undegraded":"2024-04-26T23:18:37.044240+0000","last_fullsized":"2024-04-26T23:18:37.044240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:55:55.562630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029015+0000","last_change":"2024-04-26T23:18:32.000678+0000","last_active":"2024-04-26T23:18:37.029015+0000","last_peered":"2024-04-26T23:18:37.029015+0000","last_clean":"2024-04-26T23:18:37.029015+0000","last_became_active":"2024-04-26T23:18:32.000354+0000","last_became_peered":"2024-04-26T23:18:32.000354+0000","last_unstale":"2024-04-26T23:18:37.029015+0000","last_undegraded":"2024-04-26T23:18:37.029015+0000","last_fullsized":"2024-04-26T23:18:37.029015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:51:42.870329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.043891+0000","last_change":"2024-04-26T23:18:31.999586+0000","last_active":"2024-04-26T23:18:37.043891+0000","last_peered":"2024-04-26T23:18:37.043891+0000","last_clean":"2024-04-26T23:18:37.043891+0000","last_became_active":"2024-04-26T23:18:31.999185+0000","last_became_peered":"2024-04-26T23:18:31.999185+0000","last_unstale":"2024-04-26T23:18:37.043891+0000","last_undegraded":"2024-04-26T23:18:37.043891+0000","last_fullsized":"2024-04-26T23:18:37.043891+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:51:42.870329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029862+0000","last_change":"2024-04-26T23:18:32.024681+0000","last_active":"2024-04-26T23:18:37.029862+0000","last_peered":"2024-04-26T23:18:37.029862+0000","last_clean":"2024-04-26T23:18:37.029862+0000","last_became_active":"2024-04-26T23:18:32.024372+0000","last_became_peered":"2024-04-26T23:18:32.024372+0000","last_unstale":"2024-04-26T23:18:37.029862+0000","last_undegraded":"2024-04-26T23:18:37.029862+0000","last_fullsized":"2024-04-26T23:18:37.029862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:49.567077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045305+0000","last_change":"2024-04-26T23:18:32.011333+0000","last_active":"2024-04-26T23:18:37.045305+0000","last_peered":"2024-04-26T23:18:37.045305+0000","last_clean":"2024-04-26T23:18:37.045305+0000","last_became_active":"2024-04-26T23:18:32.011052+0000","last_became_peered":"2024-04-26T23:18:32.011052+0000","last_unstale":"2024-04-26T23:18:37.045305+0000","last_undegraded":"2024-04-26T23:18:37.045305+0000","last_fullsized":"2024-04-26T23:18:37.045305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:29.991591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046025+0000","last_change":"2024-04-26T23:18:32.021692+0000","last_active":"2024-04-26T23:18:37.046025+0000","last_peered":"2024-04-26T23:18:37.046025+0000","last_clean":"2024-04-26T23:18:37.046025+0000","last_became_active":"2024-04-26T23:18:32.021426+0000","last_became_peered":"2024-04-26T23:18:32.021426+0000","last_unstale":"2024-04-26T23:18:37.046025+0000","last_undegraded":"2024-04-26T23:18:37.046025+0000","last_fullsized":"2024-04-26T23:18:37.046025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:57:02.596299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046525+0000","last_change":"2024-04-26T23:18:32.020976+0000","last_active":"2024-04-26T23:18:37.046525+0000","last_peered":"2024-04-26T23:18:37.046525+0000","last_clean":"2024-04-26T23:18:37.046525+0000","last_became_active":"2024-04-26T23:18:32.020653+0000","last_became_peered":"2024-04-26T23:18:32.020653+0000","last_unstale":"2024-04-26T23:18:37.046525+0000","last_undegraded":"2024-04-26T23:18:37.046525+0000","last_fullsized":"2024-04-26T23:18:37.046525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:51:42.870329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.047206+0000","last_change":"2024-04-26T23:18:32.030726+0000","last_active":"2024-04-26T23:18:37.047206+0000","last_peered":"2024-04-26T23:18:37.047206+0000","last_clean":"2024-04-26T23:18:37.047206+0000","last_became_active":"2024-04-26T23:18:32.030451+0000","last_became_peered":"2024-04-26T23:18:32.030451+0000","last_unstale":"2024-04-26T23:18:37.047206+0000","last_undegraded":"2024-04-26T23:18:37.047206+0000","last_fullsized":"2024-04-26T23:18:37.047206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:26.533404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030676+0000","last_change":"2024-04-26T23:18:32.020062+0000","last_active":"2024-04-26T23:18:37.030676+0000","last_peered":"2024-04-26T23:18:37.030676+0000","last_clean":"2024-04-26T23:18:37.030676+0000","last_became_active":"2024-04-26T23:18:32.019651+0000","last_became_peered":"2024-04-26T23:18:32.019651+0000","last_unstale":"2024-04-26T23:18:37.030676+0000","last_undegraded":"2024-04-26T23:18:37.030676+0000","last_fullsized":"2024-04-26T23:18:37.030676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:17:04.527690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046652+0000","last_change":"2024-04-26T23:18:32.017808+0000","last_active":"2024-04-26T23:18:37.046652+0000","last_peered":"2024-04-26T23:18:37.046652+0000","last_clean":"2024-04-26T23:18:37.046652+0000","last_became_active":"2024-04-26T23:18:32.017162+0000","last_became_peered":"2024-04-26T23:18:32.017162+0000","last_unstale":"2024-04-26T23:18:37.046652+0000","last_undegraded":"2024-04-26T23:18:37.046652+0000","last_fullsized":"2024-04-26T23:18:37.046652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:48:51.877664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044589+0000","last_change":"2024-04-26T23:18:32.023064+0000","last_active":"2024-04-26T23:18:37.044589+0000","last_peered":"2024-04-26T23:18:37.044589+0000","last_clean":"2024-04-26T23:18:37.044589+0000","last_became_active":"2024-04-26T23:18:32.022771+0000","last_became_peered":"2024-04-26T23:18:32.022771+0000","last_unstale":"2024-04-26T23:18:37.044589+0000","last_undegraded":"2024-04-26T23:18:37.044589+0000","last_fullsized":"2024-04-26T23:18:37.044589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:37:21.828417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046963+0000","last_change":"2024-04-26T23:18:32.022130+0000","last_active":"2024-04-26T23:18:37.046963+0000","last_peered":"2024-04-26T23:18:37.046963+0000","last_clean":"2024-04-26T23:18:37.046963+0000","last_became_active":"2024-04-26T23:18:32.021842+0000","last_became_peered":"2024-04-26T23:18:32.021842+0000","last_unstale":"2024-04-26T23:18:37.046963+0000","last_undegraded":"2024-04-26T23:18:37.046963+0000","last_fullsized":"2024-04-26T23:18:37.046963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:10.370646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044765+0000","last_change":"2024-04-26T23:18:32.020676+0000","last_active":"2024-04-26T23:18:37.044765+0000","last_peered":"2024-04-26T23:18:37.044765+0000","last_clean":"2024-04-26T23:18:37.044765+0000","last_became_active":"2024-04-26T23:18:32.020391+0000","last_became_peered":"2024-04-26T23:18:32.020391+0000","last_unstale":"2024-04-26T23:18:37.044765+0000","last_undegraded":"2024-04-26T23:18:37.044765+0000","last_fullsized":"2024-04-26T23:18:37.044765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:05.295882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044140+0000","last_change":"2024-04-26T23:18:32.006201+0000","last_active":"2024-04-26T23:18:37.044140+0000","last_peered":"2024-04-26T23:18:37.044140+0000","last_clean":"2024-04-26T23:18:37.044140+0000","last_became_active":"2024-04-26T23:18:32.005906+0000","last_became_peered":"2024-04-26T23:18:32.005906+0000","last_unstale":"2024-04-26T23:18:37.044140+0000","last_undegraded":"2024-04-26T23:18:37.044140+0000","last_fullsized":"2024-04-26T23:18:37.044140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:59:18.268816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030075+0000","last_change":"2024-04-26T23:18:32.003546+0000","last_active":"2024-04-26T23:18:37.030075+0000","last_peered":"2024-04-26T23:18:37.030075+0000","last_clean":"2024-04-26T23:18:37.030075+0000","last_became_active":"2024-04-26T23:18:32.003204+0000","last_became_peered":"2024-04-26T23:18:32.003204+0000","last_unstale":"2024-04-26T23:18:37.030075+0000","last_undegraded":"2024-04-26T23:18:37.030075+0000","last_fullsized":"2024-04-26T23:18:37.030075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:37:21.828417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.047079+0000","last_change":"2024-04-26T23:18:32.026678+0000","last_active":"2024-04-26T23:18:37.047079+0000","last_peered":"2024-04-26T23:18:37.047079+0000","last_clean":"2024-04-26T23:18:37.047079+0000","last_became_active":"2024-04-26T23:18:32.026390+0000","last_became_peered":"2024-04-26T23:18:32.026390+0000","last_unstale":"2024-04-26T23:18:37.047079+0000","last_undegraded":"2024-04-26T23:18:37.047079+0000","last_fullsized":"2024-04-26T23:18:37.047079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:16:12.520377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542153,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":847872,"data_stored":690086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542155,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":262144,"data_stored":102978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542154,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":99226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542154,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":99262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31432,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340408,"statfs":{"total":96636764160,"available":96604577792,"internally_reserved":0,"allocated":262144,"data_stored":102978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":249856,"data_stored":99136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31432,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340408,"statfs":{"total":96636764160,"available":96604577792,"internally_reserved":0,"allocated":258048,"data_stored":99296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":856064,"data_stored":690142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:18:55.078 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T23:18:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.213+0000 7f9fae07b640 1 Processor -- start 2024-04-26T23:18:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.213+0000 7f9fae07b640 1 -- start start 2024-04-26T23:18:55.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.214+0000 7f9fae07b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9fa80b5f20 0x7f9fa80d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:55.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.214+0000 7f9fae07b640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fa80b7e50 con 0x7f9fa8162880 2024-04-26T23:18:55.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.214+0000 7f9fae07b640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fa80d1800 con 0x7f9fa80d3bc0 2024-04-26T23:18:55.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.214+0000 7f9fae07b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f9fa80d0580 con 0x7f9fa80b5f20 2024-04-26T23:18:55.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.214+0000 7f9fa77fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9fa8162880 0x7f9fa80d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:54520/0 (socket says 172.21.15.47:54520) 2024-04-26T23:18:55.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.214+0000 7f9fa77fe640 1 -- 172.21.15.47:0/3888184450 learned_addr learned my addr 172.21.15.47:0/3888184450 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:18:55.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.214+0000 7f9fa7fff640 1 --2- 172.21.15.47:0/3888184450 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9fa80b5f20 0x7f9fa80d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:55.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.214+0000 7f9fa67fc640 1 -- 172.21.15.47:0/3888184450 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2736588967 0 0) 0x7f9fa80b7e50 con 0x7f9fa8162880 2024-04-26T23:18:55.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.214+0000 7f9fa67fc640 1 -- 172.21.15.47:0/3888184450 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f90003520 con 0x7f9fa8162880 2024-04-26T23:18:55.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.215+0000 7f9fa67fc640 1 -- 172.21.15.47:0/3888184450 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3220207981 0 0) 0x7f9f9c001110 con 0x7f9fa80d3bc0 2024-04-26T23:18:55.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.215+0000 7f9fa67fc640 1 -- 172.21.15.47:0/3888184450 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9fa80b7e50 con 0x7f9fa80d3bc0 2024-04-26T23:18:55.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.215+0000 7f9fa67fc640 1 -- 172.21.15.47:0/3888184450 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 675361409 0 0) 0x7f9f90003520 con 0x7f9fa8162880 2024-04-26T23:18:55.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.215+0000 7f9fa67fc640 1 -- 172.21.15.47:0/3888184450 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9fa80b5f20 msgr2=0x7f9fa80d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:55.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.215+0000 7f9fa67fc640 1 --2- 172.21.15.47:0/3888184450 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9fa80b5f20 0x7f9fa80d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.215+0000 7f9fa67fc640 1 -- 172.21.15.47:0/3888184450 >> v1:172.21.15.203:6789/0 conn(0x7f9fa80d3bc0 legacy=0x7f9fa80b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.215+0000 7f9fa67fc640 1 -- 172.21.15.47:0/3888184450 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fa80d07d0 con 0x7f9fa8162880 2024-04-26T23:18:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.215+0000 7f9fa67fc640 1 -- 172.21.15.47:0/3888184450 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7f9f98001670 con 0x7f9fa8162880 2024-04-26T23:18:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.215+0000 7f9fa67fc640 1 -- 172.21.15.47:0/3888184450 >> v1:172.21.15.47:6789/0 conn(0x7f9fa8162880 legacy=0x7f9fa80d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.215+0000 7f9fa67fc640 1 --2- 172.21.15.47:0/3888184450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f90003b70 0x7f9f90003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.215+0000 7f9fa67fc640 1 --2- 172.21.15.47:0/3888184450 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9fa80b5f20 0x7f9f90004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.215+0000 7f9fa67fc640 1 --2- 172.21.15.47:0/3888184450 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f90004d40 0x7f9f900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.215+0000 7f9fa67fc640 1 -- 172.21.15.47:0/3888184450 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa80d0580 con 0x7f9f90003b70 2024-04-26T23:18:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.215+0000 7f9fa67fc640 1 -- 172.21.15.47:0/3888184450 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9f9000ba70 con 0x7f9f90004d40 2024-04-26T23:18:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.215+0000 7f9fa67fc640 1 -- 172.21.15.47:0/3888184450 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f9f9000bc70 con 0x7f9fa80b5f20 2024-04-26T23:18:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.215+0000 7f9fa77fe640 1 --2- 172.21.15.47:0/3888184450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f90003b70 0x7f9f90003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.215+0000 7f9fa6ffd640 1 --2- 172.21.15.47:0/3888184450 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9fa80b5f20 0x7f9f90004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.215+0000 7f9fa7fff640 1 --2- 172.21.15.47:0/3888184450 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f90004d40 0x7f9f900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.215+0000 7f9fa77fe640 1 -- 172.21.15.47:0/3888184450 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9fa80b5f20 msgr2=0x7f9f90004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:55.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.215+0000 7f9fa77fe640 1 --2- 172.21.15.47:0/3888184450 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9fa80b5f20 0x7f9f90004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:55.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.215+0000 7f9fa77fe640 1 -- 172.21.15.47:0/3888184450 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f90004d40 msgr2=0x7f9f900051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:55.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.215+0000 7f9fa77fe640 1 --2- 172.21.15.47:0/3888184450 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f90004d40 0x7f9f900051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:55.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.215+0000 7f9fa77fe640 1 -- 172.21.15.47:0/3888184450 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f90003520 con 0x7f9f90003b70 2024-04-26T23:18:55.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.215+0000 7f9fa7fff640 1 --2- 172.21.15.47:0/3888184450 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f90004d40 0x7f9f900051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:18:55.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.216+0000 7f9fa77fe640 1 --2- 172.21.15.47:0/3888184450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f90003b70 0x7f9f90003f90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9f98007fd0 tx=0x7f9f90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2719847d5c2df53 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:55.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.216+0000 7f9fa67fc640 1 -- 172.21.15.47:0/3888184450 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f98007ca0 con 0x7f9f90003b70 2024-04-26T23:18:55.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.216+0000 7f9fa67fc640 1 -- 172.21.15.47:0/3888184450 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f98005ad0 con 0x7f9f90003b70 2024-04-26T23:18:55.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.216+0000 7f9fa67fc640 1 -- 172.21.15.47:0/3888184450 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f98007070 con 0x7f9f90003b70 2024-04-26T23:18:55.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.216+0000 7f9fae07b640 1 -- 172.21.15.47:0/3888184450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f90003b70 msgr2=0x7f9f90003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:55.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.216+0000 7f9fae07b640 1 --2- 172.21.15.47:0/3888184450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f90003b70 0x7f9f90003f90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9f98007fd0 tx=0x7f9f90005830 comp rx=0 tx=0).stop 2024-04-26T23:18:55.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.216+0000 7f9fa7fff640 1 -- 172.21.15.47:0/3888184450 reap_dead start 2024-04-26T23:18:55.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.217+0000 7f9fae07b640 1 -- 172.21.15.47:0/3888184450 shutdown_connections 2024-04-26T23:18:55.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.217+0000 7f9fae07b640 1 -- 172.21.15.47:0/3888184450 >> 172.21.15.47:0/3888184450 conn(0x7f9fa80d4f60 msgr2=0x7f9fa80d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:55.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.217+0000 7f9fae07b640 1 -- 172.21.15.47:0/3888184450 shutdown_connections 2024-04-26T23:18:55.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.217+0000 7f9fae07b640 1 -- 172.21.15.47:0/3888184450 wait complete. 2024-04-26T23:18:55.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.218+0000 7f9fae07b640 1 Processor -- start 2024-04-26T23:18:55.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.218+0000 7f9fae07b640 1 -- start start 2024-04-26T23:18:55.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.219+0000 7f9fae07b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f90003b70 0x7f9fa806cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:55.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.219+0000 7f9fae07b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9fa806d2e0 0x7f9fa81f88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:55.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.219+0000 7f9fae07b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fa81f8e00 0x7f9fa80cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.219+0000 7f9fae07b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa8162c60 con 0x7f9f90003b70 2024-04-26T23:18:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.219+0000 7f9fae07b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa80d0d90 con 0x7f9fa81f8e00 2024-04-26T23:18:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.219+0000 7f9fae07b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f9fa8162de0 con 0x7f9fa806d2e0 2024-04-26T23:18:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.219+0000 7f9fa77fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f90003b70 0x7f9fa806cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.219+0000 7f9fa77fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f90003b70 0x7f9fa806cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58546/0 (socket says 172.21.15.47:58546) 2024-04-26T23:18:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.219+0000 7f9fa77fe640 1 -- 172.21.15.47:0/157940504 learned_addr learned my addr 172.21.15.47:0/157940504 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.219+0000 7f9fa6ffd640 1 --2- 172.21.15.47:0/157940504 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9fa806d2e0 0x7f9fa81f88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.219+0000 7f9fa7fff640 1 --2- 172.21.15.47:0/157940504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fa81f8e00 0x7f9fa80cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.219+0000 7f9fa77fe640 1 -- 172.21.15.47:0/157940504 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9fa806d2e0 msgr2=0x7f9fa81f88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.219+0000 7f9fa77fe640 1 --2- 172.21.15.47:0/157940504 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9fa806d2e0 0x7f9fa81f88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.219+0000 7f9fa77fe640 1 -- 172.21.15.47:0/157940504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fa81f8e00 msgr2=0x7f9fa80cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.219+0000 7f9fa77fe640 1 --2- 172.21.15.47:0/157940504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fa81f8e00 0x7f9fa80cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.219+0000 7f9fa77fe640 1 -- 172.21.15.47:0/157940504 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fa81f5440 con 0x7f9f90003b70 2024-04-26T23:18:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.219+0000 7f9fa6ffd640 1 --2- 172.21.15.47:0/157940504 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9fa806d2e0 0x7f9fa81f88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:18:55.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.219+0000 7f9fa7fff640 1 --2- 172.21.15.47:0/157940504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fa81f8e00 0x7f9fa80cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:18:55.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.219+0000 7f9fa77fe640 1 --2- 172.21.15.47:0/157940504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f90003b70 0x7f9fa806cda0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9f98000c00 tx=0x7f9fa80b6830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:55.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.219+0000 7f9f8ffff640 1 -- 172.21.15.47:0/157940504 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f98041020 con 0x7f9f90003b70 2024-04-26T23:18:55.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.219+0000 7f9f8ffff640 1 -- 172.21.15.47:0/157940504 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f98038040 con 0x7f9f90003b70 2024-04-26T23:18:55.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.219+0000 7f9fae07b640 1 -- 172.21.15.47:0/157940504 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fa81f5720 con 0x7f9f90003b70 2024-04-26T23:18:55.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.219+0000 7f9f8ffff640 1 -- 172.21.15.47:0/157940504 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f98001670 con 0x7f9f90003b70 2024-04-26T23:18:55.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.220+0000 7f9fae07b640 1 -- 172.21.15.47:0/157940504 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fa81f5ce0 con 0x7f9f90003b70 2024-04-26T23:18:55.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.220+0000 7f9fae07b640 1 -- 172.21.15.47:0/157940504 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fa80d3100 con 0x7f9f90003b70 2024-04-26T23:18:55.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.221+0000 7f9f8ffff640 1 -- 172.21.15.47:0/157940504 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9f98044020 con 0x7f9f90003b70 2024-04-26T23:18:55.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.221+0000 7f9f8ffff640 1 --2- 172.21.15.47:0/157940504 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9f9406c920 0x7f9f9406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:55.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.221+0000 7f9f8ffff640 1 -- 172.21.15.47:0/157940504 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f9f980ac190 con 0x7f9f90003b70 2024-04-26T23:18:55.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.222+0000 7f9fa6ffd640 1 --2- 172.21.15.47:0/157940504 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9f9406c920 0x7f9f9406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:55.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.222+0000 7f9fa6ffd640 1 --2- 172.21.15.47:0/157940504 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9f9406c920 0x7f9f9406ede0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9fa80b2bd0 tx=0x7f9f9c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:55.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.226+0000 7f9f8ffff640 1 -- 172.21.15.47:0/157940504 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9fa80d3100 con 0x7f9f90003b70 2024-04-26T23:18:55.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.407+0000 7f9fae07b640 1 -- 172.21.15.47:0/157940504 --> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9fa81f6050 con 0x7f9f9406c920 2024-04-26T23:18:55.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.416+0000 7f9f8ffff640 1 -- 172.21.15.47:0/157940504 <== mgr.4099 v2:172.21.15.203:6832/992162732 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332530 (secure 0 0 0) 0x7f9fa81f6050 con 0x7f9f9406c920 2024-04-26T23:18:55.420 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:18:55.421 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-26T23:18:55.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.419+0000 7f9fae07b640 1 -- 172.21.15.47:0/157940504 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9f9406c920 msgr2=0x7f9f9406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:55.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.419+0000 7f9fae07b640 1 --2- 172.21.15.47:0/157940504 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9f9406c920 0x7f9f9406ede0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9fa80b2bd0 tx=0x7f9f9c009040 comp rx=0 tx=0).stop 2024-04-26T23:18:55.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.419+0000 7f9fae07b640 1 -- 172.21.15.47:0/157940504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f90003b70 msgr2=0x7f9fa806cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:55.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.419+0000 7f9fae07b640 1 --2- 172.21.15.47:0/157940504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f90003b70 0x7f9fa806cda0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9f98000c00 tx=0x7f9fa80b6830 comp rx=0 tx=0).stop 2024-04-26T23:18:55.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.419+0000 7f9fae07b640 1 -- 172.21.15.47:0/157940504 shutdown_connections 2024-04-26T23:18:55.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.419+0000 7f9fae07b640 1 --2- 172.21.15.47:0/157940504 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9f9406c920 0x7f9f9406ede0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:55.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.419+0000 7f9fae07b640 1 --2- 172.21.15.47:0/157940504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fa81f8e00 0x7f9fa80cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:55.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.419+0000 7f9fae07b640 1 --2- 172.21.15.47:0/157940504 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9fa806d2e0 0x7f9fa81f88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:55.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.419+0000 7f9fae07b640 1 --2- 172.21.15.47:0/157940504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f90003b70 0x7f9fa806cda0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:55.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.419+0000 7f9fae07b640 1 -- 172.21.15.47:0/157940504 >> 172.21.15.47:0/157940504 conn(0x7f9fa80d4f60 msgr2=0x7f9fa81604f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:55.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.419+0000 7f9fae07b640 1 -- 172.21.15.47:0/157940504 shutdown_connections 2024-04-26T23:18:55.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.419+0000 7f9fae07b640 1 -- 172.21.15.47:0/157940504 wait complete. 2024-04-26T23:18:55.431 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-04-26T23:18:54.929134+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":231832,"kb_used_data":3176,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754742888,"statfs":{"total":773094113280,"available":772856717312,"internally_reserved":0,"allocated":3252224,"data_stored":1983104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":234016304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003517"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044475+0000","last_change":"2024-04-26T23:18:31.011895+0000","last_active":"2024-04-26T23:18:37.044475+0000","last_peered":"2024-04-26T23:18:37.044475+0000","last_clean":"2024-04-26T23:18:37.044475+0000","last_became_active":"2024-04-26T23:18:31.010976+0000","last_became_peered":"2024-04-26T23:18:31.010976+0000","last_unstale":"2024-04-26T23:18:37.044475+0000","last_undegraded":"2024-04-26T23:18:37.044475+0000","last_fullsized":"2024-04-26T23:18:37.044475+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:57:14.165871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030065+0000","last_change":"2024-04-26T23:18:32.020612+0000","last_active":"2024-04-26T23:18:37.030065+0000","last_peered":"2024-04-26T23:18:37.030065+0000","last_clean":"2024-04-26T23:18:37.030065+0000","last_became_active":"2024-04-26T23:18:32.018903+0000","last_became_peered":"2024-04-26T23:18:32.018903+0000","last_unstale":"2024-04-26T23:18:37.030065+0000","last_undegraded":"2024-04-26T23:18:37.030065+0000","last_fullsized":"2024-04-26T23:18:37.030065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:55:55.562630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029701+0000","last_change":"2024-04-26T23:18:36.051662+0000","last_active":"2024-04-26T23:18:37.029701+0000","last_peered":"2024-04-26T23:18:37.029701+0000","last_clean":"2024-04-26T23:18:37.029701+0000","last_became_active":"2024-04-26T23:18:36.051204+0000","last_became_peered":"2024-04-26T23:18:36.051204+0000","last_unstale":"2024-04-26T23:18:37.029701+0000","last_undegraded":"2024-04-26T23:18:37.029701+0000","last_fullsized":"2024-04-26T23:18:37.029701+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:24:24.209562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030279+0000","last_change":"2024-04-26T23:18:31.008954+0000","last_active":"2024-04-26T23:18:37.030279+0000","last_peered":"2024-04-26T23:18:37.030279+0000","last_clean":"2024-04-26T23:18:37.030279+0000","last_became_active":"2024-04-26T23:18:31.008597+0000","last_became_peered":"2024-04-26T23:18:31.008597+0000","last_unstale":"2024-04-26T23:18:37.030279+0000","last_undegraded":"2024-04-26T23:18:37.030279+0000","last_fullsized":"2024-04-26T23:18:37.030279+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:16:07.026710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044341+0000","last_change":"2024-04-26T23:18:32.019526+0000","last_active":"2024-04-26T23:18:37.044341+0000","last_peered":"2024-04-26T23:18:37.044341+0000","last_clean":"2024-04-26T23:18:37.044341+0000","last_became_active":"2024-04-26T23:18:32.019264+0000","last_became_peered":"2024-04-26T23:18:32.019264+0000","last_unstale":"2024-04-26T23:18:37.044341+0000","last_undegraded":"2024-04-26T23:18:37.044341+0000","last_fullsized":"2024-04-26T23:18:37.044341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:37:21.828417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044512+0000","last_change":"2024-04-26T23:18:36.041188+0000","last_active":"2024-04-26T23:18:37.044512+0000","last_peered":"2024-04-26T23:18:37.044512+0000","last_clean":"2024-04-26T23:18:37.044512+0000","last_became_active":"2024-04-26T23:18:36.040788+0000","last_became_peered":"2024-04-26T23:18:36.040788+0000","last_unstale":"2024-04-26T23:18:37.044512+0000","last_undegraded":"2024-04-26T23:18:37.044512+0000","last_fullsized":"2024-04-26T23:18:37.044512+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:49:51.239332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.047074+0000","last_change":"2024-04-26T23:18:31.011094+0000","last_active":"2024-04-26T23:18:37.047074+0000","last_peered":"2024-04-26T23:18:37.047074+0000","last_clean":"2024-04-26T23:18:37.047074+0000","last_became_active":"2024-04-26T23:18:31.010756+0000","last_became_peered":"2024-04-26T23:18:31.010756+0000","last_unstale":"2024-04-26T23:18:37.047074+0000","last_undegraded":"2024-04-26T23:18:37.047074+0000","last_fullsized":"2024-04-26T23:18:37.047074+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:28.986242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044870+0000","last_change":"2024-04-26T23:18:32.019790+0000","last_active":"2024-04-26T23:18:37.044870+0000","last_peered":"2024-04-26T23:18:37.044870+0000","last_clean":"2024-04-26T23:18:37.044870+0000","last_became_active":"2024-04-26T23:18:32.019535+0000","last_became_peered":"2024-04-26T23:18:32.019535+0000","last_unstale":"2024-04-26T23:18:37.044870+0000","last_undegraded":"2024-04-26T23:18:37.044870+0000","last_fullsized":"2024-04-26T23:18:37.044870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:10.370646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045799+0000","last_change":"2024-04-26T23:18:36.046954+0000","last_active":"2024-04-26T23:18:37.045799+0000","last_peered":"2024-04-26T23:18:37.045799+0000","last_clean":"2024-04-26T23:18:37.045799+0000","last_became_active":"2024-04-26T23:18:36.046588+0000","last_became_peered":"2024-04-26T23:18:36.046588+0000","last_unstale":"2024-04-26T23:18:37.045799+0000","last_undegraded":"2024-04-26T23:18:37.045799+0000","last_fullsized":"2024-04-26T23:18:37.045799+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:48:46.371866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046759+0000","last_change":"2024-04-26T23:18:31.011351+0000","last_active":"2024-04-26T23:18:37.046759+0000","last_peered":"2024-04-26T23:18:37.046759+0000","last_clean":"2024-04-26T23:18:37.046759+0000","last_became_active":"2024-04-26T23:18:31.010990+0000","last_became_peered":"2024-04-26T23:18:31.010990+0000","last_unstale":"2024-04-26T23:18:37.046759+0000","last_undegraded":"2024-04-26T23:18:37.046759+0000","last_fullsized":"2024-04-26T23:18:37.046759+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:08.243628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046200+0000","last_change":"2024-04-26T23:18:32.018999+0000","last_active":"2024-04-26T23:18:37.046200+0000","last_peered":"2024-04-26T23:18:37.046200+0000","last_clean":"2024-04-26T23:18:37.046200+0000","last_became_active":"2024-04-26T23:18:32.018713+0000","last_became_peered":"2024-04-26T23:18:32.018713+0000","last_unstale":"2024-04-26T23:18:37.046200+0000","last_undegraded":"2024-04-26T23:18:37.046200+0000","last_fullsized":"2024-04-26T23:18:37.046200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:39:01.950309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045585+0000","last_change":"2024-04-26T23:18:36.042244+0000","last_active":"2024-04-26T23:18:37.045585+0000","last_peered":"2024-04-26T23:18:37.045585+0000","last_clean":"2024-04-26T23:18:37.045585+0000","last_became_active":"2024-04-26T23:18:36.041833+0000","last_became_peered":"2024-04-26T23:18:36.041833+0000","last_unstale":"2024-04-26T23:18:37.045585+0000","last_undegraded":"2024-04-26T23:18:37.045585+0000","last_fullsized":"2024-04-26T23:18:37.045585+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:39:05.974905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045365+0000","last_change":"2024-04-26T23:18:32.017614+0000","last_active":"2024-04-26T23:18:37.045365+0000","last_peered":"2024-04-26T23:18:37.045365+0000","last_clean":"2024-04-26T23:18:37.045365+0000","last_became_active":"2024-04-26T23:18:32.017251+0000","last_became_peered":"2024-04-26T23:18:32.017251+0000","last_unstale":"2024-04-26T23:18:37.045365+0000","last_undegraded":"2024-04-26T23:18:37.045365+0000","last_fullsized":"2024-04-26T23:18:37.045365+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:37:21.828417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.047093+0000","last_change":"2024-04-26T23:18:31.011768+0000","last_active":"2024-04-26T23:18:37.047093+0000","last_peered":"2024-04-26T23:18:37.047093+0000","last_clean":"2024-04-26T23:18:37.047093+0000","last_became_active":"2024-04-26T23:18:31.011495+0000","last_became_peered":"2024-04-26T23:18:31.011495+0000","last_unstale":"2024-04-26T23:18:37.047093+0000","last_undegraded":"2024-04-26T23:18:37.047093+0000","last_fullsized":"2024-04-26T23:18:37.047093+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:25:38.515775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045600+0000","last_change":"2024-04-26T23:18:36.039079+0000","last_active":"2024-04-26T23:18:37.045600+0000","last_peered":"2024-04-26T23:18:37.045600+0000","last_clean":"2024-04-26T23:18:37.045600+0000","last_became_active":"2024-04-26T23:18:36.038665+0000","last_became_peered":"2024-04-26T23:18:36.038665+0000","last_unstale":"2024-04-26T23:18:37.045600+0000","last_undegraded":"2024-04-26T23:18:37.045600+0000","last_fullsized":"2024-04-26T23:18:37.045600+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:45:26.386750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029982+0000","last_change":"2024-04-26T23:18:32.012705+0000","last_active":"2024-04-26T23:18:37.029982+0000","last_peered":"2024-04-26T23:18:37.029982+0000","last_clean":"2024-04-26T23:18:37.029982+0000","last_became_active":"2024-04-26T23:18:32.012421+0000","last_became_peered":"2024-04-26T23:18:32.012421+0000","last_unstale":"2024-04-26T23:18:37.029982+0000","last_undegraded":"2024-04-26T23:18:37.029982+0000","last_fullsized":"2024-04-26T23:18:37.029982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:49:47.214737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030444+0000","last_change":"2024-04-26T23:18:31.011273+0000","last_active":"2024-04-26T23:18:37.030444+0000","last_peered":"2024-04-26T23:18:37.030444+0000","last_clean":"2024-04-26T23:18:37.030444+0000","last_became_active":"2024-04-26T23:18:31.010978+0000","last_became_peered":"2024-04-26T23:18:31.010978+0000","last_unstale":"2024-04-26T23:18:37.030444+0000","last_undegraded":"2024-04-26T23:18:37.030444+0000","last_fullsized":"2024-04-26T23:18:37.030444+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:13.713713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045147+0000","last_change":"2024-04-26T23:18:36.040606+0000","last_active":"2024-04-26T23:18:37.045147+0000","last_peered":"2024-04-26T23:18:37.045147+0000","last_clean":"2024-04-26T23:18:37.045147+0000","last_became_active":"2024-04-26T23:18:36.040058+0000","last_became_peered":"2024-04-26T23:18:36.040058+0000","last_unstale":"2024-04-26T23:18:37.045147+0000","last_undegraded":"2024-04-26T23:18:37.045147+0000","last_fullsized":"2024-04-26T23:18:37.045147+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:48:55.902259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030935+0000","last_change":"2024-04-26T23:18:32.015298+0000","last_active":"2024-04-26T23:18:37.030935+0000","last_peered":"2024-04-26T23:18:37.030935+0000","last_clean":"2024-04-26T23:18:37.030935+0000","last_became_active":"2024-04-26T23:18:32.014508+0000","last_became_peered":"2024-04-26T23:18:32.014508+0000","last_unstale":"2024-04-26T23:18:37.030935+0000","last_undegraded":"2024-04-26T23:18:37.030935+0000","last_fullsized":"2024-04-26T23:18:37.030935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:57:02.596299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029055+0000","last_change":"2024-04-26T23:18:31.008495+0000","last_active":"2024-04-26T23:18:37.029055+0000","last_peered":"2024-04-26T23:18:37.029055+0000","last_clean":"2024-04-26T23:18:37.029055+0000","last_became_active":"2024-04-26T23:18:31.008116+0000","last_became_peered":"2024-04-26T23:18:31.008116+0000","last_unstale":"2024-04-26T23:18:37.029055+0000","last_undegraded":"2024-04-26T23:18:37.029055+0000","last_fullsized":"2024-04-26T23:18:37.029055+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:34:02.747044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030423+0000","last_change":"2024-04-26T23:18:36.045871+0000","last_active":"2024-04-26T23:18:37.030423+0000","last_peered":"2024-04-26T23:18:37.030423+0000","last_clean":"2024-04-26T23:18:37.030423+0000","last_became_active":"2024-04-26T23:18:36.044888+0000","last_became_peered":"2024-04-26T23:18:36.044888+0000","last_unstale":"2024-04-26T23:18:37.030423+0000","last_undegraded":"2024-04-26T23:18:37.030423+0000","last_fullsized":"2024-04-26T23:18:37.030423+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:08:51.442254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045209+0000","last_change":"2024-04-26T23:18:32.015241+0000","last_active":"2024-04-26T23:18:37.045209+0000","last_peered":"2024-04-26T23:18:37.045209+0000","last_clean":"2024-04-26T23:18:37.045209+0000","last_became_active":"2024-04-26T23:18:32.013246+0000","last_became_peered":"2024-04-26T23:18:32.013246+0000","last_unstale":"2024-04-26T23:18:37.045209+0000","last_undegraded":"2024-04-26T23:18:37.045209+0000","last_fullsized":"2024-04-26T23:18:37.045209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:21.586522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045696+0000","last_change":"2024-04-26T23:18:31.015485+0000","last_active":"2024-04-26T23:18:37.045696+0000","last_peered":"2024-04-26T23:18:37.045696+0000","last_clean":"2024-04-26T23:18:37.045696+0000","last_became_active":"2024-04-26T23:18:31.015187+0000","last_became_peered":"2024-04-26T23:18:31.015187+0000","last_unstale":"2024-04-26T23:18:37.045696+0000","last_undegraded":"2024-04-26T23:18:37.045696+0000","last_fullsized":"2024-04-26T23:18:37.045696+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:53:15.812510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044493+0000","last_change":"2024-04-26T23:18:36.040493+0000","last_active":"2024-04-26T23:18:37.044493+0000","last_peered":"2024-04-26T23:18:37.044493+0000","last_clean":"2024-04-26T23:18:37.044493+0000","last_became_active":"2024-04-26T23:18:36.040040+0000","last_became_peered":"2024-04-26T23:18:36.040040+0000","last_unstale":"2024-04-26T23:18:37.044493+0000","last_undegraded":"2024-04-26T23:18:37.044493+0000","last_fullsized":"2024-04-26T23:18:37.044493+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:37:25.853012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044514+0000","last_change":"2024-04-26T23:18:32.001467+0000","last_active":"2024-04-26T23:18:37.044514+0000","last_peered":"2024-04-26T23:18:37.044514+0000","last_clean":"2024-04-26T23:18:37.044514+0000","last_became_active":"2024-04-26T23:18:32.001170+0000","last_became_peered":"2024-04-26T23:18:32.001170+0000","last_unstale":"2024-04-26T23:18:37.044514+0000","last_undegraded":"2024-04-26T23:18:37.044514+0000","last_fullsized":"2024-04-26T23:18:37.044514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:17:04.527690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046567+0000","last_change":"2024-04-26T23:18:31.014468+0000","last_active":"2024-04-26T23:18:37.046567+0000","last_peered":"2024-04-26T23:18:37.046567+0000","last_clean":"2024-04-26T23:18:37.046567+0000","last_became_active":"2024-04-26T23:18:31.014192+0000","last_became_peered":"2024-04-26T23:18:31.014192+0000","last_unstale":"2024-04-26T23:18:37.046567+0000","last_undegraded":"2024-04-26T23:18:37.046567+0000","last_fullsized":"2024-04-26T23:18:37.046567+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:20.581173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044988+0000","last_change":"2024-04-26T23:18:36.044805+0000","last_active":"2024-04-26T23:18:37.044988+0000","last_peered":"2024-04-26T23:18:37.044988+0000","last_clean":"2024-04-26T23:18:37.044988+0000","last_became_active":"2024-04-26T23:18:36.044409+0000","last_became_peered":"2024-04-26T23:18:36.044409+0000","last_unstale":"2024-04-26T23:18:37.044988+0000","last_undegraded":"2024-04-26T23:18:37.044988+0000","last_fullsized":"2024-04-26T23:18:37.044988+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:04:17.836184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030862+0000","last_change":"2024-04-26T23:18:32.018289+0000","last_active":"2024-04-26T23:18:37.030862+0000","last_peered":"2024-04-26T23:18:37.030862+0000","last_clean":"2024-04-26T23:18:37.030862+0000","last_became_active":"2024-04-26T23:18:32.017788+0000","last_became_peered":"2024-04-26T23:18:32.017788+0000","last_unstale":"2024-04-26T23:18:37.030862+0000","last_undegraded":"2024-04-26T23:18:37.030862+0000","last_fullsized":"2024-04-26T23:18:37.030862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:55.832329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030196+0000","last_change":"2024-04-26T23:18:31.009677+0000","last_active":"2024-04-26T23:18:37.030196+0000","last_peered":"2024-04-26T23:18:37.030196+0000","last_clean":"2024-04-26T23:18:37.030196+0000","last_became_active":"2024-04-26T23:18:31.009368+0000","last_became_peered":"2024-04-26T23:18:31.009368+0000","last_unstale":"2024-04-26T23:18:37.030196+0000","last_undegraded":"2024-04-26T23:18:37.030196+0000","last_fullsized":"2024-04-26T23:18:37.030196+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:28.986242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029507+0000","last_change":"2024-04-26T23:18:36.050766+0000","last_active":"2024-04-26T23:18:37.029507+0000","last_peered":"2024-04-26T23:18:37.029507+0000","last_clean":"2024-04-26T23:18:37.029507+0000","last_became_active":"2024-04-26T23:18:36.050091+0000","last_became_peered":"2024-04-26T23:18:36.050091+0000","last_unstale":"2024-04-26T23:18:37.029507+0000","last_undegraded":"2024-04-26T23:18:37.029507+0000","last_fullsized":"2024-04-26T23:18:37.029507+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:16:16.544972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029722+0000","last_change":"2024-04-26T23:18:32.007502+0000","last_active":"2024-04-26T23:18:37.029722+0000","last_peered":"2024-04-26T23:18:37.029722+0000","last_clean":"2024-04-26T23:18:37.029722+0000","last_became_active":"2024-04-26T23:18:32.006451+0000","last_became_peered":"2024-04-26T23:18:32.006451+0000","last_unstale":"2024-04-26T23:18:37.029722+0000","last_undegraded":"2024-04-26T23:18:37.029722+0000","last_fullsized":"2024-04-26T23:18:37.029722+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:55:55.562630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.047238+0000","last_change":"2024-04-26T23:18:31.012702+0000","last_active":"2024-04-26T23:18:37.047238+0000","last_peered":"2024-04-26T23:18:37.047238+0000","last_clean":"2024-04-26T23:18:37.047238+0000","last_became_active":"2024-04-26T23:18:31.012425+0000","last_became_peered":"2024-04-26T23:18:31.012425+0000","last_unstale":"2024-04-26T23:18:37.047238+0000","last_undegraded":"2024-04-26T23:18:37.047238+0000","last_fullsized":"2024-04-26T23:18:37.047238+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:08:31.793547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044060+0000","last_change":"2024-04-26T23:18:36.049119+0000","last_active":"2024-04-26T23:18:37.044060+0000","last_peered":"2024-04-26T23:18:37.044060+0000","last_clean":"2024-04-26T23:18:37.044060+0000","last_became_active":"2024-04-26T23:18:36.047621+0000","last_became_peered":"2024-04-26T23:18:36.047621+0000","last_unstale":"2024-04-26T23:18:37.044060+0000","last_undegraded":"2024-04-26T23:18:37.044060+0000","last_fullsized":"2024-04-26T23:18:37.044060+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:48:46.371866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046280+0000","last_change":"2024-04-26T23:18:32.021370+0000","last_active":"2024-04-26T23:18:37.046280+0000","last_peered":"2024-04-26T23:18:37.046280+0000","last_clean":"2024-04-26T23:18:37.046280+0000","last_became_active":"2024-04-26T23:18:32.020816+0000","last_became_peered":"2024-04-26T23:18:32.020816+0000","last_unstale":"2024-04-26T23:18:37.046280+0000","last_undegraded":"2024-04-26T23:18:37.046280+0000","last_fullsized":"2024-04-26T23:18:37.046280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:45:22.362155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029882+0000","last_change":"2024-04-26T23:18:31.007687+0000","last_active":"2024-04-26T23:18:37.029882+0000","last_peered":"2024-04-26T23:18:37.029882+0000","last_clean":"2024-04-26T23:18:37.029882+0000","last_became_active":"2024-04-26T23:18:31.006664+0000","last_became_peered":"2024-04-26T23:18:31.006664+0000","last_unstale":"2024-04-26T23:18:37.029882+0000","last_undegraded":"2024-04-26T23:18:37.029882+0000","last_fullsized":"2024-04-26T23:18:37.029882+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:44.844711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029232+0000","last_change":"2024-04-26T23:18:36.047590+0000","last_active":"2024-04-26T23:18:37.029232+0000","last_peered":"2024-04-26T23:18:37.029232+0000","last_clean":"2024-04-26T23:18:37.029232+0000","last_became_active":"2024-04-26T23:18:36.046796+0000","last_became_peered":"2024-04-26T23:18:36.046796+0000","last_unstale":"2024-04-26T23:18:37.029232+0000","last_undegraded":"2024-04-26T23:18:37.029232+0000","last_fullsized":"2024-04-26T23:18:37.029232+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:09.320477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046041+0000","last_change":"2024-04-26T23:18:32.002280+0000","last_active":"2024-04-26T23:18:37.046041+0000","last_peered":"2024-04-26T23:18:37.046041+0000","last_clean":"2024-04-26T23:18:37.046041+0000","last_became_active":"2024-04-26T23:18:32.002051+0000","last_became_peered":"2024-04-26T23:18:32.002051+0000","last_unstale":"2024-04-26T23:18:37.046041+0000","last_undegraded":"2024-04-26T23:18:37.046041+0000","last_fullsized":"2024-04-26T23:18:37.046041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:09.248977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044993+0000","last_change":"2024-04-26T23:18:31.011488+0000","last_active":"2024-04-26T23:18:37.044993+0000","last_peered":"2024-04-26T23:18:37.044993+0000","last_clean":"2024-04-26T23:18:37.044993+0000","last_became_active":"2024-04-26T23:18:31.010229+0000","last_became_peered":"2024-04-26T23:18:31.010229+0000","last_unstale":"2024-04-26T23:18:37.044993+0000","last_undegraded":"2024-04-26T23:18:37.044993+0000","last_fullsized":"2024-04-26T23:18:37.044993+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:25:38.515775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029825+0000","last_change":"2024-04-26T23:18:36.042997+0000","last_active":"2024-04-26T23:18:37.029825+0000","last_peered":"2024-04-26T23:18:37.029825+0000","last_clean":"2024-04-26T23:18:37.029825+0000","last_became_active":"2024-04-26T23:18:36.042510+0000","last_became_peered":"2024-04-26T23:18:36.042510+0000","last_unstale":"2024-04-26T23:18:37.029825+0000","last_undegraded":"2024-04-26T23:18:37.029825+0000","last_fullsized":"2024-04-26T23:18:37.029825+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:30.558000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.028944+0000","last_change":"2024-04-26T23:18:32.008615+0000","last_active":"2024-04-26T23:18:37.028944+0000","last_peered":"2024-04-26T23:18:37.028944+0000","last_clean":"2024-04-26T23:18:37.028944+0000","last_became_active":"2024-04-26T23:18:32.006670+0000","last_became_peered":"2024-04-26T23:18:32.006670+0000","last_unstale":"2024-04-26T23:18:37.028944+0000","last_undegraded":"2024-04-26T23:18:37.028944+0000","last_fullsized":"2024-04-26T23:18:37.028944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:54:25.468799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.027917+0000","last_change":"2024-04-26T23:18:31.005153+0000","last_active":"2024-04-26T23:18:37.027917+0000","last_peered":"2024-04-26T23:18:37.027917+0000","last_clean":"2024-04-26T23:18:37.027917+0000","last_became_active":"2024-04-26T23:18:31.004670+0000","last_became_peered":"2024-04-26T23:18:31.004670+0000","last_unstale":"2024-04-26T23:18:37.027917+0000","last_undegraded":"2024-04-26T23:18:37.027917+0000","last_fullsized":"2024-04-26T23:18:37.027917+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:34:02.747044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030347+0000","last_change":"2024-04-26T23:18:36.048181+0000","last_active":"2024-04-26T23:18:37.030347+0000","last_peered":"2024-04-26T23:18:37.030347+0000","last_clean":"2024-04-26T23:18:37.030347+0000","last_became_active":"2024-04-26T23:18:36.047762+0000","last_became_peered":"2024-04-26T23:18:36.047762+0000","last_unstale":"2024-04-26T23:18:37.030347+0000","last_undegraded":"2024-04-26T23:18:37.030347+0000","last_fullsized":"2024-04-26T23:18:37.030347+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:39:32.810976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.043710+0000","last_change":"2024-04-26T23:18:32.005125+0000","last_active":"2024-04-26T23:18:37.043710+0000","last_peered":"2024-04-26T23:18:37.043710+0000","last_clean":"2024-04-26T23:18:37.043710+0000","last_became_active":"2024-04-26T23:18:32.004826+0000","last_became_peered":"2024-04-26T23:18:32.004826+0000","last_unstale":"2024-04-26T23:18:37.043710+0000","last_undegraded":"2024-04-26T23:18:37.043710+0000","last_fullsized":"2024-04-26T23:18:37.043710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:53:16.817858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045815+0000","last_change":"2024-04-26T23:18:31.007674+0000","last_active":"2024-04-26T23:18:37.045815+0000","last_peered":"2024-04-26T23:18:37.045815+0000","last_clean":"2024-04-26T23:18:37.045815+0000","last_became_active":"2024-04-26T23:18:31.006787+0000","last_became_peered":"2024-04-26T23:18:31.006787+0000","last_unstale":"2024-04-26T23:18:37.045815+0000","last_undegraded":"2024-04-26T23:18:37.045815+0000","last_fullsized":"2024-04-26T23:18:37.045815+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:59:17.263468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044484+0000","last_change":"2024-04-26T23:18:36.046197+0000","last_active":"2024-04-26T23:18:37.044484+0000","last_peered":"2024-04-26T23:18:37.044484+0000","last_clean":"2024-04-26T23:18:37.044484+0000","last_became_active":"2024-04-26T23:18:36.045205+0000","last_became_peered":"2024-04-26T23:18:36.045205+0000","last_unstale":"2024-04-26T23:18:37.044484+0000","last_undegraded":"2024-04-26T23:18:37.044484+0000","last_fullsized":"2024-04-26T23:18:37.044484+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:39:32.810976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030371+0000","last_change":"2024-04-26T23:18:32.025774+0000","last_active":"2024-04-26T23:18:37.030371+0000","last_peered":"2024-04-26T23:18:37.030371+0000","last_clean":"2024-04-26T23:18:37.030371+0000","last_became_active":"2024-04-26T23:18:32.025487+0000","last_became_peered":"2024-04-26T23:18:32.025487+0000","last_unstale":"2024-04-26T23:18:37.030371+0000","last_undegraded":"2024-04-26T23:18:37.030371+0000","last_fullsized":"2024-04-26T23:18:37.030371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:04:13.811589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044628+0000","last_change":"2024-04-26T23:18:31.008744+0000","last_active":"2024-04-26T23:18:37.044628+0000","last_peered":"2024-04-26T23:18:37.044628+0000","last_clean":"2024-04-26T23:18:37.044628+0000","last_became_active":"2024-04-26T23:18:31.008449+0000","last_became_peered":"2024-04-26T23:18:31.008449+0000","last_unstale":"2024-04-26T23:18:37.044628+0000","last_undegraded":"2024-04-26T23:18:37.044628+0000","last_fullsized":"2024-04-26T23:18:37.044628+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:08.243628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045855+0000","last_change":"2024-04-26T23:18:18.862631+0000","last_active":"2024-04-26T23:18:37.045855+0000","last_peered":"2024-04-26T23:18:37.045855+0000","last_clean":"2024-04-26T23:18:37.045855+0000","last_became_active":"2024-04-26T23:18:18.861536+0000","last_became_peered":"2024-04-26T23:18:18.861536+0000","last_unstale":"2024-04-26T23:18:37.045855+0000","last_undegraded":"2024-04-26T23:18:37.045855+0000","last_fullsized":"2024-04-26T23:18:37.045855+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:17.808736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:17.808736+0000","last_clean_scrub_stamp":"2024-04-26T23:18:17.808736+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:59:26.359282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045887+0000","last_change":"2024-04-26T23:18:36.045014+0000","last_active":"2024-04-26T23:18:37.045887+0000","last_peered":"2024-04-26T23:18:37.045887+0000","last_clean":"2024-04-26T23:18:37.045887+0000","last_became_active":"2024-04-26T23:18:36.044467+0000","last_became_peered":"2024-04-26T23:18:36.044467+0000","last_unstale":"2024-04-26T23:18:37.045887+0000","last_undegraded":"2024-04-26T23:18:37.045887+0000","last_fullsized":"2024-04-26T23:18:37.045887+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:48:46.371866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046652+0000","last_change":"2024-04-26T23:18:32.030669+0000","last_active":"2024-04-26T23:18:37.046652+0000","last_peered":"2024-04-26T23:18:37.046652+0000","last_clean":"2024-04-26T23:18:37.046652+0000","last_became_active":"2024-04-26T23:18:32.030359+0000","last_became_peered":"2024-04-26T23:18:32.030359+0000","last_unstale":"2024-04-26T23:18:37.046652+0000","last_undegraded":"2024-04-26T23:18:37.046652+0000","last_fullsized":"2024-04-26T23:18:37.046652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:05.295882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.028346+0000","last_change":"2024-04-26T23:18:31.011343+0000","last_active":"2024-04-26T23:18:37.028346+0000","last_peered":"2024-04-26T23:18:37.028346+0000","last_clean":"2024-04-26T23:18:37.028346+0000","last_became_active":"2024-04-26T23:18:31.011011+0000","last_became_peered":"2024-04-26T23:18:31.011011+0000","last_unstale":"2024-04-26T23:18:37.028346+0000","last_undegraded":"2024-04-26T23:18:37.028346+0000","last_fullsized":"2024-04-26T23:18:37.028346+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:22.746946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045303+0000","last_change":"2024-04-26T23:18:36.045541+0000","last_active":"2024-04-26T23:18:37.045303+0000","last_peered":"2024-04-26T23:18:37.045303+0000","last_clean":"2024-04-26T23:18:37.045303+0000","last_became_active":"2024-04-26T23:18:36.045133+0000","last_became_peered":"2024-04-26T23:18:36.045133+0000","last_unstale":"2024-04-26T23:18:37.045303+0000","last_undegraded":"2024-04-26T23:18:37.045303+0000","last_fullsized":"2024-04-26T23:18:37.045303+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:45:26.386750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045276+0000","last_change":"2024-04-26T23:18:32.018614+0000","last_active":"2024-04-26T23:18:37.045276+0000","last_peered":"2024-04-26T23:18:37.045276+0000","last_clean":"2024-04-26T23:18:37.045276+0000","last_became_active":"2024-04-26T23:18:32.018088+0000","last_became_peered":"2024-04-26T23:18:32.018088+0000","last_unstale":"2024-04-26T23:18:37.045276+0000","last_undegraded":"2024-04-26T23:18:37.045276+0000","last_fullsized":"2024-04-26T23:18:37.045276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:26.533404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030223+0000","last_change":"2024-04-26T23:18:31.010377+0000","last_active":"2024-04-26T23:18:37.030223+0000","last_peered":"2024-04-26T23:18:37.030223+0000","last_clean":"2024-04-26T23:18:37.030223+0000","last_became_active":"2024-04-26T23:18:31.010084+0000","last_became_peered":"2024-04-26T23:18:31.010084+0000","last_unstale":"2024-04-26T23:18:37.030223+0000","last_undegraded":"2024-04-26T23:18:37.030223+0000","last_fullsized":"2024-04-26T23:18:37.030223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:08.243628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044284+0000","last_change":"2024-04-26T23:18:36.047754+0000","last_active":"2024-04-26T23:18:37.044284+0000","last_peered":"2024-04-26T23:18:37.044284+0000","last_clean":"2024-04-26T23:18:37.044284+0000","last_became_active":"2024-04-26T23:18:36.047105+0000","last_became_peered":"2024-04-26T23:18:36.047105+0000","last_unstale":"2024-04-26T23:18:37.044284+0000","last_undegraded":"2024-04-26T23:18:37.044284+0000","last_fullsized":"2024-04-26T23:18:37.044284+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:14.395241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.047247+0000","last_change":"2024-04-26T23:18:32.026649+0000","last_active":"2024-04-26T23:18:37.047247+0000","last_peered":"2024-04-26T23:18:37.047247+0000","last_clean":"2024-04-26T23:18:37.047247+0000","last_became_active":"2024-04-26T23:18:32.026014+0000","last_became_peered":"2024-04-26T23:18:32.026014+0000","last_unstale":"2024-04-26T23:18:37.047247+0000","last_undegraded":"2024-04-26T23:18:37.047247+0000","last_fullsized":"2024-04-26T23:18:37.047247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:08:47.417659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.028103+0000","last_change":"2024-04-26T23:18:31.006215+0000","last_active":"2024-04-26T23:18:37.028103+0000","last_peered":"2024-04-26T23:18:37.028103+0000","last_clean":"2024-04-26T23:18:37.028103+0000","last_became_active":"2024-04-26T23:18:31.005847+0000","last_became_peered":"2024-04-26T23:18:31.005847+0000","last_unstale":"2024-04-26T23:18:37.028103+0000","last_undegraded":"2024-04-26T23:18:37.028103+0000","last_fullsized":"2024-04-26T23:18:37.028103+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:44.844711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044109+0000","last_change":"2024-04-26T23:18:36.045812+0000","last_active":"2024-04-26T23:18:37.044109+0000","last_peered":"2024-04-26T23:18:37.044109+0000","last_clean":"2024-04-26T23:18:37.044109+0000","last_became_active":"2024-04-26T23:18:36.045445+0000","last_became_peered":"2024-04-26T23:18:36.045445+0000","last_unstale":"2024-04-26T23:18:37.044109+0000","last_undegraded":"2024-04-26T23:18:37.044109+0000","last_fullsized":"2024-04-26T23:18:37.044109+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:16:16.544972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046893+0000","last_change":"2024-04-26T23:18:32.025072+0000","last_active":"2024-04-26T23:18:37.046893+0000","last_peered":"2024-04-26T23:18:37.046893+0000","last_clean":"2024-04-26T23:18:37.046893+0000","last_became_active":"2024-04-26T23:18:32.024754+0000","last_became_peered":"2024-04-26T23:18:32.024754+0000","last_unstale":"2024-04-26T23:18:37.046893+0000","last_undegraded":"2024-04-26T23:18:37.046893+0000","last_fullsized":"2024-04-26T23:18:37.046893+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:45:22.362155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044833+0000","last_change":"2024-04-26T23:18:31.013340+0000","last_active":"2024-04-26T23:18:37.044833+0000","last_peered":"2024-04-26T23:18:37.044833+0000","last_clean":"2024-04-26T23:18:37.044833+0000","last_became_active":"2024-04-26T23:18:31.012824+0000","last_became_peered":"2024-04-26T23:18:31.012824+0000","last_unstale":"2024-04-26T23:18:37.044833+0000","last_undegraded":"2024-04-26T23:18:37.044833+0000","last_fullsized":"2024-04-26T23:18:37.044833+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:17.988227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044762+0000","last_change":"2024-04-26T23:18:36.041004+0000","last_active":"2024-04-26T23:18:37.044762+0000","last_peered":"2024-04-26T23:18:37.044762+0000","last_clean":"2024-04-26T23:18:37.044762+0000","last_became_active":"2024-04-26T23:18:36.040155+0000","last_became_peered":"2024-04-26T23:18:36.040155+0000","last_unstale":"2024-04-26T23:18:37.044762+0000","last_undegraded":"2024-04-26T23:18:37.044762+0000","last_fullsized":"2024-04-26T23:18:37.044762+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:59.856924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046692+0000","last_change":"2024-04-26T23:18:32.025214+0000","last_active":"2024-04-26T23:18:37.046692+0000","last_peered":"2024-04-26T23:18:37.046692+0000","last_clean":"2024-04-26T23:18:37.046692+0000","last_became_active":"2024-04-26T23:18:32.024952+0000","last_became_peered":"2024-04-26T23:18:32.024952+0000","last_unstale":"2024-04-26T23:18:37.046692+0000","last_undegraded":"2024-04-26T23:18:37.046692+0000","last_fullsized":"2024-04-26T23:18:37.046692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:39:01.950309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.027616+0000","last_change":"2024-04-26T23:18:31.001581+0000","last_active":"2024-04-26T23:18:37.027616+0000","last_peered":"2024-04-26T23:18:37.027616+0000","last_clean":"2024-04-26T23:18:37.027616+0000","last_became_active":"2024-04-26T23:18:31.001256+0000","last_became_peered":"2024-04-26T23:18:31.001256+0000","last_unstale":"2024-04-26T23:18:37.027616+0000","last_undegraded":"2024-04-26T23:18:37.027616+0000","last_fullsized":"2024-04-26T23:18:37.027616+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:13.713713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044283+0000","last_change":"2024-04-26T23:18:36.040673+0000","last_active":"2024-04-26T23:18:37.044283+0000","last_peered":"2024-04-26T23:18:37.044283+0000","last_clean":"2024-04-26T23:18:37.044283+0000","last_became_active":"2024-04-26T23:18:36.040246+0000","last_became_peered":"2024-04-26T23:18:36.040246+0000","last_unstale":"2024-04-26T23:18:37.044283+0000","last_undegraded":"2024-04-26T23:18:37.044283+0000","last_fullsized":"2024-04-26T23:18:37.044283+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:30.558000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044353+0000","last_change":"2024-04-26T23:18:32.013824+0000","last_active":"2024-04-26T23:18:37.044353+0000","last_peered":"2024-04-26T23:18:37.044353+0000","last_clean":"2024-04-26T23:18:37.044353+0000","last_became_active":"2024-04-26T23:18:32.012613+0000","last_became_peered":"2024-04-26T23:18:32.012613+0000","last_unstale":"2024-04-26T23:18:37.044353+0000","last_undegraded":"2024-04-26T23:18:37.044353+0000","last_fullsized":"2024-04-26T23:18:37.044353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:49:47.214737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044914+0000","last_change":"2024-04-26T23:18:31.011248+0000","last_active":"2024-04-26T23:18:37.044914+0000","last_peered":"2024-04-26T23:18:37.044914+0000","last_clean":"2024-04-26T23:18:37.044914+0000","last_became_active":"2024-04-26T23:18:31.010965+0000","last_became_peered":"2024-04-26T23:18:31.010965+0000","last_unstale":"2024-04-26T23:18:37.044914+0000","last_undegraded":"2024-04-26T23:18:37.044914+0000","last_fullsized":"2024-04-26T23:18:37.044914+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:13.713713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030962+0000","last_change":"2024-04-26T23:18:36.045330+0000","last_active":"2024-04-26T23:18:37.030962+0000","last_peered":"2024-04-26T23:18:37.030962+0000","last_clean":"2024-04-26T23:18:37.030962+0000","last_became_active":"2024-04-26T23:18:36.044903+0000","last_became_peered":"2024-04-26T23:18:36.044903+0000","last_unstale":"2024-04-26T23:18:37.030962+0000","last_undegraded":"2024-04-26T23:18:37.030962+0000","last_fullsized":"2024-04-26T23:18:37.030962+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:53:15.682467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.047083+0000","last_change":"2024-04-26T23:18:32.026417+0000","last_active":"2024-04-26T23:18:37.047083+0000","last_peered":"2024-04-26T23:18:37.047083+0000","last_clean":"2024-04-26T23:18:37.047083+0000","last_became_active":"2024-04-26T23:18:32.025839+0000","last_became_peered":"2024-04-26T23:18:32.025839+0000","last_unstale":"2024-04-26T23:18:37.047083+0000","last_undegraded":"2024-04-26T23:18:37.047083+0000","last_fullsized":"2024-04-26T23:18:37.047083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:39:28.786381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044878+0000","last_change":"2024-04-26T23:18:31.011956+0000","last_active":"2024-04-26T23:18:37.044878+0000","last_peered":"2024-04-26T23:18:37.044878+0000","last_clean":"2024-04-26T23:18:37.044878+0000","last_became_active":"2024-04-26T23:18:31.010493+0000","last_became_peered":"2024-04-26T23:18:31.010493+0000","last_unstale":"2024-04-26T23:18:37.044878+0000","last_undegraded":"2024-04-26T23:18:37.044878+0000","last_fullsized":"2024-04-26T23:18:37.044878+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:59:17.263468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030567+0000","last_change":"2024-04-26T23:18:36.047105+0000","last_active":"2024-04-26T23:18:37.030567+0000","last_peered":"2024-04-26T23:18:37.030567+0000","last_clean":"2024-04-26T23:18:37.030567+0000","last_became_active":"2024-04-26T23:18:36.046724+0000","last_became_peered":"2024-04-26T23:18:36.046724+0000","last_unstale":"2024-04-26T23:18:37.030567+0000","last_undegraded":"2024-04-26T23:18:37.030567+0000","last_fullsized":"2024-04-26T23:18:37.030567+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:46:07.146408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.043961+0000","last_change":"2024-04-26T23:18:32.015408+0000","last_active":"2024-04-26T23:18:37.043961+0000","last_peered":"2024-04-26T23:18:37.043961+0000","last_clean":"2024-04-26T23:18:37.043961+0000","last_became_active":"2024-04-26T23:18:32.015037+0000","last_became_peered":"2024-04-26T23:18:32.015037+0000","last_unstale":"2024-04-26T23:18:37.043961+0000","last_undegraded":"2024-04-26T23:18:37.043961+0000","last_fullsized":"2024-04-26T23:18:37.043961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:49.567077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044345+0000","last_change":"2024-04-26T23:18:31.010413+0000","last_active":"2024-04-26T23:18:37.044345+0000","last_peered":"2024-04-26T23:18:37.044345+0000","last_clean":"2024-04-26T23:18:37.044345+0000","last_became_active":"2024-04-26T23:18:31.010138+0000","last_became_peered":"2024-04-26T23:18:31.010138+0000","last_unstale":"2024-04-26T23:18:37.044345+0000","last_undegraded":"2024-04-26T23:18:37.044345+0000","last_fullsized":"2024-04-26T23:18:37.044345+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:15:53.470195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046231+0000","last_change":"2024-04-26T23:18:36.044782+0000","last_active":"2024-04-26T23:18:37.046231+0000","last_peered":"2024-04-26T23:18:37.046231+0000","last_clean":"2024-04-26T23:18:37.046231+0000","last_became_active":"2024-04-26T23:18:36.044157+0000","last_became_peered":"2024-04-26T23:18:36.044157+0000","last_unstale":"2024-04-26T23:18:37.046231+0000","last_undegraded":"2024-04-26T23:18:37.046231+0000","last_fullsized":"2024-04-26T23:18:37.046231+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:24:24.209562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046631+0000","last_change":"2024-04-26T23:18:32.023453+0000","last_active":"2024-04-26T23:18:37.046631+0000","last_peered":"2024-04-26T23:18:37.046631+0000","last_clean":"2024-04-26T23:18:37.046631+0000","last_became_active":"2024-04-26T23:18:32.023170+0000","last_became_peered":"2024-04-26T23:18:32.023170+0000","last_unstale":"2024-04-26T23:18:37.046631+0000","last_undegraded":"2024-04-26T23:18:37.046631+0000","last_fullsized":"2024-04-26T23:18:37.046631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:48:42.347270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:41.489012+0000","last_change":"2024-04-26T23:18:31.013165+0000","last_active":"2024-04-26T23:18:41.489012+0000","last_peered":"2024-04-26T23:18:41.489012+0000","last_clean":"2024-04-26T23:18:41.489012+0000","last_became_active":"2024-04-26T23:18:31.012790+0000","last_became_peered":"2024-04-26T23:18:31.012790+0000","last_unstale":"2024-04-26T23:18:41.489012+0000","last_undegraded":"2024-04-26T23:18:41.489012+0000","last_fullsized":"2024-04-26T23:18:41.489012+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:44.844711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029334+0000","last_change":"2024-04-26T23:18:36.047231+0000","last_active":"2024-04-26T23:18:37.029334+0000","last_peered":"2024-04-26T23:18:37.029334+0000","last_clean":"2024-04-26T23:18:37.029334+0000","last_became_active":"2024-04-26T23:18:36.046579+0000","last_became_peered":"2024-04-26T23:18:36.046579+0000","last_unstale":"2024-04-26T23:18:37.029334+0000","last_undegraded":"2024-04-26T23:18:37.029334+0000","last_fullsized":"2024-04-26T23:18:37.029334+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:14.395241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045390+0000","last_change":"2024-04-26T23:18:32.022720+0000","last_active":"2024-04-26T23:18:37.045390+0000","last_peered":"2024-04-26T23:18:37.045390+0000","last_clean":"2024-04-26T23:18:37.045390+0000","last_became_active":"2024-04-26T23:18:32.022450+0000","last_became_peered":"2024-04-26T23:18:32.022450+0000","last_unstale":"2024-04-26T23:18:37.045390+0000","last_undegraded":"2024-04-26T23:18:37.045390+0000","last_fullsized":"2024-04-26T23:18:37.045390+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:25:39.521124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029950+0000","last_change":"2024-04-26T23:18:31.008154+0000","last_active":"2024-04-26T23:18:37.029950+0000","last_peered":"2024-04-26T23:18:37.029950+0000","last_clean":"2024-04-26T23:18:37.029950+0000","last_became_active":"2024-04-26T23:18:31.007881+0000","last_became_peered":"2024-04-26T23:18:31.007881+0000","last_unstale":"2024-04-26T23:18:37.029950+0000","last_undegraded":"2024-04-26T23:18:37.029950+0000","last_fullsized":"2024-04-26T23:18:37.029950+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:20.581173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046058+0000","last_change":"2024-04-26T23:18:36.047010+0000","last_active":"2024-04-26T23:18:37.046058+0000","last_peered":"2024-04-26T23:18:37.046058+0000","last_clean":"2024-04-26T23:18:37.046058+0000","last_became_active":"2024-04-26T23:18:36.045988+0000","last_became_peered":"2024-04-26T23:18:36.045988+0000","last_unstale":"2024-04-26T23:18:37.046058+0000","last_undegraded":"2024-04-26T23:18:37.046058+0000","last_fullsized":"2024-04-26T23:18:37.046058+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:09.320477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044476+0000","last_change":"2024-04-26T23:18:32.019460+0000","last_active":"2024-04-26T23:18:37.044476+0000","last_peered":"2024-04-26T23:18:37.044476+0000","last_clean":"2024-04-26T23:18:37.044476+0000","last_became_active":"2024-04-26T23:18:32.019146+0000","last_became_peered":"2024-04-26T23:18:32.019146+0000","last_unstale":"2024-04-26T23:18:37.044476+0000","last_undegraded":"2024-04-26T23:18:37.044476+0000","last_fullsized":"2024-04-26T23:18:37.044476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:55:55.562630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045137+0000","last_change":"2024-04-26T23:18:31.010513+0000","last_active":"2024-04-26T23:18:37.045137+0000","last_peered":"2024-04-26T23:18:37.045137+0000","last_clean":"2024-04-26T23:18:37.045137+0000","last_became_active":"2024-04-26T23:18:31.010206+0000","last_became_peered":"2024-04-26T23:18:31.010206+0000","last_unstale":"2024-04-26T23:18:37.045137+0000","last_undegraded":"2024-04-26T23:18:37.045137+0000","last_fullsized":"2024-04-26T23:18:37.045137+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:17:03.522342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030875+0000","last_change":"2024-04-26T23:18:36.039670+0000","last_active":"2024-04-26T23:18:37.030875+0000","last_peered":"2024-04-26T23:18:37.030875+0000","last_clean":"2024-04-26T23:18:37.030875+0000","last_became_active":"2024-04-26T23:18:36.039247+0000","last_became_peered":"2024-04-26T23:18:36.039247+0000","last_unstale":"2024-04-26T23:18:37.030875+0000","last_undegraded":"2024-04-26T23:18:37.030875+0000","last_fullsized":"2024-04-26T23:18:37.030875+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:45:37.196730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045040+0000","last_change":"2024-04-26T23:18:32.029408+0000","last_active":"2024-04-26T23:18:37.045040+0000","last_peered":"2024-04-26T23:18:37.045040+0000","last_clean":"2024-04-26T23:18:37.045040+0000","last_became_active":"2024-04-26T23:18:32.029150+0000","last_became_peered":"2024-04-26T23:18:32.029150+0000","last_unstale":"2024-04-26T23:18:37.045040+0000","last_undegraded":"2024-04-26T23:18:37.045040+0000","last_fullsized":"2024-04-26T23:18:37.045040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:48:42.347270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046379+0000","last_change":"2024-04-26T23:18:31.013885+0000","last_active":"2024-04-26T23:18:37.046379+0000","last_peered":"2024-04-26T23:18:37.046379+0000","last_clean":"2024-04-26T23:18:37.046379+0000","last_became_active":"2024-04-26T23:18:31.013617+0000","last_became_peered":"2024-04-26T23:18:31.013617+0000","last_unstale":"2024-04-26T23:18:37.046379+0000","last_undegraded":"2024-04-26T23:18:37.046379+0000","last_fullsized":"2024-04-26T23:18:37.046379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:08.243628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029721+0000","last_change":"2024-04-26T23:18:36.049794+0000","last_active":"2024-04-26T23:18:37.029721+0000","last_peered":"2024-04-26T23:18:37.029721+0000","last_clean":"2024-04-26T23:18:37.029721+0000","last_became_active":"2024-04-26T23:18:36.044164+0000","last_became_peered":"2024-04-26T23:18:36.044164+0000","last_unstale":"2024-04-26T23:18:37.029721+0000","last_undegraded":"2024-04-26T23:18:37.029721+0000","last_fullsized":"2024-04-26T23:18:37.029721+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:41:38.190204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044157+0000","last_change":"2024-04-26T23:18:36.046931+0000","last_active":"2024-04-26T23:18:37.044157+0000","last_peered":"2024-04-26T23:18:37.044157+0000","last_clean":"2024-04-26T23:18:37.044157+0000","last_became_active":"2024-04-26T23:18:36.045786+0000","last_became_peered":"2024-04-26T23:18:36.045786+0000","last_unstale":"2024-04-26T23:18:37.044157+0000","last_undegraded":"2024-04-26T23:18:37.044157+0000","last_fullsized":"2024-04-26T23:18:37.044157+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:09.320477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.028930+0000","last_change":"2024-04-26T23:18:32.011300+0000","last_active":"2024-04-26T23:18:37.028930+0000","last_peered":"2024-04-26T23:18:37.028930+0000","last_clean":"2024-04-26T23:18:37.028930+0000","last_became_active":"2024-04-26T23:18:32.010998+0000","last_became_peered":"2024-04-26T23:18:32.010998+0000","last_unstale":"2024-04-26T23:18:37.028930+0000","last_undegraded":"2024-04-26T23:18:37.028930+0000","last_fullsized":"2024-04-26T23:18:37.028930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:57:02.596299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044365+0000","last_change":"2024-04-26T23:18:31.011198+0000","last_active":"2024-04-26T23:18:37.044365+0000","last_peered":"2024-04-26T23:18:37.044365+0000","last_clean":"2024-04-26T23:18:37.044365+0000","last_became_active":"2024-04-26T23:18:31.010908+0000","last_became_peered":"2024-04-26T23:18:31.010908+0000","last_unstale":"2024-04-26T23:18:37.044365+0000","last_undegraded":"2024-04-26T23:18:37.044365+0000","last_fullsized":"2024-04-26T23:18:37.044365+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:22.746946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046251+0000","last_change":"2024-04-26T23:18:36.042049+0000","last_active":"2024-04-26T23:18:37.046251+0000","last_peered":"2024-04-26T23:18:37.046251+0000","last_clean":"2024-04-26T23:18:37.046251+0000","last_became_active":"2024-04-26T23:18:36.041661+0000","last_became_peered":"2024-04-26T23:18:36.041661+0000","last_unstale":"2024-04-26T23:18:37.046251+0000","last_undegraded":"2024-04-26T23:18:37.046251+0000","last_fullsized":"2024-04-26T23:18:37.046251+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:09.320477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030801+0000","last_change":"2024-04-26T23:18:32.013285+0000","last_active":"2024-04-26T23:18:37.030801+0000","last_peered":"2024-04-26T23:18:37.030801+0000","last_clean":"2024-04-26T23:18:37.030801+0000","last_became_active":"2024-04-26T23:18:32.013013+0000","last_became_peered":"2024-04-26T23:18:32.013013+0000","last_unstale":"2024-04-26T23:18:37.030801+0000","last_undegraded":"2024-04-26T23:18:37.030801+0000","last_fullsized":"2024-04-26T23:18:37.030801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:54:25.468799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029792+0000","last_change":"2024-04-26T23:18:31.011691+0000","last_active":"2024-04-26T23:18:37.029792+0000","last_peered":"2024-04-26T23:18:37.029792+0000","last_clean":"2024-04-26T23:18:37.029792+0000","last_became_active":"2024-04-26T23:18:31.009521+0000","last_became_peered":"2024-04-26T23:18:31.009521+0000","last_unstale":"2024-04-26T23:18:37.029792+0000","last_undegraded":"2024-04-26T23:18:37.029792+0000","last_fullsized":"2024-04-26T23:18:37.029792+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:22.746946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045573+0000","last_change":"2024-04-26T23:18:31.010360+0000","last_active":"2024-04-26T23:18:37.045573+0000","last_peered":"2024-04-26T23:18:37.045573+0000","last_clean":"2024-04-26T23:18:37.045573+0000","last_became_active":"2024-04-26T23:18:31.010083+0000","last_became_peered":"2024-04-26T23:18:31.010083+0000","last_unstale":"2024-04-26T23:18:37.045573+0000","last_undegraded":"2024-04-26T23:18:37.045573+0000","last_fullsized":"2024-04-26T23:18:37.045573+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:34:02.747044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044258+0000","last_change":"2024-04-26T23:18:32.000871+0000","last_active":"2024-04-26T23:18:37.044258+0000","last_peered":"2024-04-26T23:18:37.044258+0000","last_clean":"2024-04-26T23:18:37.044258+0000","last_became_active":"2024-04-26T23:18:31.999865+0000","last_became_peered":"2024-04-26T23:18:31.999865+0000","last_unstale":"2024-04-26T23:18:37.044258+0000","last_undegraded":"2024-04-26T23:18:37.044258+0000","last_fullsized":"2024-04-26T23:18:37.044258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:57:02.596299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.028327+0000","last_change":"2024-04-26T23:18:36.040907+0000","last_active":"2024-04-26T23:18:37.028327+0000","last_peered":"2024-04-26T23:18:37.028327+0000","last_clean":"2024-04-26T23:18:37.028327+0000","last_became_active":"2024-04-26T23:18:36.040494+0000","last_became_peered":"2024-04-26T23:18:36.040494+0000","last_unstale":"2024-04-26T23:18:37.028327+0000","last_undegraded":"2024-04-26T23:18:37.028327+0000","last_fullsized":"2024-04-26T23:18:37.028327+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:17:08.552285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030078+0000","last_change":"2024-04-26T23:18:31.012855+0000","last_active":"2024-04-26T23:18:37.030078+0000","last_peered":"2024-04-26T23:18:37.030078+0000","last_clean":"2024-04-26T23:18:37.030078+0000","last_became_active":"2024-04-26T23:18:31.012510+0000","last_became_peered":"2024-04-26T23:18:31.012510+0000","last_unstale":"2024-04-26T23:18:37.030078+0000","last_undegraded":"2024-04-26T23:18:37.030078+0000","last_fullsized":"2024-04-26T23:18:37.030078+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:15:53.470195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045692+0000","last_change":"2024-04-26T23:18:32.002285+0000","last_active":"2024-04-26T23:18:37.045692+0000","last_peered":"2024-04-26T23:18:37.045692+0000","last_clean":"2024-04-26T23:18:37.045692+0000","last_became_active":"2024-04-26T23:18:32.001991+0000","last_became_peered":"2024-04-26T23:18:32.001991+0000","last_unstale":"2024-04-26T23:18:37.045692+0000","last_undegraded":"2024-04-26T23:18:37.045692+0000","last_fullsized":"2024-04-26T23:18:37.045692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:45.850060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030744+0000","last_change":"2024-04-26T23:18:36.038108+0000","last_active":"2024-04-26T23:18:37.030744+0000","last_peered":"2024-04-26T23:18:37.030744+0000","last_clean":"2024-04-26T23:18:37.030744+0000","last_became_active":"2024-04-26T23:18:36.037655+0000","last_became_peered":"2024-04-26T23:18:36.037655+0000","last_unstale":"2024-04-26T23:18:37.030744+0000","last_undegraded":"2024-04-26T23:18:37.030744+0000","last_fullsized":"2024-04-26T23:18:37.030744+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:02.473474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044156+0000","last_change":"2024-04-26T23:18:32.014058+0000","last_active":"2024-04-26T23:18:37.044156+0000","last_peered":"2024-04-26T23:18:37.044156+0000","last_clean":"2024-04-26T23:18:37.044156+0000","last_became_active":"2024-04-26T23:18:32.013500+0000","last_became_peered":"2024-04-26T23:18:32.013500+0000","last_unstale":"2024-04-26T23:18:37.044156+0000","last_undegraded":"2024-04-26T23:18:37.044156+0000","last_fullsized":"2024-04-26T23:18:37.044156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:54:25.468799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.027802+0000","last_change":"2024-04-26T23:18:32.018280+0000","last_active":"2024-04-26T23:18:37.027802+0000","last_peered":"2024-04-26T23:18:37.027802+0000","last_clean":"2024-04-26T23:18:37.027802+0000","last_became_active":"2024-04-26T23:18:32.017993+0000","last_became_peered":"2024-04-26T23:18:32.017993+0000","last_unstale":"2024-04-26T23:18:37.027802+0000","last_undegraded":"2024-04-26T23:18:37.027802+0000","last_fullsized":"2024-04-26T23:18:37.027802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:08:32.798896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044791+0000","last_change":"2024-04-26T23:18:32.022457+0000","last_active":"2024-04-26T23:18:37.044791+0000","last_peered":"2024-04-26T23:18:37.044791+0000","last_clean":"2024-04-26T23:18:37.044791+0000","last_became_active":"2024-04-26T23:18:32.021766+0000","last_became_peered":"2024-04-26T23:18:32.021766+0000","last_unstale":"2024-04-26T23:18:37.044791+0000","last_undegraded":"2024-04-26T23:18:37.044791+0000","last_fullsized":"2024-04-26T23:18:37.044791+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:48:51.877664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044133+0000","last_change":"2024-04-26T23:18:32.011541+0000","last_active":"2024-04-26T23:18:37.044133+0000","last_peered":"2024-04-26T23:18:37.044133+0000","last_clean":"2024-04-26T23:18:37.044133+0000","last_became_active":"2024-04-26T23:18:32.009252+0000","last_became_peered":"2024-04-26T23:18:32.009252+0000","last_unstale":"2024-04-26T23:18:37.044133+0000","last_undegraded":"2024-04-26T23:18:37.044133+0000","last_fullsized":"2024-04-26T23:18:37.044133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:54:25.468799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030659+0000","last_change":"2024-04-26T23:18:32.018905+0000","last_active":"2024-04-26T23:18:37.030659+0000","last_peered":"2024-04-26T23:18:37.030659+0000","last_clean":"2024-04-26T23:18:37.030659+0000","last_became_active":"2024-04-26T23:18:32.018586+0000","last_became_peered":"2024-04-26T23:18:32.018586+0000","last_unstale":"2024-04-26T23:18:37.030659+0000","last_undegraded":"2024-04-26T23:18:37.030659+0000","last_fullsized":"2024-04-26T23:18:37.030659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:37:21.828417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029798+0000","last_change":"2024-04-26T23:18:32.011775+0000","last_active":"2024-04-26T23:18:37.029798+0000","last_peered":"2024-04-26T23:18:37.029798+0000","last_clean":"2024-04-26T23:18:37.029798+0000","last_became_active":"2024-04-26T23:18:32.011443+0000","last_became_peered":"2024-04-26T23:18:32.011443+0000","last_unstale":"2024-04-26T23:18:37.029798+0000","last_undegraded":"2024-04-26T23:18:37.029798+0000","last_fullsized":"2024-04-26T23:18:37.029798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:49.567077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046666+0000","last_change":"2024-04-26T23:18:32.026884+0000","last_active":"2024-04-26T23:18:37.046666+0000","last_peered":"2024-04-26T23:18:37.046666+0000","last_clean":"2024-04-26T23:18:37.046666+0000","last_became_active":"2024-04-26T23:18:32.026579+0000","last_became_peered":"2024-04-26T23:18:32.026579+0000","last_unstale":"2024-04-26T23:18:37.046666+0000","last_undegraded":"2024-04-26T23:18:37.046666+0000","last_fullsized":"2024-04-26T23:18:37.046666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:55:55.562630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030747+0000","last_change":"2024-04-26T23:18:32.018063+0000","last_active":"2024-04-26T23:18:37.030747+0000","last_peered":"2024-04-26T23:18:37.030747+0000","last_clean":"2024-04-26T23:18:37.030747+0000","last_became_active":"2024-04-26T23:18:32.017430+0000","last_became_peered":"2024-04-26T23:18:32.017430+0000","last_unstale":"2024-04-26T23:18:37.030747+0000","last_undegraded":"2024-04-26T23:18:37.030747+0000","last_fullsized":"2024-04-26T23:18:37.030747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:48:51.877664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044241+0000","last_change":"2024-04-26T23:18:32.018383+0000","last_active":"2024-04-26T23:18:37.044241+0000","last_peered":"2024-04-26T23:18:37.044241+0000","last_clean":"2024-04-26T23:18:37.044241+0000","last_became_active":"2024-04-26T23:18:32.018101+0000","last_became_peered":"2024-04-26T23:18:32.018101+0000","last_unstale":"2024-04-26T23:18:37.044241+0000","last_undegraded":"2024-04-26T23:18:37.044241+0000","last_fullsized":"2024-04-26T23:18:37.044241+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:49:47.214737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029898+0000","last_change":"2024-04-26T23:18:32.020368+0000","last_active":"2024-04-26T23:18:37.029898+0000","last_peered":"2024-04-26T23:18:37.029898+0000","last_clean":"2024-04-26T23:18:37.029898+0000","last_became_active":"2024-04-26T23:18:32.019278+0000","last_became_peered":"2024-04-26T23:18:32.019278+0000","last_unstale":"2024-04-26T23:18:37.029898+0000","last_undegraded":"2024-04-26T23:18:37.029898+0000","last_fullsized":"2024-04-26T23:18:37.029898+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:51:42.870329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044697+0000","last_change":"2024-04-26T23:18:32.029085+0000","last_active":"2024-04-26T23:18:37.044697+0000","last_peered":"2024-04-26T23:18:37.044697+0000","last_clean":"2024-04-26T23:18:37.044697+0000","last_became_active":"2024-04-26T23:18:32.028769+0000","last_became_peered":"2024-04-26T23:18:32.028769+0000","last_unstale":"2024-04-26T23:18:37.044697+0000","last_undegraded":"2024-04-26T23:18:37.044697+0000","last_fullsized":"2024-04-26T23:18:37.044697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:55.832329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.043961+0000","last_change":"2024-04-26T23:18:32.014926+0000","last_active":"2024-04-26T23:18:37.043961+0000","last_peered":"2024-04-26T23:18:37.043961+0000","last_clean":"2024-04-26T23:18:37.043961+0000","last_became_active":"2024-04-26T23:18:32.014414+0000","last_became_peered":"2024-04-26T23:18:32.014414+0000","last_unstale":"2024-04-26T23:18:37.043961+0000","last_undegraded":"2024-04-26T23:18:37.043961+0000","last_fullsized":"2024-04-26T23:18:37.043961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:58:44.698418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045548+0000","last_change":"2024-04-26T23:18:32.019959+0000","last_active":"2024-04-26T23:18:37.045548+0000","last_peered":"2024-04-26T23:18:37.045548+0000","last_clean":"2024-04-26T23:18:37.045548+0000","last_became_active":"2024-04-26T23:18:32.019690+0000","last_became_peered":"2024-04-26T23:18:32.019690+0000","last_unstale":"2024-04-26T23:18:37.045548+0000","last_undegraded":"2024-04-26T23:18:37.045548+0000","last_fullsized":"2024-04-26T23:18:37.045548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:49.567077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.047090+0000","last_change":"2024-04-26T23:18:32.023444+0000","last_active":"2024-04-26T23:18:37.047090+0000","last_peered":"2024-04-26T23:18:37.047090+0000","last_clean":"2024-04-26T23:18:37.047090+0000","last_became_active":"2024-04-26T23:18:32.023211+0000","last_became_peered":"2024-04-26T23:18:32.023211+0000","last_unstale":"2024-04-26T23:18:37.047090+0000","last_undegraded":"2024-04-26T23:18:37.047090+0000","last_fullsized":"2024-04-26T23:18:37.047090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:57:19.300614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.027980+0000","last_change":"2024-04-26T23:18:32.011378+0000","last_active":"2024-04-26T23:18:37.027980+0000","last_peered":"2024-04-26T23:18:37.027980+0000","last_clean":"2024-04-26T23:18:37.027980+0000","last_became_active":"2024-04-26T23:18:32.011101+0000","last_became_peered":"2024-04-26T23:18:32.011101+0000","last_unstale":"2024-04-26T23:18:37.027980+0000","last_undegraded":"2024-04-26T23:18:37.027980+0000","last_fullsized":"2024-04-26T23:18:37.027980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:53:16.817858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045910+0000","last_change":"2024-04-26T23:18:32.028996+0000","last_active":"2024-04-26T23:18:37.045910+0000","last_peered":"2024-04-26T23:18:37.045910+0000","last_clean":"2024-04-26T23:18:37.045910+0000","last_became_active":"2024-04-26T23:18:32.028697+0000","last_became_peered":"2024-04-26T23:18:32.028697+0000","last_unstale":"2024-04-26T23:18:37.045910+0000","last_undegraded":"2024-04-26T23:18:37.045910+0000","last_fullsized":"2024-04-26T23:18:37.045910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:04:13.811589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044240+0000","last_change":"2024-04-26T23:18:32.018149+0000","last_active":"2024-04-26T23:18:37.044240+0000","last_peered":"2024-04-26T23:18:37.044240+0000","last_clean":"2024-04-26T23:18:37.044240+0000","last_became_active":"2024-04-26T23:18:32.017657+0000","last_became_peered":"2024-04-26T23:18:32.017657+0000","last_unstale":"2024-04-26T23:18:37.044240+0000","last_undegraded":"2024-04-26T23:18:37.044240+0000","last_fullsized":"2024-04-26T23:18:37.044240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:55:55.562630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029015+0000","last_change":"2024-04-26T23:18:32.000678+0000","last_active":"2024-04-26T23:18:37.029015+0000","last_peered":"2024-04-26T23:18:37.029015+0000","last_clean":"2024-04-26T23:18:37.029015+0000","last_became_active":"2024-04-26T23:18:32.000354+0000","last_became_peered":"2024-04-26T23:18:32.000354+0000","last_unstale":"2024-04-26T23:18:37.029015+0000","last_undegraded":"2024-04-26T23:18:37.029015+0000","last_fullsized":"2024-04-26T23:18:37.029015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:51:42.870329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.043891+0000","last_change":"2024-04-26T23:18:31.999586+0000","last_active":"2024-04-26T23:18:37.043891+0000","last_peered":"2024-04-26T23:18:37.043891+0000","last_clean":"2024-04-26T23:18:37.043891+0000","last_became_active":"2024-04-26T23:18:31.999185+0000","last_became_peered":"2024-04-26T23:18:31.999185+0000","last_unstale":"2024-04-26T23:18:37.043891+0000","last_undegraded":"2024-04-26T23:18:37.043891+0000","last_fullsized":"2024-04-26T23:18:37.043891+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:51:42.870329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029862+0000","last_change":"2024-04-26T23:18:32.024681+0000","last_active":"2024-04-26T23:18:37.029862+0000","last_peered":"2024-04-26T23:18:37.029862+0000","last_clean":"2024-04-26T23:18:37.029862+0000","last_became_active":"2024-04-26T23:18:32.024372+0000","last_became_peered":"2024-04-26T23:18:32.024372+0000","last_unstale":"2024-04-26T23:18:37.029862+0000","last_undegraded":"2024-04-26T23:18:37.029862+0000","last_fullsized":"2024-04-26T23:18:37.029862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:49.567077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045305+0000","last_change":"2024-04-26T23:18:32.011333+0000","last_active":"2024-04-26T23:18:37.045305+0000","last_peered":"2024-04-26T23:18:37.045305+0000","last_clean":"2024-04-26T23:18:37.045305+0000","last_became_active":"2024-04-26T23:18:32.011052+0000","last_became_peered":"2024-04-26T23:18:32.011052+0000","last_unstale":"2024-04-26T23:18:37.045305+0000","last_undegraded":"2024-04-26T23:18:37.045305+0000","last_fullsized":"2024-04-26T23:18:37.045305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:29.991591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046025+0000","last_change":"2024-04-26T23:18:32.021692+0000","last_active":"2024-04-26T23:18:37.046025+0000","last_peered":"2024-04-26T23:18:37.046025+0000","last_clean":"2024-04-26T23:18:37.046025+0000","last_became_active":"2024-04-26T23:18:32.021426+0000","last_became_peered":"2024-04-26T23:18:32.021426+0000","last_unstale":"2024-04-26T23:18:37.046025+0000","last_undegraded":"2024-04-26T23:18:37.046025+0000","last_fullsized":"2024-04-26T23:18:37.046025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:57:02.596299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046525+0000","last_change":"2024-04-26T23:18:32.020976+0000","last_active":"2024-04-26T23:18:37.046525+0000","last_peered":"2024-04-26T23:18:37.046525+0000","last_clean":"2024-04-26T23:18:37.046525+0000","last_became_active":"2024-04-26T23:18:32.020653+0000","last_became_peered":"2024-04-26T23:18:32.020653+0000","last_unstale":"2024-04-26T23:18:37.046525+0000","last_undegraded":"2024-04-26T23:18:37.046525+0000","last_fullsized":"2024-04-26T23:18:37.046525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:51:42.870329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.047206+0000","last_change":"2024-04-26T23:18:32.030726+0000","last_active":"2024-04-26T23:18:37.047206+0000","last_peered":"2024-04-26T23:18:37.047206+0000","last_clean":"2024-04-26T23:18:37.047206+0000","last_became_active":"2024-04-26T23:18:32.030451+0000","last_became_peered":"2024-04-26T23:18:32.030451+0000","last_unstale":"2024-04-26T23:18:37.047206+0000","last_undegraded":"2024-04-26T23:18:37.047206+0000","last_fullsized":"2024-04-26T23:18:37.047206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:26.533404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030676+0000","last_change":"2024-04-26T23:18:32.020062+0000","last_active":"2024-04-26T23:18:37.030676+0000","last_peered":"2024-04-26T23:18:37.030676+0000","last_clean":"2024-04-26T23:18:37.030676+0000","last_became_active":"2024-04-26T23:18:32.019651+0000","last_became_peered":"2024-04-26T23:18:32.019651+0000","last_unstale":"2024-04-26T23:18:37.030676+0000","last_undegraded":"2024-04-26T23:18:37.030676+0000","last_fullsized":"2024-04-26T23:18:37.030676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:17:04.527690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046652+0000","last_change":"2024-04-26T23:18:32.017808+0000","last_active":"2024-04-26T23:18:37.046652+0000","last_peered":"2024-04-26T23:18:37.046652+0000","last_clean":"2024-04-26T23:18:37.046652+0000","last_became_active":"2024-04-26T23:18:32.017162+0000","last_became_peered":"2024-04-26T23:18:32.017162+0000","last_unstale":"2024-04-26T23:18:37.046652+0000","last_undegraded":"2024-04-26T23:18:37.046652+0000","last_fullsized":"2024-04-26T23:18:37.046652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:48:51.877664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044589+0000","last_change":"2024-04-26T23:18:32.023064+0000","last_active":"2024-04-26T23:18:37.044589+0000","last_peered":"2024-04-26T23:18:37.044589+0000","last_clean":"2024-04-26T23:18:37.044589+0000","last_became_active":"2024-04-26T23:18:32.022771+0000","last_became_peered":"2024-04-26T23:18:32.022771+0000","last_unstale":"2024-04-26T23:18:37.044589+0000","last_undegraded":"2024-04-26T23:18:37.044589+0000","last_fullsized":"2024-04-26T23:18:37.044589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:37:21.828417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046963+0000","last_change":"2024-04-26T23:18:32.022130+0000","last_active":"2024-04-26T23:18:37.046963+0000","last_peered":"2024-04-26T23:18:37.046963+0000","last_clean":"2024-04-26T23:18:37.046963+0000","last_became_active":"2024-04-26T23:18:32.021842+0000","last_became_peered":"2024-04-26T23:18:32.021842+0000","last_unstale":"2024-04-26T23:18:37.046963+0000","last_undegraded":"2024-04-26T23:18:37.046963+0000","last_fullsized":"2024-04-26T23:18:37.046963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:10.370646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044765+0000","last_change":"2024-04-26T23:18:32.020676+0000","last_active":"2024-04-26T23:18:37.044765+0000","last_peered":"2024-04-26T23:18:37.044765+0000","last_clean":"2024-04-26T23:18:37.044765+0000","last_became_active":"2024-04-26T23:18:32.020391+0000","last_became_peered":"2024-04-26T23:18:32.020391+0000","last_unstale":"2024-04-26T23:18:37.044765+0000","last_undegraded":"2024-04-26T23:18:37.044765+0000","last_fullsized":"2024-04-26T23:18:37.044765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:05.295882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044140+0000","last_change":"2024-04-26T23:18:32.006201+0000","last_active":"2024-04-26T23:18:37.044140+0000","last_peered":"2024-04-26T23:18:37.044140+0000","last_clean":"2024-04-26T23:18:37.044140+0000","last_became_active":"2024-04-26T23:18:32.005906+0000","last_became_peered":"2024-04-26T23:18:32.005906+0000","last_unstale":"2024-04-26T23:18:37.044140+0000","last_undegraded":"2024-04-26T23:18:37.044140+0000","last_fullsized":"2024-04-26T23:18:37.044140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:59:18.268816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030075+0000","last_change":"2024-04-26T23:18:32.003546+0000","last_active":"2024-04-26T23:18:37.030075+0000","last_peered":"2024-04-26T23:18:37.030075+0000","last_clean":"2024-04-26T23:18:37.030075+0000","last_became_active":"2024-04-26T23:18:32.003204+0000","last_became_peered":"2024-04-26T23:18:32.003204+0000","last_unstale":"2024-04-26T23:18:37.030075+0000","last_undegraded":"2024-04-26T23:18:37.030075+0000","last_fullsized":"2024-04-26T23:18:37.030075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:37:21.828417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.047079+0000","last_change":"2024-04-26T23:18:32.026678+0000","last_active":"2024-04-26T23:18:37.047079+0000","last_peered":"2024-04-26T23:18:37.047079+0000","last_clean":"2024-04-26T23:18:37.047079+0000","last_became_active":"2024-04-26T23:18:32.026390+0000","last_became_peered":"2024-04-26T23:18:32.026390+0000","last_unstale":"2024-04-26T23:18:37.047079+0000","last_undegraded":"2024-04-26T23:18:37.047079+0000","last_fullsized":"2024-04-26T23:18:37.047079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:16:12.520377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542153,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":847872,"data_stored":690086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542155,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":262144,"data_stored":102978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542154,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":99226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542154,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":99262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31432,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340408,"statfs":{"total":96636764160,"available":96604577792,"internally_reserved":0,"allocated":262144,"data_stored":102978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":249856,"data_stored":99136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31432,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340408,"statfs":{"total":96636764160,"available":96604577792,"internally_reserved":0,"allocated":258048,"data_stored":99296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":856064,"data_stored":690142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:18:55.435 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T23:18:55.436 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-26T23:18:55.436 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-26T23:18:55.436 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-26T23:18:55.436 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-26T23:18:55.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.571+0000 7fd8b97b6640 1 Processor -- start 2024-04-26T23:18:55.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.571+0000 7fd8b97b6640 1 -- start start 2024-04-26T23:18:55.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.572+0000 7fd8b97b6640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd8b4070750 0x7fd8b40b56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:55.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.572+0000 7fd8b97b6640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd8b40af6e0 0x7fd8b40afb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:55.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.572+0000 7fd8b97b6640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8b40b5e30 con 0x7fd8b4072fb0 2024-04-26T23:18:55.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.572+0000 7fd8b97b6640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fd8b40b5c00 con 0x7fd8b4070750 2024-04-26T23:18:55.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.572+0000 7fd8b97b6640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fd8b406f2b0 con 0x7fd8b40af6e0 2024-04-26T23:18:55.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.572+0000 7fd8b27fc640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd8b4070750 0x7fd8b40b56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:55.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.572+0000 7fd8b27fc640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd8b4070750 0x7fd8b40b56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58550/0 (socket says 172.21.15.47:58550) 2024-04-26T23:18:55.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.572+0000 7fd8b27fc640 1 -- 172.21.15.47:0/664205940 learned_addr learned my addr 172.21.15.47:0/664205940 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:55.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.573+0000 7fd8b37fe640 1 --2- 172.21.15.47:0/664205940 >> v2:172.21.15.203:3300/0 conn(0x7fd8b40af6e0 0x7fd8b40afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:55.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.573+0000 7fd8b27fc640 1 -- 172.21.15.47:0/664205940 >> v2:172.21.15.203:3300/0 conn(0x7fd8b40af6e0 msgr2=0x7fd8b40afb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:55.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.573+0000 7fd8b27fc640 1 --2- 172.21.15.47:0/664205940 >> v2:172.21.15.203:3300/0 conn(0x7fd8b40af6e0 0x7fd8b40afb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:55.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.573+0000 7fd8b27fc640 1 -- 172.21.15.47:0/664205940 >> v1:172.21.15.203:6789/0 conn(0x7fd8b4072fb0 legacy=0x7fd8b4073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:55.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.573+0000 7fd8b27fc640 1 -- 172.21.15.47:0/664205940 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8b40b6110 con 0x7fd8b4070750 2024-04-26T23:18:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.573+0000 7fd8b27fc640 1 --2- 172.21.15.47:0/664205940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8b4070750 0x7fd8b40b56c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4071b40 tx=0x7fd89c02c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=335a77e8a3835aae server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.574+0000 7fd8b1ffb640 1 -- 172.21.15.47:0/664205940 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd89c036020 con 0x7fd8b4070750 2024-04-26T23:18:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.574+0000 7fd8b1ffb640 1 -- 172.21.15.47:0/664205940 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd89c032640 con 0x7fd8b4070750 2024-04-26T23:18:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.574+0000 7fd8b1ffb640 1 -- 172.21.15.47:0/664205940 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd89c032a10 con 0x7fd8b4070750 2024-04-26T23:18:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.574+0000 7fd8b97b6640 1 -- 172.21.15.47:0/664205940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8b4070750 msgr2=0x7fd8b40b56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.574+0000 7fd8b97b6640 1 --2- 172.21.15.47:0/664205940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8b4070750 0x7fd8b40b56c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4071b40 tx=0x7fd89c02c940 comp rx=0 tx=0).stop 2024-04-26T23:18:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.574+0000 7fd8b97b6640 1 -- 172.21.15.47:0/664205940 shutdown_connections 2024-04-26T23:18:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.574+0000 7fd8b97b6640 1 --2- 172.21.15.47:0/664205940 >> v2:172.21.15.203:3300/0 conn(0x7fd8b40af6e0 0x7fd8b40afb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.574+0000 7fd8b97b6640 1 --2- 172.21.15.47:0/664205940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8b4070750 0x7fd8b40b56c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.574+0000 7fd8b97b6640 1 -- 172.21.15.47:0/664205940 >> 172.21.15.47:0/664205940 conn(0x7fd8b40b39b0 msgr2=0x7fd8b40b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.574+0000 7fd8b97b6640 1 -- 172.21.15.47:0/664205940 shutdown_connections 2024-04-26T23:18:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.574+0000 7fd8b97b6640 1 -- 172.21.15.47:0/664205940 wait complete. 2024-04-26T23:18:55.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.576+0000 7fd8b97b6640 1 Processor -- start 2024-04-26T23:18:55.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.576+0000 7fd8b97b6640 1 -- start start 2024-04-26T23:18:55.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.576+0000 7fd8b97b6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8b4070750 0x7fd8b40cb030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:55.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.576+0000 7fd8b97b6640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8b4072fb0 0x7fd8b41498f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:55.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.576+0000 7fd8b97b6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8b40af6e0 0x7fd8b41f5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:55.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.576+0000 7fd8b97b6640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b415c2e0 con 0x7fd8b40af6e0 2024-04-26T23:18:55.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.576+0000 7fd8b97b6640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b415c160 con 0x7fd8b4070750 2024-04-26T23:18:55.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.576+0000 7fd8b97b6640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd8b40b00b0 con 0x7fd8b4072fb0 2024-04-26T23:18:55.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.576+0000 7fd8b37fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8b40af6e0 0x7fd8b41f5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:55.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.576+0000 7fd8b37fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8b40af6e0 0x7fd8b41f5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58556/0 (socket says 172.21.15.47:58556) 2024-04-26T23:18:55.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.576+0000 7fd8b37fe640 1 -- 172.21.15.47:0/4264679981 learned_addr learned my addr 172.21.15.47:0/4264679981 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:55.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.577+0000 7fd8b27fc640 1 --2- 172.21.15.47:0/4264679981 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8b4072fb0 0x7fd8b41498f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:55.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.577+0000 7fd8b2ffd640 1 --2- 172.21.15.47:0/4264679981 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8b4070750 0x7fd8b40cb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:55.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.577+0000 7fd8b37fe640 1 -- 172.21.15.47:0/4264679981 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8b4072fb0 msgr2=0x7fd8b41498f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:55.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.577+0000 7fd8b37fe640 1 --2- 172.21.15.47:0/4264679981 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8b4072fb0 0x7fd8b41498f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:55.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.577+0000 7fd8b37fe640 1 -- 172.21.15.47:0/4264679981 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8b4070750 msgr2=0x7fd8b40cb030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:55.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.577+0000 7fd8b37fe640 1 --2- 172.21.15.47:0/4264679981 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8b4070750 0x7fd8b40cb030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:55.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.577+0000 7fd8b37fe640 1 -- 172.21.15.47:0/4264679981 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8b40cb6c0 con 0x7fd8b40af6e0 2024-04-26T23:18:55.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.577+0000 7fd8b2ffd640 1 --2- 172.21.15.47:0/4264679981 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8b4070750 0x7fd8b40cb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:18:55.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.577+0000 7fd8b37fe640 1 --2- 172.21.15.47:0/4264679981 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8b40af6e0 0x7fd8b41f5440 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4007e60 tx=0x7fd8a400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:55.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.577+0000 7fd8ab7fe640 1 -- 172.21.15.47:0/4264679981 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a4014420 con 0x7fd8b40af6e0 2024-04-26T23:18:55.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.577+0000 7fd8ab7fe640 1 -- 172.21.15.47:0/4264679981 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8a4019070 con 0x7fd8b40af6e0 2024-04-26T23:18:55.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.577+0000 7fd8b97b6640 1 -- 172.21.15.47:0/4264679981 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8b40cd080 con 0x7fd8b40af6e0 2024-04-26T23:18:55.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.577+0000 7fd8ab7fe640 1 -- 172.21.15.47:0/4264679981 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a4014820 con 0x7fd8b40af6e0 2024-04-26T23:18:55.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.577+0000 7fd8b97b6640 1 -- 172.21.15.47:0/4264679981 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8b40ccd50 con 0x7fd8b40af6e0 2024-04-26T23:18:55.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.578+0000 7fd8b97b6640 1 -- 172.21.15.47:0/4264679981 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8b40694f0 con 0x7fd8b40af6e0 2024-04-26T23:18:55.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.579+0000 7fd8ab7fe640 1 -- 172.21.15.47:0/4264679981 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd8a40149c0 con 0x7fd8b40af6e0 2024-04-26T23:18:55.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.579+0000 7fd8ab7fe640 1 --2- 172.21.15.47:0/4264679981 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd89406c8f0 0x7fd89406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:55.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.579+0000 7fd8ab7fe640 1 -- 172.21.15.47:0/4264679981 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd8a408c6f0 con 0x7fd8b40af6e0 2024-04-26T23:18:55.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.579+0000 7fd8b2ffd640 1 --2- 172.21.15.47:0/4264679981 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd89406c8f0 0x7fd89406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:55.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.580+0000 7fd8b2ffd640 1 --2- 172.21.15.47:0/4264679981 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd89406c8f0 0x7fd89406edb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac001680 tx=0x7fd8ac009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:55.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.583+0000 7fd8ab7fe640 1 -- 172.21.15.47:0/4264679981 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd8a405a7e0 con 0x7fd8b40af6e0 2024-04-26T23:18:55.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.817+0000 7fd8b97b6640 1 -- 172.21.15.47:0/4264679981 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fd8b40b59e0 con 0x7fd8b40af6e0 2024-04-26T23:18:55.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.820+0000 7fd8ab7fe640 1 -- 172.21.15.47:0/4264679981 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+174141 (secure 0 0 0) 0x7fd8a405a0e0 con 0x7fd8b40af6e0 2024-04-26T23:18:55.823 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:18:55.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.822+0000 7fd8b97b6640 1 -- 172.21.15.47:0/4264679981 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd89406c8f0 msgr2=0x7fd89406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:55.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.822+0000 7fd8b97b6640 1 --2- 172.21.15.47:0/4264679981 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd89406c8f0 0x7fd89406edb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac001680 tx=0x7fd8ac009040 comp rx=0 tx=0).stop 2024-04-26T23:18:55.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.822+0000 7fd8b97b6640 1 -- 172.21.15.47:0/4264679981 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8b40af6e0 msgr2=0x7fd8b41f5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:55.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.822+0000 7fd8b97b6640 1 --2- 172.21.15.47:0/4264679981 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8b40af6e0 0x7fd8b41f5440 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4007e60 tx=0x7fd8a400e4b0 comp rx=0 tx=0).stop 2024-04-26T23:18:55.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.822+0000 7fd8b97b6640 1 -- 172.21.15.47:0/4264679981 shutdown_connections 2024-04-26T23:18:55.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.822+0000 7fd8b97b6640 1 --2- 172.21.15.47:0/4264679981 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd89406c8f0 0x7fd89406edb0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:55.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.822+0000 7fd8b97b6640 1 --2- 172.21.15.47:0/4264679981 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8b40af6e0 0x7fd8b41f5440 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:55.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.823+0000 7fd8b97b6640 1 --2- 172.21.15.47:0/4264679981 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8b4072fb0 0x7fd8b41498f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:55.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.823+0000 7fd8b97b6640 1 --2- 172.21.15.47:0/4264679981 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8b4070750 0x7fd8b40cb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:55.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.823+0000 7fd8b97b6640 1 -- 172.21.15.47:0/4264679981 >> 172.21.15.47:0/4264679981 conn(0x7fd8b40b39b0 msgr2=0x7fd8b40b1920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:55.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.823+0000 7fd8b97b6640 1 -- 172.21.15.47:0/4264679981 shutdown_connections 2024-04-26T23:18:55.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.823+0000 7fd8b97b6640 1 -- 172.21.15.47:0/4264679981 wait complete. 2024-04-26T23:18:55.834 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":5,"flags":0,"active_gid":4099,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":992162732},{"type":"v1","addr":"172.21.15.203:6833","nonce":992162732}]},"active_addr":"172.21.15.203:6833/992162732","active_change":"2024-04-26T23:18:10.912154+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4116,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.203:0","nonce":2792186024}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.203:0","nonce":2759121318}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.203:0","nonce":2656553071}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.203:0","nonce":1872320562}]}]} 2024-04-26T23:18:55.837 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-26T23:18:55.837 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-26T23:18:55.837 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:18:55.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.938+0000 7fbb55daf640 1 Processor -- start 2024-04-26T23:18:55.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.938+0000 7fbb55daf640 1 -- start start 2024-04-26T23:18:55.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.938+0000 7fbb55daf640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbb5015f4f0 0x7fbb5006e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:55.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.938+0000 7fbb55daf640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb50163b90 0x7fbb50162ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:55.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.938+0000 7fbb55daf640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbb50163530 0x7fbb500d0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:55.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.938+0000 7fbb55daf640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fbb5015f8d0 con 0x7fbb5015f4f0 2024-04-26T23:18:55.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.938+0000 7fbb55daf640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fbb500b2890 con 0x7fbb50163530 2024-04-26T23:18:55.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.938+0000 7fbb55daf640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fbb500b2b70 con 0x7fbb50163b90 2024-04-26T23:18:55.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.939+0000 7fbb4f7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbb5015f4f0 0x7fbb5006e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:55.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.939+0000 7fbb4f7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbb5015f4f0 0x7fbb5006e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58572/0 (socket says 172.21.15.47:58572) 2024-04-26T23:18:55.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.939+0000 7fbb4f7fe640 1 -- 172.21.15.47:0/89771964 learned_addr learned my addr 172.21.15.47:0/89771964 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:55.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.939+0000 7fbb4effd640 1 --2- 172.21.15.47:0/89771964 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb50163b90 0x7fbb50162ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:55.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.939+0000 7fbb4ffff640 1 --2- 172.21.15.47:0/89771964 >> v2:172.21.15.203:3300/0 conn(0x7fbb50163530 0x7fbb500d0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:55.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.939+0000 7fbb4f7fe640 1 -- 172.21.15.47:0/89771964 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb50163b90 msgr2=0x7fbb50162ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:55.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.939+0000 7fbb4f7fe640 1 --2- 172.21.15.47:0/89771964 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb50163b90 0x7fbb50162ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:55.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.939+0000 7fbb4f7fe640 1 -- 172.21.15.47:0/89771964 >> v2:172.21.15.203:3300/0 conn(0x7fbb50163530 msgr2=0x7fbb500d0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:55.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.939+0000 7fbb4f7fe640 1 --2- 172.21.15.47:0/89771964 >> v2:172.21.15.203:3300/0 conn(0x7fbb50163530 0x7fbb500d0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:55.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.939+0000 7fbb4f7fe640 1 -- 172.21.15.47:0/89771964 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb500b2e50 con 0x7fbb5015f4f0 2024-04-26T23:18:55.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.940+0000 7fbb4f7fe640 1 --2- 172.21.15.47:0/89771964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb5015f4f0 0x7fbb5006e3b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbb50164090 tx=0x7fbb4002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8271c7106dc30bf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:55.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.940+0000 7fbb4e7fc640 1 -- 172.21.15.47:0/89771964 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb40037070 con 0x7fbb5015f4f0 2024-04-26T23:18:55.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.940+0000 7fbb4e7fc640 1 -- 172.21.15.47:0/89771964 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb40032440 con 0x7fbb5015f4f0 2024-04-26T23:18:55.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.940+0000 7fbb4e7fc640 1 -- 172.21.15.47:0/89771964 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb40032810 con 0x7fbb5015f4f0 2024-04-26T23:18:55.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.940+0000 7fbb55daf640 1 -- 172.21.15.47:0/89771964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb5015f4f0 msgr2=0x7fbb5006e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:55.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.940+0000 7fbb55daf640 1 --2- 172.21.15.47:0/89771964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb5015f4f0 0x7fbb5006e3b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbb50164090 tx=0x7fbb4002a140 comp rx=0 tx=0).stop 2024-04-26T23:18:55.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.940+0000 7fbb55daf640 1 -- 172.21.15.47:0/89771964 shutdown_connections 2024-04-26T23:18:55.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.940+0000 7fbb55daf640 1 --2- 172.21.15.47:0/89771964 >> v2:172.21.15.203:3300/0 conn(0x7fbb50163530 0x7fbb500d0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:55.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.940+0000 7fbb55daf640 1 --2- 172.21.15.47:0/89771964 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb50163b90 0x7fbb50162ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:55.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.940+0000 7fbb55daf640 1 --2- 172.21.15.47:0/89771964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb5015f4f0 0x7fbb5006e3b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:55.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.940+0000 7fbb55daf640 1 -- 172.21.15.47:0/89771964 >> 172.21.15.47:0/89771964 conn(0x7fbb500d2e00 msgr2=0x7fbb500d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:55.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.940+0000 7fbb55daf640 1 -- 172.21.15.47:0/89771964 shutdown_connections 2024-04-26T23:18:55.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.940+0000 7fbb55daf640 1 -- 172.21.15.47:0/89771964 wait complete. 2024-04-26T23:18:55.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.942+0000 7fbb55daf640 1 Processor -- start 2024-04-26T23:18:55.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.942+0000 7fbb55daf640 1 -- start start 2024-04-26T23:18:55.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.942+0000 7fbb55daf640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb5015f4f0 0x7fbb5015b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:55.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.942+0000 7fbb55daf640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb50163530 0x7fbb5009f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:55.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.943+0000 7fbb55daf640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb50163b90 0x7fbb500c9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:55.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.943+0000 7fbb55daf640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fbb5006e8f0 con 0x7fbb50163530 2024-04-26T23:18:55.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.943+0000 7fbb55daf640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbb500afbb0 con 0x7fbb5015f4f0 2024-04-26T23:18:55.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.943+0000 7fbb55daf640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fbb5006ea70 con 0x7fbb50163b90 2024-04-26T23:18:55.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.943+0000 7fbb4effd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb50163530 0x7fbb5009f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:55.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.943+0000 7fbb4effd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb50163530 0x7fbb5009f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58578/0 (socket says 172.21.15.47:58578) 2024-04-26T23:18:55.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.943+0000 7fbb4effd640 1 -- 172.21.15.47:0/3206329824 learned_addr learned my addr 172.21.15.47:0/3206329824 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:55.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.943+0000 7fbb4f7fe640 1 --2- 172.21.15.47:0/3206329824 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb5015f4f0 0x7fbb5015b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:55.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.943+0000 7fbb4ffff640 1 --2- 172.21.15.47:0/3206329824 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb50163b90 0x7fbb500c9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:55.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.943+0000 7fbb4effd640 1 -- 172.21.15.47:0/3206329824 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb50163b90 msgr2=0x7fbb500c9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:55.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.943+0000 7fbb4effd640 1 --2- 172.21.15.47:0/3206329824 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb50163b90 0x7fbb500c9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:55.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.943+0000 7fbb4effd640 1 -- 172.21.15.47:0/3206329824 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb5015f4f0 msgr2=0x7fbb5015b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:55.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.943+0000 7fbb4effd640 1 --2- 172.21.15.47:0/3206329824 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb5015f4f0 0x7fbb5015b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:55.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.943+0000 7fbb4effd640 1 -- 172.21.15.47:0/3206329824 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb501f8990 con 0x7fbb50163530 2024-04-26T23:18:55.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.943+0000 7fbb4ffff640 1 --2- 172.21.15.47:0/3206329824 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb50163b90 0x7fbb500c9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:18:55.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.943+0000 7fbb4f7fe640 1 --2- 172.21.15.47:0/3206329824 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb5015f4f0 0x7fbb5015b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:18:55.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.943+0000 7fbb4effd640 1 --2- 172.21.15.47:0/3206329824 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb50163530 0x7fbb5009f520 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbb4400d5c0 tx=0x7fbb4400da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:55.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.943+0000 7fbb37fff640 1 -- 172.21.15.47:0/3206329824 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb440125f0 con 0x7fbb50163530 2024-04-26T23:18:55.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.943+0000 7fbb37fff640 1 -- 172.21.15.47:0/3206329824 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb44019070 con 0x7fbb50163530 2024-04-26T23:18:55.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.943+0000 7fbb55daf640 1 -- 172.21.15.47:0/3206329824 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb501f8c70 con 0x7fbb50163530 2024-04-26T23:18:55.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.943+0000 7fbb37fff640 1 -- 172.21.15.47:0/3206329824 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb440129d0 con 0x7fbb50163530 2024-04-26T23:18:55.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.943+0000 7fbb55daf640 1 -- 172.21.15.47:0/3206329824 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb501f9230 con 0x7fbb50163530 2024-04-26T23:18:55.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.944+0000 7fbb55daf640 1 -- 172.21.15.47:0/3206329824 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb500694f0 con 0x7fbb50163530 2024-04-26T23:18:55.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.945+0000 7fbb37fff640 1 -- 172.21.15.47:0/3206329824 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbb44012b70 con 0x7fbb50163530 2024-04-26T23:18:55.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.945+0000 7fbb37fff640 1 --2- 172.21.15.47:0/3206329824 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fbb3c06c8f0 0x7fbb3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:55.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.945+0000 7fbb37fff640 1 -- 172.21.15.47:0/3206329824 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fbb4408c260 con 0x7fbb50163530 2024-04-26T23:18:55.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.946+0000 7fbb4f7fe640 1 --2- 172.21.15.47:0/3206329824 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fbb3c06c8f0 0x7fbb3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:55.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.946+0000 7fbb4f7fe640 1 --2- 172.21.15.47:0/3206329824 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fbb3c06c8f0 0x7fbb3c06edb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbb40034000 tx=0x7fbb4002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:55.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:55.950+0000 7fbb37fff640 1 -- 172.21.15.47:0/3206329824 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbb4405a340 con 0x7fbb50163530 2024-04-26T23:18:56.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.140+0000 7fbb55daf640 1 -- 172.21.15.47:0/3206329824 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbb500b8d80 con 0x7fbb50163530 2024-04-26T23:18:56.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.141+0000 7fbb37fff640 1 -- 172.21.15.47:0/3206329824 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18478 (secure 0 0 0) 0x7fbb44059c40 con 0x7fbb50163530 2024-04-26T23:18:56.142 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:18:56.142 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":24,"fsid":"399006d3-4db1-43e6-ba81-a79a276eabfd","created":"2024-04-26T23:18:06.731211+0000","modified":"2024-04-26T23:18:36.402132+0000","last_up_change":"2024-04-26T23:18:17.808736+0000","last_in_change":"2024-04-26T23:18:08.945859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:18:16.937492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:18:29.033575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:18:30.318499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:18:34.014536+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ee091567-fea0-4a14-bf4e-0e16c20bb2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6819","nonce":326520779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6821","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6823","nonce":326520779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6829","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6831","nonce":326520779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6825","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6827","nonce":326520779}]},"public_addr":"172.21.15.47:6819/326520779","cluster_addr":"172.21.15.47:6823/326520779","heartbeat_back_addr":"172.21.15.47:6831/326520779","heartbeat_front_addr":"172.21.15.47:6827/326520779","state":["exists","up"]},{"osd":1,"uuid":"c14440a0-bba5-4931-a6db-b736c08e869b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6809","nonce":1912610931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6811","nonce":1912610931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6815","nonce":1912610931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6813","nonce":1912610931}]},"public_addr":"172.21.15.47:6809/1912610931","cluster_addr":"172.21.15.47:6811/1912610931","heartbeat_back_addr":"172.21.15.47:6815/1912610931","heartbeat_front_addr":"172.21.15.47:6813/1912610931","state":["exists","up"]},{"osd":2,"uuid":"3940f8de-c9b2-4cd3-a911-9c301729eb30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6801","nonce":2569869305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6803","nonce":2569869305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6807","nonce":2569869305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6805","nonce":2569869305}]},"public_addr":"172.21.15.47:6801/2569869305","cluster_addr":"172.21.15.47:6803/2569869305","heartbeat_back_addr":"172.21.15.47:6807/2569869305","heartbeat_front_addr":"172.21.15.47:6805/2569869305","state":["exists","up"]},{"osd":3,"uuid":"1286a67e-8958-426a-8c2e-bc85498c619c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6818","nonce":2791411683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6822","nonce":2791411683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6830","nonce":2791411683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6826","nonce":2791411683}]},"public_addr":"172.21.15.47:6818/2791411683","cluster_addr":"172.21.15.47:6822/2791411683","heartbeat_back_addr":"172.21.15.47:6830/2791411683","heartbeat_front_addr":"172.21.15.47:6826/2791411683","state":["exists","up"]},{"osd":4,"uuid":"d8ee2c20-e5b7-46c2-9999-b234070126cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6809","nonce":441313104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6811","nonce":441313104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6815","nonce":441313104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6813","nonce":441313104}]},"public_addr":"172.21.15.203:6809/441313104","cluster_addr":"172.21.15.203:6811/441313104","heartbeat_back_addr":"172.21.15.203:6815/441313104","heartbeat_front_addr":"172.21.15.203:6813/441313104","state":["exists","up"]},{"osd":5,"uuid":"b453a8f0-f324-4840-9ae0-d67bdadb9f8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6801","nonce":56838444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6803","nonce":56838444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6807","nonce":56838444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6805","nonce":56838444}]},"public_addr":"172.21.15.203:6801/56838444","cluster_addr":"172.21.15.203:6803/56838444","heartbeat_back_addr":"172.21.15.203:6807/56838444","heartbeat_front_addr":"172.21.15.203:6805/56838444","state":["exists","up"]},{"osd":6,"uuid":"a6d31951-2d4b-4e8d-a510-6283b01dc158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6825","nonce":2738517080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6827","nonce":2738517080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6831","nonce":2738517080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6829","nonce":2738517080}]},"public_addr":"172.21.15.203:6825/2738517080","cluster_addr":"172.21.15.203:6827/2738517080","heartbeat_back_addr":"172.21.15.203:6831/2738517080","heartbeat_front_addr":"172.21.15.203:6829/2738517080","state":["exists","up"]},{"osd":7,"uuid":"7119b8c0-fe96-402d-9e75-7b509d9d44d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6817","nonce":1315565058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6819","nonce":1315565058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6823","nonce":1315565058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6821","nonce":1315565058}]},"public_addr":"172.21.15.203:6817/1315565058","cluster_addr":"172.21.15.203:6819/1315565058","heartbeat_back_addr":"172.21.15.203:6823/1315565058","heartbeat_front_addr":"172.21.15.203:6821/1315565058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.121921+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.137494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.084814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.111593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.072430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.125066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.097867+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.057061+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:18:56.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.143+0000 7fbb55daf640 1 -- 172.21.15.47:0/3206329824 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fbb3c06c8f0 msgr2=0x7fbb3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.143+0000 7fbb55daf640 1 --2- 172.21.15.47:0/3206329824 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fbb3c06c8f0 0x7fbb3c06edb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbb40034000 tx=0x7fbb4002e000 comp rx=0 tx=0).stop 2024-04-26T23:18:56.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.143+0000 7fbb55daf640 1 -- 172.21.15.47:0/3206329824 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb50163530 msgr2=0x7fbb5009f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.143+0000 7fbb55daf640 1 --2- 172.21.15.47:0/3206329824 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb50163530 0x7fbb5009f520 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbb4400d5c0 tx=0x7fbb4400da80 comp rx=0 tx=0).stop 2024-04-26T23:18:56.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.143+0000 7fbb55daf640 1 -- 172.21.15.47:0/3206329824 shutdown_connections 2024-04-26T23:18:56.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.143+0000 7fbb55daf640 1 --2- 172.21.15.47:0/3206329824 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fbb3c06c8f0 0x7fbb3c06edb0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.143+0000 7fbb55daf640 1 --2- 172.21.15.47:0/3206329824 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb50163b90 0x7fbb500c9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.143+0000 7fbb55daf640 1 --2- 172.21.15.47:0/3206329824 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb50163530 0x7fbb5009f520 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.143+0000 7fbb55daf640 1 --2- 172.21.15.47:0/3206329824 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb5015f4f0 0x7fbb5015b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.143+0000 7fbb55daf640 1 -- 172.21.15.47:0/3206329824 >> 172.21.15.47:0/3206329824 conn(0x7fbb500d2e00 msgr2=0x7fbb500b0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:56.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.143+0000 7fbb55daf640 1 -- 172.21.15.47:0/3206329824 shutdown_connections 2024-04-26T23:18:56.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.144+0000 7fbb55daf640 1 -- 172.21.15.47:0/3206329824 wait complete. 2024-04-26T23:18:56.155 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-26T23:18:56.155 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:18:56.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.255+0000 7ff3e9a62640 1 Processor -- start 2024-04-26T23:18:56.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.256+0000 7ff3e9a62640 1 -- start start 2024-04-26T23:18:56.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.256+0000 7ff3e9a62640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff3e415b600 0x7ff3e4071f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.256+0000 7ff3e9a62640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff3e40cf5e0 0x7ff3e40ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.256+0000 7ff3e9a62640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff3e40cbbd0 0x7ff3e40cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.256+0000 7ff3e9a62640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7ff3e40cb9b0 con 0x7ff3e40cbbd0 2024-04-26T23:18:56.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.256+0000 7ff3e9a62640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7ff3e4072450 con 0x7ff3e415b600 2024-04-26T23:18:56.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.256+0000 7ff3e9a62640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7ff3e415b9e0 con 0x7ff3e40cf5e0 2024-04-26T23:18:56.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.257+0000 7ff3e3fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff3e40cbbd0 0x7ff3e40cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.257+0000 7ff3e3fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff3e40cbbd0 0x7ff3e40cef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58580/0 (socket says 172.21.15.47:58580) 2024-04-26T23:18:56.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.257+0000 7ff3e3fff640 1 -- 172.21.15.47:0/4038255250 learned_addr learned my addr 172.21.15.47:0/4038255250 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:56.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.257+0000 7ff3e2ffd640 1 --2- 172.21.15.47:0/4038255250 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff3e40cf5e0 0x7ff3e40ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.257+0000 7ff3e37fe640 1 --2- 172.21.15.47:0/4038255250 >> v2:172.21.15.203:3300/0 conn(0x7ff3e415b600 0x7ff3e4071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.257+0000 7ff3e3fff640 1 -- 172.21.15.47:0/4038255250 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff3e40cf5e0 msgr2=0x7ff3e40ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:56.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.257+0000 7ff3e3fff640 1 --2- 172.21.15.47:0/4038255250 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff3e40cf5e0 0x7ff3e40ce9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.257+0000 7ff3e3fff640 1 -- 172.21.15.47:0/4038255250 >> v2:172.21.15.203:3300/0 conn(0x7ff3e415b600 msgr2=0x7ff3e4071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:56.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.257+0000 7ff3e3fff640 1 --2- 172.21.15.47:0/4038255250 >> v2:172.21.15.203:3300/0 conn(0x7ff3e415b600 0x7ff3e4071f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.257+0000 7ff3e3fff640 1 -- 172.21.15.47:0/4038255250 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3e40d0740 con 0x7ff3e40cbbd0 2024-04-26T23:18:56.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.257+0000 7ff3e2ffd640 1 --2- 172.21.15.47:0/4038255250 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff3e40cf5e0 0x7ff3e40ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:18:56.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.257+0000 7ff3e37fe640 1 --2- 172.21.15.47:0/4038255250 >> v2:172.21.15.203:3300/0 conn(0x7ff3e415b600 0x7ff3e4071f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:18:56.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.258+0000 7ff3e3fff640 1 --2- 172.21.15.47:0/4038255250 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3e40cbbd0 0x7ff3e40cef10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4071ad0 tx=0x7ff3dc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f96ade204892a8df server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:56.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.258+0000 7ff3e27fc640 1 -- 172.21.15.47:0/4038255250 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc036070 con 0x7ff3e40cbbd0 2024-04-26T23:18:56.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.258+0000 7ff3e27fc640 1 -- 172.21.15.47:0/4038255250 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3dc031600 con 0x7ff3e40cbbd0 2024-04-26T23:18:56.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.258+0000 7ff3e27fc640 1 -- 172.21.15.47:0/4038255250 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc0319d0 con 0x7ff3e40cbbd0 2024-04-26T23:18:56.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.258+0000 7ff3e9a62640 1 -- 172.21.15.47:0/4038255250 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3e40cbbd0 msgr2=0x7ff3e40cef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.258+0000 7ff3e9a62640 1 --2- 172.21.15.47:0/4038255250 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3e40cbbd0 0x7ff3e40cef10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4071ad0 tx=0x7ff3dc02a6a0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.258+0000 7ff3e9a62640 1 -- 172.21.15.47:0/4038255250 shutdown_connections 2024-04-26T23:18:56.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.258+0000 7ff3e9a62640 1 --2- 172.21.15.47:0/4038255250 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3e40cbbd0 0x7ff3e40cef10 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.258+0000 7ff3e9a62640 1 --2- 172.21.15.47:0/4038255250 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff3e40cf5e0 0x7ff3e40ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.258+0000 7ff3e9a62640 1 --2- 172.21.15.47:0/4038255250 >> v2:172.21.15.203:3300/0 conn(0x7ff3e415b600 0x7ff3e4071f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.258+0000 7ff3e9a62640 1 -- 172.21.15.47:0/4038255250 >> 172.21.15.47:0/4038255250 conn(0x7ff3e40d0be0 msgr2=0x7ff3e40d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:56.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.258+0000 7ff3e9a62640 1 -- 172.21.15.47:0/4038255250 shutdown_connections 2024-04-26T23:18:56.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.258+0000 7ff3e9a62640 1 -- 172.21.15.47:0/4038255250 wait complete. 2024-04-26T23:18:56.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.260+0000 7ff3e9a62640 1 Processor -- start 2024-04-26T23:18:56.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.260+0000 7ff3e9a62640 1 -- start start 2024-04-26T23:18:56.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.260+0000 7ff3e9a62640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff3e40cbbd0 0x7ff3e40c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.260+0000 7ff3e9a62640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff3e40cf5e0 0x7ff3e4153e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.260+0000 7ff3e9a62640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3e415b600 0x7ff3e4154380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.261+0000 7ff3e9a62640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e40d02f0 con 0x7ff3e415b600 2024-04-26T23:18:56.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.261+0000 7ff3e9a62640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e40cb640 con 0x7ff3e40cbbd0 2024-04-26T23:18:56.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.261+0000 7ff3e9a62640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7ff3e40d0470 con 0x7ff3e40cf5e0 2024-04-26T23:18:56.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.261+0000 7ff3e3fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3e415b600 0x7ff3e4154380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.261+0000 7ff3e3fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3e415b600 0x7ff3e4154380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58594/0 (socket says 172.21.15.47:58594) 2024-04-26T23:18:56.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.261+0000 7ff3e3fff640 1 -- 172.21.15.47:0/509365690 learned_addr learned my addr 172.21.15.47:0/509365690 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:56.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.261+0000 7ff3e2ffd640 1 --2- 172.21.15.47:0/509365690 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff3e40cf5e0 0x7ff3e4153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.261+0000 7ff3e37fe640 1 --2- 172.21.15.47:0/509365690 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff3e40cbbd0 0x7ff3e40c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.261+0000 7ff3e3fff640 1 -- 172.21.15.47:0/509365690 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff3e40cf5e0 msgr2=0x7ff3e4153e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.261+0000 7ff3e3fff640 1 --2- 172.21.15.47:0/509365690 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff3e40cf5e0 0x7ff3e4153e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.261+0000 7ff3e3fff640 1 -- 172.21.15.47:0/509365690 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff3e40cbbd0 msgr2=0x7ff3e40c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.261+0000 7ff3e3fff640 1 --2- 172.21.15.47:0/509365690 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff3e40cbbd0 0x7ff3e40c7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.261+0000 7ff3e3fff640 1 -- 172.21.15.47:0/509365690 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3e415a9e0 con 0x7ff3e415b600 2024-04-26T23:18:56.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.261+0000 7ff3e3fff640 1 --2- 172.21.15.47:0/509365690 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3e415b600 0x7ff3e4154380 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc034040 tx=0x7ff3dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:56.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.261+0000 7ff3cbfff640 1 -- 172.21.15.47:0/509365690 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc041020 con 0x7ff3e415b600 2024-04-26T23:18:56.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.261+0000 7ff3cbfff640 1 -- 172.21.15.47:0/509365690 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3dc0077a0 con 0x7ff3e415b600 2024-04-26T23:18:56.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.261+0000 7ff3cbfff640 1 -- 172.21.15.47:0/509365690 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc036040 con 0x7ff3e415b600 2024-04-26T23:18:56.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.261+0000 7ff3e9a62640 1 -- 172.21.15.47:0/509365690 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3e41501e0 con 0x7ff3e415b600 2024-04-26T23:18:56.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.261+0000 7ff3e9a62640 1 -- 172.21.15.47:0/509365690 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3e415a700 con 0x7ff3e415b600 2024-04-26T23:18:56.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.263+0000 7ff3e9a62640 1 -- 172.21.15.47:0/509365690 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3e4072010 con 0x7ff3e415b600 2024-04-26T23:18:56.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.263+0000 7ff3cbfff640 1 -- 172.21.15.47:0/509365690 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff3dc044020 con 0x7ff3e415b600 2024-04-26T23:18:56.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.263+0000 7ff3cbfff640 1 --2- 172.21.15.47:0/509365690 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7ff3d006c8f0 0x7ff3d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.263+0000 7ff3cbfff640 1 -- 172.21.15.47:0/509365690 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7ff3dc0aca80 con 0x7ff3e415b600 2024-04-26T23:18:56.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.264+0000 7ff3e37fe640 1 --2- 172.21.15.47:0/509365690 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7ff3d006c8f0 0x7ff3d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.264+0000 7ff3e37fe640 1 --2- 172.21.15.47:0/509365690 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7ff3d006c8f0 0x7ff3d006edb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff3e40b6b20 tx=0x7ff3d4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:56.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.268+0000 7ff3cbfff640 1 -- 172.21.15.47:0/509365690 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff3e4072010 con 0x7ff3e415b600 2024-04-26T23:18:56.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.452+0000 7ff3e9a62640 1 -- 172.21.15.47:0/509365690 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff3e40694f0 con 0x7ff3e415b600 2024-04-26T23:18:56.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.454+0000 7ff3cbfff640 1 -- 172.21.15.47:0/509365690 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18478 (secure 0 0 0) 0x7ff3dc04b030 con 0x7ff3e415b600 2024-04-26T23:18:56.455 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:18:56.455 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":24,"fsid":"399006d3-4db1-43e6-ba81-a79a276eabfd","created":"2024-04-26T23:18:06.731211+0000","modified":"2024-04-26T23:18:36.402132+0000","last_up_change":"2024-04-26T23:18:17.808736+0000","last_in_change":"2024-04-26T23:18:08.945859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:18:16.937492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:18:29.033575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:18:30.318499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:18:34.014536+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ee091567-fea0-4a14-bf4e-0e16c20bb2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6819","nonce":326520779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6821","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6823","nonce":326520779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6829","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6831","nonce":326520779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6825","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6827","nonce":326520779}]},"public_addr":"172.21.15.47:6819/326520779","cluster_addr":"172.21.15.47:6823/326520779","heartbeat_back_addr":"172.21.15.47:6831/326520779","heartbeat_front_addr":"172.21.15.47:6827/326520779","state":["exists","up"]},{"osd":1,"uuid":"c14440a0-bba5-4931-a6db-b736c08e869b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6809","nonce":1912610931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6811","nonce":1912610931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6815","nonce":1912610931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6813","nonce":1912610931}]},"public_addr":"172.21.15.47:6809/1912610931","cluster_addr":"172.21.15.47:6811/1912610931","heartbeat_back_addr":"172.21.15.47:6815/1912610931","heartbeat_front_addr":"172.21.15.47:6813/1912610931","state":["exists","up"]},{"osd":2,"uuid":"3940f8de-c9b2-4cd3-a911-9c301729eb30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6801","nonce":2569869305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6803","nonce":2569869305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6807","nonce":2569869305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6805","nonce":2569869305}]},"public_addr":"172.21.15.47:6801/2569869305","cluster_addr":"172.21.15.47:6803/2569869305","heartbeat_back_addr":"172.21.15.47:6807/2569869305","heartbeat_front_addr":"172.21.15.47:6805/2569869305","state":["exists","up"]},{"osd":3,"uuid":"1286a67e-8958-426a-8c2e-bc85498c619c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6818","nonce":2791411683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6822","nonce":2791411683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6830","nonce":2791411683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6826","nonce":2791411683}]},"public_addr":"172.21.15.47:6818/2791411683","cluster_addr":"172.21.15.47:6822/2791411683","heartbeat_back_addr":"172.21.15.47:6830/2791411683","heartbeat_front_addr":"172.21.15.47:6826/2791411683","state":["exists","up"]},{"osd":4,"uuid":"d8ee2c20-e5b7-46c2-9999-b234070126cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6809","nonce":441313104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6811","nonce":441313104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6815","nonce":441313104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6813","nonce":441313104}]},"public_addr":"172.21.15.203:6809/441313104","cluster_addr":"172.21.15.203:6811/441313104","heartbeat_back_addr":"172.21.15.203:6815/441313104","heartbeat_front_addr":"172.21.15.203:6813/441313104","state":["exists","up"]},{"osd":5,"uuid":"b453a8f0-f324-4840-9ae0-d67bdadb9f8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6801","nonce":56838444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6803","nonce":56838444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6807","nonce":56838444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6805","nonce":56838444}]},"public_addr":"172.21.15.203:6801/56838444","cluster_addr":"172.21.15.203:6803/56838444","heartbeat_back_addr":"172.21.15.203:6807/56838444","heartbeat_front_addr":"172.21.15.203:6805/56838444","state":["exists","up"]},{"osd":6,"uuid":"a6d31951-2d4b-4e8d-a510-6283b01dc158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6825","nonce":2738517080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6827","nonce":2738517080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6831","nonce":2738517080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6829","nonce":2738517080}]},"public_addr":"172.21.15.203:6825/2738517080","cluster_addr":"172.21.15.203:6827/2738517080","heartbeat_back_addr":"172.21.15.203:6831/2738517080","heartbeat_front_addr":"172.21.15.203:6829/2738517080","state":["exists","up"]},{"osd":7,"uuid":"7119b8c0-fe96-402d-9e75-7b509d9d44d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6817","nonce":1315565058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6819","nonce":1315565058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6823","nonce":1315565058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6821","nonce":1315565058}]},"public_addr":"172.21.15.203:6817/1315565058","cluster_addr":"172.21.15.203:6819/1315565058","heartbeat_back_addr":"172.21.15.203:6823/1315565058","heartbeat_front_addr":"172.21.15.203:6821/1315565058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.121921+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.137494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.084814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.111593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.072430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.125066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.097867+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.057061+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:18:56.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.456+0000 7ff3e9a62640 1 -- 172.21.15.47:0/509365690 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7ff3d006c8f0 msgr2=0x7ff3d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.456+0000 7ff3e9a62640 1 --2- 172.21.15.47:0/509365690 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7ff3d006c8f0 0x7ff3d006edb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff3e40b6b20 tx=0x7ff3d4008040 comp rx=0 tx=0).stop 2024-04-26T23:18:56.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.456+0000 7ff3e9a62640 1 -- 172.21.15.47:0/509365690 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3e415b600 msgr2=0x7ff3e4154380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.456+0000 7ff3e9a62640 1 --2- 172.21.15.47:0/509365690 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3e415b600 0x7ff3e4154380 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc034040 tx=0x7ff3dc004a40 comp rx=0 tx=0).stop 2024-04-26T23:18:56.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.456+0000 7ff3e9a62640 1 -- 172.21.15.47:0/509365690 shutdown_connections 2024-04-26T23:18:56.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.456+0000 7ff3e9a62640 1 --2- 172.21.15.47:0/509365690 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7ff3d006c8f0 0x7ff3d006edb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.456+0000 7ff3e9a62640 1 --2- 172.21.15.47:0/509365690 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3e415b600 0x7ff3e4154380 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.456+0000 7ff3e9a62640 1 --2- 172.21.15.47:0/509365690 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff3e40cf5e0 0x7ff3e4153e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.456+0000 7ff3e9a62640 1 --2- 172.21.15.47:0/509365690 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff3e40cbbd0 0x7ff3e40c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.456+0000 7ff3e9a62640 1 -- 172.21.15.47:0/509365690 >> 172.21.15.47:0/509365690 conn(0x7ff3e40d0be0 msgr2=0x7ff3e40cea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:56.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.456+0000 7ff3e9a62640 1 -- 172.21.15.47:0/509365690 shutdown_connections 2024-04-26T23:18:56.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.456+0000 7ff3e9a62640 1 -- 172.21.15.47:0/509365690 wait complete. 2024-04-26T23:18:56.469 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-26T23:18:56.469 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-26T23:18:56.470 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-26T23:18:56.470 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-26T23:18:56.471 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-26T23:18:56.471 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-26T23:18:56.472 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-26T23:18:56.472 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-26T23:18:56.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.646+0000 7f73f8afc640 1 Processor -- start 2024-04-26T23:18:56.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.646+0000 7f73f8afc640 1 -- start start 2024-04-26T23:18:56.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.647+0000 7f602ac28640 1 Processor -- start 2024-04-26T23:18:56.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.647+0000 7fe75b647640 1 Processor -- start 2024-04-26T23:18:56.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.647+0000 7fe75b647640 1 -- start start 2024-04-26T23:18:56.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.647+0000 7f7edf69c640 1 Processor -- start 2024-04-26T23:18:56.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.647+0000 7f73f8afc640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f73f40d1700 0x7f73f40722f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.647+0000 7fc34b9af640 1 Processor -- start 2024-04-26T23:18:56.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.648+0000 7fc34b9af640 1 -- start start 2024-04-26T23:18:56.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.648+0000 7f73f8afc640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f73f40aff60 0x7f73f406f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.648+0000 7f73f8afc640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73f4073ce0 con 0x7f73f406ddf0 2024-04-26T23:18:56.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.648+0000 7f73f8afc640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f73f4071e60 con 0x7f73f40aff60 2024-04-26T23:18:56.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.648+0000 7f73f8afc640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f73f40d06a0 con 0x7f73f40d1700 2024-04-26T23:18:56.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.648+0000 7fe75b647640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe7540d1700 0x7fe7540722f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.648+0000 7fe75b647640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7540cf5d0 con 0x7fe7540aff60 2024-04-26T23:18:56.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.648+0000 7fe75b647640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe754073ce0 con 0x7fe75406f600 2024-04-26T23:18:56.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.648+0000 7f73f2d9d640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f73f406ddf0 0x7f73f406e200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:54534/0 (socket says 172.21.15.47:54534) 2024-04-26T23:18:56.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.648+0000 7f73f2d9d640 1 -- 172.21.15.47:0/3797558189 learned_addr learned my addr 172.21.15.47:0/3797558189 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:18:56.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.648+0000 7f73f259c640 1 --2- 172.21.15.47:0/3797558189 >> v2:172.21.15.203:3300/0 conn(0x7f73f40d1700 0x7f73f40722f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.648+0000 7f73f1d9b640 1 --2- 172.21.15.47:0/3797558189 >> v2:172.21.15.47:3300/0 conn(0x7f73f40aff60 0x7f73f406f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.648+0000 7fe75b647640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fe754071e60 con 0x7fe7540d1700 2024-04-26T23:18:56.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.647+0000 7f602ac28640 1 -- start start 2024-04-26T23:18:56.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.649+0000 7fe758be2640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe7540aff60 0x7fe7540ce7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:54540/0 (socket says 172.21.15.47:54540) 2024-04-26T23:18:56.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.649+0000 7fe758be2640 1 -- 172.21.15.47:0/3717837201 learned_addr learned my addr 172.21.15.47:0/3717837201 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:18:56.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.648+0000 7f7edf69c640 1 -- start start 2024-04-26T23:18:56.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.649+0000 7fe7593e3640 1 --2- 172.21.15.47:0/3717837201 >> v2:172.21.15.203:3300/0 conn(0x7fe7540d1700 0x7fe7540722f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.649+0000 7f602ac28640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f60240d1700 0x7f60240722f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.649+0000 7f602ac28640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60240cf5d0 con 0x7f60240aff60 2024-04-26T23:18:56.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.649+0000 7f602ac28640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6024073ce0 con 0x7f602406f600 2024-04-26T23:18:56.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.649+0000 7f602ac28640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f6024071e60 con 0x7f60240d1700 2024-04-26T23:18:56.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.649+0000 7f7edf69c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7ed00e33a0 0x7f7ed0053eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.649+0000 7f7edf69c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7ed0007260 0x7f7ed00076f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.649+0000 7f7edf69c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ed0057d80 con 0x7f7ed00543f0 2024-04-26T23:18:56.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.649+0000 7f7edf69c640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f7ed0058060 con 0x7f7ed0007260 2024-04-26T23:18:56.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.649+0000 7f6023fff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f60240aff60 0x7f60240ce7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:54554/0 (socket says 172.21.15.47:54554) 2024-04-26T23:18:56.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.649+0000 7f6023fff640 1 -- 172.21.15.47:0/1202497799 learned_addr learned my addr 172.21.15.47:0/1202497799 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:18:56.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.649+0000 7f7edf69c640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f7ed0058340 con 0x7f7ed00e33a0 2024-04-26T23:18:56.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.649+0000 7f7aa2e6d640 1 Processor -- start 2024-04-26T23:18:56.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.649+0000 7f7eddc39640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7ed0007260 0x7f7ed00076f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.649+0000 7f2c5359e640 1 Processor -- start 2024-04-26T23:18:56.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.649+0000 7f7eddc39640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7ed0007260 0x7f7ed00076f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58610/0 (socket says 172.21.15.47:58610) 2024-04-26T23:18:56.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.649+0000 7f7eddc39640 1 -- 172.21.15.47:0/1878323778 learned_addr learned my addr 172.21.15.47:0/1878323778 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:56.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.649+0000 7f73f259c640 1 -- 172.21.15.47:0/3797558189 >> v2:172.21.15.47:3300/0 conn(0x7f73f40aff60 msgr2=0x7f73f406f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:56.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.649+0000 7fe74bfff640 1 -- 172.21.15.47:0/3717837201 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3543220015 0 0) 0x7fe7540cf5d0 con 0x7fe7540aff60 2024-04-26T23:18:56.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.649+0000 7fe74bfff640 1 -- 172.21.15.47:0/3717837201 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe740003520 con 0x7fe7540aff60 2024-04-26T23:18:56.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.649+0000 7f7aa2e6d640 1 -- start start 2024-04-26T23:18:56.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.649+0000 7fc34b9af640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc3440b6b60 0x7fc3440d2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.650+0000 7f60289c4640 1 --2- 172.21.15.47:0/1202497799 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f60240d1700 0x7f60240722f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.650+0000 7f7edd438640 1 --2- 172.21.15.47:0/1878323778 >> v2:172.21.15.203:3300/0 conn(0x7f7ed00e33a0 0x7f7ed0053eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.649+0000 7f73f259c640 1 --2- 172.21.15.47:0/3797558189 >> v2:172.21.15.47:3300/0 conn(0x7f73f40aff60 0x7f73f406f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.649+0000 7f73f259c640 1 -- 172.21.15.47:0/3797558189 >> v1:172.21.15.47:6789/0 conn(0x7f73f406ddf0 legacy=0x7f73f406e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.649+0000 7f73f259c640 1 -- 172.21.15.47:0/3797558189 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73f4073f80 con 0x7f73f40d1700 2024-04-26T23:18:56.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.650+0000 7f73f1d9b640 1 --2- 172.21.15.47:0/3797558189 >> v2:172.21.15.47:3300/0 conn(0x7f73f40aff60 0x7f73f406f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:18:56.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.649+0000 7fc34b9af640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc3440b8720 0x7fc344175990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.649+0000 7fc34b9af640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3440d3020 con 0x7fc3440d4460 2024-04-26T23:18:56.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.649+0000 7fc34b9af640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fc3440d3300 con 0x7fc3440b6b60 2024-04-26T23:18:56.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.649+0000 7fc34b9af640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fc3440d35e0 con 0x7fc3440b8720 2024-04-26T23:18:56.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.650+0000 7fc348f4a640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc3440d4460 0x7fc3440b8010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:54558/0 (socket says 172.21.15.47:54558) 2024-04-26T23:18:56.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.650+0000 7fc348f4a640 1 -- 172.21.15.47:0/4139125699 learned_addr learned my addr 172.21.15.47:0/4139125699 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:18:56.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.651+0000 7f0a3d3af640 1 Processor -- start 2024-04-26T23:18:56.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.651+0000 7f60237fe640 1 -- 172.21.15.47:0/1202497799 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2724785504 0 0) 0x7f60240cf5d0 con 0x7f60240aff60 2024-04-26T23:18:56.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.651+0000 7fc34974b640 1 --2- 172.21.15.47:0/4139125699 >> v2:172.21.15.47:3300/0 conn(0x7fc3440b6b60 0x7fc3440d2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.650+0000 7f7eddc39640 1 -- 172.21.15.47:0/1878323778 >> v2:172.21.15.203:3300/0 conn(0x7f7ed00e33a0 msgr2=0x7f7ed0053eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:56.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.650+0000 7f7eddc39640 1 --2- 172.21.15.47:0/1878323778 >> v2:172.21.15.203:3300/0 conn(0x7f7ed00e33a0 0x7f7ed0053eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.650+0000 7f7eddc39640 1 -- 172.21.15.47:0/1878323778 >> v1:172.21.15.203:6789/0 conn(0x7f7ed00543f0 legacy=0x7f7ed003c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:56.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.650+0000 7f7eddc39640 1 -- 172.21.15.47:0/1878323778 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed0058620 con 0x7f7ed0007260 2024-04-26T23:18:56.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.651+0000 7f60237fe640 1 -- 172.21.15.47:0/1202497799 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f600c005730 con 0x7f60240aff60 2024-04-26T23:18:56.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.651+0000 7f0a3d3af640 1 -- start start 2024-04-26T23:18:56.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.650+0000 7fe74bfff640 1 -- 172.21.15.47:0/3717837201 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1736327072 0 0) 0x7fe754073ce0 con 0x7fe75406f600 2024-04-26T23:18:56.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.651+0000 7f60289c4640 1 -- 172.21.15.47:0/1202497799 >> v1:172.21.15.203:6789/0 conn(0x7f602406f600 legacy=0x7f60240ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:56.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.651+0000 7f73f259c640 1 --2- 172.21.15.47:0/3797558189 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73f40d1700 0x7f73f40722f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f73f406c4e0 tx=0x7f73e402a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b5b1da0c883c79 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:56.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.651+0000 7f60289c4640 1 -- 172.21.15.47:0/1202497799 >> v1:172.21.15.47:6789/0 conn(0x7f60240aff60 legacy=0x7f60240ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.650+0000 7f2c5359e640 1 -- start start 2024-04-26T23:18:56.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.651+0000 7f60237fe640 1 -- 172.21.15.47:0/1202497799 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1752849123 0 0) 0x7f600c005730 con 0x7f60240aff60 2024-04-26T23:18:56.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.650+0000 7fe7593e3640 1 -- 172.21.15.47:0/3717837201 >> v1:172.21.15.203:6789/0 conn(0x7fe75406f600 legacy=0x7fe7540ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.651+0000 7f60289c4640 1 -- 172.21.15.47:0/1202497799 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f602406fa50 con 0x7f60240d1700 2024-04-26T23:18:56.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.650+0000 7fe7593e3640 1 -- 172.21.15.47:0/3717837201 >> v1:172.21.15.47:6789/0 conn(0x7fe7540aff60 legacy=0x7fe7540ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.650+0000 7fe7593e3640 1 -- 172.21.15.47:0/3717837201 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7540d06a0 con 0x7fe7540d1700 2024-04-26T23:18:56.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.651+0000 7f7eddc39640 1 --2- 172.21.15.47:0/1878323778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ed0007260 0x7f7ed00076f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8050be0 tx=0x7f7ed80a6cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb9e8196f8251c00 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:56.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.651+0000 7f2c5359e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c540cfdb0 0x7f2c54070bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.651+0000 7f2c5359e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2c54071d40 0x7f2c54071130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.651+0000 7f2c5359e640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c540d3950 con 0x7f2c540ceff0 2024-04-26T23:18:56.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.651+0000 7f2c5359e640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f2c54160ab0 con 0x7f2c54071d40 2024-04-26T23:18:56.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.651+0000 7f2c5359e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f2c540cf3d0 con 0x7f2c540cfdb0 2024-04-26T23:18:56.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.651+0000 7f7aa2e6d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7a9c1633b0 0x7f7a9c0737c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.651+0000 7f7aa2e6d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7a9c0cd680 0x7f7a9c06c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.651+0000 7f7aa2e6d640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a9c0d28a0 con 0x7f7a9c06c780 2024-04-26T23:18:56.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.651+0000 7f7aa2e6d640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f7a9c06cbd0 con 0x7f7a9c0cd680 2024-04-26T23:18:56.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.651+0000 7f7aa2e6d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f7a9c0cf6c0 con 0x7f7a9c1633b0 2024-04-26T23:18:56.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.651+0000 7f7aa140a640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7a9c06c780 0x7f7a9c0b0670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:54570/0 (socket says 172.21.15.47:54570) 2024-04-26T23:18:56.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.651+0000 7f7aa140a640 1 -- 172.21.15.47:0/976309669 learned_addr learned my addr 172.21.15.47:0/976309669 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:18:56.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.651+0000 7f7a9bfff640 1 --2- 172.21.15.47:0/976309669 >> v2:172.21.15.47:3300/0 conn(0x7f7a9c0cd680 0x7f7a9c06c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.652+0000 7f7aa0c09640 1 --2- 172.21.15.47:0/976309669 >> v2:172.21.15.203:3300/0 conn(0x7f7a9c1633b0 0x7f7a9c0737c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.651+0000 7f2c5259c640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f2c540ceff0 0x7f2c540cf6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:54584/0 (socket says 172.21.15.47:54584) 2024-04-26T23:18:56.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.652+0000 7f2c5259c640 1 -- 172.21.15.47:0/3837612841 learned_addr learned my addr 172.21.15.47:0/3837612841 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:18:56.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.652+0000 7f73f159a640 1 -- 172.21.15.47:0/3797558189 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e4037070 con 0x7f73f40d1700 2024-04-26T23:18:56.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.652+0000 7f73f159a640 1 -- 172.21.15.47:0/3797558189 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73e4032440 con 0x7f73f40d1700 2024-04-26T23:18:56.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.652+0000 7f7a9b7fe640 1 -- 172.21.15.47:0/976309669 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3677383018 0 0) 0x7f7a94001490 con 0x7f7a9c06c780 2024-04-26T23:18:56.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.652+0000 7f7a9b7fe640 1 -- 172.21.15.47:0/976309669 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a84003520 con 0x7f7a9c06c780 2024-04-26T23:18:56.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.652+0000 7fe7593e3640 1 --2- 172.21.15.47:0/3717837201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7540d1700 0x7fe7540722f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe74c00aea0 tx=0x7fe74c02b720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efeeb03d7f323286 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:56.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.651+0000 7fc34974b640 1 -- 172.21.15.47:0/4139125699 >> v2:172.21.15.203:3300/0 conn(0x7fc3440b8720 msgr2=0x7fc344175990 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T23:18:56.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.651+0000 7fc34974b640 1 --2- 172.21.15.47:0/4139125699 >> v2:172.21.15.203:3300/0 conn(0x7fc3440b8720 0x7fc344175990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.651+0000 7fc34974b640 1 -- 172.21.15.47:0/4139125699 >> v1:172.21.15.47:6789/0 conn(0x7fc3440d4460 legacy=0x7fc3440b8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:56.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.651+0000 7fc34974b640 1 -- 172.21.15.47:0/4139125699 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3440d38c0 con 0x7fc3440b6b60 2024-04-26T23:18:56.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.652+0000 7f73f159a640 1 -- 172.21.15.47:0/3797558189 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e402ae60 con 0x7f73f40d1700 2024-04-26T23:18:56.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.652+0000 7f7ecffff640 1 -- 172.21.15.47:0/1878323778 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed80b4070 con 0x7f7ed0007260 2024-04-26T23:18:56.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.652+0000 7f7ecffff640 1 -- 172.21.15.47:0/1878323778 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ed80a7ae0 con 0x7f7ed0007260 2024-04-26T23:18:56.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.652+0000 7f7ecffff640 1 -- 172.21.15.47:0/1878323778 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed80af440 con 0x7f7ed0007260 2024-04-26T23:18:56.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.652+0000 7f7aa0c09640 1 -- 172.21.15.47:0/976309669 >> v2:172.21.15.47:3300/0 conn(0x7f7a9c0cd680 msgr2=0x7f7a9c06c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:56.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.652+0000 7f7aa0c09640 1 --2- 172.21.15.47:0/976309669 >> v2:172.21.15.47:3300/0 conn(0x7f7a9c0cd680 0x7f7a9c06c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.652+0000 7f0a3d3af640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0a3815d980 0x7f0a38167a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.652+0000 7f7aa0c09640 1 -- 172.21.15.47:0/976309669 >> v1:172.21.15.47:6789/0 conn(0x7f7a9c06c780 legacy=0x7f7a9c0b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.652+0000 7f7aa0c09640 1 -- 172.21.15.47:0/976309669 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a9c073d00 con 0x7f7a9c1633b0 2024-04-26T23:18:56.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.652+0000 7f7a9bfff640 1 --2- 172.21.15.47:0/976309669 >> v2:172.21.15.47:3300/0 conn(0x7f7a9c0cd680 0x7f7a9c06c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:18:56.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.652+0000 7fe74bfff640 1 -- 172.21.15.47:0/3717837201 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7500015f0 con 0x7fe7540d1700 2024-04-26T23:18:56.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.652+0000 7f2c51d9b640 1 --2- 172.21.15.47:0/3837612841 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c540cfdb0 0x7f2c54070bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.652+0000 7f60289c4640 1 --2- 172.21.15.47:0/1202497799 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f60240d1700 0x7f60240722f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f601c00aed0 tx=0x7f601c02a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e79eb11d288dc2d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:56.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.652+0000 7f0a3d3af640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0a380b4650 0x7f0a38072430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.652+0000 7f2c52d9d640 1 --2- 172.21.15.47:0/3837612841 >> v2:172.21.15.47:3300/0 conn(0x7f2c54071d40 0x7f2c54071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.653+0000 7f0a3d3af640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a38163a50 con 0x7f0a380b6130 2024-04-26T23:18:56.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.653+0000 7fe74bfff640 1 -- 172.21.15.47:0/3717837201 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe74c00a0d0 con 0x7fe7540d1700 2024-04-26T23:18:56.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.653+0000 7f60237fe640 1 -- 172.21.15.47:0/1202497799 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601c036070 con 0x7f60240d1700 2024-04-26T23:18:56.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.653+0000 7f2c51d9b640 1 -- 172.21.15.47:0/3837612841 >> v2:172.21.15.47:3300/0 conn(0x7f2c54071d40 msgr2=0x7f2c54071130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:56.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.653+0000 7fe74bfff640 1 -- 172.21.15.47:0/3717837201 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74c008660 con 0x7fe7540d1700 2024-04-26T23:18:56.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.653+0000 7f0a377fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0a380b4650 0x7f0a38072430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.653+0000 7f60237fe640 1 -- 172.21.15.47:0/1202497799 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f601c008440 con 0x7f60240d1700 2024-04-26T23:18:56.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.653+0000 7f60237fe640 1 -- 172.21.15.47:0/1202497799 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6018001670 con 0x7f60240d1700 2024-04-26T23:18:56.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.653+0000 7f0a377fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0a380b4650 0x7f0a38072430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58652/0 (socket says 172.21.15.47:58652) 2024-04-26T23:18:56.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.653+0000 7f0a377fe640 1 -- 172.21.15.47:0/4086350355 learned_addr learned my addr 172.21.15.47:0/4086350355 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:56.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.653+0000 7f0a3d3af640 1 -- 172.21.15.47:0/4086350355 --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f0a38164920 con 0x7f0a380b4650 2024-04-26T23:18:56.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.652+0000 7f0a367fc640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0a380b6130 0x7f0a380b3f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:54586/0 (socket says 172.21.15.47:54586) 2024-04-26T23:18:56.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.653+0000 7f0a3d3af640 1 -- 172.21.15.47:0/4086350355 --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f0a38163d10 con 0x7f0a3815d980 2024-04-26T23:18:56.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.653+0000 7f7aa0c09640 1 --2- 172.21.15.47:0/976309669 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a9c1633b0 0x7f7a9c0737c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c0cf670 tx=0x7f7a9002a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b5937534f1b7f68 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:56.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.653+0000 7f7edf69c640 1 -- 172.21.15.47:0/1878323778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ed0007260 msgr2=0x7f7ed00076f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.653+0000 7f7edf69c640 1 --2- 172.21.15.47:0/1878323778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ed0007260 0x7f7ed00076f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8050be0 tx=0x7f7ed80a6cd0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.653+0000 7f73f8afc640 1 -- 172.21.15.47:0/3797558189 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73f40d1700 msgr2=0x7f73f40722f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.653+0000 7f73f8afc640 1 --2- 172.21.15.47:0/3797558189 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73f40d1700 0x7f73f40722f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f73f406c4e0 tx=0x7f73e402a110 comp rx=0 tx=0).stop 2024-04-26T23:18:56.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.653+0000 7f7edf69c640 1 -- 172.21.15.47:0/1878323778 shutdown_connections 2024-04-26T23:18:56.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.653+0000 7f0a36ffd640 1 --2- 172.21.15.47:0/4086350355 >> v2:172.21.15.203:3300/0 conn(0x7f0a3815d980 0x7f0a38167a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.653+0000 7f7edf69c640 1 --2- 172.21.15.47:0/1878323778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ed0007260 0x7f7ed00076f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.653+0000 7f7edf69c640 1 --2- 172.21.15.47:0/1878323778 >> v2:172.21.15.203:3300/0 conn(0x7f7ed00e33a0 0x7f7ed0053eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.653+0000 7f2c51d9b640 1 --2- 172.21.15.47:0/3837612841 >> v2:172.21.15.47:3300/0 conn(0x7f2c54071d40 0x7f2c54071130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.653+0000 7f2c51d9b640 1 -- 172.21.15.47:0/3837612841 >> v1:172.21.15.47:6789/0 conn(0x7f2c540ceff0 legacy=0x7f2c540cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.653+0000 7f2c51d9b640 1 -- 172.21.15.47:0/3837612841 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c54070110 con 0x7f2c540cfdb0 2024-04-26T23:18:56.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.653+0000 7f2c52d9d640 1 --2- 172.21.15.47:0/3837612841 >> v2:172.21.15.47:3300/0 conn(0x7f2c54071d40 0x7f2c54071130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:18:56.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.653+0000 7f7edf69c640 1 -- 172.21.15.47:0/1878323778 >> 172.21.15.47:0/1878323778 conn(0x7f7ed003a9c0 msgr2=0x7f7ed003add0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:56.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.653+0000 7f7a9b7fe640 1 -- 172.21.15.47:0/976309669 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a90037070 con 0x7f7a9c1633b0 2024-04-26T23:18:56.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.653+0000 7f7a9b7fe640 1 -- 172.21.15.47:0/976309669 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a90032440 con 0x7f7a9c1633b0 2024-04-26T23:18:56.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.654+0000 7f7edf69c640 1 -- 172.21.15.47:0/1878323778 shutdown_connections 2024-04-26T23:18:56.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.654+0000 7f73f8afc640 1 -- 172.21.15.47:0/3797558189 shutdown_connections 2024-04-26T23:18:56.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.654+0000 7f73f8afc640 1 --2- 172.21.15.47:0/3797558189 >> v2:172.21.15.47:3300/0 conn(0x7f73f40aff60 0x7f73f406f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.654+0000 7f73f8afc640 1 --2- 172.21.15.47:0/3797558189 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73f40d1700 0x7f73f40722f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.654+0000 7f73f8afc640 1 -- 172.21.15.47:0/3797558189 >> 172.21.15.47:0/3797558189 conn(0x7f73f40d0be0 msgr2=0x7f73f40d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:56.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.654+0000 7fc34974b640 1 --2- 172.21.15.47:0/4139125699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3440b6b60 0x7fc3440d2a40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc3440cf450 tx=0x7fc33402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96eb11c8ea0f6573 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:56.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.654+0000 7f0a36ffd640 1 -- 172.21.15.47:0/4086350355 >> v2:172.21.15.47:3300/0 conn(0x7f0a380b4650 msgr2=0x7f0a38072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:56.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.654+0000 7f0a36ffd640 1 --2- 172.21.15.47:0/4086350355 >> v2:172.21.15.47:3300/0 conn(0x7f0a380b4650 0x7f0a38072430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.654+0000 7f0a36ffd640 1 -- 172.21.15.47:0/4086350355 >> v1:172.21.15.47:6789/0 conn(0x7f0a380b6130 legacy=0x7f0a380b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.654+0000 7f7edf69c640 1 -- 172.21.15.47:0/1878323778 wait complete. 2024-04-26T23:18:56.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.654+0000 7f0a35ffb640 1 -- 172.21.15.47:0/4086350355 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1040488793 0 0) 0x7f0a38163a50 con 0x7f0a380b6130 2024-04-26T23:18:56.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.654+0000 7f0a36ffd640 1 -- 172.21.15.47:0/4086350355 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a38163ff0 con 0x7f0a3815d980 2024-04-26T23:18:56.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.654+0000 7f602ac28640 1 -- 172.21.15.47:0/1202497799 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f60240d1700 msgr2=0x7f60240722f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.654+0000 7f602ac28640 1 --2- 172.21.15.47:0/1202497799 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f60240d1700 0x7f60240722f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f601c00aed0 tx=0x7f601c02a5d0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.654+0000 7f73f8afc640 1 -- 172.21.15.47:0/3797558189 shutdown_connections 2024-04-26T23:18:56.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.654+0000 7f7a9b7fe640 1 -- 172.21.15.47:0/976309669 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9002ae60 con 0x7f7a9c1633b0 2024-04-26T23:18:56.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.654+0000 7fc33bfff640 1 -- 172.21.15.47:0/4139125699 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc334037070 con 0x7fc3440b6b60 2024-04-26T23:18:56.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.654+0000 7fc33bfff640 1 -- 172.21.15.47:0/4139125699 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc33402ae10 con 0x7fc3440b6b60 2024-04-26T23:18:56.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.654+0000 7fc33bfff640 1 -- 172.21.15.47:0/4139125699 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc334032660 con 0x7fc3440b6b60 2024-04-26T23:18:56.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.654+0000 7fe75b647640 1 -- 172.21.15.47:0/3717837201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7540d1700 msgr2=0x7fe7540722f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.654+0000 7fe75b647640 1 --2- 172.21.15.47:0/3717837201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7540d1700 0x7fe7540722f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe74c00aea0 tx=0x7fe74c02b720 comp rx=0 tx=0).stop 2024-04-26T23:18:56.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.654+0000 7f7aa2e6d640 1 -- 172.21.15.47:0/976309669 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a9c1633b0 msgr2=0x7f7a9c0737c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.654+0000 7f7aa2e6d640 1 --2- 172.21.15.47:0/976309669 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a9c1633b0 0x7f7a9c0737c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c0cf670 tx=0x7f7a9002a110 comp rx=0 tx=0).stop 2024-04-26T23:18:56.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.655+0000 7f7aa2e6d640 1 -- 172.21.15.47:0/976309669 shutdown_connections 2024-04-26T23:18:56.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.655+0000 7fe75b647640 1 -- 172.21.15.47:0/3717837201 shutdown_connections 2024-04-26T23:18:56.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.655+0000 7fe75b647640 1 --2- 172.21.15.47:0/3717837201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7540d1700 0x7fe7540722f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.655+0000 7fe75b647640 1 -- 172.21.15.47:0/3717837201 >> 172.21.15.47:0/3717837201 conn(0x7fe7540d0be0 msgr2=0x7fe7540d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:56.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.655+0000 7f7aa2e6d640 1 --2- 172.21.15.47:0/976309669 >> v2:172.21.15.47:3300/0 conn(0x7f7a9c0cd680 0x7f7a9c06c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.655+0000 7f7aa2e6d640 1 --2- 172.21.15.47:0/976309669 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a9c1633b0 0x7f7a9c0737c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.655+0000 7f7aa2e6d640 1 -- 172.21.15.47:0/976309669 >> 172.21.15.47:0/976309669 conn(0x7f7a9c0d1af0 msgr2=0x7f7a9c0d1f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:56.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.654+0000 7f2c51d9b640 1 --2- 172.21.15.47:0/3837612841 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c540cfdb0 0x7f2c54070bf0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c009d90 tx=0x7f2c4c02a550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bee62b54f6b7d693 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:56.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.655+0000 7f7aa2e6d640 1 -- 172.21.15.47:0/976309669 shutdown_connections 2024-04-26T23:18:56.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.655+0000 7f7aa2e6d640 1 -- 172.21.15.47:0/976309669 wait complete. 2024-04-26T23:18:56.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.654+0000 7f73f8afc640 1 -- 172.21.15.47:0/3797558189 wait complete. 2024-04-26T23:18:56.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.655+0000 7f2c5159a640 1 -- 172.21.15.47:0/3837612841 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c007740 con 0x7f2c540cfdb0 2024-04-26T23:18:56.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.655+0000 7f2c5159a640 1 -- 172.21.15.47:0/3837612841 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c4c0078e0 con 0x7f2c540cfdb0 2024-04-26T23:18:56.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.655+0000 7f2c5159a640 1 -- 172.21.15.47:0/3837612841 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c007cb0 con 0x7f2c540cfdb0 2024-04-26T23:18:56.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.655+0000 7fe75b647640 1 -- 172.21.15.47:0/3717837201 shutdown_connections 2024-04-26T23:18:56.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.655+0000 7f602ac28640 1 -- 172.21.15.47:0/1202497799 shutdown_connections 2024-04-26T23:18:56.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.655+0000 7f602ac28640 1 --2- 172.21.15.47:0/1202497799 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f60240d1700 0x7f60240722f0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.655+0000 7f602ac28640 1 -- 172.21.15.47:0/1202497799 >> 172.21.15.47:0/1202497799 conn(0x7f60240d0be0 msgr2=0x7f60240d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:56.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.655+0000 7f602ac28640 1 -- 172.21.15.47:0/1202497799 shutdown_connections 2024-04-26T23:18:56.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.655+0000 7fe75b647640 1 -- 172.21.15.47:0/3717837201 wait complete. 2024-04-26T23:18:56.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.655+0000 7f602ac28640 1 -- 172.21.15.47:0/1202497799 wait complete. 2024-04-26T23:18:56.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.655+0000 7fc34b9af640 1 -- 172.21.15.47:0/4139125699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3440b6b60 msgr2=0x7fc3440d2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.655+0000 7fc34b9af640 1 --2- 172.21.15.47:0/4139125699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3440b6b60 0x7fc3440d2a40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc3440cf450 tx=0x7fc33402a100 comp rx=0 tx=0).stop 2024-04-26T23:18:56.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.656+0000 7f7edf69c640 1 Processor -- start 2024-04-26T23:18:56.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.656+0000 7f7edf69c640 1 -- start start 2024-04-26T23:18:56.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.655+0000 7fc34b9af640 1 -- 172.21.15.47:0/4139125699 shutdown_connections 2024-04-26T23:18:56.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.656+0000 7fc34b9af640 1 --2- 172.21.15.47:0/4139125699 >> v2:172.21.15.203:3300/0 conn(0x7fc3440b8720 0x7fc344175990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.654+0000 7f0a377fe640 1 --2- 172.21.15.47:0/4086350355 >> v2:172.21.15.47:3300/0 conn(0x7f0a380b4650 0x7f0a38072430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:18:56.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.656+0000 7f0a36ffd640 1 --2- 172.21.15.47:0/4086350355 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a3815d980 0x7f0a38167a50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0a380d1f20 tx=0x7f0a2802a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a005818f12f0105 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:56.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.656+0000 7fc34b9af640 1 --2- 172.21.15.47:0/4139125699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3440b6b60 0x7fc3440d2a40 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.656+0000 7fc34b9af640 1 -- 172.21.15.47:0/4139125699 >> 172.21.15.47:0/4139125699 conn(0x7fc3440d18f0 msgr2=0x7fc3440d1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:56.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.656+0000 7f73f8afc640 1 Processor -- start 2024-04-26T23:18:56.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.656+0000 7f0a35ffb640 1 -- 172.21.15.47:0/4086350355 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a28037070 con 0x7f0a3815d980 2024-04-26T23:18:56.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.656+0000 7f73f8afc640 1 -- start start 2024-04-26T23:18:56.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.656+0000 7f0a35ffb640 1 -- 172.21.15.47:0/4086350355 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a28032440 con 0x7f0a3815d980 2024-04-26T23:18:56.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.656+0000 7f7aa2e6d640 1 Processor -- start 2024-04-26T23:18:56.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.656+0000 7f0a35ffb640 1 -- 172.21.15.47:0/4086350355 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a28032810 con 0x7f0a3815d980 2024-04-26T23:18:56.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.657+0000 7f2c5359e640 1 -- 172.21.15.47:0/3837612841 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c540cfdb0 msgr2=0x7f2c54070bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.657+0000 7f2c5359e640 1 --2- 172.21.15.47:0/3837612841 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c540cfdb0 0x7f2c54070bf0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c009d90 tx=0x7f2c4c02a550 comp rx=0 tx=0).stop 2024-04-26T23:18:56.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.656+0000 7f7aa2e6d640 1 -- start start 2024-04-26T23:18:56.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.657+0000 7fe75b647640 1 Processor -- start 2024-04-26T23:18:56.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.657+0000 7fe75b647640 1 -- start start 2024-04-26T23:18:56.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.657+0000 7f7edf69c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ed0007260 0x7f7ed00d72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.657+0000 7f7edf69c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ed00543f0 0x7f7ed0039540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.657+0000 7f7edf69c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ed00e33a0 0x7f7ed0039a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.657+0000 7f7edf69c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed0053a70 con 0x7f7ed00543f0 2024-04-26T23:18:56.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.657+0000 7f7edf69c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed00538f0 con 0x7f7ed00e33a0 2024-04-26T23:18:56.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.657+0000 7f7edf69c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f7ed003c810 con 0x7f7ed0007260 2024-04-26T23:18:56.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.657+0000 7f2c5359e640 1 -- 172.21.15.47:0/3837612841 shutdown_connections 2024-04-26T23:18:56.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.657+0000 7f2c5359e640 1 --2- 172.21.15.47:0/3837612841 >> v2:172.21.15.47:3300/0 conn(0x7f2c54071d40 0x7f2c54071130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.657+0000 7f2c5359e640 1 --2- 172.21.15.47:0/3837612841 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c540cfdb0 0x7f2c54070bf0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.657+0000 7f2c5359e640 1 -- 172.21.15.47:0/3837612841 >> 172.21.15.47:0/3837612841 conn(0x7f2c540d2c90 msgr2=0x7f2c540d30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:56.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.657+0000 7f602ac28640 1 Processor -- start 2024-04-26T23:18:56.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.657+0000 7f73f8afc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73f406ddf0 0x7f73f41582a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.657+0000 7f7edd438640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ed0007260 0x7f7ed00d72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.657+0000 7f7edcc37640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ed00543f0 0x7f7ed0039540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.657+0000 7f7eddc39640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ed00e33a0 0x7f7ed0039a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.657+0000 7f7edd438640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ed0007260 0x7f7ed00d72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.47:42514/0 (socket says 172.21.15.47:42514) 2024-04-26T23:18:56.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.657+0000 7f7edcc37640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ed00543f0 0x7f7ed0039540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58660/0 (socket says 172.21.15.47:58660) 2024-04-26T23:18:56.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.657+0000 7f7edd438640 1 -- 172.21.15.47:0/225730950 learned_addr learned my addr 172.21.15.47:0/225730950 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:56.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.657+0000 7f7eddc39640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ed00e33a0 0x7f7ed0039a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.47:58136/0 (socket says 172.21.15.47:58136) 2024-04-26T23:18:56.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.657+0000 7f0a3d3af640 1 -- 172.21.15.47:0/4086350355 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a3815d980 msgr2=0x7f0a38167a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.657+0000 7f0a3d3af640 1 --2- 172.21.15.47:0/4086350355 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a3815d980 0x7f0a38167a50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0a380d1f20 tx=0x7f0a2802a1a0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.658+0000 7fc34b9af640 1 -- 172.21.15.47:0/4139125699 shutdown_connections 2024-04-26T23:18:56.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.657+0000 7f73f8afc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73f40aff60 0x7f73f4155090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.658+0000 7f73f259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73f406ddf0 0x7f73f41582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.658+0000 7f2c5359e640 1 -- 172.21.15.47:0/3837612841 shutdown_connections 2024-04-26T23:18:56.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.658+0000 7f73f8afc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73f40d1700 0x7f73f40b4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.658+0000 7f73f8afc640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f73f406ea90 con 0x7f73f406ddf0 2024-04-26T23:18:56.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.658+0000 7f73f259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73f406ddf0 0x7f73f41582a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58664/0 (socket says 172.21.15.47:58664) 2024-04-26T23:18:56.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.658+0000 7f73f259c640 1 -- 172.21.15.47:0/1378842018 learned_addr learned my addr 172.21.15.47:0/1378842018 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:56.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.658+0000 7f7edcc37640 1 -- 172.21.15.47:0/225730950 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ed0007260 msgr2=0x7f7ed00d72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.658+0000 7f7edcc37640 1 --2- 172.21.15.47:0/225730950 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ed0007260 0x7f7ed00d72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.658+0000 7f7edcc37640 1 -- 172.21.15.47:0/225730950 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ed00e33a0 msgr2=0x7f7ed0039a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.658+0000 7f7edcc37640 1 --2- 172.21.15.47:0/225730950 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ed00e33a0 0x7f7ed0039a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.658+0000 7f7edcc37640 1 -- 172.21.15.47:0/225730950 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed0051040 con 0x7f7ed00543f0 2024-04-26T23:18:56.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.658+0000 7f73f8afc640 1 -- 172.21.15.47:0/1378842018 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f73f406e910 con 0x7f73f40d1700 2024-04-26T23:18:56.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.658+0000 7f7aa2e6d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a9c06c780 0x7f7a9c162b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.658+0000 7f73f2d9d640 1 --2- 172.21.15.47:0/1378842018 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73f40d1700 0x7f73f40b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.658+0000 7f7aa2e6d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a9c0cd680 0x7f7a9c0700f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.658+0000 7f7aa2e6d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a9c1633b0 0x7f7a9c070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.658+0000 7f2c5359e640 1 -- 172.21.15.47:0/3837612841 wait complete. 2024-04-26T23:18:56.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.658+0000 7f7aa2e6d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9c0cdef0 con 0x7f7a9c0cd680 2024-04-26T23:18:56.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.658+0000 7f73f8afc640 1 -- 172.21.15.47:0/1378842018 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f73f415ece0 con 0x7f73f40aff60 2024-04-26T23:18:56.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.658+0000 7f73f1d9b640 1 --2- 172.21.15.47:0/1378842018 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73f40aff60 0x7f73f4155090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.658+0000 7f7aa2e6d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9c0cf9a0 con 0x7f7a9c1633b0 2024-04-26T23:18:56.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.658+0000 7f7aa2e6d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f7a9c0ce1d0 con 0x7f7a9c06c780 2024-04-26T23:18:56.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.658+0000 7f7a9bfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a9c0cd680 0x7f7a9c0700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.658+0000 7f7a9bfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a9c0cd680 0x7f7a9c0700f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58676/0 (socket says 172.21.15.47:58676) 2024-04-26T23:18:56.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.658+0000 7f7a9bfff640 1 -- 172.21.15.47:0/491440119 learned_addr learned my addr 172.21.15.47:0/491440119 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:56.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.658+0000 7f7edcc37640 1 --2- 172.21.15.47:0/225730950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ed00543f0 0x7f7ed0039540 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7ec800c440 tx=0x7f7ec800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:56.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.658+0000 7f7aa140a640 1 --2- 172.21.15.47:0/491440119 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a9c1633b0 0x7f7a9c070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.658+0000 7f7aa0c09640 1 --2- 172.21.15.47:0/491440119 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a9c06c780 0x7f7a9c162b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.659+0000 7fe75b647640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe75406f600 0x7fe7540b0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.659+0000 7f7a9bfff640 1 -- 172.21.15.47:0/491440119 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a9c06c780 msgr2=0x7f7a9c162b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.659+0000 7f7ecdffb640 1 -- 172.21.15.47:0/225730950 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec8011790 con 0x7f7ed00543f0 2024-04-26T23:18:56.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.659+0000 7f7a9bfff640 1 --2- 172.21.15.47:0/491440119 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a9c06c780 0x7f7a9c162b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.659+0000 7f7a9bfff640 1 -- 172.21.15.47:0/491440119 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a9c1633b0 msgr2=0x7f7a9c070630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.659+0000 7f7a9bfff640 1 --2- 172.21.15.47:0/491440119 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a9c1633b0 0x7f7a9c070630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.659+0000 7f7a9bfff640 1 -- 172.21.15.47:0/491440119 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a9c0ae330 con 0x7f7a9c0cd680 2024-04-26T23:18:56.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.659+0000 7fe75b647640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7540aff60 0x7fe7540c3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.659+0000 7f7aa0c09640 1 --2- 172.21.15.47:0/491440119 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a9c06c780 0x7f7a9c162b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:18:56.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.659+0000 7f7edf69c640 1 -- 172.21.15.47:0/225730950 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ed004ffa0 con 0x7f7ed00543f0 2024-04-26T23:18:56.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.659+0000 7f7ecdffb640 1 -- 172.21.15.47:0/225730950 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ec8017070 con 0x7f7ed00543f0 2024-04-26T23:18:56.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.659+0000 7fe7593e3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe75406f600 0x7fe7540b0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.659+0000 7fe75b647640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7540d1700 0x7fe75415a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.659+0000 7fe75b647640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe7540d0980 con 0x7fe7540aff60 2024-04-26T23:18:56.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.659+0000 7fe75b647640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe75406fa10 con 0x7fe7540d1700 2024-04-26T23:18:56.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.659+0000 7fe75b647640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fe75406ddf0 con 0x7fe75406f600 2024-04-26T23:18:56.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.659+0000 7fe758be2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7540aff60 0x7fe7540c3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.659+0000 7fe7593e3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe75406f600 0x7fe7540b0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.47:42534/0 (socket says 172.21.15.47:42534) 2024-04-26T23:18:56.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.659+0000 7fe7593e3640 1 -- 172.21.15.47:0/4001770385 learned_addr learned my addr 172.21.15.47:0/4001770385 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:56.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.660+0000 7fe758be2640 1 -- 172.21.15.47:0/4001770385 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe75406f600 msgr2=0x7fe7540b0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.660+0000 7fe758be2640 1 --2- 172.21.15.47:0/4001770385 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe75406f600 0x7fe7540b0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.660+0000 7fe758be2640 1 -- 172.21.15.47:0/4001770385 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7540d1700 msgr2=0x7fe75415a970 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T23:18:56.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.660+0000 7fe758be2640 1 --2- 172.21.15.47:0/4001770385 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7540d1700 0x7fe75415a970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.660+0000 7fe758be2640 1 -- 172.21.15.47:0/4001770385 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7540c41c0 con 0x7fe7540aff60 2024-04-26T23:18:56.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.660+0000 7f602ac28640 1 -- start start 2024-04-26T23:18:56.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.659+0000 7f73f259c640 1 -- 172.21.15.47:0/1378842018 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73f40aff60 msgr2=0x7f73f4155090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.659+0000 7f73f259c640 1 --2- 172.21.15.47:0/1378842018 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73f40aff60 0x7f73f4155090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.659+0000 7f73f259c640 1 -- 172.21.15.47:0/1378842018 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73f40d1700 msgr2=0x7f73f40b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.659+0000 7f73f259c640 1 --2- 172.21.15.47:0/1378842018 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73f40d1700 0x7f73f40b4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.659+0000 7f73f259c640 1 -- 172.21.15.47:0/1378842018 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73f4157200 con 0x7f73f406ddf0 2024-04-26T23:18:56.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.660+0000 7f73f2d9d640 1 --2- 172.21.15.47:0/1378842018 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73f40d1700 0x7f73f40b4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:18:56.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.660+0000 7f73f259c640 1 --2- 172.21.15.47:0/1378842018 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73f406ddf0 0x7f73f41582a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f73e4009890 tx=0x7f73e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:56.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.660+0000 7f73e2ffd640 1 -- 172.21.15.47:0/1378842018 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e4037050 con 0x7f73f406ddf0 2024-04-26T23:18:56.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.660+0000 7f2c5359e640 1 Processor -- start 2024-04-26T23:18:56.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.659+0000 7f0a3d3af640 1 -- 172.21.15.47:0/4086350355 shutdown_connections 2024-04-26T23:18:56.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.659+0000 7f0a3d3af640 1 --2- 172.21.15.47:0/4086350355 >> v2:172.21.15.47:3300/0 conn(0x7f0a380b4650 0x7f0a38072430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.659+0000 7f0a3d3af640 1 --2- 172.21.15.47:0/4086350355 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a3815d980 0x7f0a38167a50 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.659+0000 7f0a3d3af640 1 -- 172.21.15.47:0/4086350355 >> 172.21.15.47:0/4086350355 conn(0x7f0a38068910 msgr2=0x7f0a3806e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:56.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.659+0000 7f0a3d3af640 1 -- 172.21.15.47:0/4086350355 shutdown_connections 2024-04-26T23:18:56.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.659+0000 7f0a3d3af640 1 -- 172.21.15.47:0/4086350355 wait complete. 2024-04-26T23:18:56.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.659+0000 7f7a9bfff640 1 --2- 172.21.15.47:0/491440119 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a9c0cd680 0x7f7a9c0700f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c00ec10 tx=0x7f7a8c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:56.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.660+0000 7f7a997fa640 1 -- 172.21.15.47:0/491440119 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c017070 con 0x7f7a9c0cd680 2024-04-26T23:18:56.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.660+0000 7f7a997fa640 1 -- 172.21.15.47:0/491440119 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a8c00cc30 con 0x7f7a9c0cd680 2024-04-26T23:18:56.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.660+0000 7f7a997fa640 1 -- 172.21.15.47:0/491440119 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c013470 con 0x7f7a9c0cd680 2024-04-26T23:18:56.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.660+0000 7f73e2ffd640 1 -- 172.21.15.47:0/1378842018 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73e4043070 con 0x7f73f406ddf0 2024-04-26T23:18:56.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.660+0000 7f73e2ffd640 1 -- 172.21.15.47:0/1378842018 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e4030ab0 con 0x7f73f406ddf0 2024-04-26T23:18:56.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.660+0000 7f7aa2e6d640 1 -- 172.21.15.47:0/491440119 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a9c0ae610 con 0x7f7a9c0cd680 2024-04-26T23:18:56.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.660+0000 7f7aa2e6d640 1 -- 172.21.15.47:0/491440119 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a9c0ad5d0 con 0x7f7a9c0cd680 2024-04-26T23:18:56.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.659+0000 7f7edf69c640 1 -- 172.21.15.47:0/225730950 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ed0050560 con 0x7f7ed00543f0 2024-04-26T23:18:56.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.660+0000 7f73f8afc640 1 -- 172.21.15.47:0/1378842018 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73f41574e0 con 0x7f73f406ddf0 2024-04-26T23:18:56.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.660+0000 7f2c5359e640 1 -- start start 2024-04-26T23:18:56.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.660+0000 7f7ecdffb640 1 -- 172.21.15.47:0/225730950 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec8011b70 con 0x7f7ed00543f0 2024-04-26T23:18:56.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.658+0000 7fc34b9af640 1 -- 172.21.15.47:0/4139125699 wait complete. 2024-04-26T23:18:56.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.660+0000 7fc34b9af640 1 Processor -- start 2024-04-26T23:18:56.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.661+0000 7f2c5359e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c54071d40 0x7f2c541552d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.662+0000 7f602ac28640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f602406f600 0x7f60240c9be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.662+0000 7f73f8afc640 1 -- 172.21.15.47:0/1378842018 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73f4157aa0 con 0x7f73f406ddf0 2024-04-26T23:18:56.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.660+0000 7fc34b9af640 1 -- start start 2024-04-26T23:18:56.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.661+0000 7fc34b9af640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3440b8720 0x7fc3441635d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.661+0000 7fc34b9af640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3440d4460 0x7fc344166a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.661+0000 7fc34b9af640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc34406dda0 0x7fc3440c89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.661+0000 7fc34b9af640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc3440d40c0 con 0x7fc34406dda0 2024-04-26T23:18:56.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.661+0000 7fc34b9af640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc34409e0e0 con 0x7fc3440b8720 2024-04-26T23:18:56.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.661+0000 7fc34b9af640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc3440d4240 con 0x7fc3440d4460 2024-04-26T23:18:56.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.662+0000 7fc34974b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3440b8720 0x7fc3441635d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.662+0000 7f2c5359e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c540ceff0 0x7f2c5409cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.662+0000 7f7aa2e6d640 1 -- 172.21.15.47:0/491440119 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7a9c069a80 con 0x7f7a9c0cd680 2024-04-26T23:18:56.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.662+0000 7fc34974b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3440b8720 0x7fc3441635d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.47:58172/0 (socket says 172.21.15.47:58172) 2024-04-26T23:18:56.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.662+0000 7fc34974b640 1 -- 172.21.15.47:0/2334268724 learned_addr learned my addr 172.21.15.47:0/2334268724 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:56.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.662+0000 7fc348f4a640 1 --2- 172.21.15.47:0/2334268724 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3440d4460 0x7fc344166a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.662+0000 7f0a3d3af640 1 Processor -- start 2024-04-26T23:18:56.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.662+0000 7f2c5259c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c54071d40 0x7f2c541552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.662+0000 7f0a3d3af640 1 -- start start 2024-04-26T23:18:56.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.662+0000 7f2c5259c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c54071d40 0x7f2c541552d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.47:58180/0 (socket says 172.21.15.47:58180) 2024-04-26T23:18:56.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.662+0000 7f2c5259c640 1 -- 172.21.15.47:0/1920633277 learned_addr learned my addr 172.21.15.47:0/1920633277 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:56.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.662+0000 7f2c5359e640 1 --2- 172.21.15.47:0/1920633277 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c540cfdb0 0x7f2c54153ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.662+0000 7fe758be2640 1 --2- 172.21.15.47:0/4001770385 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7540aff60 0x7fe7540c3c80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe75000c7b0 tx=0x7fe75000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:56.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.662+0000 7f7ecdffb640 1 -- 172.21.15.47:0/225730950 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7ec8011d10 con 0x7f7ed00543f0 2024-04-26T23:18:56.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.662+0000 7f2c5359e640 1 -- 172.21.15.47:0/1920633277 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2c540719b0 con 0x7f2c540ceff0 2024-04-26T23:18:56.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.662+0000 7f602ac28640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60240aff60 0x7f60240c5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.662+0000 7f2c5359e640 1 -- 172.21.15.47:0/1920633277 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2c540706d0 con 0x7f2c54071d40 2024-04-26T23:18:56.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.662+0000 7f2c5359e640 1 -- 172.21.15.47:0/1920633277 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f2c540cd6c0 con 0x7f2c540cfdb0 2024-04-26T23:18:56.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.662+0000 7f2c52d9d640 1 --2- 172.21.15.47:0/1920633277 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c540cfdb0 0x7f2c54153ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.662+0000 7f60289c4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f602406f600 0x7f60240c9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.662+0000 7fc34974b640 1 -- 172.21.15.47:0/2334268724 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3440d4460 msgr2=0x7fc344166a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.662+0000 7fc34974b640 1 --2- 172.21.15.47:0/2334268724 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3440d4460 0x7fc344166a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.662+0000 7fc34974b640 1 -- 172.21.15.47:0/2334268724 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc34406dda0 msgr2=0x7fc3440c89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.662+0000 7fc34974b640 1 --2- 172.21.15.47:0/2334268724 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc34406dda0 0x7fc3440c89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.663+0000 7fc34974b640 1 -- 172.21.15.47:0/2334268724 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc344162b30 con 0x7fc3440b8720 2024-04-26T23:18:56.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.662+0000 7f60289c4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f602406f600 0x7f60240c9be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.47:42560/0 (socket says 172.21.15.47:42560) 2024-04-26T23:18:56.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.663+0000 7f73f1d9b640 1 --2- 172.21.15.47:0/1378842018 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73f40aff60 0x7f73f4155090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:18:56.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.662+0000 7f602ac28640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60240d1700 0x7f60240c5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.662+0000 7f60289c4640 1 -- 172.21.15.47:0/3356465508 learned_addr learned my addr 172.21.15.47:0/3356465508 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:56.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.662+0000 7f602ac28640 1 -- 172.21.15.47:0/3356465508 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f60240d0980 con 0x7f60240d1700 2024-04-26T23:18:56.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.662+0000 7f602ac28640 1 -- 172.21.15.47:0/3356465508 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6024073f80 con 0x7f60240aff60 2024-04-26T23:18:56.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.662+0000 7f602ac28640 1 -- 172.21.15.47:0/3356465508 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f602406ddf0 con 0x7f602406f600 2024-04-26T23:18:56.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.662+0000 7f6023fff640 1 --2- 172.21.15.47:0/3356465508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60240aff60 0x7f60240c5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.663+0000 7fc34974b640 1 --2- 172.21.15.47:0/2334268724 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3440b8720 0x7fc3441635d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc33402a670 tx=0x7fc3340046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:56.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.663+0000 7f7ecdffb640 1 --2- 172.21.15.47:0/225730950 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f7ebc06c8f0 0x7f7ebc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.663+0000 7f60289c4640 1 -- 172.21.15.47:0/3356465508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60240aff60 msgr2=0x7f60240c5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.663+0000 7f7edf69c640 1 -- 172.21.15.47:0/225730950 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7eb8001030 con 0x7f7ed00543f0 2024-04-26T23:18:56.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.663+0000 7f60289c4640 1 --2- 172.21.15.47:0/3356465508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60240aff60 0x7f60240c5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.663+0000 7f60289c4640 1 -- 172.21.15.47:0/3356465508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60240d1700 msgr2=0x7f60240c5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.663+0000 7fc339ffb640 1 -- 172.21.15.47:0/2334268724 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc334042070 con 0x7fc3440b8720 2024-04-26T23:18:56.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.663+0000 7f2c5259c640 1 -- 172.21.15.47:0/1920633277 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c540cfdb0 msgr2=0x7f2c54153ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.663+0000 7f2c5259c640 1 --2- 172.21.15.47:0/1920633277 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c540cfdb0 0x7f2c54153ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.663+0000 7f2c5259c640 1 -- 172.21.15.47:0/1920633277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c540ceff0 msgr2=0x7f2c5409cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.663+0000 7f2c5259c640 1 --2- 172.21.15.47:0/1920633277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c540ceff0 0x7f2c5409cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.663+0000 7f2c5259c640 1 -- 172.21.15.47:0/1920633277 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c5409d0b0 con 0x7f2c54071d40 2024-04-26T23:18:56.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.663+0000 7f60289c4640 1 --2- 172.21.15.47:0/3356465508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60240d1700 0x7f60240c5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.663+0000 7f60289c4640 1 -- 172.21.15.47:0/3356465508 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f602415c4a0 con 0x7f602406f600 2024-04-26T23:18:56.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.663+0000 7f7edd438640 1 --2- 172.21.15.47:0/225730950 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f7ebc06c8f0 0x7f7ebc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.663+0000 7f0a3d3af640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a380b4650 0x7f0a38162540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.663+0000 7f2c5259c640 1 --2- 172.21.15.47:0/1920633277 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c54071d40 0x7f2c541552d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2c4400c780 tx=0x7f2c4400cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:56.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.663+0000 7fc34b9af640 1 -- 172.21.15.47:0/2334268724 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc34408cab0 con 0x7fc3440b8720 2024-04-26T23:18:56.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.664+0000 7fc339ffb640 1 -- 172.21.15.47:0/2334268724 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc33403e660 con 0x7fc3440b8720 2024-04-26T23:18:56.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.664+0000 7f0a3d3af640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a380b6130 0x7f0a38153bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.664+0000 7f2c42ffd640 1 -- 172.21.15.47:0/1920633277 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c44018070 con 0x7f2c54071d40 2024-04-26T23:18:56.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.664+0000 7f2c5359e640 1 -- 172.21.15.47:0/1920633277 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c5409d350 con 0x7f2c54071d40 2024-04-26T23:18:56.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.664+0000 7f7ecdffb640 1 -- 172.21.15.47:0/225730950 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f7ec8056db0 con 0x7f7ed00543f0 2024-04-26T23:18:56.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.664+0000 7f73f8afc640 1 -- 172.21.15.47:0/1378842018 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f73d8001030 con 0x7f73f406ddf0 2024-04-26T23:18:56.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.664+0000 7f2c5359e640 1 -- 172.21.15.47:0/1920633277 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c5409eff0 con 0x7f2c54071d40 2024-04-26T23:18:56.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.664+0000 7f0a3d3af640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a3815d980 0x7f0a38154110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.664+0000 7f0a3d3af640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3815dd60 con 0x7f0a380b6130 2024-04-26T23:18:56.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.664+0000 7f0a3d3af640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0a381645b0 con 0x7f0a3815d980 2024-04-26T23:18:56.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.664+0000 7f0a3d3af640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f0a3815dee0 con 0x7f0a380b4650 2024-04-26T23:18:56.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.664+0000 7f7ecdffb640 1 --2- 172.21.15.47:0/225730950 >> [v2:172.21.15.203:6824/2738517080,v1:172.21.15.203:6825/2738517080] conn(0x7f7ebc075ff0 0x7f7ebc078430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.664+0000 7f0a36ffd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a380b4650 0x7f0a38162540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.664+0000 7f7ecdffb640 1 -- 172.21.15.47:0/225730950 --> [v2:172.21.15.203:6824/2738517080,v1:172.21.15.203:6825/2738517080] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7ec8017070 con 0x7f7ebc075ff0 2024-04-26T23:18:56.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.664+0000 7f2c42ffd640 1 -- 172.21.15.47:0/1920633277 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c440072d0 con 0x7f2c54071d40 2024-04-26T23:18:56.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.664+0000 7f0a36ffd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a380b4650 0x7f0a38162540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.47:42576/0 (socket says 172.21.15.47:42576) 2024-04-26T23:18:56.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.664+0000 7f0a36ffd640 1 -- 172.21.15.47:0/2700425415 learned_addr learned my addr 172.21.15.47:0/2700425415 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:56.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.664+0000 7f7ecdffb640 1 -- 172.21.15.47:0/225730950 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7ec8090060 con 0x7f7ed00543f0 2024-04-26T23:18:56.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.665+0000 7fe749ffb640 1 -- 172.21.15.47:0/4001770385 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe750018070 con 0x7fe7540aff60 2024-04-26T23:18:56.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.665+0000 7f0a367fc640 1 --2- 172.21.15.47:0/2700425415 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a380b6130 0x7f0a38153bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.664+0000 7fc34b9af640 1 -- 172.21.15.47:0/2334268724 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc344162dd0 con 0x7fc3440b8720 2024-04-26T23:18:56.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.665+0000 7fc339ffb640 1 -- 172.21.15.47:0/2334268724 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc334037040 con 0x7fc3440b8720 2024-04-26T23:18:56.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.664+0000 7f7a997fa640 1 -- 172.21.15.47:0/491440119 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7a8c01a070 con 0x7f7a9c0cd680 2024-04-26T23:18:56.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.665+0000 7fe75b647640 1 -- 172.21.15.47:0/4001770385 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7540c4470 con 0x7fe7540aff60 2024-04-26T23:18:56.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.664+0000 7f7a997fa640 1 --2- 172.21.15.47:0/491440119 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f7a8806c9f0 0x7f7a8806eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.665+0000 7f7a997fa640 1 -- 172.21.15.47:0/491440119 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f7a8c08c010 con 0x7f7a9c0cd680 2024-04-26T23:18:56.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.665+0000 7f7eddc39640 1 --2- 172.21.15.47:0/225730950 >> [v2:172.21.15.203:6824/2738517080,v1:172.21.15.203:6825/2738517080] conn(0x7f7ebc075ff0 0x7f7ebc078430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.665+0000 7f7a997fa640 1 --2- 172.21.15.47:0/491440119 >> [v2:172.21.15.203:6800/56838444,v1:172.21.15.203:6801/56838444] conn(0x7f7a880760f0 0x7f7a88078530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.665+0000 7f7a997fa640 1 -- 172.21.15.47:0/491440119 --> [v2:172.21.15.203:6800/56838444,v1:172.21.15.203:6801/56838444] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7a88078c00 con 0x7f7a880760f0 2024-04-26T23:18:56.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.665+0000 7f7a997fa640 1 -- 172.21.15.47:0/491440119 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7a8c093050 con 0x7f7a9c0cd680 2024-04-26T23:18:56.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.665+0000 7fe75b647640 1 -- 172.21.15.47:0/4001770385 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe75415dfe0 con 0x7fe7540aff60 2024-04-26T23:18:56.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.665+0000 7f7aa140a640 1 --2- 172.21.15.47:0/491440119 >> [v2:172.21.15.203:6800/56838444,v1:172.21.15.203:6801/56838444] conn(0x7f7a880760f0 0x7f7a88078530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.665+0000 7fe749ffb640 1 -- 172.21.15.47:0/4001770385 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7500072d0 con 0x7fe7540aff60 2024-04-26T23:18:56.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.665+0000 7f60289c4640 1 --2- 172.21.15.47:0/3356465508 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f602406f600 0x7f60240c9be0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f601c02abc0 tx=0x7f601c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:56.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.666+0000 7f60217fa640 1 -- 172.21.15.47:0/3356465508 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601c036050 con 0x7f602406f600 2024-04-26T23:18:56.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.665+0000 7f0a377fe640 1 --2- 172.21.15.47:0/2700425415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a3815d980 0x7f0a38154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.666+0000 7f0a367fc640 1 -- 172.21.15.47:0/2700425415 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a380b4650 msgr2=0x7f0a38162540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.666+0000 7f0a367fc640 1 --2- 172.21.15.47:0/2700425415 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a380b4650 0x7f0a38162540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.666+0000 7f0a367fc640 1 -- 172.21.15.47:0/2700425415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a3815d980 msgr2=0x7f0a38154110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.666+0000 7f0a367fc640 1 --2- 172.21.15.47:0/2700425415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a3815d980 0x7f0a38154110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.666+0000 7f0a367fc640 1 -- 172.21.15.47:0/2700425415 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a381ef820 con 0x7f0a380b6130 2024-04-26T23:18:56.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.666+0000 7f60217fa640 1 -- 172.21.15.47:0/3356465508 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f601c037040 con 0x7f602406f600 2024-04-26T23:18:56.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.666+0000 7f0a367fc640 1 --2- 172.21.15.47:0/2700425415 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a380b6130 0x7f0a38153bd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0a380d1a30 tx=0x7f0a2c00c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:56.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.666+0000 7fc34b9af640 1 -- 172.21.15.47:0/2334268724 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc344166f60 con 0x7fc3440b8720 2024-04-26T23:18:56.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.666+0000 7f60217fa640 1 -- 172.21.15.47:0/3356465508 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601c004040 con 0x7f602406f600 2024-04-26T23:18:56.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.666+0000 7f602ac28640 1 -- 172.21.15.47:0/3356465508 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f602415d910 con 0x7f602406f600 2024-04-26T23:18:56.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.666+0000 7f602ac28640 1 -- 172.21.15.47:0/3356465508 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f602415ded0 con 0x7f602406f600 2024-04-26T23:18:56.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.666+0000 7fe749ffb640 1 -- 172.21.15.47:0/4001770385 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7500047e0 con 0x7fe7540aff60 2024-04-26T23:18:56.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.666+0000 7f73e2ffd640 1 -- 172.21.15.47:0/1378842018 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f73e4030c50 con 0x7f73f406ddf0 2024-04-26T23:18:56.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.666+0000 7f73e2ffd640 1 --2- 172.21.15.47:0/1378842018 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f73d406c9c0 0x7f73d406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.666+0000 7f7aa0c09640 1 --2- 172.21.15.47:0/491440119 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f7a8806c9f0 0x7f7a8806eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.667+0000 7f73e2ffd640 1 -- 172.21.15.47:0/1378842018 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f73e40ac960 con 0x7f73f406ddf0 2024-04-26T23:18:56.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.666+0000 7f2c42ffd640 1 -- 172.21.15.47:0/1920633277 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c440047e0 con 0x7f2c54071d40 2024-04-26T23:18:56.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.667+0000 7fe75b647640 1 -- 172.21.15.47:0/4001770385 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe7540694f0 con 0x7fe7540aff60 2024-04-26T23:18:56.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.667+0000 7f73e2ffd640 1 --2- 172.21.15.47:0/1378842018 >> [v2:172.21.15.47:6800/2569869305,v1:172.21.15.47:6801/2569869305] conn(0x7f73d4076130 0x7f73d4078570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.667+0000 7f73e2ffd640 1 -- 172.21.15.47:0/1378842018 --> [v2:172.21.15.47:6800/2569869305,v1:172.21.15.47:6801/2569869305] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f73e4008eb0 con 0x7f73d4076130 2024-04-26T23:18:56.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.667+0000 7f73e2ffd640 1 -- 172.21.15.47:0/1378842018 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f73e40b12a0 con 0x7f73f406ddf0 2024-04-26T23:18:56.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.667+0000 7f7aa140a640 1 --2- 172.21.15.47:0/491440119 >> [v2:172.21.15.203:6800/56838444,v1:172.21.15.203:6801/56838444] conn(0x7f7a880760f0 0x7f7a88078530 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:56.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.667+0000 7f2c42ffd640 1 -- 172.21.15.47:0/1920633277 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2c4401b070 con 0x7f2c54071d40 2024-04-26T23:18:56.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.667+0000 7f73f1d9b640 1 --2- 172.21.15.47:0/1378842018 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f73d406c9c0 0x7f73d406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.666+0000 7f7eddc39640 1 --2- 172.21.15.47:0/225730950 >> [v2:172.21.15.203:6824/2738517080,v1:172.21.15.203:6825/2738517080] conn(0x7f7ebc075ff0 0x7f7ebc078430 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:56.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.666+0000 7f7edd438640 1 --2- 172.21.15.47:0/225730950 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f7ebc06c8f0 0x7f7ebc06edb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4007a60 tx=0x7f7ed4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:56.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.667+0000 7f7ecdffb640 1 -- 172.21.15.47:0/225730950 <== osd.6 v2:172.21.15.203:6824/2738517080 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7ec8017070 con 0x7f7ebc075ff0 2024-04-26T23:18:56.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.667+0000 7f2c5359e640 1 -- 172.21.15.47:0/1920633277 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2c38001030 con 0x7f2c54071d40 2024-04-26T23:18:56.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.666+0000 7fc339ffb640 1 -- 172.21.15.47:0/2334268724 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc334045070 con 0x7fc3440b8720 2024-04-26T23:18:56.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.667+0000 7fc339ffb640 1 --2- 172.21.15.47:0/2334268724 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fc32806c9c0 0x7fc32806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.667+0000 7fc339ffb640 1 -- 172.21.15.47:0/2334268724 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fc3340abe10 con 0x7fc3440b8720 2024-04-26T23:18:56.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.667+0000 7fc339ffb640 1 --2- 172.21.15.47:0/2334268724 >> [v2:172.21.15.203:6816/1315565058,v1:172.21.15.203:6817/1315565058] conn(0x7fc3280760c0 0x7fc328078500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.667+0000 7fc348f4a640 1 --2- 172.21.15.47:0/2334268724 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fc32806c9c0 0x7fc32806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.667+0000 7fc339ffb640 1 -- 172.21.15.47:0/2334268724 --> [v2:172.21.15.203:6816/1315565058,v1:172.21.15.203:6817/1315565058] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc334033e50 con 0x7fc3280760c0 2024-04-26T23:18:56.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.667+0000 7f73f2d9d640 1 --2- 172.21.15.47:0/1378842018 >> [v2:172.21.15.47:6800/2569869305,v1:172.21.15.47:6801/2569869305] conn(0x7f73d4076130 0x7f73d4078570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.667+0000 7fc339ffb640 1 -- 172.21.15.47:0/2334268724 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc334075bc0 con 0x7fc3440b8720 2024-04-26T23:18:56.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.667+0000 7f0a1f7fe640 1 -- 172.21.15.47:0/2700425415 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c0074e0 con 0x7f0a380b6130 2024-04-26T23:18:56.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.667+0000 7fc349f4c640 1 --2- 172.21.15.47:0/2334268724 >> [v2:172.21.15.203:6816/1315565058,v1:172.21.15.203:6817/1315565058] conn(0x7fc3280760c0 0x7fc328078500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.668+0000 7f0a1f7fe640 1 -- 172.21.15.47:0/2700425415 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a2c007680 con 0x7f0a380b6130 2024-04-26T23:18:56.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.668+0000 7f7a997fa640 1 -- 172.21.15.47:0/491440119 <== osd.5 v2:172.21.15.203:6800/56838444 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7a88078c00 con 0x7f7a880760f0 2024-04-26T23:18:56.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.668+0000 7f0a1f7fe640 1 -- 172.21.15.47:0/2700425415 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c0049e0 con 0x7f0a380b6130 2024-04-26T23:18:56.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.668+0000 7f0a3d3af640 1 -- 172.21.15.47:0/2700425415 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a381ef540 con 0x7f0a380b6130 2024-04-26T23:18:56.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.668+0000 7f73f2d9d640 1 --2- 172.21.15.47:0/1378842018 >> [v2:172.21.15.47:6800/2569869305,v1:172.21.15.47:6801/2569869305] conn(0x7f73d4076130 0x7f73d4078570 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:56.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.668+0000 7f60217fa640 1 -- 172.21.15.47:0/3356465508 <== mon.2 v2:172.21.15.203:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f601c04d060 con 0x7f602406f600 2024-04-26T23:18:56.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.668+0000 7fc348f4a640 1 --2- 172.21.15.47:0/2334268724 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fc32806c9c0 0x7fc32806ee80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc340007c90 tx=0x7fc340007be0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:56.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.668+0000 7fe749ffb640 1 -- 172.21.15.47:0/4001770385 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe75001b070 con 0x7fe7540aff60 2024-04-26T23:18:56.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.669+0000 7f0a3d3af640 1 -- 172.21.15.47:0/2700425415 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a380b17f0 con 0x7f0a380b6130 2024-04-26T23:18:56.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.669+0000 7f60217fa640 1 --2- 172.21.15.47:0/3356465508 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f601006c9c0 0x7f601006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.669+0000 7fe749ffb640 1 --2- 172.21.15.47:0/4001770385 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fe73806c940 0x7fe73806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.669+0000 7f60217fa640 1 -- 172.21.15.47:0/3356465508 <== mon.2 v2:172.21.15.203:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f601c0b0200 con 0x7f602406f600 2024-04-26T23:18:56.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.669+0000 7fe749ffb640 1 -- 172.21.15.47:0/4001770385 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fe75008b3b0 con 0x7fe7540aff60 2024-04-26T23:18:56.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.669+0000 7fc349f4c640 1 --2- 172.21.15.47:0/2334268724 >> [v2:172.21.15.203:6816/1315565058,v1:172.21.15.203:6817/1315565058] conn(0x7fc3280760c0 0x7fc328078500 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:56.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.669+0000 7f602ac28640 1 --2- 172.21.15.47:0/3356465508 >> [v2:172.21.15.47:6808/1912610931,v1:172.21.15.47:6809/1912610931] conn(0x7f6004001650 0x7f6004003b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.669+0000 7fe749ffb640 1 --2- 172.21.15.47:0/4001770385 >> [v2:172.21.15.47:6817/326520779,v1:172.21.15.47:6819/326520779] conn(0x7fe7380760b0 0x7fe7380784f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.669+0000 7fe7593e3640 1 --2- 172.21.15.47:0/4001770385 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fe73806c940 0x7fe73806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.669+0000 7f6023fff640 1 --2- 172.21.15.47:0/3356465508 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f601006c9c0 0x7f601006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.670+0000 7f602ac28640 1 -- 172.21.15.47:0/3356465508 --> [v2:172.21.15.47:6808/1912610931,v1:172.21.15.47:6809/1912610931] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6004006bb0 con 0x7f6004001650 2024-04-26T23:18:56.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.670+0000 7fe749ffb640 1 -- 172.21.15.47:0/4001770385 --> [v2:172.21.15.47:6817/326520779,v1:172.21.15.47:6819/326520779] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe738078bc0 con 0x7fe7380760b0 2024-04-26T23:18:56.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.670+0000 7fe749ffb640 1 -- 172.21.15.47:0/4001770385 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe750092050 con 0x7fe7540aff60 2024-04-26T23:18:56.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.670+0000 7f73e2ffd640 1 -- 172.21.15.47:0/1378842018 <== osd.2 v2:172.21.15.47:6800/2569869305 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f73e4008eb0 con 0x7f73d4076130 2024-04-26T23:18:56.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.670+0000 7fe759be4640 1 --2- 172.21.15.47:0/4001770385 >> [v2:172.21.15.47:6817/326520779,v1:172.21.15.47:6819/326520779] conn(0x7fe7380760b0 0x7fe7380784f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.670+0000 7f7aa0c09640 1 --2- 172.21.15.47:0/491440119 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f7a8806c9f0 0x7f7a8806eeb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7a90034000 tx=0x7f7a9002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:56.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.670+0000 7f0a3d3af640 1 -- 172.21.15.47:0/2700425415 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0a38069a80 con 0x7f0a380b6130 2024-04-26T23:18:56.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.670+0000 7f60291c5640 1 --2- 172.21.15.47:0/3356465508 >> [v2:172.21.15.47:6808/1912610931,v1:172.21.15.47:6809/1912610931] conn(0x7f6004001650 0x7f6004003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.670+0000 7fe759be4640 1 --2- 172.21.15.47:0/4001770385 >> [v2:172.21.15.47:6817/326520779,v1:172.21.15.47:6819/326520779] conn(0x7fe7380760b0 0x7fe7380784f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:56.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.670+0000 7f0a1f7fe640 1 -- 172.21.15.47:0/2700425415 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0a2c01a020 con 0x7f0a380b6130 2024-04-26T23:18:56.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.671+0000 7f60291c5640 1 --2- 172.21.15.47:0/3356465508 >> [v2:172.21.15.47:6808/1912610931,v1:172.21.15.47:6809/1912610931] conn(0x7f6004001650 0x7f6004003b10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:56.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.671+0000 7f2c42ffd640 1 --2- 172.21.15.47:0/1920633277 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f2c3406c9c0 0x7f2c3406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.671+0000 7fc339ffb640 1 -- 172.21.15.47:0/2334268724 <== osd.7 v2:172.21.15.203:6816/1315565058 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc334033e50 con 0x7fc3280760c0 2024-04-26T23:18:56.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.671+0000 7f2c42ffd640 1 -- 172.21.15.47:0/1920633277 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f2c4408b2b0 con 0x7f2c54071d40 2024-04-26T23:18:56.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.671+0000 7f0a1f7fe640 1 --2- 172.21.15.47:0/2700425415 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f0a2406c9c0 0x7f0a2406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.671+0000 7f2c42ffd640 1 --2- 172.21.15.47:0/1920633277 >> [v2:172.21.15.47:6816/2791411683,v1:172.21.15.47:6818/2791411683] conn(0x7f2c34076140 0x7f2c34078580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.671+0000 7f2c51d9b640 1 --2- 172.21.15.47:0/1920633277 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f2c3406c9c0 0x7f2c3406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.671+0000 7f2c42ffd640 1 -- 172.21.15.47:0/1920633277 --> [v2:172.21.15.47:6816/2791411683,v1:172.21.15.47:6818/2791411683] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2c4400ee50 con 0x7f2c34076140 2024-04-26T23:18:56.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.671+0000 7f2c42ffd640 1 -- 172.21.15.47:0/1920633277 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2c44092050 con 0x7f2c54071d40 2024-04-26T23:18:56.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.671+0000 7f6023fff640 1 --2- 172.21.15.47:0/3356465508 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f601006c9c0 0x7f601006ee80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f60240c4650 tx=0x7f60180016d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:56.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.671+0000 7f0a1f7fe640 1 -- 172.21.15.47:0/2700425415 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f0a2c08b5e0 con 0x7f0a380b6130 2024-04-26T23:18:56.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.671+0000 7f73f1d9b640 1 --2- 172.21.15.47:0/1378842018 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f73d406c9c0 0x7f73d406ee80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f73f4158a10 tx=0x7f73e8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:56.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.671+0000 7f0a36ffd640 1 --2- 172.21.15.47:0/2700425415 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f0a2406c9c0 0x7f0a2406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.671+0000 7fe7593e3640 1 --2- 172.21.15.47:0/4001770385 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fe73806c940 0x7fe73806ee00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe74c00ace0 tx=0x7fe74c00a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:56.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.672+0000 7f2c52d9d640 1 --2- 172.21.15.47:0/1920633277 >> [v2:172.21.15.47:6816/2791411683,v1:172.21.15.47:6818/2791411683] conn(0x7f2c34076140 0x7f2c34078580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.672+0000 7f0a1f7fe640 1 --2- 172.21.15.47:0/2700425415 >> [v2:172.21.15.203:6808/441313104,v1:172.21.15.203:6809/441313104] conn(0x7f0a24076130 0x7f0a24078570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.672+0000 7f0a1f7fe640 1 -- 172.21.15.47:0/2700425415 --> [v2:172.21.15.203:6808/441313104,v1:172.21.15.203:6809/441313104] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0a24078c40 con 0x7f0a24076130 2024-04-26T23:18:56.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.672+0000 7f0a1f7fe640 1 -- 172.21.15.47:0/2700425415 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0a2c092050 con 0x7f0a380b6130 2024-04-26T23:18:56.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.672+0000 7f0a377fe640 1 --2- 172.21.15.47:0/2700425415 >> [v2:172.21.15.203:6808/441313104,v1:172.21.15.203:6809/441313104] conn(0x7f0a24076130 0x7f0a24078570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.672+0000 7f2c52d9d640 1 --2- 172.21.15.47:0/1920633277 >> [v2:172.21.15.47:6816/2791411683,v1:172.21.15.47:6818/2791411683] conn(0x7f2c34076140 0x7f2c34078580 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:56.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.672+0000 7fe749ffb640 1 -- 172.21.15.47:0/4001770385 <== osd.0 v2:172.21.15.47:6817/326520779 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe738078bc0 con 0x7fe7380760b0 2024-04-26T23:18:56.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.672+0000 7f60217fa640 1 -- 172.21.15.47:0/3356465508 <== osd.1 v2:172.21.15.47:6808/1912610931 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6004006bb0 con 0x7f6004001650 2024-04-26T23:18:56.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.672+0000 7f0a377fe640 1 --2- 172.21.15.47:0/2700425415 >> [v2:172.21.15.203:6808/441313104,v1:172.21.15.203:6809/441313104] conn(0x7f0a24076130 0x7f0a24078570 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:56.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.672+0000 7f0a36ffd640 1 --2- 172.21.15.47:0/2700425415 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f0a2406c9c0 0x7f0a2406ee80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0a28009560 tx=0x7f0a2802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:56.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.673+0000 7f0a1f7fe640 1 -- 172.21.15.47:0/2700425415 <== osd.4 v2:172.21.15.203:6808/441313104 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0a24078c40 con 0x7f0a24076130 2024-04-26T23:18:56.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.672+0000 7f2c51d9b640 1 --2- 172.21.15.47:0/1920633277 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f2c3406c9c0 0x7f2c3406ee80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c009d90 tx=0x7f2c4c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:56.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.673+0000 7f2c42ffd640 1 -- 172.21.15.47:0/1920633277 <== osd.3 v2:172.21.15.47:6816/2791411683 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2c4400ee50 con 0x7f2c34076140 2024-04-26T23:18:56.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.700+0000 7f7edf69c640 1 -- 172.21.15.47:0/225730950 --> [v2:172.21.15.203:6824/2738517080,v1:172.21.15.203:6825/2738517080] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7eb8002d50 con 0x7f7ebc075ff0 2024-04-26T23:18:56.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.701+0000 7f7aa2e6d640 1 -- 172.21.15.47:0/491440119 --> [v2:172.21.15.203:6800/56838444,v1:172.21.15.203:6801/56838444] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7a9c073fe0 con 0x7f7a880760f0 2024-04-26T23:18:56.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.701+0000 7f7ecdffb640 1 -- 172.21.15.47:0/225730950 <== osd.6 v2:172.21.15.203:6824/2738517080 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7eb8002d50 con 0x7f7ebc075ff0 2024-04-26T23:18:56.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.702+0000 7f7a997fa640 1 -- 172.21.15.47:0/491440119 <== osd.5 v2:172.21.15.203:6800/56838444 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7a9c073fe0 con 0x7f7a880760f0 2024-04-26T23:18:56.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.702+0000 7f7edf69c640 1 -- 172.21.15.47:0/225730950 >> [v2:172.21.15.203:6824/2738517080,v1:172.21.15.203:6825/2738517080] conn(0x7f7ebc075ff0 msgr2=0x7f7ebc078430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.702+0000 7f7edf69c640 1 --2- 172.21.15.47:0/225730950 >> [v2:172.21.15.203:6824/2738517080,v1:172.21.15.203:6825/2738517080] conn(0x7f7ebc075ff0 0x7f7ebc078430 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.702+0000 7f7edf69c640 1 -- 172.21.15.47:0/225730950 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f7ebc06c8f0 msgr2=0x7f7ebc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.702+0000 7f7edf69c640 1 --2- 172.21.15.47:0/225730950 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f7ebc06c8f0 0x7f7ebc06edb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4007a60 tx=0x7f7ed4008040 comp rx=0 tx=0).stop 2024-04-26T23:18:56.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.702+0000 7f7edf69c640 1 -- 172.21.15.47:0/225730950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ed00543f0 msgr2=0x7f7ed0039540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.702+0000 7f7edf69c640 1 --2- 172.21.15.47:0/225730950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ed00543f0 0x7f7ed0039540 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7ec800c440 tx=0x7f7ec800c900 comp rx=0 tx=0).stop 2024-04-26T23:18:56.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.702+0000 7f7eddc39640 1 -- 172.21.15.47:0/225730950 reap_dead start 2024-04-26T23:18:56.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.702+0000 7f7edf69c640 1 -- 172.21.15.47:0/225730950 shutdown_connections 2024-04-26T23:18:56.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.702+0000 7f7edf69c640 1 -- 172.21.15.47:0/225730950 >> 172.21.15.47:0/225730950 conn(0x7f7ed003a9c0 msgr2=0x7f7ed0038a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:56.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.702+0000 7f7edf69c640 1 -- 172.21.15.47:0/225730950 shutdown_connections 2024-04-26T23:18:56.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.702+0000 7f7edf69c640 1 -- 172.21.15.47:0/225730950 wait complete. 2024-04-26T23:18:56.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.703+0000 7f7aa2e6d640 1 -- 172.21.15.47:0/491440119 >> [v2:172.21.15.203:6800/56838444,v1:172.21.15.203:6801/56838444] conn(0x7f7a880760f0 msgr2=0x7f7a88078530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.703+0000 7f7aa2e6d640 1 --2- 172.21.15.47:0/491440119 >> [v2:172.21.15.203:6800/56838444,v1:172.21.15.203:6801/56838444] conn(0x7f7a880760f0 0x7f7a88078530 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.703+0000 7f7aa2e6d640 1 -- 172.21.15.47:0/491440119 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f7a8806c9f0 msgr2=0x7f7a8806eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.703+0000 7f7aa2e6d640 1 --2- 172.21.15.47:0/491440119 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f7a8806c9f0 0x7f7a8806eeb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7a90034000 tx=0x7f7a9002e000 comp rx=0 tx=0).stop 2024-04-26T23:18:56.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.703+0000 7f7aa2e6d640 1 -- 172.21.15.47:0/491440119 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a9c0cd680 msgr2=0x7f7a9c0700f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.703+0000 7f7aa2e6d640 1 --2- 172.21.15.47:0/491440119 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a9c0cd680 0x7f7a9c0700f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c00ec10 tx=0x7f7a8c00c520 comp rx=0 tx=0).stop 2024-04-26T23:18:56.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.703+0000 7f7aa140a640 1 -- 172.21.15.47:0/491440119 reap_dead start 2024-04-26T23:18:56.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.703+0000 7f7aa2e6d640 1 -- 172.21.15.47:0/491440119 shutdown_connections 2024-04-26T23:18:56.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.703+0000 7f7aa2e6d640 1 -- 172.21.15.47:0/491440119 >> 172.21.15.47:0/491440119 conn(0x7f7a9c0d1af0 msgr2=0x7f7a9c0c7590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:56.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.703+0000 7f73f8afc640 1 -- 172.21.15.47:0/1378842018 --> [v2:172.21.15.47:6800/2569869305,v1:172.21.15.47:6801/2569869305] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f73d8002dd0 con 0x7f73d4076130 2024-04-26T23:18:56.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.704+0000 7f73e2ffd640 1 -- 172.21.15.47:0/1378842018 <== osd.2 v2:172.21.15.47:6800/2569869305 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f73d8002dd0 con 0x7f73d4076130 2024-04-26T23:18:56.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.704+0000 7f7aa2e6d640 1 -- 172.21.15.47:0/491440119 shutdown_connections 2024-04-26T23:18:56.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.704+0000 7f7aa2e6d640 1 -- 172.21.15.47:0/491440119 wait complete. 2024-04-26T23:18:56.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.705+0000 7fe75b647640 1 -- 172.21.15.47:0/4001770385 --> [v2:172.21.15.47:6817/326520779,v1:172.21.15.47:6819/326520779] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe7540b1360 con 0x7fe7380760b0 2024-04-26T23:18:56.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.705+0000 7f73f8afc640 1 -- 172.21.15.47:0/1378842018 >> [v2:172.21.15.47:6800/2569869305,v1:172.21.15.47:6801/2569869305] conn(0x7f73d4076130 msgr2=0x7f73d4078570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.705+0000 7fc34b9af640 1 -- 172.21.15.47:0/2334268724 --> [v2:172.21.15.203:6816/1315565058,v1:172.21.15.203:6817/1315565058] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc344069a80 con 0x7fc3280760c0 2024-04-26T23:18:56.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.705+0000 7f73f8afc640 1 --2- 172.21.15.47:0/1378842018 >> [v2:172.21.15.47:6800/2569869305,v1:172.21.15.47:6801/2569869305] conn(0x7f73d4076130 0x7f73d4078570 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.705+0000 7fe749ffb640 1 -- 172.21.15.47:0/4001770385 <== osd.0 v2:172.21.15.47:6817/326520779 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe7540b1360 con 0x7fe7380760b0 2024-04-26T23:18:56.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.705+0000 7f73f8afc640 1 -- 172.21.15.47:0/1378842018 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f73d406c9c0 msgr2=0x7f73d406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.705+0000 7f73f8afc640 1 --2- 172.21.15.47:0/1378842018 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f73d406c9c0 0x7f73d406ee80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f73f4158a10 tx=0x7f73e8008040 comp rx=0 tx=0).stop 2024-04-26T23:18:56.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.705+0000 7f73f8afc640 1 -- 172.21.15.47:0/1378842018 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73f406ddf0 msgr2=0x7f73f41582a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.705+0000 7f73f8afc640 1 --2- 172.21.15.47:0/1378842018 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73f406ddf0 0x7f73f41582a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f73e4009890 tx=0x7f73e40046c0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.705+0000 7f73f2d9d640 1 -- 172.21.15.47:0/1378842018 reap_dead start 2024-04-26T23:18:56.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.705+0000 7f73f8afc640 1 -- 172.21.15.47:0/1378842018 shutdown_connections 2024-04-26T23:18:56.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.705+0000 7f73f8afc640 1 -- 172.21.15.47:0/1378842018 >> 172.21.15.47:0/1378842018 conn(0x7f73f40d0be0 msgr2=0x7f73f40c4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:56.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.706+0000 7fc339ffb640 1 -- 172.21.15.47:0/2334268724 <== osd.7 v2:172.21.15.203:6816/1315565058 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc344069a80 con 0x7fc3280760c0 2024-04-26T23:18:56.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.706+0000 7fe75b647640 1 -- 172.21.15.47:0/4001770385 >> [v2:172.21.15.47:6817/326520779,v1:172.21.15.47:6819/326520779] conn(0x7fe7380760b0 msgr2=0x7fe7380784f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.706+0000 7fe75b647640 1 --2- 172.21.15.47:0/4001770385 >> [v2:172.21.15.47:6817/326520779,v1:172.21.15.47:6819/326520779] conn(0x7fe7380760b0 0x7fe7380784f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.706+0000 7f73f8afc640 1 -- 172.21.15.47:0/1378842018 shutdown_connections 2024-04-26T23:18:56.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.706+0000 7f73f8afc640 1 -- 172.21.15.47:0/1378842018 wait complete. 2024-04-26T23:18:56.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.706+0000 7fe75b647640 1 -- 172.21.15.47:0/4001770385 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fe73806c940 msgr2=0x7fe73806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.706+0000 7fc34b9af640 1 -- 172.21.15.47:0/2334268724 >> [v2:172.21.15.203:6816/1315565058,v1:172.21.15.203:6817/1315565058] conn(0x7fc3280760c0 msgr2=0x7fc328078500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.706+0000 7fc34b9af640 1 --2- 172.21.15.47:0/2334268724 >> [v2:172.21.15.203:6816/1315565058,v1:172.21.15.203:6817/1315565058] conn(0x7fc3280760c0 0x7fc328078500 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.706+0000 7fe75b647640 1 --2- 172.21.15.47:0/4001770385 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fe73806c940 0x7fe73806ee00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe74c00ace0 tx=0x7fe74c00a000 comp rx=0 tx=0).stop 2024-04-26T23:18:56.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.706+0000 7f2c5359e640 1 -- 172.21.15.47:0/1920633277 --> [v2:172.21.15.47:6816/2791411683,v1:172.21.15.47:6818/2791411683] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2c38002dd0 con 0x7f2c34076140 2024-04-26T23:18:56.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.706+0000 7fe75b647640 1 -- 172.21.15.47:0/4001770385 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7540aff60 msgr2=0x7fe7540c3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.706+0000 7fe75b647640 1 --2- 172.21.15.47:0/4001770385 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7540aff60 0x7fe7540c3c80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe75000c7b0 tx=0x7fe75000cc70 comp rx=0 tx=0).stop 2024-04-26T23:18:56.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.706+0000 7fe759be4640 1 -- 172.21.15.47:0/4001770385 reap_dead start 2024-04-26T23:18:56.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.707+0000 7fe75b647640 1 -- 172.21.15.47:0/4001770385 shutdown_connections 2024-04-26T23:18:56.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.707+0000 7fe75b647640 1 -- 172.21.15.47:0/4001770385 >> 172.21.15.47:0/4001770385 conn(0x7fe7540d0be0 msgr2=0x7fe7540ce6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:56.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.707+0000 7f2c42ffd640 1 -- 172.21.15.47:0/1920633277 <== osd.3 v2:172.21.15.47:6816/2791411683 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2c38002dd0 con 0x7f2c34076140 2024-04-26T23:18:56.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.707+0000 7fc34b9af640 1 -- 172.21.15.47:0/2334268724 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fc32806c9c0 msgr2=0x7fc32806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.707+0000 7fc34b9af640 1 --2- 172.21.15.47:0/2334268724 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fc32806c9c0 0x7fc32806ee80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc340007c90 tx=0x7fc340007be0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.707+0000 7fe75b647640 1 -- 172.21.15.47:0/4001770385 shutdown_connections 2024-04-26T23:18:56.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.707+0000 7fc34b9af640 1 -- 172.21.15.47:0/2334268724 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3440b8720 msgr2=0x7fc3441635d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.707+0000 7fc34b9af640 1 --2- 172.21.15.47:0/2334268724 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3440b8720 0x7fc3441635d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc33402a670 tx=0x7fc3340046c0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.707+0000 7fc349f4c640 1 -- 172.21.15.47:0/2334268724 reap_dead start 2024-04-26T23:18:56.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.707+0000 7fe75b647640 1 -- 172.21.15.47:0/4001770385 wait complete. 2024-04-26T23:18:56.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.707+0000 7fc34b9af640 1 -- 172.21.15.47:0/2334268724 shutdown_connections 2024-04-26T23:18:56.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.707+0000 7fc34b9af640 1 -- 172.21.15.47:0/2334268724 >> 172.21.15.47:0/2334268724 conn(0x7fc3440d18f0 msgr2=0x7fc3440b3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:56.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.707+0000 7fc34b9af640 1 -- 172.21.15.47:0/2334268724 shutdown_connections 2024-04-26T23:18:56.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.707+0000 7fc34b9af640 1 -- 172.21.15.47:0/2334268724 wait complete. 2024-04-26T23:18:56.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.707+0000 7f602ac28640 1 -- 172.21.15.47:0/3356465508 --> [v2:172.21.15.47:6808/1912610931,v1:172.21.15.47:6809/1912610931] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6004005c90 con 0x7f6004001650 2024-04-26T23:18:56.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.708+0000 7f60217fa640 1 -- 172.21.15.47:0/3356465508 <== osd.1 v2:172.21.15.47:6808/1912610931 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6004005c90 con 0x7f6004001650 2024-04-26T23:18:56.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.708+0000 7f2c5359e640 1 -- 172.21.15.47:0/1920633277 >> [v2:172.21.15.47:6816/2791411683,v1:172.21.15.47:6818/2791411683] conn(0x7f2c34076140 msgr2=0x7f2c34078580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.708+0000 7f2c5359e640 1 --2- 172.21.15.47:0/1920633277 >> [v2:172.21.15.47:6816/2791411683,v1:172.21.15.47:6818/2791411683] conn(0x7f2c34076140 0x7f2c34078580 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.708+0000 7f0a3d3af640 1 -- 172.21.15.47:0/2700425415 --> [v2:172.21.15.203:6808/441313104,v1:172.21.15.203:6809/441313104] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0a38167830 con 0x7f0a24076130 2024-04-26T23:18:56.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.708+0000 7f2c5359e640 1 -- 172.21.15.47:0/1920633277 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f2c3406c9c0 msgr2=0x7f2c3406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.708+0000 7f2c5359e640 1 --2- 172.21.15.47:0/1920633277 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f2c3406c9c0 0x7f2c3406ee80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c009d90 tx=0x7f2c4c02e000 comp rx=0 tx=0).stop 2024-04-26T23:18:56.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.708+0000 7f2c5359e640 1 -- 172.21.15.47:0/1920633277 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c54071d40 msgr2=0x7f2c541552d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.708+0000 7f2c5359e640 1 --2- 172.21.15.47:0/1920633277 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c54071d40 0x7f2c541552d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2c4400c780 tx=0x7f2c4400cc40 comp rx=0 tx=0).stop 2024-04-26T23:18:56.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.708+0000 7f2c52d9d640 1 -- 172.21.15.47:0/1920633277 reap_dead start 2024-04-26T23:18:56.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.709+0000 7f0a1f7fe640 1 -- 172.21.15.47:0/2700425415 <== osd.4 v2:172.21.15.203:6808/441313104 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0a38167830 con 0x7f0a24076130 2024-04-26T23:18:56.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.709+0000 7f2c5359e640 1 -- 172.21.15.47:0/1920633277 shutdown_connections 2024-04-26T23:18:56.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.709+0000 7f2c5359e640 1 -- 172.21.15.47:0/1920633277 >> 172.21.15.47:0/1920633277 conn(0x7f2c540d2c90 msgr2=0x7f2c54159890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:56.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.709+0000 7f602ac28640 1 -- 172.21.15.47:0/3356465508 >> [v2:172.21.15.47:6808/1912610931,v1:172.21.15.47:6809/1912610931] conn(0x7f6004001650 msgr2=0x7f6004003b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.709+0000 7f602ac28640 1 --2- 172.21.15.47:0/3356465508 >> [v2:172.21.15.47:6808/1912610931,v1:172.21.15.47:6809/1912610931] conn(0x7f6004001650 0x7f6004003b10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.709+0000 7f2c5359e640 1 -- 172.21.15.47:0/1920633277 shutdown_connections 2024-04-26T23:18:56.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.709+0000 7f2c5359e640 1 -- 172.21.15.47:0/1920633277 wait complete. 2024-04-26T23:18:56.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.709+0000 7f602ac28640 1 -- 172.21.15.47:0/3356465508 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f601006c9c0 msgr2=0x7f601006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.709+0000 7f602ac28640 1 --2- 172.21.15.47:0/3356465508 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f601006c9c0 0x7f601006ee80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f60240c4650 tx=0x7f60180016d0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.709+0000 7f602ac28640 1 -- 172.21.15.47:0/3356465508 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f602406f600 msgr2=0x7f60240c9be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.709+0000 7f602ac28640 1 --2- 172.21.15.47:0/3356465508 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f602406f600 0x7f60240c9be0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f601c02abc0 tx=0x7f601c004a40 comp rx=0 tx=0).stop 2024-04-26T23:18:56.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.709+0000 7f60291c5640 1 -- 172.21.15.47:0/3356465508 reap_dead start 2024-04-26T23:18:56.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.710+0000 7f602ac28640 1 -- 172.21.15.47:0/3356465508 shutdown_connections 2024-04-26T23:18:56.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.710+0000 7f602ac28640 1 -- 172.21.15.47:0/3356465508 >> 172.21.15.47:0/3356465508 conn(0x7f60240d0be0 msgr2=0x7f60240c6570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:56.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.710+0000 7f602ac28640 1 -- 172.21.15.47:0/3356465508 shutdown_connections 2024-04-26T23:18:56.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.710+0000 7f602ac28640 1 -- 172.21.15.47:0/3356465508 wait complete. 2024-04-26T23:18:56.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.710+0000 7f0a3d3af640 1 -- 172.21.15.47:0/2700425415 >> [v2:172.21.15.203:6808/441313104,v1:172.21.15.203:6809/441313104] conn(0x7f0a24076130 msgr2=0x7f0a24078570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.710+0000 7f0a3d3af640 1 --2- 172.21.15.47:0/2700425415 >> [v2:172.21.15.203:6808/441313104,v1:172.21.15.203:6809/441313104] conn(0x7f0a24076130 0x7f0a24078570 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:56.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.710+0000 7f0a3d3af640 1 -- 172.21.15.47:0/2700425415 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f0a2406c9c0 msgr2=0x7f0a2406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.710+0000 7f0a3d3af640 1 --2- 172.21.15.47:0/2700425415 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f0a2406c9c0 0x7f0a2406ee80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0a28009560 tx=0x7f0a2802e000 comp rx=0 tx=0).stop 2024-04-26T23:18:56.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.710+0000 7f0a3d3af640 1 -- 172.21.15.47:0/2700425415 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a380b6130 msgr2=0x7f0a38153bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.710+0000 7f0a3d3af640 1 --2- 172.21.15.47:0/2700425415 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a380b6130 0x7f0a38153bd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0a380d1a30 tx=0x7f0a2c00c870 comp rx=0 tx=0).stop 2024-04-26T23:18:56.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.711+0000 7f0a377fe640 1 -- 172.21.15.47:0/2700425415 reap_dead start 2024-04-26T23:18:56.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.711+0000 7f0a3d3af640 1 -- 172.21.15.47:0/2700425415 shutdown_connections 2024-04-26T23:18:56.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.711+0000 7f0a3d3af640 1 -- 172.21.15.47:0/2700425415 >> 172.21.15.47:0/2700425415 conn(0x7f0a38068910 msgr2=0x7f0a381654e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:56.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.711+0000 7f0a3d3af640 1 -- 172.21.15.47:0/2700425415 shutdown_connections 2024-04-26T23:18:56.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.711+0000 7f0a3d3af640 1 -- 172.21.15.47:0/2700425415 wait complete. 2024-04-26T23:18:56.774 INFO:teuthology.orchestra.run.smithi047.stdout:60129542156 2024-04-26T23:18:56.774 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-26T23:18:56.775 INFO:teuthology.orchestra.run.smithi047.stdout:60129542155 2024-04-26T23:18:56.775 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-26T23:18:56.776 INFO:teuthology.orchestra.run.smithi047.stdout:55834574858 2024-04-26T23:18:56.776 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-26T23:18:56.777 INFO:teuthology.orchestra.run.smithi047.stdout:55834574858 2024-04-26T23:18:56.777 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-26T23:18:56.778 INFO:teuthology.orchestra.run.smithi047.stdout:60129542154 2024-04-26T23:18:56.778 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-26T23:18:56.779 INFO:teuthology.orchestra.run.smithi047.stdout:55834574858 2024-04-26T23:18:56.779 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-26T23:18:56.780 INFO:teuthology.orchestra.run.smithi047.stdout:55834574857 2024-04-26T23:18:56.780 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-26T23:18:56.781 INFO:teuthology.orchestra.run.smithi047.stdout:60129542155 2024-04-26T23:18:56.782 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-26T23:18:56.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.989+0000 7f96a1166640 1 Processor -- start 2024-04-26T23:18:56.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.989+0000 7f96a1166640 1 -- start start 2024-04-26T23:18:56.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.990+0000 7f3628af9640 1 Processor -- start 2024-04-26T23:18:56.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.990+0000 7f6d59b9d640 1 Processor -- start 2024-04-26T23:18:56.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.990+0000 7f96a1166640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f969c070930 0x7f969c160ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.990+0000 7f96a1166640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f969c0b0990 con 0x7f969c15d100 2024-04-26T23:18:56.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.990+0000 7f96a1166640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f969c0b0c70 con 0x7f969c06c950 2024-04-26T23:18:56.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.990+0000 7f96a1166640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f969c0b0f50 con 0x7f969c070930 2024-04-26T23:18:56.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.990+0000 7f969b59e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f969c070930 0x7f969c160ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:56.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.990+0000 7f969ad9d640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f969c15d100 0x7f969c06c240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:54592/0 (socket says 172.21.15.47:54592) 2024-04-26T23:18:56.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.990+0000 7f969ad9d640 1 -- 172.21.15.47:0/3594122014 learned_addr learned my addr 172.21.15.47:0/3594122014 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:18:56.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.990+0000 7f3628af9640 1 -- start start 2024-04-26T23:18:56.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.991+0000 7f6d59b9d640 1 -- start start 2024-04-26T23:18:56.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.991+0000 7f969b59e640 1 -- 172.21.15.47:0/3594122014 >> v1:172.21.15.203:6789/0 conn(0x7f969c06c950 legacy=0x7f969c073160 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T23:18:56.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.991+0000 7f969b59e640 1 -- 172.21.15.47:0/3594122014 >> v1:172.21.15.47:6789/0 conn(0x7f969c15d100 legacy=0x7f969c06c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:56.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.991+0000 7f969b59e640 1 -- 172.21.15.47:0/3594122014 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f969c0b1230 con 0x7f969c070930 2024-04-26T23:18:56.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.991+0000 7f329e63e640 1 Processor -- start 2024-04-26T23:18:56.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.991+0000 7ff78b59e640 1 Processor -- start 2024-04-26T23:18:56.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.991+0000 7ff78b59e640 1 -- start start 2024-04-26T23:18:56.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.992+0000 7f3628af9640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f36240c4620 0x7f362415f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.992+0000 7f3628af9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36240cf180 0x7f36240cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.992+0000 7f6d59b9d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f6d54072fb0 0x7f6d540b56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.992+0000 7f6d59b9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6d54073390 0x7f6d5406f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:56.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.992+0000 7f6d59b9d640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d540b5e30 con 0x7f6d5406fc80 2024-04-26T23:18:56.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.992+0000 7f6d59b9d640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f6d540b5c00 con 0x7f6d54073390 2024-04-26T23:18:56.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.992+0000 7f6d59b9d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f6d5406f2b0 con 0x7f6d54072fb0 2024-04-26T23:18:56.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.992+0000 7f558b59e640 1 Processor -- start 2024-04-26T23:18:56.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.992+0000 7f558b59e640 1 -- start start 2024-04-26T23:18:56.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.993+0000 7f362259c640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f3624070a30 0x7f362415ed90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.47:54114/0 (socket says 172.21.15.47:54114) 2024-04-26T23:18:56.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.993+0000 7f362259c640 1 -- 172.21.15.47:0/1331015590 learned_addr learned my addr 172.21.15.47:0/1331015590 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:18:57.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.993+0000 7fe77c359640 1 Processor -- start 2024-04-26T23:18:57.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.993+0000 7f3628af9640 1 -- 172.21.15.47:0/1331015590 --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36240ccbf0 con 0x7f3624070a30 2024-04-26T23:18:57.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.993+0000 7f3628af9640 1 -- 172.21.15.47:0/1331015590 --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f36240cd820 con 0x7f36240c4620 2024-04-26T23:18:57.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.993+0000 7fe77c359640 1 -- start start 2024-04-26T23:18:57.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.993+0000 7f3628af9640 1 -- 172.21.15.47:0/1331015590 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3624072480 con 0x7f36240cf180 2024-04-26T23:18:57.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.993+0000 7f6d52ffd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6d54073390 0x7f6d5406f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:57.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.993+0000 7f6d537fe640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f6d54072fb0 0x7f6d540b56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:57.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.993+0000 7f6d537fe640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f6d54072fb0 0x7f6d540b56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.47:58212/0 (socket says 172.21.15.47:58212) 2024-04-26T23:18:57.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.993+0000 7f6d537fe640 1 -- 172.21.15.47:0/3568697616 learned_addr learned my addr 172.21.15.47:0/3568697616 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:57.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.993+0000 7f6d52ffd640 1 --2- 172.21.15.47:0/3568697616 >> v2:172.21.15.47:3300/0 conn(0x7f6d54073390 0x7f6d5406f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58726/0 (socket says 172.21.15.47:58726) 2024-04-26T23:18:57.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.993+0000 7f3621d9b640 1 -- 172.21.15.47:0/1331015590 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2123216461 0 0) 0x7f360c001110 con 0x7f3624070a30 2024-04-26T23:18:57.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.993+0000 7f969b59e640 1 --2- 172.21.15.47:0/3594122014 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f969c070930 0x7f969c160ba0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9694009d90 tx=0x7f969402a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=884bf9b9c6c27378 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:57.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.993+0000 7f3621d9b640 1 -- 172.21.15.47:0/1331015590 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3610003520 con 0x7f3624070a30 2024-04-26T23:18:57.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.993+0000 7f3622d9d640 1 --2- 172.21.15.47:0/1331015590 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36240cf180 0x7f36240cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:57.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.992+0000 7f329e63e640 1 -- start start 2024-04-26T23:18:57.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.993+0000 7f329e63e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f32980d1d60 0x7f32980c8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:57.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.993+0000 7f329e63e640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32980d5a90 con 0x7f32980d1980 2024-04-26T23:18:57.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.993+0000 7f329e63e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32980d0690 con 0x7f32980d3570 2024-04-26T23:18:57.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.993+0000 7f329e63e640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f32980d5d70 con 0x7f32980d1d60 2024-04-26T23:18:57.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.993+0000 7f329d63c640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f32980d1980 0x7f32980d2e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:54594/0 (socket says 172.21.15.47:54594) 2024-04-26T23:18:57.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.993+0000 7f329d63c640 1 -- 172.21.15.47:0/862679811 learned_addr learned my addr 172.21.15.47:0/862679811 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:18:57.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.993+0000 7f9699d9b640 1 -- 172.21.15.47:0/3594122014 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9694007740 con 0x7f969c070930 2024-04-26T23:18:57.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.994+0000 7f9699d9b640 1 -- 172.21.15.47:0/3594122014 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96940078e0 con 0x7f969c070930 2024-04-26T23:18:57.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.994+0000 7f3621d9b640 1 -- 172.21.15.47:0/1331015590 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1286879382 0 0) 0x7f3610003520 con 0x7f3624070a30 2024-04-26T23:18:57.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.994+0000 7f3621d9b640 1 -- 172.21.15.47:0/1331015590 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36240cf180 msgr2=0x7f36240cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:57.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.994+0000 7f3621d9b640 1 --2- 172.21.15.47:0/1331015590 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36240cf180 0x7f36240cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.994+0000 7fc3171f2640 1 Processor -- start 2024-04-26T23:18:57.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.994+0000 7f3621d9b640 1 -- 172.21.15.47:0/1331015590 >> v2:172.21.15.203:3300/0 conn(0x7f36240c4620 msgr2=0x7f362415f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:57.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.994+0000 7f361bfff640 1 --2- 172.21.15.47:0/1331015590 >> v2:172.21.15.203:3300/0 conn(0x7f36240c4620 0x7f362415f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:57.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.994+0000 7f558b59e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f55840501a0 0x7f5584038080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:57.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.994+0000 7f3621d9b640 1 --2- 172.21.15.47:0/1331015590 >> v2:172.21.15.203:3300/0 conn(0x7f36240c4620 0x7f362415f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.994+0000 7f558b59e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5584053c70 0x7f5584006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:57.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.994+0000 7fc3171f2640 1 -- start start 2024-04-26T23:18:57.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.994+0000 7f3621d9b640 1 -- 172.21.15.47:0/1331015590 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36240cd540 con 0x7f3624070a30 2024-04-26T23:18:57.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.994+0000 7f558b59e640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55840544c0 con 0x7f55840df220 2024-04-26T23:18:57.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.994+0000 7f558b59e640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f55840541e0 con 0x7f55840501a0 2024-04-26T23:18:57.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.993+0000 7ff78b59e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff784050ab0 0x7ff7840f2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:57.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.993+0000 7ff78b59e640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff784034be0 con 0x7ff7840e76e0 2024-04-26T23:18:57.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.993+0000 7ff78b59e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff784034900 con 0x7ff784034400 2024-04-26T23:18:57.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.994+0000 7f558b59e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f5584005f30 con 0x7f5584053c70 2024-04-26T23:18:57.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.993+0000 7ff78b59e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7ff784054040 con 0x7ff784050ab0 2024-04-26T23:18:57.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.994+0000 7f3621d9b640 1 -- 172.21.15.47:0/1331015590 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7f360c001840 con 0x7f3624070a30 2024-04-26T23:18:57.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.994+0000 7f329ce3b640 1 --2- 172.21.15.47:0/862679811 >> v2:172.21.15.47:3300/0 conn(0x7f32980d1d60 0x7f32980c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:57.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.994+0000 7f3621d9b640 1 -- 172.21.15.47:0/1331015590 >> v1:172.21.15.203:6789/0 conn(0x7f3624070a30 legacy=0x7f362415ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:57.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.994+0000 7f3621d9b640 1 --2- 172.21.15.47:0/1331015590 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36100039b0 0x7f3610003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:57.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.994+0000 7f9699d9b640 1 -- 172.21.15.47:0/3594122014 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9694007cb0 con 0x7f969c070930 2024-04-26T23:18:57.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.994+0000 7f3621d9b640 1 --2- 172.21.15.47:0/1331015590 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3610004800 0x7f3610004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:57.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.995+0000 7f3621d9b640 1 --2- 172.21.15.47:0/1331015590 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36240cf180 0x7f36100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:57.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.995+0000 7f3621d9b640 1 -- 172.21.15.47:0/1331015590 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f36240cd820 con 0x7f3610004800 2024-04-26T23:18:57.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.995+0000 7f3621d9b640 1 -- 172.21.15.47:0/1331015590 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3624072480 con 0x7f36100039b0 2024-04-26T23:18:57.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.995+0000 7f3621d9b640 1 -- 172.21.15.47:0/1331015590 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f361000bb20 con 0x7f36240cf180 2024-04-26T23:18:57.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.995+0000 7f361bfff640 1 --2- 172.21.15.47:0/1331015590 >> v2:172.21.15.203:3300/0 conn(0x7f36240c4620 0x7f362415f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:18:57.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.994+0000 7fe77c359640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7740c41d0 0x7fe7740cfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:57.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.994+0000 7fe77c359640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe774073f30 0x7fe7741671e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:57.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.994+0000 7fe77c359640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe774163a50 con 0x7fe774167720 2024-04-26T23:18:57.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.994+0000 7fe77c359640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fe7740cf7a0 con 0x7fe774073f30 2024-04-26T23:18:57.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.994+0000 7fe77c359640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fe7740ceb90 con 0x7fe7740c41d0 2024-04-26T23:18:57.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.995+0000 7fe77a0f5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7740c41d0 0x7fe7740cfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:57.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.995+0000 7fe77a0f5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7740c41d0 0x7fe7740cfc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.47:42602/0 (socket says 172.21.15.47:42602) 2024-04-26T23:18:57.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.995+0000 7fe77a0f5640 1 -- 172.21.15.47:0/2535691144 learned_addr learned my addr 172.21.15.47:0/2535691144 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:57.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.994+0000 7f5589d9b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f55840501a0 0x7f5584038080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:57.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.995+0000 7f5589d9b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f55840501a0 0x7f5584038080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58750/0 (socket says 172.21.15.47:58750) 2024-04-26T23:18:57.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.995+0000 7f5589d9b640 1 -- 172.21.15.47:0/654950736 learned_addr learned my addr 172.21.15.47:0/654950736 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:57.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.995+0000 7f558ad9d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5584053c70 0x7f5584006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:57.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.994+0000 7f558a59c640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f55840df220 0x7f558404fa90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:54602/0 (socket says 172.21.15.47:54602) 2024-04-26T23:18:57.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.995+0000 7ff78a59c640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff7840e76e0 0x7ff784052f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:54608/0 (socket says 172.21.15.47:54608) 2024-04-26T23:18:57.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.995+0000 7ff78a59c640 1 -- 172.21.15.47:0/2870217089 learned_addr learned my addr 172.21.15.47:0/2870217089 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:18:57.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.994+0000 7f6d527fc640 1 -- 172.21.15.47:0/3568697616 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3009191222 0 0) 0x7f6d540b5e30 con 0x7f6d5406fc80 2024-04-26T23:18:57.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.994+0000 7f6d527fc640 1 -- 172.21.15.47:0/3568697616 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6d3c005730 con 0x7f6d5406fc80 2024-04-26T23:18:57.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.994+0000 7f6d537fe640 1 -- 172.21.15.47:0/3568697616 >> v2:172.21.15.47:3300/0 conn(0x7f6d54073390 msgr2=0x7f6d5406f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:57.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.994+0000 7f6d537fe640 1 --2- 172.21.15.47:0/3568697616 >> v2:172.21.15.47:3300/0 conn(0x7f6d54073390 0x7f6d5406f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.994+0000 7f6d537fe640 1 -- 172.21.15.47:0/3568697616 >> v1:172.21.15.203:6789/0 conn(0x7f6d5406fc80 legacy=0x7f6d54070110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:57.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.994+0000 7f6d527fc640 1 -- 172.21.15.47:0/3568697616 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3110038332 0 0) 0x7f6d3c005730 con 0x7f6d5406fc80 2024-04-26T23:18:57.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.995+0000 7f6d537fe640 1 -- 172.21.15.47:0/3568697616 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d540b6110 con 0x7f6d54072fb0 2024-04-26T23:18:57.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.995+0000 7f3622d9d640 1 --2- 172.21.15.47:0/1331015590 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36240cf180 0x7f36100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:57.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.995+0000 7f362259c640 1 --2- 172.21.15.47:0/1331015590 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36100039b0 0x7f3610003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:57.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.995+0000 7f361bfff640 1 --2- 172.21.15.47:0/1331015590 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3610004800 0x7f3610004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:57.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.995+0000 7f329ce3b640 1 -- 172.21.15.47:0/862679811 >> v1:172.21.15.203:6789/0 conn(0x7f32980d3570 legacy=0x7f32980b7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:57.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.995+0000 7f329ce3b640 1 -- 172.21.15.47:0/862679811 >> v1:172.21.15.47:6789/0 conn(0x7f32980d1980 legacy=0x7f32980d2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:57.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.995+0000 7fe7798f4640 1 --2- 172.21.15.47:0/2535691144 >> v2:172.21.15.47:3300/0 conn(0x7fe774073f30 0x7fe7741671e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:57.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.995+0000 7fe77a0f5640 1 -- 172.21.15.47:0/2535691144 >> v2:172.21.15.47:3300/0 conn(0x7fe774073f30 msgr2=0x7fe7741671e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:57.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.996+0000 7fe77a0f5640 1 --2- 172.21.15.47:0/2535691144 >> v2:172.21.15.47:3300/0 conn(0x7fe774073f30 0x7fe7741671e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.996+0000 7fe77a0f5640 1 -- 172.21.15.47:0/2535691144 >> v1:172.21.15.47:6789/0 conn(0x7fe774167720 legacy=0x7fe77406ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:57.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.996+0000 7fe77a0f5640 1 -- 172.21.15.47:0/2535691144 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7740cee70 con 0x7fe7740c41d0 2024-04-26T23:18:57.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.995+0000 7f5589d9b640 1 -- 172.21.15.47:0/654950736 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5584053c70 msgr2=0x7f5584006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:57.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.995+0000 7f5589d9b640 1 --2- 172.21.15.47:0/654950736 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5584053c70 0x7f5584006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.995+0000 7f5589d9b640 1 -- 172.21.15.47:0/654950736 >> v1:172.21.15.47:6789/0 conn(0x7f55840df220 legacy=0x7f558404fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:57.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.995+0000 7f5589d9b640 1 -- 172.21.15.47:0/654950736 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5584006210 con 0x7f55840501a0 2024-04-26T23:18:57.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.995+0000 7f96a1166640 1 -- 172.21.15.47:0/3594122014 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f969c070930 msgr2=0x7f969c160ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:57.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.995+0000 7f96a1166640 1 --2- 172.21.15.47:0/3594122014 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f969c070930 0x7f969c160ba0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9694009d90 tx=0x7f969402a5b0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.995+0000 7f96a1166640 1 -- 172.21.15.47:0/3594122014 shutdown_connections 2024-04-26T23:18:57.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.995+0000 7f96a1166640 1 --2- 172.21.15.47:0/3594122014 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f969c070930 0x7f969c160ba0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.995+0000 7f96a1166640 1 -- 172.21.15.47:0/3594122014 >> 172.21.15.47:0/3594122014 conn(0x7f969c0b39b0 msgr2=0x7f969c0b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:57.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.995+0000 7f96a1166640 1 -- 172.21.15.47:0/3594122014 shutdown_connections 2024-04-26T23:18:57.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.996+0000 7ff78ad9d640 1 --2- 172.21.15.47:0/2870217089 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff784050ab0 0x7ff7840f2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:57.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.995+0000 7fc3171f2640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc30804fbe0 0x7fc3080e7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:57.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.996+0000 7fc30ffff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc30804fbe0 0x7fc3080e7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:57.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.996+0000 7fc3171f2640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc308054580 con 0x7fc30803aa10 2024-04-26T23:18:57.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.996+0000 7fc30ffff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc30804fbe0 0x7fc3080e7740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58764/0 (socket says 172.21.15.47:58764) 2024-04-26T23:18:57.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.996+0000 7fc30ffff640 1 -- 172.21.15.47:0/559169318 learned_addr learned my addr 172.21.15.47:0/559169318 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:57.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.996+0000 7fc3171f2640 1 -- 172.21.15.47:0/559169318 --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc308054860 con 0x7fc3080e3160 2024-04-26T23:18:57.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.996+0000 7fc3171f2640 1 -- 172.21.15.47:0/559169318 --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fc308050150 con 0x7fc30804fbe0 2024-04-26T23:18:57.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.995+0000 7f329ce3b640 1 -- 172.21.15.47:0/862679811 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32980d0970 con 0x7f32980d1d60 2024-04-26T23:18:57.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.996+0000 7f362259c640 1 -- 172.21.15.47:0/1331015590 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36240cf180 msgr2=0x7f36100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:57.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.996+0000 7f362259c640 1 --2- 172.21.15.47:0/1331015590 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36240cf180 0x7f36100051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.996+0000 7f362259c640 1 -- 172.21.15.47:0/1331015590 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3610004800 msgr2=0x7f3610004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:57.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.996+0000 7f6d537fe640 1 --2- 172.21.15.47:0/3568697616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d54072fb0 0x7f6d540b56c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6d54050730 tx=0x7f6d48032a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6474b384e8d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:57.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.996+0000 7f362259c640 1 --2- 172.21.15.47:0/1331015590 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3610004800 0x7f3610004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.996+0000 7f362259c640 1 -- 172.21.15.47:0/1331015590 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3610003520 con 0x7f36100039b0 2024-04-26T23:18:57.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.997+0000 7f361bfff640 1 --2- 172.21.15.47:0/1331015590 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3610004800 0x7f3610004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:18:57.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.997+0000 7fe77a0f5640 1 --2- 172.21.15.47:0/2535691144 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7740c41d0 0x7fe7740cfc30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe7740cf920 tx=0x7fe76402a1f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b48e1b0a68ecf6ee server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:57.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.996+0000 7f5589d9b640 1 --2- 172.21.15.47:0/654950736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55840501a0 0x7f5584038080 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f558c06b750 tx=0x7f558c0a8840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9b00fb2c4efee67 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:57.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.996+0000 7ff78ad9d640 1 -- 172.21.15.47:0/2870217089 >> v1:172.21.15.203:6789/0 conn(0x7ff784034400 legacy=0x7ff784053630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:57.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.996+0000 7ff78ad9d640 1 -- 172.21.15.47:0/2870217089 >> v1:172.21.15.47:6789/0 conn(0x7ff7840e76e0 legacy=0x7ff784052f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:57.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.996+0000 7ff78ad9d640 1 -- 172.21.15.47:0/2870217089 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff784054320 con 0x7ff784050ab0 2024-04-26T23:18:57.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.997+0000 7fc30ffff640 1 -- 172.21.15.47:0/559169318 >> v1:172.21.15.203:6789/0 conn(0x7fc3080e3160 legacy=0x7fc30804f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:57.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.997+0000 7fc30ffff640 1 -- 172.21.15.47:0/559169318 >> v1:172.21.15.47:6789/0 conn(0x7fc30803aa10 legacy=0x7fc308006a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:57.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.997+0000 7fc30ffff640 1 -- 172.21.15.47:0/559169318 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc308054b40 con 0x7fc30804fbe0 2024-04-26T23:18:57.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.995+0000 7f96a1166640 1 -- 172.21.15.47:0/3594122014 wait complete. 2024-04-26T23:18:57.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.997+0000 7fe7790f3640 1 -- 172.21.15.47:0/2535691144 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe764037070 con 0x7fe7740c41d0 2024-04-26T23:18:57.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.997+0000 7fe7790f3640 1 -- 172.21.15.47:0/2535691144 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe764032440 con 0x7fe7740c41d0 2024-04-26T23:18:57.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.997+0000 7f558959a640 1 -- 172.21.15.47:0/654950736 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558c0b7070 con 0x7f55840501a0 2024-04-26T23:18:57.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.997+0000 7f6d527fc640 1 -- 172.21.15.47:0/3568697616 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4803c020 con 0x7f6d54072fb0 2024-04-26T23:18:57.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.997+0000 7f6d527fc640 1 -- 172.21.15.47:0/3568697616 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d48034040 con 0x7f6d54072fb0 2024-04-26T23:18:57.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.997+0000 7f6d527fc640 1 -- 172.21.15.47:0/3568697616 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d44001670 con 0x7f6d54072fb0 2024-04-26T23:18:57.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.997+0000 7fe7790f3640 1 -- 172.21.15.47:0/2535691144 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe764032810 con 0x7fe7740c41d0 2024-04-26T23:18:57.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.997+0000 7f329ce3b640 1 --2- 172.21.15.47:0/862679811 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32980d1d60 0x7f32980c8350 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f32880097a0 tx=0x7f328802a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f79349cfef6e480 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:57.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.997+0000 7ff78ad9d640 1 --2- 172.21.15.47:0/2870217089 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff784050ab0 0x7ff7840f2280 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff78c06b190 tx=0x7ff78c0a6c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4965acc63f04f2a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:57.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.998+0000 7f328ffff640 1 -- 172.21.15.47:0/862679811 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3288037070 con 0x7f32980d1d60 2024-04-26T23:18:57.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.998+0000 7f328ffff640 1 -- 172.21.15.47:0/862679811 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f328802adc0 con 0x7f32980d1d60 2024-04-26T23:18:57.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.998+0000 7f96a1166640 1 Processor -- start 2024-04-26T23:18:57.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.998+0000 7f362259c640 1 --2- 172.21.15.47:0/1331015590 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36100039b0 0x7f3610003d90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f360c005200 tx=0x7f3610005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7c3dfc1da22c45c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:57.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.998+0000 7f558959a640 1 -- 172.21.15.47:0/654950736 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f558c0ae050 con 0x7f55840501a0 2024-04-26T23:18:57.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.998+0000 7f558959a640 1 -- 172.21.15.47:0/654950736 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558c0ac030 con 0x7f55840501a0 2024-04-26T23:18:57.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.998+0000 7f3621d9b640 1 -- 172.21.15.47:0/1331015590 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f360c004e80 con 0x7f36100039b0 2024-04-26T23:18:57.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.998+0000 7fc30ffff640 1 --2- 172.21.15.47:0/559169318 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc30804fbe0 0x7fc3080e7740 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc009870 tx=0x7fc2fc02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c4dfa7f15ba060b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:57.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.998+0000 7f6d59b9d640 1 -- 172.21.15.47:0/3568697616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d54072fb0 msgr2=0x7f6d540b56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:57.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.998+0000 7f6d59b9d640 1 --2- 172.21.15.47:0/3568697616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d54072fb0 0x7f6d540b56c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6d54050730 tx=0x7f6d48032a20 comp rx=0 tx=0).stop 2024-04-26T23:18:57.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.998+0000 7ff78959a640 1 -- 172.21.15.47:0/2870217089 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff78c0b4070 con 0x7ff784050ab0 2024-04-26T23:18:57.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.998+0000 7ff78959a640 1 -- 172.21.15.47:0/2870217089 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff78c0a7a50 con 0x7ff784050ab0 2024-04-26T23:18:57.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.998+0000 7ff78959a640 1 -- 172.21.15.47:0/2870217089 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff78c0af440 con 0x7ff784050ab0 2024-04-26T23:18:57.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.998+0000 7f3621d9b640 1 -- 172.21.15.47:0/1331015590 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f360c02fc50 con 0x7f36100039b0 2024-04-26T23:18:57.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.998+0000 7fe77c359640 1 -- 172.21.15.47:0/2535691144 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7740c41d0 msgr2=0x7fe7740cfc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:57.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.998+0000 7fe77c359640 1 --2- 172.21.15.47:0/2535691144 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7740c41d0 0x7fe7740cfc30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe7740cf920 tx=0x7fe76402a1f0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.998+0000 7f3621d9b640 1 -- 172.21.15.47:0/1331015590 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f360c0058d0 con 0x7f36100039b0 2024-04-26T23:18:57.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.998+0000 7fe77c359640 1 -- 172.21.15.47:0/2535691144 shutdown_connections 2024-04-26T23:18:57.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.998+0000 7fe77c359640 1 --2- 172.21.15.47:0/2535691144 >> v2:172.21.15.47:3300/0 conn(0x7fe774073f30 0x7fe7741671e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.998+0000 7fe77c359640 1 --2- 172.21.15.47:0/2535691144 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7740c41d0 0x7fe7740cfc30 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.998+0000 7fe77c359640 1 -- 172.21.15.47:0/2535691144 >> 172.21.15.47:0/2535691144 conn(0x7fe7740d0be0 msgr2=0x7fe7740d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:57.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.998+0000 7fc30f7fe640 1 -- 172.21.15.47:0/559169318 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc037070 con 0x7fc30804fbe0 2024-04-26T23:18:57.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.998+0000 7fe77c359640 1 -- 172.21.15.47:0/2535691144 shutdown_connections 2024-04-26T23:18:57.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.998+0000 7fc30f7fe640 1 -- 172.21.15.47:0/559169318 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2fc02ae50 con 0x7fc30804fbe0 2024-04-26T23:18:57.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.999+0000 7f329e63e640 1 -- 172.21.15.47:0/862679811 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32980d1d60 msgr2=0x7f32980c8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:57.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.999+0000 7f329e63e640 1 --2- 172.21.15.47:0/862679811 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32980d1d60 0x7f32980c8350 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f32880097a0 tx=0x7f328802a130 comp rx=0 tx=0).stop 2024-04-26T23:18:57.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.999+0000 7f329e63e640 1 -- 172.21.15.47:0/862679811 shutdown_connections 2024-04-26T23:18:57.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.999+0000 7f329e63e640 1 --2- 172.21.15.47:0/862679811 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32980d1d60 0x7f32980c8350 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.999+0000 7f329e63e640 1 -- 172.21.15.47:0/862679811 >> 172.21.15.47:0/862679811 conn(0x7f32980d4dd0 msgr2=0x7f32980d51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:57.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.999+0000 7fc30f7fe640 1 -- 172.21.15.47:0/559169318 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc037070 con 0x7fc30804fbe0 2024-04-26T23:18:57.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.999+0000 7f558b59e640 1 -- 172.21.15.47:0/654950736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55840501a0 msgr2=0x7f5584038080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:57.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.999+0000 7f558b59e640 1 --2- 172.21.15.47:0/654950736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55840501a0 0x7f5584038080 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f558c06b750 tx=0x7f558c0a8840 comp rx=0 tx=0).stop 2024-04-26T23:18:57.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.999+0000 7f329e63e640 1 -- 172.21.15.47:0/862679811 shutdown_connections 2024-04-26T23:18:57.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.999+0000 7fe77c359640 1 -- 172.21.15.47:0/2535691144 wait complete. 2024-04-26T23:18:57.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.998+0000 7f96a1166640 1 -- start start 2024-04-26T23:18:57.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.999+0000 7f3628af9640 1 -- 172.21.15.47:0/1331015590 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36100039b0 msgr2=0x7f3610003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:57.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.999+0000 7f3628af9640 1 --2- 172.21.15.47:0/1331015590 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36100039b0 0x7f3610003d90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f360c005200 tx=0x7f3610005830 comp rx=0 tx=0).stop 2024-04-26T23:18:57.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.999+0000 7ff78b59e640 1 -- 172.21.15.47:0/2870217089 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff784050ab0 msgr2=0x7ff7840f2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:57.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.999+0000 7ff78b59e640 1 --2- 172.21.15.47:0/2870217089 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff784050ab0 0x7ff7840f2280 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff78c06b190 tx=0x7ff78c0a6c40 comp rx=0 tx=0).stop 2024-04-26T23:18:57.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.999+0000 7ff78b59e640 1 -- 172.21.15.47:0/2870217089 shutdown_connections 2024-04-26T23:18:57.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.999+0000 7ff78b59e640 1 --2- 172.21.15.47:0/2870217089 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff784050ab0 0x7ff7840f2280 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.999+0000 7ff78b59e640 1 -- 172.21.15.47:0/2870217089 >> 172.21.15.47:0/2870217089 conn(0x7ff784055350 msgr2=0x7ff784055760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:57.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.999+0000 7f558b59e640 1 -- 172.21.15.47:0/654950736 shutdown_connections 2024-04-26T23:18:57.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.999+0000 7f558b59e640 1 --2- 172.21.15.47:0/654950736 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5584053c70 0x7f5584006a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.999+0000 7f558b59e640 1 --2- 172.21.15.47:0/654950736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55840501a0 0x7f5584038080 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.999+0000 7f329e63e640 1 -- 172.21.15.47:0/862679811 wait complete. 2024-04-26T23:18:57.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.999+0000 7f558b59e640 1 -- 172.21.15.47:0/654950736 >> 172.21.15.47:0/654950736 conn(0x7f5584009fb0 msgr2=0x7f558400a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:57.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.999+0000 7fc3171f2640 1 -- 172.21.15.47:0/559169318 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc30804fbe0 msgr2=0x7fc3080e7740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:57.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.999+0000 7fc3171f2640 1 --2- 172.21.15.47:0/559169318 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc30804fbe0 0x7fc3080e7740 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc009870 tx=0x7fc2fc02a300 comp rx=0 tx=0).stop 2024-04-26T23:18:57.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.999+0000 7f3622d9d640 1 -- 172.21.15.47:0/1331015590 reap_dead start 2024-04-26T23:18:57.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.999+0000 7f3628af9640 1 -- 172.21.15.47:0/1331015590 shutdown_connections 2024-04-26T23:18:57.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.999+0000 7f3628af9640 1 -- 172.21.15.47:0/1331015590 >> 172.21.15.47:0/1331015590 conn(0x7f36240d0ff0 msgr2=0x7f36240d13d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:57.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.999+0000 7ff78b59e640 1 -- 172.21.15.47:0/2870217089 shutdown_connections 2024-04-26T23:18:57.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.000+0000 7ff78b59e640 1 -- 172.21.15.47:0/2870217089 wait complete. 2024-04-26T23:18:57.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.000+0000 7fc3171f2640 1 -- 172.21.15.47:0/559169318 shutdown_connections 2024-04-26T23:18:57.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.999+0000 7f6d59b9d640 1 -- 172.21.15.47:0/3568697616 shutdown_connections 2024-04-26T23:18:57.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.999+0000 7f6d59b9d640 1 --2- 172.21.15.47:0/3568697616 >> v2:172.21.15.47:3300/0 conn(0x7f6d54073390 0x7f6d5406f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.999+0000 7f6d59b9d640 1 --2- 172.21.15.47:0/3568697616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d54072fb0 0x7f6d540b56c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.999+0000 7f6d59b9d640 1 -- 172.21.15.47:0/3568697616 >> 172.21.15.47:0/3568697616 conn(0x7f6d540b39b0 msgr2=0x7f6d540b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:57.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.000+0000 7f6d59b9d640 1 -- 172.21.15.47:0/3568697616 shutdown_connections 2024-04-26T23:18:57.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:56.999+0000 7f96a1166640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f969c06c950 0x7f969c14b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:57.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.000+0000 7f96a1166640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f969c070930 0x7f969c0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:57.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.000+0000 7f96a1166640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f969c15d100 0x7f969c1583d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:57.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.000+0000 7f96a1166640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f969c0aef10 con 0x7f969c070930 2024-04-26T23:18:57.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.000+0000 7f96a1166640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f969c0aed90 con 0x7f969c06c950 2024-04-26T23:18:57.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.000+0000 7f96a1166640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f969c0af090 con 0x7f969c15d100 2024-04-26T23:18:57.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.000+0000 7f3628af9640 1 -- 172.21.15.47:0/1331015590 shutdown_connections 2024-04-26T23:18:57.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.000+0000 7f3628af9640 1 -- 172.21.15.47:0/1331015590 wait complete. 2024-04-26T23:18:57.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.000+0000 7fc3171f2640 1 --2- 172.21.15.47:0/559169318 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc30804fbe0 0x7fc3080e7740 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.000+0000 7fc3171f2640 1 -- 172.21.15.47:0/559169318 >> 172.21.15.47:0/559169318 conn(0x7fc30800a400 msgr2=0x7fc30800a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:57.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.000+0000 7fc3171f2640 1 -- 172.21.15.47:0/559169318 shutdown_connections 2024-04-26T23:18:57.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.000+0000 7f969a59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f969c070930 0x7f969c0b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:57.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.000+0000 7f969a59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f969c070930 0x7f969c0b4db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58766/0 (socket says 172.21.15.47:58766) 2024-04-26T23:18:57.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.000+0000 7f969a59c640 1 -- 172.21.15.47:0/230842618 learned_addr learned my addr 172.21.15.47:0/230842618 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:57.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.000+0000 7f969b59e640 1 --2- 172.21.15.47:0/230842618 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f969c15d100 0x7f969c1583d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:57.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.000+0000 7f969ad9d640 1 --2- 172.21.15.47:0/230842618 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f969c06c950 0x7f969c14b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:57.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.000+0000 7fe77c359640 1 Processor -- start 2024-04-26T23:18:57.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.000+0000 7fe77c359640 1 -- start start 2024-04-26T23:18:57.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.000+0000 7f969a59c640 1 -- 172.21.15.47:0/230842618 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f969c15d100 msgr2=0x7f969c1583d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:57.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.000+0000 7f969a59c640 1 --2- 172.21.15.47:0/230842618 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f969c15d100 0x7f969c1583d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.001+0000 7f969a59c640 1 -- 172.21.15.47:0/230842618 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f969c06c950 msgr2=0x7f969c14b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:57.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.001+0000 7f969a59c640 1 --2- 172.21.15.47:0/230842618 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f969c06c950 0x7f969c14b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.001+0000 7f969a59c640 1 -- 172.21.15.47:0/230842618 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f969c0b5790 con 0x7f969c070930 2024-04-26T23:18:57.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.001+0000 7f6d59b9d640 1 -- 172.21.15.47:0/3568697616 wait complete. 2024-04-26T23:18:57.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.001+0000 7f329e63e640 1 Processor -- start 2024-04-26T23:18:57.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.001+0000 7f969a59c640 1 --2- 172.21.15.47:0/230842618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f969c070930 0x7f969c0b4db0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f968c00b410 tx=0x7f968c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:57.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.001+0000 7f96837fe640 1 -- 172.21.15.47:0/230842618 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968c015020 con 0x7f969c070930 2024-04-26T23:18:57.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.001+0000 7f96837fe640 1 -- 172.21.15.47:0/230842618 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f968c011830 con 0x7f969c070930 2024-04-26T23:18:57.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.001+0000 7f96837fe640 1 -- 172.21.15.47:0/230842618 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968c011c20 con 0x7f969c070930 2024-04-26T23:18:57.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.001+0000 7f96a1166640 1 -- 172.21.15.47:0/230842618 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f969c0b5a70 con 0x7f969c070930 2024-04-26T23:18:57.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.001+0000 7f96a1166640 1 -- 172.21.15.47:0/230842618 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f969c0b6030 con 0x7f969c070930 2024-04-26T23:18:57.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.002+0000 7fe77c359640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe774073f30 0x7fe77414e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:57.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.000+0000 7fc3171f2640 1 -- 172.21.15.47:0/559169318 wait complete. 2024-04-26T23:18:57.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.002+0000 7fc3171f2640 1 Processor -- start 2024-04-26T23:18:57.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.002+0000 7f3628af9640 1 Processor -- start 2024-04-26T23:18:57.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.002+0000 7f3628af9640 1 -- start start 2024-04-26T23:18:57.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.002+0000 7fe77c359640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7740c41d0 0x7fe7740b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:57.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.002+0000 7fe77c359640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe774167720 0x7fe774163c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:57.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.001+0000 7f329e63e640 1 -- start start 2024-04-26T23:18:57.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.002+0000 7fe77c359640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe7740d0170 con 0x7fe774167720 2024-04-26T23:18:57.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.002+0000 7f329e63e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32980d1980 0x7f32981f8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:57.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.002+0000 7f329e63e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32980d1d60 0x7f32980c8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:57.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.002+0000 7f329e63e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f32980d3570 0x7f32980c8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:57.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.002+0000 7f329e63e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f32980d0ff0 con 0x7f32980d1d60 2024-04-26T23:18:57.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.002+0000 7f329e63e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f32980d0e70 con 0x7f32980d1980 2024-04-26T23:18:57.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.002+0000 7f329e63e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f32980b62c0 con 0x7f32980d3570 2024-04-26T23:18:57.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.003+0000 7f6d59b9d640 1 Processor -- start 2024-04-26T23:18:57.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.002+0000 7f329d63c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32980d1980 0x7f32981f8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:57.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.002+0000 7f329d63c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32980d1980 0x7f32981f8920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.47:58248/0 (socket says 172.21.15.47:58248) 2024-04-26T23:18:57.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.002+0000 7f329d63c640 1 -- 172.21.15.47:0/3691387343 learned_addr learned my addr 172.21.15.47:0/3691387343 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:57.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.003+0000 7fe77a8f6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe774167720 0x7fe774163c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:57.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.003+0000 7fe77a8f6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe774167720 0x7fe774163c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58770/0 (socket says 172.21.15.47:58770) 2024-04-26T23:18:57.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.003+0000 7fe77a8f6640 1 -- 172.21.15.47:0/2728888840 learned_addr learned my addr 172.21.15.47:0/2728888840 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:57.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.003+0000 7f329d63c640 1 -- 172.21.15.47:0/3691387343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f32980d3570 msgr2=0x7f32980c8e40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T23:18:57.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.003+0000 7f329d63c640 1 --2- 172.21.15.47:0/3691387343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f32980d3570 0x7f32980c8e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.003+0000 7f329d63c640 1 -- 172.21.15.47:0/3691387343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32980d1d60 msgr2=0x7f32980c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:57.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.003+0000 7f329d63c640 1 --2- 172.21.15.47:0/3691387343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32980d1d60 0x7f32980c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.003+0000 7f329d63c640 1 -- 172.21.15.47:0/3691387343 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3298150d90 con 0x7f32980d1980 2024-04-26T23:18:57.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.003+0000 7fe77c359640 1 -- 172.21.15.47:0/2728888840 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe7740cf430 con 0x7fe7740c41d0 2024-04-26T23:18:57.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.003+0000 7fe77a0f5640 1 --2- 172.21.15.47:0/2728888840 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe774073f30 0x7fe77414e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:57.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.003+0000 7fe77c359640 1 -- 172.21.15.47:0/2728888840 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fe7740ce7b0 con 0x7fe774073f30 2024-04-26T23:18:57.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.003+0000 7f329d63c640 1 --2- 172.21.15.47:0/3691387343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32980d1980 0x7f32981f8920 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f329400caf0 tx=0x7f3294002090 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:57.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.003+0000 7f6d59b9d640 1 -- start start 2024-04-26T23:18:57.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.004+0000 7f328dffb640 1 -- 172.21.15.47:0/3691387343 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3294018070 con 0x7f32980d1980 2024-04-26T23:18:57.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.004+0000 7f558b59e640 1 -- 172.21.15.47:0/654950736 shutdown_connections 2024-04-26T23:18:57.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.002+0000 7fc3171f2640 1 -- start start 2024-04-26T23:18:57.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.004+0000 7f3628af9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3624158d40 0x7f36241f0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:57.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.004+0000 7fc3171f2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc30803aa10 0x7fc30804ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:57.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.004+0000 7fc3171f2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc30804fbe0 0x7fc3080e57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:57.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.004+0000 7fc3171f2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3080e3160 0x7fc30804d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:57.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.004+0000 7fc3171f2640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc3080e3540 con 0x7fc30803aa10 2024-04-26T23:18:57.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.004+0000 7fc3171f2640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc3080062e0 con 0x7fc30804fbe0 2024-04-26T23:18:57.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.004+0000 7fc3171f2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc308007160 con 0x7fc3080e3160 2024-04-26T23:18:57.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.004+0000 7f558b59e640 1 -- 172.21.15.47:0/654950736 wait complete. 2024-04-26T23:18:57.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.004+0000 7f3628af9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3624159120 0x7f36241f41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:57.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.004+0000 7f362259c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3624158d40 0x7f36241f0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:57.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.004+0000 7f362259c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3624158d40 0x7f36241f0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.47:42678/0 (socket says 172.21.15.47:42678) 2024-04-26T23:18:57.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.004+0000 7f362259c640 1 -- 172.21.15.47:0/2566093308 learned_addr learned my addr 172.21.15.47:0/2566093308 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:57.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.004+0000 7fe7798f4640 1 --2- 172.21.15.47:0/2728888840 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7740c41d0 0x7fe7740b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:57.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.004+0000 7f6d59b9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d5406fc80 0x7f6d540d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:57.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.004+0000 7f6d59b9d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d54072fb0 0x7f6d5415c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:57.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.002+0000 7ff78b59e640 1 Processor -- start 2024-04-26T23:18:57.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.002+0000 7ff78b59e640 1 -- start start 2024-04-26T23:18:57.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.003+0000 7ff78b59e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff784034400 0x7ff78404d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:57.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.003+0000 7ff78b59e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff784050ab0 0x7ff78404b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:57.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.003+0000 7ff78b59e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff7840e76e0 0x7ff7840db7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:57.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.003+0000 7ff78b59e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff7840e8ad0 con 0x7ff784034400 2024-04-26T23:18:57.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.003+0000 7ff78b59e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff784053d40 con 0x7ff784050ab0 2024-04-26T23:18:57.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.003+0000 7ff78a59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff784034400 0x7ff78404d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:57.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.003+0000 7ff78b59e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7ff7840e8c50 con 0x7ff7840e76e0 2024-04-26T23:18:57.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.003+0000 7ff78a59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff784034400 0x7ff78404d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58792/0 (socket says 172.21.15.47:58792) 2024-04-26T23:18:57.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.003+0000 7ff78ad9d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff7840e76e0 0x7ff7840db7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:57.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.003+0000 7ff78a59c640 1 -- 172.21.15.47:0/30704947 learned_addr learned my addr 172.21.15.47:0/30704947 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:57.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.003+0000 7ff789d9b640 1 --2- 172.21.15.47:0/30704947 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff784050ab0 0x7ff78404b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:57.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.004+0000 7ff78a59c640 1 -- 172.21.15.47:0/30704947 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff7840e76e0 msgr2=0x7ff7840db7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:57.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.004+0000 7ff78a59c640 1 --2- 172.21.15.47:0/30704947 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff7840e76e0 0x7ff7840db7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.004+0000 7ff78a59c640 1 -- 172.21.15.47:0/30704947 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff784050ab0 msgr2=0x7ff78404b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:57.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.004+0000 7ff78a59c640 1 --2- 172.21.15.47:0/30704947 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff784050ab0 0x7ff78404b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.004+0000 7ff78a59c640 1 -- 172.21.15.47:0/30704947 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff78404bd90 con 0x7ff784034400 2024-04-26T23:18:57.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.004+0000 7f6d59b9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d54073390 0x7f6d5415ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:57.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.004+0000 7f3628af9640 1 --2- 172.21.15.47:0/2566093308 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36240a0210 0x7f36240b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:57.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.004+0000 7f361bfff640 1 --2- 172.21.15.47:0/2566093308 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3624159120 0x7f36241f41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:57.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.004+0000 7fe77a0f5640 1 -- 172.21.15.47:0/2728888840 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7740c41d0 msgr2=0x7fe7740b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:57.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.004+0000 7fe77a0f5640 1 --2- 172.21.15.47:0/2728888840 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7740c41d0 0x7fe7740b3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.004+0000 7fe77a0f5640 1 -- 172.21.15.47:0/2728888840 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe774167720 msgr2=0x7fe774163c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:57.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.004+0000 7fc31578f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3080e3160 0x7fc30804d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:57.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.004+0000 7fe77a0f5640 1 --2- 172.21.15.47:0/2728888840 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe774167720 0x7fe774163c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.004+0000 7fe77a0f5640 1 -- 172.21.15.47:0/2728888840 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7740b49d0 con 0x7fe774073f30 2024-04-26T23:18:57.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.004+0000 7f6d59b9d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5415c160 con 0x7f6d54073390 2024-04-26T23:18:57.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.005+0000 7f6d59b9d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6d540737f0 con 0x7f6d5406fc80 2024-04-26T23:18:57.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.005+0000 7f3628af9640 1 -- 172.21.15.47:0/2566093308 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f362415fb60 con 0x7f36240a0210 2024-04-26T23:18:57.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.005+0000 7f6d59b9d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f6d5415c2e0 con 0x7f6d54072fb0 2024-04-26T23:18:57.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.005+0000 7f3628af9640 1 -- 172.21.15.47:0/2566093308 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f362415f9e0 con 0x7f3624159120 2024-04-26T23:18:57.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.005+0000 7f6d52ffd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d54072fb0 0x7f6d5415c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:57.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.005+0000 7f3628af9640 1 -- 172.21.15.47:0/2566093308 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3624069a80 con 0x7f3624158d40 2024-04-26T23:18:57.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.005+0000 7f6d52ffd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d54072fb0 0x7f6d5415c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.47:42682/0 (socket says 172.21.15.47:42682) 2024-04-26T23:18:57.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.004+0000 7fc31578f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3080e3160 0x7fc30804d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.47:42668/0 (socket says 172.21.15.47:42668) 2024-04-26T23:18:57.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.005+0000 7f6d52ffd640 1 -- 172.21.15.47:0/951585067 learned_addr learned my addr 172.21.15.47:0/951585067 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:57.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.004+0000 7f96837fe640 1 -- 172.21.15.47:0/230842618 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f968c018020 con 0x7f969c070930 2024-04-26T23:18:57.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.005+0000 7f3622d9d640 1 --2- 172.21.15.47:0/2566093308 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36240a0210 0x7f36240b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:57.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.005+0000 7f6d53fff640 1 --2- 172.21.15.47:0/951585067 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d54073390 0x7f6d5415ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:57.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.004+0000 7fc31578f640 1 -- 172.21.15.47:0/4007414522 learned_addr learned my addr 172.21.15.47:0/4007414522 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:57.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.004+0000 7fc30ffff640 1 --2- 172.21.15.47:0/4007414522 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc30804fbe0 0x7fc3080e57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:57.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.004+0000 7fc314f8e640 1 --2- 172.21.15.47:0/4007414522 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc30803aa10 0x7fc30804ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:57.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.005+0000 7fc31578f640 1 -- 172.21.15.47:0/4007414522 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc30804fbe0 msgr2=0x7fc3080e57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:57.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.005+0000 7fc31578f640 1 --2- 172.21.15.47:0/4007414522 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc30804fbe0 0x7fc3080e57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.005+0000 7fc31578f640 1 -- 172.21.15.47:0/4007414522 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc30803aa10 msgr2=0x7fc30804ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:57.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.005+0000 7fc31578f640 1 --2- 172.21.15.47:0/4007414522 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc30803aa10 0x7fc30804ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.005+0000 7fc31578f640 1 -- 172.21.15.47:0/4007414522 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc30804d8a0 con 0x7fc3080e3160 2024-04-26T23:18:57.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.005+0000 7fe77a0f5640 1 --2- 172.21.15.47:0/2728888840 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe774073f30 0x7fe77414e150 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe764031000 tx=0x7fe7640046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:57.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.005+0000 7f6d537fe640 1 --2- 172.21.15.47:0/951585067 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d5406fc80 0x7f6d540d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:57.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.005+0000 7f96837fe640 1 --2- 172.21.15.47:0/230842618 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f968806c9f0 0x7f968806eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:57.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.005+0000 7f96837fe640 1 -- 172.21.15.47:0/230842618 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f968c08bcb0 con 0x7f969c070930 2024-04-26T23:18:57.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.005+0000 7f362259c640 1 -- 172.21.15.47:0/2566093308 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3624159120 msgr2=0x7f36241f41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:57.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.005+0000 7f362259c640 1 --2- 172.21.15.47:0/2566093308 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3624159120 0x7f36241f41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.005+0000 7f362259c640 1 -- 172.21.15.47:0/2566093308 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36240a0210 msgr2=0x7f36240b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:57.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.005+0000 7f362259c640 1 --2- 172.21.15.47:0/2566093308 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36240a0210 0x7f36240b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.005+0000 7f362259c640 1 -- 172.21.15.47:0/2566093308 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36241f4940 con 0x7f3624158d40 2024-04-26T23:18:57.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.005+0000 7f3622d9d640 1 --2- 172.21.15.47:0/2566093308 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36240a0210 0x7f36240b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:18:57.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.005+0000 7f96a1166640 1 -- 172.21.15.47:0/230842618 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f967c005290 con 0x7f969c070930 2024-04-26T23:18:57.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.005+0000 7ff78a59c640 1 --2- 172.21.15.47:0/30704947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff784034400 0x7ff78404d7d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff77c00c480 tx=0x7ff77c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:57.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.005+0000 7fc31578f640 1 --2- 172.21.15.47:0/4007414522 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3080e3160 0x7fc30804d360 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc31006b190 tx=0x7fc310075270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:57.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.005+0000 7f969ad9d640 1 --2- 172.21.15.47:0/230842618 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f968806c9f0 0x7f968806eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:57.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.005+0000 7fe76affd640 1 -- 172.21.15.47:0/2728888840 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe764041020 con 0x7fe774073f30 2024-04-26T23:18:57.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.006+0000 7ff77affd640 1 -- 172.21.15.47:0/30704947 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff77c017070 con 0x7ff784034400 2024-04-26T23:18:57.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.006+0000 7ff78b59e640 1 -- 172.21.15.47:0/30704947 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff78404f460 con 0x7ff784034400 2024-04-26T23:18:57.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.006+0000 7ff78b59e640 1 -- 172.21.15.47:0/30704947 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff78404aa90 con 0x7ff784034400 2024-04-26T23:18:57.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.006+0000 7ff77affd640 1 -- 172.21.15.47:0/30704947 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff77c005ce0 con 0x7ff784034400 2024-04-26T23:18:57.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.006+0000 7f558b59e640 1 Processor -- start 2024-04-26T23:18:57.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.006+0000 7ff77affd640 1 -- 172.21.15.47:0/30704947 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff77c0072d0 con 0x7ff784034400 2024-04-26T23:18:57.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.006+0000 7f6d52ffd640 1 -- 172.21.15.47:0/951585067 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d5406fc80 msgr2=0x7f6d540d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:57.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.006+0000 7f6d52ffd640 1 --2- 172.21.15.47:0/951585067 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d5406fc80 0x7f6d540d0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.006+0000 7f6d52ffd640 1 -- 172.21.15.47:0/951585067 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d54073390 msgr2=0x7f6d5415ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:57.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.006+0000 7f6d52ffd640 1 --2- 172.21.15.47:0/951585067 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d54073390 0x7f6d5415ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.006+0000 7f6d52ffd640 1 -- 172.21.15.47:0/951585067 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d54159840 con 0x7f6d54072fb0 2024-04-26T23:18:57.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.006+0000 7f6d52ffd640 1 --2- 172.21.15.47:0/951585067 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d54072fb0 0x7f6d5415c4f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c009d10 tx=0x7f6d4c00c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:57.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.006+0000 7f6d3bfff640 1 -- 172.21.15.47:0/951585067 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c018020 con 0x7f6d54072fb0 2024-04-26T23:18:57.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.006+0000 7f558b59e640 1 -- start start 2024-04-26T23:18:57.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.006+0000 7f6d59b9d640 1 -- 172.21.15.47:0/951585067 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d54159ae0 con 0x7f6d54072fb0 2024-04-26T23:18:57.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.006+0000 7fe76affd640 1 -- 172.21.15.47:0/2728888840 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe764030780 con 0x7fe774073f30 2024-04-26T23:18:57.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.006+0000 7fe76affd640 1 -- 172.21.15.47:0/2728888840 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe764037040 con 0x7fe774073f30 2024-04-26T23:18:57.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.006+0000 7fe77c359640 1 -- 172.21.15.47:0/2728888840 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7740b4cb0 con 0x7fe774073f30 2024-04-26T23:18:57.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.005+0000 7f329e63e640 1 -- 172.21.15.47:0/3691387343 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32981537e0 con 0x7f32980d1980 2024-04-26T23:18:57.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.007+0000 7fe77c359640 1 -- 172.21.15.47:0/2728888840 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7740b5270 con 0x7fe774073f30 2024-04-26T23:18:57.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.007+0000 7f6d59b9d640 1 -- 172.21.15.47:0/951585067 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d5415a0a0 con 0x7f6d54072fb0 2024-04-26T23:18:57.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.007+0000 7f6d3bfff640 1 -- 172.21.15.47:0/951585067 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d4c014480 con 0x7f6d54072fb0 2024-04-26T23:18:57.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.008+0000 7f6d3bfff640 1 -- 172.21.15.47:0/951585067 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c014830 con 0x7f6d54072fb0 2024-04-26T23:18:57.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.007+0000 7f558b59e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5584053c70 0x7f55840cca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:57.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.008+0000 7f558b59e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55840df220 0x7f558404dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:57.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.008+0000 7f558a59c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5584053c70 0x7f55840cca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:57.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.008+0000 7f558a59c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5584053c70 0x7f55840cca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.47:42692/0 (socket says 172.21.15.47:42692) 2024-04-26T23:18:57.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.008+0000 7f558a59c640 1 -- 172.21.15.47:0/3427278210 learned_addr learned my addr 172.21.15.47:0/3427278210 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:57.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.008+0000 7f5589d9b640 1 --2- 172.21.15.47:0/3427278210 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55840df220 0x7f558404dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:57.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.008+0000 7f558b59e640 1 --2- 172.21.15.47:0/3427278210 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f558404a880 0x7f558404e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:57.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.007+0000 7fc30d7fa640 1 -- 172.21.15.47:0/4007414522 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc310072440 con 0x7fc3080e3160 2024-04-26T23:18:57.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.007+0000 7fc30d7fa640 1 -- 172.21.15.47:0/4007414522 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3100725e0 con 0x7fc3080e3160 2024-04-26T23:18:57.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.007+0000 7fc30d7fa640 1 -- 172.21.15.47:0/4007414522 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc310072990 con 0x7fc3080e3160 2024-04-26T23:18:57.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.008+0000 7fc3171f2640 1 -- 172.21.15.47:0/4007414522 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc30804db40 con 0x7fc3080e3160 2024-04-26T23:18:57.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.008+0000 7fc3171f2640 1 -- 172.21.15.47:0/4007414522 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc30804e100 con 0x7fc3080e3160 2024-04-26T23:18:57.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.008+0000 7f362259c640 1 --2- 172.21.15.47:0/2566093308 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3624158d40 0x7f36241f0c00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f360c005320 tx=0x7f360c004c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:57.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.008+0000 7f558b59e640 1 -- 172.21.15.47:0/3427278210 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5584037a90 con 0x7f558404a880 2024-04-26T23:18:57.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.008+0000 7f558ad9d640 1 --2- 172.21.15.47:0/3427278210 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f558404a880 0x7f558404e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:57.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.008+0000 7ff78b59e640 1 -- 172.21.15.47:0/30704947 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff770005290 con 0x7ff784034400 2024-04-26T23:18:57.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.005+0000 7f329e63e640 1 -- 172.21.15.47:0/3691387343 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32980b7f80 con 0x7f32980d1980 2024-04-26T23:18:57.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.008+0000 7f558b59e640 1 -- 172.21.15.47:0/3427278210 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5584037910 con 0x7f55840df220 2024-04-26T23:18:57.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.008+0000 7f558b59e640 1 -- 172.21.15.47:0/3427278210 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f55840df600 con 0x7f5584053c70 2024-04-26T23:18:57.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.008+0000 7f969ad9d640 1 --2- 172.21.15.47:0/230842618 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f968806c9f0 0x7f968806eeb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9690000f60 tx=0x7f9690000bc0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:57.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.008+0000 7f6d39ffb640 1 -- 172.21.15.47:0/951585067 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d540694f0 con 0x7f6d54072fb0 2024-04-26T23:18:57.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.008+0000 7fe77c359640 1 -- 172.21.15.47:0/2728888840 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe774167510 con 0x7fe774073f30 2024-04-26T23:18:57.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.006+0000 7f328dffb640 1 -- 172.21.15.47:0/3691387343 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f329400a700 con 0x7f32980d1980 2024-04-26T23:18:57.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.006+0000 7f328dffb640 1 -- 172.21.15.47:0/3691387343 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329400ab70 con 0x7f32980d1980 2024-04-26T23:18:57.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.008+0000 7f328dffb640 1 -- 172.21.15.47:0/3691387343 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f329401b070 con 0x7f32980d1980 2024-04-26T23:18:57.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.009+0000 7f329e63e640 1 -- 172.21.15.47:0/3691387343 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3278005290 con 0x7f32980d1980 2024-04-26T23:18:57.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.009+0000 7f328dffb640 1 --2- 172.21.15.47:0/3691387343 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f327c06c9c0 0x7f327c06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:57.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.009+0000 7f5589d9b640 1 -- 172.21.15.47:0/3427278210 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5584053c70 msgr2=0x7f55840cca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:57.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.009+0000 7f5589d9b640 1 --2- 172.21.15.47:0/3427278210 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5584053c70 0x7f55840cca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.009+0000 7f5589d9b640 1 -- 172.21.15.47:0/3427278210 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f558404a880 msgr2=0x7f558404e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:57.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.009+0000 7f5589d9b640 1 --2- 172.21.15.47:0/3427278210 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f558404a880 0x7f558404e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.009+0000 7f5589d9b640 1 -- 172.21.15.47:0/3427278210 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5584049b10 con 0x7f55840df220 2024-04-26T23:18:57.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.009+0000 7f328dffb640 1 -- 172.21.15.47:0/3691387343 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f329408b750 con 0x7f32980d1980 2024-04-26T23:18:57.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.009+0000 7f5589d9b640 1 --2- 172.21.15.47:0/3427278210 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55840df220 0x7f558404dd80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f558c06a760 tx=0x7f558c06a0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:57.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.009+0000 7f329ce3b640 1 --2- 172.21.15.47:0/3691387343 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f327c06c9c0 0x7f327c06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:57.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.009+0000 7f361affd640 1 -- 172.21.15.47:0/2566093308 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f360c00f1d0 con 0x7f3624158d40 2024-04-26T23:18:57.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.009+0000 7f3628af9640 1 -- 172.21.15.47:0/2566093308 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36241f4c20 con 0x7f3624158d40 2024-04-26T23:18:57.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.009+0000 7f557affd640 1 -- 172.21.15.47:0/3427278210 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558c0c1020 con 0x7f55840df220 2024-04-26T23:18:57.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.010+0000 7fc3171f2640 1 -- 172.21.15.47:0/4007414522 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc30800aa30 con 0x7fc3080e3160 2024-04-26T23:18:57.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.010+0000 7f557affd640 1 -- 172.21.15.47:0/3427278210 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f558c0b26c0 con 0x7f55840df220 2024-04-26T23:18:57.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.010+0000 7f558b59e640 1 -- 172.21.15.47:0/3427278210 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f558404adb0 con 0x7f55840df220 2024-04-26T23:18:57.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.010+0000 7f558b59e640 1 -- 172.21.15.47:0/3427278210 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f558404b370 con 0x7f55840df220 2024-04-26T23:18:57.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.009+0000 7f3628af9640 1 -- 172.21.15.47:0/2566093308 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f362409fcf0 con 0x7f3624158d40 2024-04-26T23:18:57.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.010+0000 7f329ce3b640 1 --2- 172.21.15.47:0/3691387343 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f327c06c9c0 0x7f327c06ee80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3288034000 tx=0x7f328802c000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:57.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.010+0000 7fe76affd640 1 -- 172.21.15.47:0/2728888840 <== mon.2 v2:172.21.15.203:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe764044020 con 0x7fe774073f30 2024-04-26T23:18:57.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.010+0000 7f6d3bfff640 1 -- 172.21.15.47:0/951585067 <== mon.2 v2:172.21.15.203:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6d4c005d10 con 0x7f6d54072fb0 2024-04-26T23:18:57.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.010+0000 7f557affd640 1 -- 172.21.15.47:0/3427278210 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558c0c1020 con 0x7f55840df220 2024-04-26T23:18:57.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.011+0000 7f6d3bfff640 1 --2- 172.21.15.47:0/951585067 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f6d4006c940 0x7f6d4006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:57.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.011+0000 7f6d3bfff640 1 -- 172.21.15.47:0/951585067 <== mon.2 v2:172.21.15.203:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f6d4c08d050 con 0x7f6d54072fb0 2024-04-26T23:18:57.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.011+0000 7f3628af9640 1 -- 172.21.15.47:0/2566093308 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f362409fae0 con 0x7f3624158d40 2024-04-26T23:18:57.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.011+0000 7f6d537fe640 1 --2- 172.21.15.47:0/951585067 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f6d4006c940 0x7f6d4006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:57.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.011+0000 7fc30d7fa640 1 -- 172.21.15.47:0/4007414522 <== mon.2 v2:172.21.15.203:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc31006f030 con 0x7fc3080e3160 2024-04-26T23:18:57.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.011+0000 7fe76affd640 1 --2- 172.21.15.47:0/2728888840 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fe75806c940 0x7fe75806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:57.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.011+0000 7f558b59e640 1 -- 172.21.15.47:0/3427278210 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5584005390 con 0x7f55840df220 2024-04-26T23:18:57.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.011+0000 7fe7798f4640 1 --2- 172.21.15.47:0/2728888840 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fe75806c940 0x7fe75806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:57.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.011+0000 7fc30d7fa640 1 --2- 172.21.15.47:0/4007414522 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fc2f406c940 0x7fc2f406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:57.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.012+0000 7f6d537fe640 1 --2- 172.21.15.47:0/951585067 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f6d4006c940 0x7f6d4006ee00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6d4803d000 tx=0x7f6d48008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:57.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.012+0000 7fc314f8e640 1 --2- 172.21.15.47:0/4007414522 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fc2f406c940 0x7fc2f406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:57.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.012+0000 7fe7798f4640 1 --2- 172.21.15.47:0/2728888840 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fe75806c940 0x7fe75806ee00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe7740ce4f0 tx=0x7fe76c003990 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:57.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.012+0000 7fc314f8e640 1 --2- 172.21.15.47:0/4007414522 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fc2f406c940 0x7fc2f406ee00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc304000f60 tx=0x7fc304009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:57.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.014+0000 7f96837fe640 1 -- 172.21.15.47:0/230842618 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f968c01f020 con 0x7f969c070930 2024-04-26T23:18:57.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.015+0000 7fe76affd640 1 -- 172.21.15.47:0/2728888840 <== mon.2 v2:172.21.15.203:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fe76403ee10 con 0x7fe774073f30 2024-04-26T23:18:57.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.015+0000 7f361affd640 1 -- 172.21.15.47:0/2566093308 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f360c02fc50 con 0x7f3624158d40 2024-04-26T23:18:57.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.015+0000 7fc30d7fa640 1 -- 172.21.15.47:0/4007414522 <== mon.2 v2:172.21.15.203:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fc3100d4500 con 0x7fc3080e3160 2024-04-26T23:18:57.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.015+0000 7f361affd640 1 -- 172.21.15.47:0/2566093308 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f360c004aa0 con 0x7f3624158d40 2024-04-26T23:18:57.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.015+0000 7f6d3bfff640 1 -- 172.21.15.47:0/951585067 <== mon.2 v2:172.21.15.203:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6d4c022070 con 0x7f6d54072fb0 2024-04-26T23:18:57.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.015+0000 7f328dffb640 1 -- 172.21.15.47:0/3691387343 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f32940598b0 con 0x7f32980d1980 2024-04-26T23:18:57.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.015+0000 7ff77affd640 1 -- 172.21.15.47:0/30704947 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff77c01a070 con 0x7ff784034400 2024-04-26T23:18:57.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.016+0000 7ff77affd640 1 --2- 172.21.15.47:0/30704947 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7ff76c06c9c0 0x7ff76c06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:57.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.016+0000 7ff77affd640 1 -- 172.21.15.47:0/30704947 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7ff77c08b940 con 0x7ff784034400 2024-04-26T23:18:57.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.017+0000 7ff789d9b640 1 --2- 172.21.15.47:0/30704947 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7ff76c06c9c0 0x7ff76c06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:57.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.017+0000 7ff789d9b640 1 --2- 172.21.15.47:0/30704947 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7ff76c06c9c0 0x7ff76c06ee80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff78403abf0 tx=0x7ff780009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:57.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.017+0000 7f557affd640 1 -- 172.21.15.47:0/3427278210 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f558c0c4020 con 0x7f55840df220 2024-04-26T23:18:57.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.018+0000 7f557affd640 1 --2- 172.21.15.47:0/3427278210 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f556c06caf0 0x7f556c06efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:57.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.018+0000 7f557affd640 1 -- 172.21.15.47:0/3427278210 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f558c12ab20 con 0x7f55840df220 2024-04-26T23:18:57.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.018+0000 7f558a59c640 1 --2- 172.21.15.47:0/3427278210 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f556c06caf0 0x7f556c06efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:57.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.018+0000 7f558a59c640 1 --2- 172.21.15.47:0/3427278210 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f556c06caf0 0x7f556c06efb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5580000f60 tx=0x7f5580007f00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:57.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.020+0000 7fe76affd640 1 -- 172.21.15.47:0/2728888840 <== mon.2 v2:172.21.15.203:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe774167510 con 0x7fe774073f30 2024-04-26T23:18:57.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.020+0000 7f557affd640 1 -- 172.21.15.47:0/3427278210 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f558c0f8c90 con 0x7f55840df220 2024-04-26T23:18:57.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.023+0000 7ff77affd640 1 -- 172.21.15.47:0/30704947 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff77c0593c0 con 0x7ff784034400 2024-04-26T23:18:57.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.025+0000 7fc30d7fa640 1 -- 172.21.15.47:0/4007414522 <== mon.2 v2:172.21.15.203:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc310090030 con 0x7fc3080e3160 2024-04-26T23:18:57.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.031+0000 7f361affd640 1 -- 172.21.15.47:0/2566093308 <== mon.2 v2:172.21.15.203:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f360c043070 con 0x7f3624158d40 2024-04-26T23:18:57.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.031+0000 7f361affd640 1 --2- 172.21.15.47:0/2566093308 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f360406c8f0 0x7f360406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:57.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.031+0000 7f361affd640 1 -- 172.21.15.47:0/2566093308 <== mon.2 v2:172.21.15.203:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f360c0b0550 con 0x7f3624158d40 2024-04-26T23:18:57.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.032+0000 7f361affd640 1 -- 172.21.15.47:0/2566093308 <== mon.2 v2:172.21.15.203:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f36240ccbf0 con 0x7f3624158d40 2024-04-26T23:18:57.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.032+0000 7f361bfff640 1 --2- 172.21.15.47:0/2566093308 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f360406c8f0 0x7f360406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:57.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.043+0000 7f361bfff640 1 --2- 172.21.15.47:0/2566093308 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f360406c8f0 0x7f360406edb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f36241599b0 tx=0x7f3614008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:57.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.440+0000 7f96a1166640 1 -- 172.21.15.47:0/230842618 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f967c005580 con 0x7f969c070930 2024-04-26T23:18:57.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.441+0000 7f96837fe640 1 -- 172.21.15.47:0/230842618 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f968c059730 con 0x7f969c070930 2024-04-26T23:18:57.443 INFO:teuthology.orchestra.run.smithi047.stdout:55834574858 2024-04-26T23:18:57.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.444+0000 7f329e63e640 1 -- 172.21.15.47:0/3691387343 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f3278005580 con 0x7f32980d1980 2024-04-26T23:18:57.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.444+0000 7f328dffb640 1 -- 172.21.15.47:0/3691387343 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f32940591b0 con 0x7f32980d1980 2024-04-26T23:18:57.446 INFO:teuthology.orchestra.run.smithi047.stdout:60129542156 2024-04-26T23:18:57.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.444+0000 7f96a1166640 1 -- 172.21.15.47:0/230842618 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f968806c9f0 msgr2=0x7f968806eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:57.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.444+0000 7f96a1166640 1 --2- 172.21.15.47:0/230842618 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f968806c9f0 0x7f968806eeb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9690000f60 tx=0x7f9690000bc0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.444+0000 7f96a1166640 1 -- 172.21.15.47:0/230842618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f969c070930 msgr2=0x7f969c0b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:57.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.444+0000 7f96a1166640 1 --2- 172.21.15.47:0/230842618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f969c070930 0x7f969c0b4db0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f968c00b410 tx=0x7f968c00b8d0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.445+0000 7f96a1166640 1 -- 172.21.15.47:0/230842618 shutdown_connections 2024-04-26T23:18:57.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.445+0000 7f96a1166640 1 --2- 172.21.15.47:0/230842618 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f968806c9f0 0x7f968806eeb0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.445+0000 7f96a1166640 1 --2- 172.21.15.47:0/230842618 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f969c15d100 0x7f969c1583d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.445+0000 7f96a1166640 1 --2- 172.21.15.47:0/230842618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f969c070930 0x7f969c0b4db0 secure :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f968c00b410 tx=0x7f968c00b8d0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.445+0000 7f96a1166640 1 --2- 172.21.15.47:0/230842618 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f969c06c950 0x7f969c14b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.445+0000 7f96a1166640 1 -- 172.21.15.47:0/230842618 >> 172.21.15.47:0/230842618 conn(0x7f969c0b39b0 msgr2=0x7f969c073160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:57.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.446+0000 7f96a1166640 1 -- 172.21.15.47:0/230842618 shutdown_connections 2024-04-26T23:18:57.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.446+0000 7f96a1166640 1 -- 172.21.15.47:0/230842618 wait complete. 2024-04-26T23:18:57.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.449+0000 7f329e63e640 1 -- 172.21.15.47:0/3691387343 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f327c06c9c0 msgr2=0x7f327c06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:57.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.449+0000 7f329e63e640 1 --2- 172.21.15.47:0/3691387343 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f327c06c9c0 0x7f327c06ee80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3288034000 tx=0x7f328802c000 comp rx=0 tx=0).stop 2024-04-26T23:18:57.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.449+0000 7f329e63e640 1 -- 172.21.15.47:0/3691387343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32980d1980 msgr2=0x7f32981f8920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:57.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.450+0000 7f329e63e640 1 --2- 172.21.15.47:0/3691387343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32980d1980 0x7f32981f8920 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f329400caf0 tx=0x7f3294002090 comp rx=0 tx=0).stop 2024-04-26T23:18:57.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.450+0000 7f329e63e640 1 -- 172.21.15.47:0/3691387343 shutdown_connections 2024-04-26T23:18:57.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.450+0000 7f329e63e640 1 --2- 172.21.15.47:0/3691387343 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f327c06c9c0 0x7f327c06ee80 secure :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3288034000 tx=0x7f328802c000 comp rx=0 tx=0).stop 2024-04-26T23:18:57.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.450+0000 7f329e63e640 1 --2- 172.21.15.47:0/3691387343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f32980d3570 0x7f32980c8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.450+0000 7f329e63e640 1 --2- 172.21.15.47:0/3691387343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32980d1d60 0x7f32980c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.450+0000 7f329e63e640 1 --2- 172.21.15.47:0/3691387343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32980d1980 0x7f32981f8920 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.450+0000 7f329e63e640 1 -- 172.21.15.47:0/3691387343 >> 172.21.15.47:0/3691387343 conn(0x7f32980d4dd0 msgr2=0x7f3298162480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:57.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.450+0000 7f6d39ffb640 1 -- 172.21.15.47:0/951585067 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f6d540cdc90 con 0x7f6d54072fb0 2024-04-26T23:18:57.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.450+0000 7f329e63e640 1 -- 172.21.15.47:0/3691387343 shutdown_connections 2024-04-26T23:18:57.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.451+0000 7f329e63e640 1 -- 172.21.15.47:0/3691387343 wait complete. 2024-04-26T23:18:57.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.451+0000 7f6d3bfff640 1 -- 172.21.15.47:0/951585067 <== mon.2 v2:172.21.15.203:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6d4c010040 con 0x7f6d54072fb0 2024-04-26T23:18:57.454 INFO:teuthology.orchestra.run.smithi047.stdout:55834574858 2024-04-26T23:18:57.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.454+0000 7f6d59b9d640 1 -- 172.21.15.47:0/951585067 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f6d4006c940 msgr2=0x7f6d4006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:57.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.455+0000 7f6d59b9d640 1 --2- 172.21.15.47:0/951585067 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f6d4006c940 0x7f6d4006ee00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6d4803d000 tx=0x7f6d48008030 comp rx=0 tx=0).stop 2024-04-26T23:18:57.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.455+0000 7f6d59b9d640 1 -- 172.21.15.47:0/951585067 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d54072fb0 msgr2=0x7f6d5415c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:57.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.455+0000 7f6d59b9d640 1 --2- 172.21.15.47:0/951585067 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d54072fb0 0x7f6d5415c4f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c009d10 tx=0x7f6d4c00c490 comp rx=0 tx=0).stop 2024-04-26T23:18:57.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.455+0000 7f6d59b9d640 1 -- 172.21.15.47:0/951585067 shutdown_connections 2024-04-26T23:18:57.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.455+0000 7f6d59b9d640 1 --2- 172.21.15.47:0/951585067 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f6d4006c940 0x7f6d4006ee00 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.455+0000 7f6d59b9d640 1 --2- 172.21.15.47:0/951585067 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d54073390 0x7f6d5415ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.455+0000 7f6d59b9d640 1 --2- 172.21.15.47:0/951585067 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d54072fb0 0x7f6d5415c4f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.455+0000 7f6d59b9d640 1 --2- 172.21.15.47:0/951585067 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d5406fc80 0x7f6d540d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.455+0000 7f6d59b9d640 1 -- 172.21.15.47:0/951585067 >> 172.21.15.47:0/951585067 conn(0x7f6d540b39b0 msgr2=0x7f6d5406c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:57.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.455+0000 7f6d59b9d640 1 -- 172.21.15.47:0/951585067 shutdown_connections 2024-04-26T23:18:57.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.455+0000 7f6d59b9d640 1 -- 172.21.15.47:0/951585067 wait complete. 2024-04-26T23:18:57.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.455+0000 7fe77c359640 1 -- 172.21.15.47:0/2728888840 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fe7740d0430 con 0x7fe774073f30 2024-04-26T23:18:57.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.456+0000 7fe76affd640 1 -- 172.21.15.47:0/2728888840 <== mon.2 v2:172.21.15.203:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe76404b030 con 0x7fe774073f30 2024-04-26T23:18:57.458 INFO:teuthology.orchestra.run.smithi047.stdout:55834574858 2024-04-26T23:18:57.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.459+0000 7fc3171f2640 1 -- 172.21.15.47:0/4007414522 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fc308009450 con 0x7fc3080e3160 2024-04-26T23:18:57.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.459+0000 7fe77c359640 1 -- 172.21.15.47:0/2728888840 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fe75806c940 msgr2=0x7fe75806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:57.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.459+0000 7fe77c359640 1 --2- 172.21.15.47:0/2728888840 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fe75806c940 0x7fe75806ee00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe7740ce4f0 tx=0x7fe76c003990 comp rx=0 tx=0).stop 2024-04-26T23:18:57.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.459+0000 7fe77c359640 1 -- 172.21.15.47:0/2728888840 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe774073f30 msgr2=0x7fe77414e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:57.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.459+0000 7fe77c359640 1 --2- 172.21.15.47:0/2728888840 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe774073f30 0x7fe77414e150 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe764031000 tx=0x7fe7640046c0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.460+0000 7fe77c359640 1 -- 172.21.15.47:0/2728888840 shutdown_connections 2024-04-26T23:18:57.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.460+0000 7fe77c359640 1 --2- 172.21.15.47:0/2728888840 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fe75806c940 0x7fe75806ee00 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.460+0000 7fe77c359640 1 --2- 172.21.15.47:0/2728888840 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe774167720 0x7fe774163c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.460+0000 7fe77c359640 1 --2- 172.21.15.47:0/2728888840 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7740c41d0 0x7fe7740b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.460+0000 7fe77c359640 1 --2- 172.21.15.47:0/2728888840 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe774073f30 0x7fe77414e150 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.460+0000 7fe77c359640 1 -- 172.21.15.47:0/2728888840 >> 172.21.15.47:0/2728888840 conn(0x7fe7740d0be0 msgr2=0x7fe7740b1590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:57.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.460+0000 7fc30d7fa640 1 -- 172.21.15.47:0/4007414522 <== mon.2 v2:172.21.15.203:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc3100d6b60 con 0x7fc3080e3160 2024-04-26T23:18:57.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.460+0000 7fe77c359640 1 -- 172.21.15.47:0/2728888840 shutdown_connections 2024-04-26T23:18:57.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.460+0000 7fe77c359640 1 -- 172.21.15.47:0/2728888840 wait complete. 2024-04-26T23:18:57.463 INFO:teuthology.orchestra.run.smithi047.stdout:60129542155 2024-04-26T23:18:57.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.462+0000 7f558b59e640 1 -- 172.21.15.47:0/3427278210 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f55840dc300 con 0x7f55840df220 2024-04-26T23:18:57.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.462+0000 7f557affd640 1 -- 172.21.15.47:0/3427278210 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f558c0f8590 con 0x7f55840df220 2024-04-26T23:18:57.464 INFO:teuthology.orchestra.run.smithi047.stdout:60129542154 2024-04-26T23:18:57.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.464+0000 7fc3171f2640 1 -- 172.21.15.47:0/4007414522 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fc2f406c940 msgr2=0x7fc2f406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:57.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.464+0000 7fc3171f2640 1 --2- 172.21.15.47:0/4007414522 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fc2f406c940 0x7fc2f406ee00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc304000f60 tx=0x7fc304009040 comp rx=0 tx=0).stop 2024-04-26T23:18:57.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.464+0000 7fc3171f2640 1 -- 172.21.15.47:0/4007414522 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3080e3160 msgr2=0x7fc30804d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:57.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.464+0000 7fc3171f2640 1 --2- 172.21.15.47:0/4007414522 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3080e3160 0x7fc30804d360 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc31006b190 tx=0x7fc310075270 comp rx=0 tx=0).stop 2024-04-26T23:18:57.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.464+0000 7fc3171f2640 1 -- 172.21.15.47:0/4007414522 shutdown_connections 2024-04-26T23:18:57.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.464+0000 7fc3171f2640 1 --2- 172.21.15.47:0/4007414522 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fc2f406c940 0x7fc2f406ee00 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.464+0000 7fc3171f2640 1 --2- 172.21.15.47:0/4007414522 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3080e3160 0x7fc30804d360 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.464+0000 7fc3171f2640 1 --2- 172.21.15.47:0/4007414522 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc30804fbe0 0x7fc3080e57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.464+0000 7fc3171f2640 1 --2- 172.21.15.47:0/4007414522 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc30803aa10 0x7fc30804ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.464+0000 7fc3171f2640 1 -- 172.21.15.47:0/4007414522 >> 172.21.15.47:0/4007414522 conn(0x7fc30800a400 msgr2=0x7fc3080352b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:57.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.465+0000 7ff78b59e640 1 -- 172.21.15.47:0/30704947 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7ff770005850 con 0x7ff784034400 2024-04-26T23:18:57.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.466+0000 7ff77affd640 1 -- 172.21.15.47:0/30704947 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff77c05d6c0 con 0x7ff784034400 2024-04-26T23:18:57.468 INFO:teuthology.orchestra.run.smithi047.stdout:55834574857 2024-04-26T23:18:57.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.466+0000 7f558b59e640 1 -- 172.21.15.47:0/3427278210 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f556c06caf0 msgr2=0x7f556c06efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:57.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.466+0000 7f558b59e640 1 --2- 172.21.15.47:0/3427278210 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f556c06caf0 0x7f556c06efb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5580000f60 tx=0x7f5580007f00 comp rx=0 tx=0).stop 2024-04-26T23:18:57.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.466+0000 7f558b59e640 1 -- 172.21.15.47:0/3427278210 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55840df220 msgr2=0x7f558404dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:57.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.466+0000 7f558b59e640 1 --2- 172.21.15.47:0/3427278210 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55840df220 0x7f558404dd80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f558c06a760 tx=0x7f558c06a0f0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.466+0000 7f558b59e640 1 -- 172.21.15.47:0/3427278210 shutdown_connections 2024-04-26T23:18:57.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.466+0000 7f558b59e640 1 --2- 172.21.15.47:0/3427278210 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f556c06caf0 0x7f556c06efb0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.466+0000 7f558b59e640 1 --2- 172.21.15.47:0/3427278210 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f558404a880 0x7f558404e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.466+0000 7f558b59e640 1 --2- 172.21.15.47:0/3427278210 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55840df220 0x7f558404dd80 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.466+0000 7f558b59e640 1 --2- 172.21.15.47:0/3427278210 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5584053c70 0x7f55840cca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.467+0000 7f558b59e640 1 -- 172.21.15.47:0/3427278210 >> 172.21.15.47:0/3427278210 conn(0x7f5584009fb0 msgr2=0x7f5584033e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:57.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.466+0000 7fc3171f2640 1 -- 172.21.15.47:0/4007414522 shutdown_connections 2024-04-26T23:18:57.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.466+0000 7fc3171f2640 1 -- 172.21.15.47:0/4007414522 wait complete. 2024-04-26T23:18:57.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.467+0000 7f558b59e640 1 -- 172.21.15.47:0/3427278210 shutdown_connections 2024-04-26T23:18:57.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.467+0000 7f558b59e640 1 -- 172.21.15.47:0/3427278210 wait complete. 2024-04-26T23:18:57.471 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.3 2024-04-26T23:18:57.471 DEBUG:teuthology.parallel:result is None 2024-04-26T23:18:57.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.469+0000 7ff78b59e640 1 -- 172.21.15.47:0/30704947 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7ff76c06c9c0 msgr2=0x7ff76c06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:57.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.469+0000 7ff78b59e640 1 --2- 172.21.15.47:0/30704947 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7ff76c06c9c0 0x7ff76c06ee80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff78403abf0 tx=0x7ff780009040 comp rx=0 tx=0).stop 2024-04-26T23:18:57.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.469+0000 7ff78b59e640 1 -- 172.21.15.47:0/30704947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff784034400 msgr2=0x7ff78404d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:57.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.469+0000 7ff78b59e640 1 --2- 172.21.15.47:0/30704947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff784034400 0x7ff78404d7d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff77c00c480 tx=0x7ff77c00c940 comp rx=0 tx=0).stop 2024-04-26T23:18:57.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.470+0000 7ff78b59e640 1 -- 172.21.15.47:0/30704947 shutdown_connections 2024-04-26T23:18:57.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.470+0000 7ff78b59e640 1 --2- 172.21.15.47:0/30704947 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7ff76c06c9c0 0x7ff76c06ee80 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.470+0000 7ff78b59e640 1 --2- 172.21.15.47:0/30704947 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff7840e76e0 0x7ff7840db7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.470+0000 7ff78b59e640 1 --2- 172.21.15.47:0/30704947 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff784050ab0 0x7ff78404b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.470+0000 7ff78b59e640 1 --2- 172.21.15.47:0/30704947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff784034400 0x7ff78404d7d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.470+0000 7ff78b59e640 1 -- 172.21.15.47:0/30704947 >> 172.21.15.47:0/30704947 conn(0x7ff784055350 msgr2=0x7ff784050e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:57.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.470+0000 7ff78b59e640 1 -- 172.21.15.47:0/30704947 shutdown_connections 2024-04-26T23:18:57.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.470+0000 7ff78b59e640 1 -- 172.21.15.47:0/30704947 wait complete. 2024-04-26T23:18:57.473 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542156 got 60129542156 for osd.6 2024-04-26T23:18:57.474 DEBUG:teuthology.parallel:result is None 2024-04-26T23:18:57.478 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.0 2024-04-26T23:18:57.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.476+0000 7f3628af9640 1 -- 172.21.15.47:0/2566093308 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f36240694f0 con 0x7f3624158d40 2024-04-26T23:18:57.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.476+0000 7f361affd640 1 -- 172.21.15.47:0/2566093308 <== mon.2 v2:172.21.15.203:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f360c07e6b0 con 0x7f3624158d40 2024-04-26T23:18:57.478 DEBUG:teuthology.parallel:result is None 2024-04-26T23:18:57.479 INFO:teuthology.orchestra.run.smithi047.stdout:60129542155 2024-04-26T23:18:57.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.480+0000 7f3628af9640 1 -- 172.21.15.47:0/2566093308 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f360406c8f0 msgr2=0x7f360406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:57.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.480+0000 7f3628af9640 1 --2- 172.21.15.47:0/2566093308 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f360406c8f0 0x7f360406edb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f36241599b0 tx=0x7f3614008040 comp rx=0 tx=0).stop 2024-04-26T23:18:57.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.480+0000 7f3628af9640 1 -- 172.21.15.47:0/2566093308 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3624158d40 msgr2=0x7f36241f0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:57.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.480+0000 7f3628af9640 1 --2- 172.21.15.47:0/2566093308 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3624158d40 0x7f36241f0c00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f360c005320 tx=0x7f360c004c40 comp rx=0 tx=0).stop 2024-04-26T23:18:57.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.480+0000 7f3628af9640 1 -- 172.21.15.47:0/2566093308 shutdown_connections 2024-04-26T23:18:57.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.480+0000 7f3628af9640 1 --2- 172.21.15.47:0/2566093308 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f360406c8f0 0x7f360406edb0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.480+0000 7f3628af9640 1 --2- 172.21.15.47:0/2566093308 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36240a0210 0x7f36240b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.480+0000 7f3628af9640 1 --2- 172.21.15.47:0/2566093308 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3624159120 0x7f36241f41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.480+0000 7f3628af9640 1 --2- 172.21.15.47:0/2566093308 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3624158d40 0x7f36241f0c00 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.480+0000 7f3628af9640 1 -- 172.21.15.47:0/2566093308 >> 172.21.15.47:0/2566093308 conn(0x7f36240d0ff0 msgr2=0x7f362406e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:57.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.480+0000 7f3628af9640 1 -- 172.21.15.47:0/2566093308 shutdown_connections 2024-04-26T23:18:57.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.480+0000 7f3628af9640 1 -- 172.21.15.47:0/2566093308 wait complete. 2024-04-26T23:18:57.484 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.2 2024-04-26T23:18:57.484 DEBUG:teuthology.parallel:result is None 2024-04-26T23:18:57.488 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542155 for osd.4 2024-04-26T23:18:57.488 DEBUG:teuthology.parallel:result is None 2024-04-26T23:18:57.489 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542154 for osd.7 2024-04-26T23:18:57.489 DEBUG:teuthology.parallel:result is None 2024-04-26T23:18:57.492 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.1 2024-04-26T23:18:57.492 DEBUG:teuthology.parallel:result is None 2024-04-26T23:18:57.499 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542155 for osd.5 2024-04-26T23:18:57.500 DEBUG:teuthology.parallel:result is None 2024-04-26T23:18:57.500 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-26T23:18:57.500 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T23:18:57.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.640+0000 7f2fc54da640 1 Processor -- start 2024-04-26T23:18:57.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.640+0000 7f2fc54da640 1 -- start start 2024-04-26T23:18:57.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.641+0000 7f2fc54da640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2fc0070750 0x7f2fc00b56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:57.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.641+0000 7f2fc54da640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2fc00af6e0 0x7f2fc00afb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:57.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.641+0000 7f2fc54da640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fc00b5e30 con 0x7f2fc0072fb0 2024-04-26T23:18:57.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.641+0000 7f2fc54da640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f2fc00b5c00 con 0x7f2fc0070750 2024-04-26T23:18:57.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.641+0000 7f2fc54da640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f2fc006f2b0 con 0x7f2fc00af6e0 2024-04-26T23:18:57.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.641+0000 7f2fbeffd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f2fc0072fb0 0x7f2fc0073390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:54630/0 (socket says 172.21.15.47:54630) 2024-04-26T23:18:57.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.641+0000 7f2fbeffd640 1 -- 172.21.15.47:0/2989714255 learned_addr learned my addr 172.21.15.47:0/2989714255 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:18:57.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.641+0000 7f2fbdffb640 1 -- 172.21.15.47:0/2989714255 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2841282141 0 0) 0x7f2fc00b5e30 con 0x7f2fc0072fb0 2024-04-26T23:18:57.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.642+0000 7f2fbe7fc640 1 --2- 172.21.15.47:0/2989714255 >> v2:172.21.15.203:3300/0 conn(0x7f2fc0070750 0x7f2fc00b56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:57.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.642+0000 7f2fbdffb640 1 -- 172.21.15.47:0/2989714255 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2fa8003520 con 0x7f2fc0072fb0 2024-04-26T23:18:57.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.642+0000 7f2fbf7fe640 1 --2- 172.21.15.47:0/2989714255 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2fc00af6e0 0x7f2fc00afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:57.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.642+0000 7f2fbdffb640 1 -- 172.21.15.47:0/2989714255 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 791272064 0 0) 0x7f2fa8003520 con 0x7f2fc0072fb0 2024-04-26T23:18:57.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.642+0000 7f2fbdffb640 1 -- 172.21.15.47:0/2989714255 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2fc00af6e0 msgr2=0x7f2fc00afb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:57.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.642+0000 7f2fbdffb640 1 --2- 172.21.15.47:0/2989714255 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2fc00af6e0 0x7f2fc00afb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.642+0000 7f2fbdffb640 1 -- 172.21.15.47:0/2989714255 >> v2:172.21.15.203:3300/0 conn(0x7f2fc0070750 msgr2=0x7f2fc00b56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:57.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.642+0000 7f2fbdffb640 1 --2- 172.21.15.47:0/2989714255 >> v2:172.21.15.203:3300/0 conn(0x7f2fc0070750 0x7f2fc00b56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.642+0000 7f2fbdffb640 1 -- 172.21.15.47:0/2989714255 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fc00b6110 con 0x7f2fc0072fb0 2024-04-26T23:18:57.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.642+0000 7f2fbdffb640 1 -- 172.21.15.47:0/2989714255 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7f2fb0001670 con 0x7f2fc0072fb0 2024-04-26T23:18:57.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.642+0000 7f2fbdffb640 1 -- 172.21.15.47:0/2989714255 >> v1:172.21.15.47:6789/0 conn(0x7f2fc0072fb0 legacy=0x7f2fc0073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:57.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.642+0000 7f2fbdffb640 1 --2- 172.21.15.47:0/2989714255 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2fa80039b0 0x7f2fa8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:57.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.642+0000 7f2fbdffb640 1 --2- 172.21.15.47:0/2989714255 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2fc00af6e0 0x7f2fa8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:57.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.642+0000 7f2fbdffb640 1 --2- 172.21.15.47:0/2989714255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fa8004d40 0x7f2fa80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:57.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.642+0000 7f2fbdffb640 1 -- 172.21.15.47:0/2989714255 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc006f2b0 con 0x7f2fa8004d40 2024-04-26T23:18:57.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.642+0000 7f2fbdffb640 1 -- 172.21.15.47:0/2989714255 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa800b9c0 con 0x7f2fa80039b0 2024-04-26T23:18:57.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.642+0000 7f2fbdffb640 1 -- 172.21.15.47:0/2989714255 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f2fa800bb40 con 0x7f2fc00af6e0 2024-04-26T23:18:57.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.642+0000 7f2fbeffd640 1 --2- 172.21.15.47:0/2989714255 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2fa80039b0 0x7f2fa8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:57.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.642+0000 7f2fbf7fe640 1 --2- 172.21.15.47:0/2989714255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fa8004d40 0x7f2fa80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:57.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.642+0000 7f2fbe7fc640 1 --2- 172.21.15.47:0/2989714255 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2fc00af6e0 0x7f2fa8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:57.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.642+0000 7f2fbe7fc640 1 -- 172.21.15.47:0/2989714255 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2fa80039b0 msgr2=0x7f2fa8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:57.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.642+0000 7f2fbe7fc640 1 --2- 172.21.15.47:0/2989714255 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2fa80039b0 0x7f2fa8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.642+0000 7f2fbe7fc640 1 -- 172.21.15.47:0/2989714255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fa8004d40 msgr2=0x7f2fa80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:57.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.642+0000 7f2fbe7fc640 1 --2- 172.21.15.47:0/2989714255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fa8004d40 0x7f2fa80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.643+0000 7f2fbe7fc640 1 -- 172.21.15.47:0/2989714255 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fa8003520 con 0x7f2fc00af6e0 2024-04-26T23:18:57.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.643+0000 7f2fbf7fe640 1 --2- 172.21.15.47:0/2989714255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fa8004d40 0x7f2fa80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:18:57.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.643+0000 7f2fbeffd640 1 --2- 172.21.15.47:0/2989714255 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2fa80039b0 0x7f2fa8003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:18:57.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.643+0000 7f2fbe7fc640 1 --2- 172.21.15.47:0/2989714255 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2fc00af6e0 0x7f2fa8004800 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4009ac0 tx=0x7f2fa8005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82c5bb57fc08811f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:57.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.643+0000 7f2fbdffb640 1 -- 172.21.15.47:0/2989714255 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb4036070 con 0x7f2fc00af6e0 2024-04-26T23:18:57.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.644+0000 7f2fbdffb640 1 -- 172.21.15.47:0/2989714255 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fb4031400 con 0x7f2fc00af6e0 2024-04-26T23:18:57.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.644+0000 7f2fbdffb640 1 -- 172.21.15.47:0/2989714255 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb4009690 con 0x7f2fc00af6e0 2024-04-26T23:18:57.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.644+0000 7f2fc54da640 1 -- 172.21.15.47:0/2989714255 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2fc00af6e0 msgr2=0x7f2fa8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:57.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.644+0000 7f2fc54da640 1 --2- 172.21.15.47:0/2989714255 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2fc00af6e0 0x7f2fa8004800 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4009ac0 tx=0x7f2fa8005830 comp rx=0 tx=0).stop 2024-04-26T23:18:57.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.644+0000 7f2fbf7fe640 1 -- 172.21.15.47:0/2989714255 reap_dead start 2024-04-26T23:18:57.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.644+0000 7f2fc54da640 1 -- 172.21.15.47:0/2989714255 shutdown_connections 2024-04-26T23:18:57.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.644+0000 7f2fc54da640 1 -- 172.21.15.47:0/2989714255 >> 172.21.15.47:0/2989714255 conn(0x7f2fc00b39b0 msgr2=0x7f2fc00b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:57.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.644+0000 7f2fc54da640 1 -- 172.21.15.47:0/2989714255 shutdown_connections 2024-04-26T23:18:57.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.644+0000 7f2fc54da640 1 -- 172.21.15.47:0/2989714255 wait complete. 2024-04-26T23:18:57.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.646+0000 7f2fc54da640 1 Processor -- start 2024-04-26T23:18:57.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.646+0000 7f2fc54da640 1 -- start start 2024-04-26T23:18:57.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.646+0000 7f2fc54da640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2fc00af6e0 0x7f2fc00c7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:57.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.646+0000 7f2fc54da640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fc009ec60 0x7f2fc0157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:57.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.646+0000 7f2fc54da640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2fc01f4ce0 0x7f2fc01f5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:57.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.646+0000 7f2fc54da640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc015c2e0 con 0x7f2fc009ec60 2024-04-26T23:18:57.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.646+0000 7f2fc54da640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc015c160 con 0x7f2fc00af6e0 2024-04-26T23:18:57.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.646+0000 7f2fc54da640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f2fc006c240 con 0x7f2fc01f4ce0 2024-04-26T23:18:57.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.646+0000 7f2fbe7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fc009ec60 0x7f2fc0157e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:57.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.647+0000 7f2fbe7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fc009ec60 0x7f2fc0157e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58852/0 (socket says 172.21.15.47:58852) 2024-04-26T23:18:57.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.647+0000 7f2fbe7fc640 1 -- 172.21.15.47:0/1783857931 learned_addr learned my addr 172.21.15.47:0/1783857931 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:57.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.647+0000 7f2fbeffd640 1 --2- 172.21.15.47:0/1783857931 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2fc00af6e0 0x7f2fc00c7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:57.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.647+0000 7f2fbf7fe640 1 --2- 172.21.15.47:0/1783857931 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2fc01f4ce0 0x7f2fc01f5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:57.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.647+0000 7f2fbe7fc640 1 -- 172.21.15.47:0/1783857931 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2fc01f4ce0 msgr2=0x7f2fc01f5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:57.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.647+0000 7f2fbe7fc640 1 --2- 172.21.15.47:0/1783857931 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2fc01f4ce0 0x7f2fc01f5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.647+0000 7f2fbe7fc640 1 -- 172.21.15.47:0/1783857931 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2fc00af6e0 msgr2=0x7f2fc00c7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:57.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.647+0000 7f2fbe7fc640 1 --2- 172.21.15.47:0/1783857931 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2fc00af6e0 0x7f2fc00c7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.647+0000 7f2fbe7fc640 1 -- 172.21.15.47:0/1783857931 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fc00c8ec0 con 0x7f2fc009ec60 2024-04-26T23:18:57.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.647+0000 7f2fbe7fc640 1 --2- 172.21.15.47:0/1783857931 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fc009ec60 0x7f2fc0157e30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4000c00 tx=0x7f2fb4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:57.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.647+0000 7f2fa77fe640 1 -- 172.21.15.47:0/1783857931 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb4040020 con 0x7f2fc009ec60 2024-04-26T23:18:57.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.647+0000 7f2fa77fe640 1 -- 172.21.15.47:0/1783857931 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fb4003b60 con 0x7f2fc009ec60 2024-04-26T23:18:57.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.647+0000 7f2fa77fe640 1 -- 172.21.15.47:0/1783857931 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb4036040 con 0x7f2fc009ec60 2024-04-26T23:18:57.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.647+0000 7f2fc54da640 1 -- 172.21.15.47:0/1783857931 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fc00c7b70 con 0x7f2fc009ec60 2024-04-26T23:18:57.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.647+0000 7f2fc54da640 1 -- 172.21.15.47:0/1783857931 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fc00c8120 con 0x7f2fc009ec60 2024-04-26T23:18:57.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.648+0000 7f2fc54da640 1 -- 172.21.15.47:0/1783857931 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fc00694f0 con 0x7f2fc009ec60 2024-04-26T23:18:57.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.649+0000 7f2fa77fe640 1 -- 172.21.15.47:0/1783857931 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2fb4043020 con 0x7f2fc009ec60 2024-04-26T23:18:57.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.649+0000 7f2fa77fe640 1 --2- 172.21.15.47:0/1783857931 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f2fac06c920 0x7f2fac06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:57.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.650+0000 7f2fa77fe640 1 -- 172.21.15.47:0/1783857931 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f2fb40ac010 con 0x7f2fc009ec60 2024-04-26T23:18:57.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.650+0000 7f2fbeffd640 1 --2- 172.21.15.47:0/1783857931 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f2fac06c920 0x7f2fac06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:57.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.650+0000 7f2fbeffd640 1 --2- 172.21.15.47:0/1783857931 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f2fac06c920 0x7f2fac06ede0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0001eb0 tx=0x7f2fb0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:57.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.654+0000 7f2fa77fe640 1 -- 172.21.15.47:0/1783857931 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2fb404a070 con 0x7f2fc009ec60 2024-04-26T23:18:57.721 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:18:57.724 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:18:57.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.840+0000 7f2fc54da640 1 -- 172.21.15.47:0/1783857931 --> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2fc015b270 con 0x7f2fac06c920 2024-04-26T23:18:57.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.849+0000 7f2fa77fe640 1 -- 172.21.15.47:0/1783857931 <== mgr.4099 v2:172.21.15.203:6832/992162732 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332529 (secure 0 0 0) 0x7f2fc015b270 con 0x7f2fac06c920 2024-04-26T23:18:57.854 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:18:57.856 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-26T23:18:57.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.852+0000 7f2fc54da640 1 -- 172.21.15.47:0/1783857931 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f2fac06c920 msgr2=0x7f2fac06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:57.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.852+0000 7f2fc54da640 1 --2- 172.21.15.47:0/1783857931 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f2fac06c920 0x7f2fac06ede0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0001eb0 tx=0x7f2fb0009040 comp rx=0 tx=0).stop 2024-04-26T23:18:57.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.852+0000 7f2fc54da640 1 -- 172.21.15.47:0/1783857931 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fc009ec60 msgr2=0x7f2fc0157e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:57.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.852+0000 7f2fc54da640 1 --2- 172.21.15.47:0/1783857931 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fc009ec60 0x7f2fc0157e30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4000c00 tx=0x7f2fb4004730 comp rx=0 tx=0).stop 2024-04-26T23:18:57.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.852+0000 7f2fc54da640 1 -- 172.21.15.47:0/1783857931 shutdown_connections 2024-04-26T23:18:57.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.852+0000 7f2fc54da640 1 --2- 172.21.15.47:0/1783857931 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f2fac06c920 0x7f2fac06ede0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.852+0000 7f2fc54da640 1 --2- 172.21.15.47:0/1783857931 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2fc01f4ce0 0x7f2fc01f5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.852+0000 7f2fc54da640 1 --2- 172.21.15.47:0/1783857931 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fc009ec60 0x7f2fc0157e30 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.852+0000 7f2fc54da640 1 --2- 172.21.15.47:0/1783857931 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2fc00af6e0 0x7f2fc00c7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:57.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.852+0000 7f2fc54da640 1 -- 172.21.15.47:0/1783857931 >> 172.21.15.47:0/1783857931 conn(0x7f2fc00b39b0 msgr2=0x7f2fc006c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:57.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.852+0000 7f2fc54da640 1 -- 172.21.15.47:0/1783857931 shutdown_connections 2024-04-26T23:18:57.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:57.852+0000 7f2fc54da640 1 -- 172.21.15.47:0/1783857931 wait complete. 2024-04-26T23:18:57.866 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2024-04-26T23:18:56.929714+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":231832,"kb_used_data":3176,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754742888,"statfs":{"total":773094113280,"available":772856717312,"internally_reserved":0,"allocated":3252224,"data_stored":1983104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":234016304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003411"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044475+0000","last_change":"2024-04-26T23:18:31.011895+0000","last_active":"2024-04-26T23:18:37.044475+0000","last_peered":"2024-04-26T23:18:37.044475+0000","last_clean":"2024-04-26T23:18:37.044475+0000","last_became_active":"2024-04-26T23:18:31.010976+0000","last_became_peered":"2024-04-26T23:18:31.010976+0000","last_unstale":"2024-04-26T23:18:37.044475+0000","last_undegraded":"2024-04-26T23:18:37.044475+0000","last_fullsized":"2024-04-26T23:18:37.044475+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:57:14.165871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030065+0000","last_change":"2024-04-26T23:18:32.020612+0000","last_active":"2024-04-26T23:18:37.030065+0000","last_peered":"2024-04-26T23:18:37.030065+0000","last_clean":"2024-04-26T23:18:37.030065+0000","last_became_active":"2024-04-26T23:18:32.018903+0000","last_became_peered":"2024-04-26T23:18:32.018903+0000","last_unstale":"2024-04-26T23:18:37.030065+0000","last_undegraded":"2024-04-26T23:18:37.030065+0000","last_fullsized":"2024-04-26T23:18:37.030065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:55:55.562630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029701+0000","last_change":"2024-04-26T23:18:36.051662+0000","last_active":"2024-04-26T23:18:37.029701+0000","last_peered":"2024-04-26T23:18:37.029701+0000","last_clean":"2024-04-26T23:18:37.029701+0000","last_became_active":"2024-04-26T23:18:36.051204+0000","last_became_peered":"2024-04-26T23:18:36.051204+0000","last_unstale":"2024-04-26T23:18:37.029701+0000","last_undegraded":"2024-04-26T23:18:37.029701+0000","last_fullsized":"2024-04-26T23:18:37.029701+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:24:24.209562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030279+0000","last_change":"2024-04-26T23:18:31.008954+0000","last_active":"2024-04-26T23:18:37.030279+0000","last_peered":"2024-04-26T23:18:37.030279+0000","last_clean":"2024-04-26T23:18:37.030279+0000","last_became_active":"2024-04-26T23:18:31.008597+0000","last_became_peered":"2024-04-26T23:18:31.008597+0000","last_unstale":"2024-04-26T23:18:37.030279+0000","last_undegraded":"2024-04-26T23:18:37.030279+0000","last_fullsized":"2024-04-26T23:18:37.030279+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:16:07.026710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044341+0000","last_change":"2024-04-26T23:18:32.019526+0000","last_active":"2024-04-26T23:18:37.044341+0000","last_peered":"2024-04-26T23:18:37.044341+0000","last_clean":"2024-04-26T23:18:37.044341+0000","last_became_active":"2024-04-26T23:18:32.019264+0000","last_became_peered":"2024-04-26T23:18:32.019264+0000","last_unstale":"2024-04-26T23:18:37.044341+0000","last_undegraded":"2024-04-26T23:18:37.044341+0000","last_fullsized":"2024-04-26T23:18:37.044341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:37:21.828417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044512+0000","last_change":"2024-04-26T23:18:36.041188+0000","last_active":"2024-04-26T23:18:37.044512+0000","last_peered":"2024-04-26T23:18:37.044512+0000","last_clean":"2024-04-26T23:18:37.044512+0000","last_became_active":"2024-04-26T23:18:36.040788+0000","last_became_peered":"2024-04-26T23:18:36.040788+0000","last_unstale":"2024-04-26T23:18:37.044512+0000","last_undegraded":"2024-04-26T23:18:37.044512+0000","last_fullsized":"2024-04-26T23:18:37.044512+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:49:51.239332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.047074+0000","last_change":"2024-04-26T23:18:31.011094+0000","last_active":"2024-04-26T23:18:37.047074+0000","last_peered":"2024-04-26T23:18:37.047074+0000","last_clean":"2024-04-26T23:18:37.047074+0000","last_became_active":"2024-04-26T23:18:31.010756+0000","last_became_peered":"2024-04-26T23:18:31.010756+0000","last_unstale":"2024-04-26T23:18:37.047074+0000","last_undegraded":"2024-04-26T23:18:37.047074+0000","last_fullsized":"2024-04-26T23:18:37.047074+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:28.986242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044870+0000","last_change":"2024-04-26T23:18:32.019790+0000","last_active":"2024-04-26T23:18:37.044870+0000","last_peered":"2024-04-26T23:18:37.044870+0000","last_clean":"2024-04-26T23:18:37.044870+0000","last_became_active":"2024-04-26T23:18:32.019535+0000","last_became_peered":"2024-04-26T23:18:32.019535+0000","last_unstale":"2024-04-26T23:18:37.044870+0000","last_undegraded":"2024-04-26T23:18:37.044870+0000","last_fullsized":"2024-04-26T23:18:37.044870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:10.370646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045799+0000","last_change":"2024-04-26T23:18:36.046954+0000","last_active":"2024-04-26T23:18:37.045799+0000","last_peered":"2024-04-26T23:18:37.045799+0000","last_clean":"2024-04-26T23:18:37.045799+0000","last_became_active":"2024-04-26T23:18:36.046588+0000","last_became_peered":"2024-04-26T23:18:36.046588+0000","last_unstale":"2024-04-26T23:18:37.045799+0000","last_undegraded":"2024-04-26T23:18:37.045799+0000","last_fullsized":"2024-04-26T23:18:37.045799+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:48:46.371866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046759+0000","last_change":"2024-04-26T23:18:31.011351+0000","last_active":"2024-04-26T23:18:37.046759+0000","last_peered":"2024-04-26T23:18:37.046759+0000","last_clean":"2024-04-26T23:18:37.046759+0000","last_became_active":"2024-04-26T23:18:31.010990+0000","last_became_peered":"2024-04-26T23:18:31.010990+0000","last_unstale":"2024-04-26T23:18:37.046759+0000","last_undegraded":"2024-04-26T23:18:37.046759+0000","last_fullsized":"2024-04-26T23:18:37.046759+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:08.243628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046200+0000","last_change":"2024-04-26T23:18:32.018999+0000","last_active":"2024-04-26T23:18:37.046200+0000","last_peered":"2024-04-26T23:18:37.046200+0000","last_clean":"2024-04-26T23:18:37.046200+0000","last_became_active":"2024-04-26T23:18:32.018713+0000","last_became_peered":"2024-04-26T23:18:32.018713+0000","last_unstale":"2024-04-26T23:18:37.046200+0000","last_undegraded":"2024-04-26T23:18:37.046200+0000","last_fullsized":"2024-04-26T23:18:37.046200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:39:01.950309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045585+0000","last_change":"2024-04-26T23:18:36.042244+0000","last_active":"2024-04-26T23:18:37.045585+0000","last_peered":"2024-04-26T23:18:37.045585+0000","last_clean":"2024-04-26T23:18:37.045585+0000","last_became_active":"2024-04-26T23:18:36.041833+0000","last_became_peered":"2024-04-26T23:18:36.041833+0000","last_unstale":"2024-04-26T23:18:37.045585+0000","last_undegraded":"2024-04-26T23:18:37.045585+0000","last_fullsized":"2024-04-26T23:18:37.045585+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:39:05.974905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045365+0000","last_change":"2024-04-26T23:18:32.017614+0000","last_active":"2024-04-26T23:18:37.045365+0000","last_peered":"2024-04-26T23:18:37.045365+0000","last_clean":"2024-04-26T23:18:37.045365+0000","last_became_active":"2024-04-26T23:18:32.017251+0000","last_became_peered":"2024-04-26T23:18:32.017251+0000","last_unstale":"2024-04-26T23:18:37.045365+0000","last_undegraded":"2024-04-26T23:18:37.045365+0000","last_fullsized":"2024-04-26T23:18:37.045365+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:37:21.828417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.047093+0000","last_change":"2024-04-26T23:18:31.011768+0000","last_active":"2024-04-26T23:18:37.047093+0000","last_peered":"2024-04-26T23:18:37.047093+0000","last_clean":"2024-04-26T23:18:37.047093+0000","last_became_active":"2024-04-26T23:18:31.011495+0000","last_became_peered":"2024-04-26T23:18:31.011495+0000","last_unstale":"2024-04-26T23:18:37.047093+0000","last_undegraded":"2024-04-26T23:18:37.047093+0000","last_fullsized":"2024-04-26T23:18:37.047093+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:25:38.515775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045600+0000","last_change":"2024-04-26T23:18:36.039079+0000","last_active":"2024-04-26T23:18:37.045600+0000","last_peered":"2024-04-26T23:18:37.045600+0000","last_clean":"2024-04-26T23:18:37.045600+0000","last_became_active":"2024-04-26T23:18:36.038665+0000","last_became_peered":"2024-04-26T23:18:36.038665+0000","last_unstale":"2024-04-26T23:18:37.045600+0000","last_undegraded":"2024-04-26T23:18:37.045600+0000","last_fullsized":"2024-04-26T23:18:37.045600+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:45:26.386750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029982+0000","last_change":"2024-04-26T23:18:32.012705+0000","last_active":"2024-04-26T23:18:37.029982+0000","last_peered":"2024-04-26T23:18:37.029982+0000","last_clean":"2024-04-26T23:18:37.029982+0000","last_became_active":"2024-04-26T23:18:32.012421+0000","last_became_peered":"2024-04-26T23:18:32.012421+0000","last_unstale":"2024-04-26T23:18:37.029982+0000","last_undegraded":"2024-04-26T23:18:37.029982+0000","last_fullsized":"2024-04-26T23:18:37.029982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:49:47.214737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030444+0000","last_change":"2024-04-26T23:18:31.011273+0000","last_active":"2024-04-26T23:18:37.030444+0000","last_peered":"2024-04-26T23:18:37.030444+0000","last_clean":"2024-04-26T23:18:37.030444+0000","last_became_active":"2024-04-26T23:18:31.010978+0000","last_became_peered":"2024-04-26T23:18:31.010978+0000","last_unstale":"2024-04-26T23:18:37.030444+0000","last_undegraded":"2024-04-26T23:18:37.030444+0000","last_fullsized":"2024-04-26T23:18:37.030444+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:13.713713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045147+0000","last_change":"2024-04-26T23:18:36.040606+0000","last_active":"2024-04-26T23:18:37.045147+0000","last_peered":"2024-04-26T23:18:37.045147+0000","last_clean":"2024-04-26T23:18:37.045147+0000","last_became_active":"2024-04-26T23:18:36.040058+0000","last_became_peered":"2024-04-26T23:18:36.040058+0000","last_unstale":"2024-04-26T23:18:37.045147+0000","last_undegraded":"2024-04-26T23:18:37.045147+0000","last_fullsized":"2024-04-26T23:18:37.045147+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:48:55.902259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030935+0000","last_change":"2024-04-26T23:18:32.015298+0000","last_active":"2024-04-26T23:18:37.030935+0000","last_peered":"2024-04-26T23:18:37.030935+0000","last_clean":"2024-04-26T23:18:37.030935+0000","last_became_active":"2024-04-26T23:18:32.014508+0000","last_became_peered":"2024-04-26T23:18:32.014508+0000","last_unstale":"2024-04-26T23:18:37.030935+0000","last_undegraded":"2024-04-26T23:18:37.030935+0000","last_fullsized":"2024-04-26T23:18:37.030935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:57:02.596299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029055+0000","last_change":"2024-04-26T23:18:31.008495+0000","last_active":"2024-04-26T23:18:37.029055+0000","last_peered":"2024-04-26T23:18:37.029055+0000","last_clean":"2024-04-26T23:18:37.029055+0000","last_became_active":"2024-04-26T23:18:31.008116+0000","last_became_peered":"2024-04-26T23:18:31.008116+0000","last_unstale":"2024-04-26T23:18:37.029055+0000","last_undegraded":"2024-04-26T23:18:37.029055+0000","last_fullsized":"2024-04-26T23:18:37.029055+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:34:02.747044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030423+0000","last_change":"2024-04-26T23:18:36.045871+0000","last_active":"2024-04-26T23:18:37.030423+0000","last_peered":"2024-04-26T23:18:37.030423+0000","last_clean":"2024-04-26T23:18:37.030423+0000","last_became_active":"2024-04-26T23:18:36.044888+0000","last_became_peered":"2024-04-26T23:18:36.044888+0000","last_unstale":"2024-04-26T23:18:37.030423+0000","last_undegraded":"2024-04-26T23:18:37.030423+0000","last_fullsized":"2024-04-26T23:18:37.030423+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:08:51.442254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045209+0000","last_change":"2024-04-26T23:18:32.015241+0000","last_active":"2024-04-26T23:18:37.045209+0000","last_peered":"2024-04-26T23:18:37.045209+0000","last_clean":"2024-04-26T23:18:37.045209+0000","last_became_active":"2024-04-26T23:18:32.013246+0000","last_became_peered":"2024-04-26T23:18:32.013246+0000","last_unstale":"2024-04-26T23:18:37.045209+0000","last_undegraded":"2024-04-26T23:18:37.045209+0000","last_fullsized":"2024-04-26T23:18:37.045209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:21.586522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045696+0000","last_change":"2024-04-26T23:18:31.015485+0000","last_active":"2024-04-26T23:18:37.045696+0000","last_peered":"2024-04-26T23:18:37.045696+0000","last_clean":"2024-04-26T23:18:37.045696+0000","last_became_active":"2024-04-26T23:18:31.015187+0000","last_became_peered":"2024-04-26T23:18:31.015187+0000","last_unstale":"2024-04-26T23:18:37.045696+0000","last_undegraded":"2024-04-26T23:18:37.045696+0000","last_fullsized":"2024-04-26T23:18:37.045696+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:53:15.812510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044493+0000","last_change":"2024-04-26T23:18:36.040493+0000","last_active":"2024-04-26T23:18:37.044493+0000","last_peered":"2024-04-26T23:18:37.044493+0000","last_clean":"2024-04-26T23:18:37.044493+0000","last_became_active":"2024-04-26T23:18:36.040040+0000","last_became_peered":"2024-04-26T23:18:36.040040+0000","last_unstale":"2024-04-26T23:18:37.044493+0000","last_undegraded":"2024-04-26T23:18:37.044493+0000","last_fullsized":"2024-04-26T23:18:37.044493+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:37:25.853012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044514+0000","last_change":"2024-04-26T23:18:32.001467+0000","last_active":"2024-04-26T23:18:37.044514+0000","last_peered":"2024-04-26T23:18:37.044514+0000","last_clean":"2024-04-26T23:18:37.044514+0000","last_became_active":"2024-04-26T23:18:32.001170+0000","last_became_peered":"2024-04-26T23:18:32.001170+0000","last_unstale":"2024-04-26T23:18:37.044514+0000","last_undegraded":"2024-04-26T23:18:37.044514+0000","last_fullsized":"2024-04-26T23:18:37.044514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:17:04.527690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046567+0000","last_change":"2024-04-26T23:18:31.014468+0000","last_active":"2024-04-26T23:18:37.046567+0000","last_peered":"2024-04-26T23:18:37.046567+0000","last_clean":"2024-04-26T23:18:37.046567+0000","last_became_active":"2024-04-26T23:18:31.014192+0000","last_became_peered":"2024-04-26T23:18:31.014192+0000","last_unstale":"2024-04-26T23:18:37.046567+0000","last_undegraded":"2024-04-26T23:18:37.046567+0000","last_fullsized":"2024-04-26T23:18:37.046567+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:20.581173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044988+0000","last_change":"2024-04-26T23:18:36.044805+0000","last_active":"2024-04-26T23:18:37.044988+0000","last_peered":"2024-04-26T23:18:37.044988+0000","last_clean":"2024-04-26T23:18:37.044988+0000","last_became_active":"2024-04-26T23:18:36.044409+0000","last_became_peered":"2024-04-26T23:18:36.044409+0000","last_unstale":"2024-04-26T23:18:37.044988+0000","last_undegraded":"2024-04-26T23:18:37.044988+0000","last_fullsized":"2024-04-26T23:18:37.044988+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:04:17.836184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030862+0000","last_change":"2024-04-26T23:18:32.018289+0000","last_active":"2024-04-26T23:18:37.030862+0000","last_peered":"2024-04-26T23:18:37.030862+0000","last_clean":"2024-04-26T23:18:37.030862+0000","last_became_active":"2024-04-26T23:18:32.017788+0000","last_became_peered":"2024-04-26T23:18:32.017788+0000","last_unstale":"2024-04-26T23:18:37.030862+0000","last_undegraded":"2024-04-26T23:18:37.030862+0000","last_fullsized":"2024-04-26T23:18:37.030862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:55.832329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030196+0000","last_change":"2024-04-26T23:18:31.009677+0000","last_active":"2024-04-26T23:18:37.030196+0000","last_peered":"2024-04-26T23:18:37.030196+0000","last_clean":"2024-04-26T23:18:37.030196+0000","last_became_active":"2024-04-26T23:18:31.009368+0000","last_became_peered":"2024-04-26T23:18:31.009368+0000","last_unstale":"2024-04-26T23:18:37.030196+0000","last_undegraded":"2024-04-26T23:18:37.030196+0000","last_fullsized":"2024-04-26T23:18:37.030196+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:28.986242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029507+0000","last_change":"2024-04-26T23:18:36.050766+0000","last_active":"2024-04-26T23:18:37.029507+0000","last_peered":"2024-04-26T23:18:37.029507+0000","last_clean":"2024-04-26T23:18:37.029507+0000","last_became_active":"2024-04-26T23:18:36.050091+0000","last_became_peered":"2024-04-26T23:18:36.050091+0000","last_unstale":"2024-04-26T23:18:37.029507+0000","last_undegraded":"2024-04-26T23:18:37.029507+0000","last_fullsized":"2024-04-26T23:18:37.029507+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:16:16.544972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029722+0000","last_change":"2024-04-26T23:18:32.007502+0000","last_active":"2024-04-26T23:18:37.029722+0000","last_peered":"2024-04-26T23:18:37.029722+0000","last_clean":"2024-04-26T23:18:37.029722+0000","last_became_active":"2024-04-26T23:18:32.006451+0000","last_became_peered":"2024-04-26T23:18:32.006451+0000","last_unstale":"2024-04-26T23:18:37.029722+0000","last_undegraded":"2024-04-26T23:18:37.029722+0000","last_fullsized":"2024-04-26T23:18:37.029722+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:55:55.562630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.047238+0000","last_change":"2024-04-26T23:18:31.012702+0000","last_active":"2024-04-26T23:18:37.047238+0000","last_peered":"2024-04-26T23:18:37.047238+0000","last_clean":"2024-04-26T23:18:37.047238+0000","last_became_active":"2024-04-26T23:18:31.012425+0000","last_became_peered":"2024-04-26T23:18:31.012425+0000","last_unstale":"2024-04-26T23:18:37.047238+0000","last_undegraded":"2024-04-26T23:18:37.047238+0000","last_fullsized":"2024-04-26T23:18:37.047238+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:08:31.793547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044060+0000","last_change":"2024-04-26T23:18:36.049119+0000","last_active":"2024-04-26T23:18:37.044060+0000","last_peered":"2024-04-26T23:18:37.044060+0000","last_clean":"2024-04-26T23:18:37.044060+0000","last_became_active":"2024-04-26T23:18:36.047621+0000","last_became_peered":"2024-04-26T23:18:36.047621+0000","last_unstale":"2024-04-26T23:18:37.044060+0000","last_undegraded":"2024-04-26T23:18:37.044060+0000","last_fullsized":"2024-04-26T23:18:37.044060+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:48:46.371866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046280+0000","last_change":"2024-04-26T23:18:32.021370+0000","last_active":"2024-04-26T23:18:37.046280+0000","last_peered":"2024-04-26T23:18:37.046280+0000","last_clean":"2024-04-26T23:18:37.046280+0000","last_became_active":"2024-04-26T23:18:32.020816+0000","last_became_peered":"2024-04-26T23:18:32.020816+0000","last_unstale":"2024-04-26T23:18:37.046280+0000","last_undegraded":"2024-04-26T23:18:37.046280+0000","last_fullsized":"2024-04-26T23:18:37.046280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:45:22.362155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029882+0000","last_change":"2024-04-26T23:18:31.007687+0000","last_active":"2024-04-26T23:18:37.029882+0000","last_peered":"2024-04-26T23:18:37.029882+0000","last_clean":"2024-04-26T23:18:37.029882+0000","last_became_active":"2024-04-26T23:18:31.006664+0000","last_became_peered":"2024-04-26T23:18:31.006664+0000","last_unstale":"2024-04-26T23:18:37.029882+0000","last_undegraded":"2024-04-26T23:18:37.029882+0000","last_fullsized":"2024-04-26T23:18:37.029882+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:44.844711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029232+0000","last_change":"2024-04-26T23:18:36.047590+0000","last_active":"2024-04-26T23:18:37.029232+0000","last_peered":"2024-04-26T23:18:37.029232+0000","last_clean":"2024-04-26T23:18:37.029232+0000","last_became_active":"2024-04-26T23:18:36.046796+0000","last_became_peered":"2024-04-26T23:18:36.046796+0000","last_unstale":"2024-04-26T23:18:37.029232+0000","last_undegraded":"2024-04-26T23:18:37.029232+0000","last_fullsized":"2024-04-26T23:18:37.029232+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:09.320477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046041+0000","last_change":"2024-04-26T23:18:32.002280+0000","last_active":"2024-04-26T23:18:37.046041+0000","last_peered":"2024-04-26T23:18:37.046041+0000","last_clean":"2024-04-26T23:18:37.046041+0000","last_became_active":"2024-04-26T23:18:32.002051+0000","last_became_peered":"2024-04-26T23:18:32.002051+0000","last_unstale":"2024-04-26T23:18:37.046041+0000","last_undegraded":"2024-04-26T23:18:37.046041+0000","last_fullsized":"2024-04-26T23:18:37.046041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:09.248977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044993+0000","last_change":"2024-04-26T23:18:31.011488+0000","last_active":"2024-04-26T23:18:37.044993+0000","last_peered":"2024-04-26T23:18:37.044993+0000","last_clean":"2024-04-26T23:18:37.044993+0000","last_became_active":"2024-04-26T23:18:31.010229+0000","last_became_peered":"2024-04-26T23:18:31.010229+0000","last_unstale":"2024-04-26T23:18:37.044993+0000","last_undegraded":"2024-04-26T23:18:37.044993+0000","last_fullsized":"2024-04-26T23:18:37.044993+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:25:38.515775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029825+0000","last_change":"2024-04-26T23:18:36.042997+0000","last_active":"2024-04-26T23:18:37.029825+0000","last_peered":"2024-04-26T23:18:37.029825+0000","last_clean":"2024-04-26T23:18:37.029825+0000","last_became_active":"2024-04-26T23:18:36.042510+0000","last_became_peered":"2024-04-26T23:18:36.042510+0000","last_unstale":"2024-04-26T23:18:37.029825+0000","last_undegraded":"2024-04-26T23:18:37.029825+0000","last_fullsized":"2024-04-26T23:18:37.029825+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:30.558000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.028944+0000","last_change":"2024-04-26T23:18:32.008615+0000","last_active":"2024-04-26T23:18:37.028944+0000","last_peered":"2024-04-26T23:18:37.028944+0000","last_clean":"2024-04-26T23:18:37.028944+0000","last_became_active":"2024-04-26T23:18:32.006670+0000","last_became_peered":"2024-04-26T23:18:32.006670+0000","last_unstale":"2024-04-26T23:18:37.028944+0000","last_undegraded":"2024-04-26T23:18:37.028944+0000","last_fullsized":"2024-04-26T23:18:37.028944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:54:25.468799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.027917+0000","last_change":"2024-04-26T23:18:31.005153+0000","last_active":"2024-04-26T23:18:37.027917+0000","last_peered":"2024-04-26T23:18:37.027917+0000","last_clean":"2024-04-26T23:18:37.027917+0000","last_became_active":"2024-04-26T23:18:31.004670+0000","last_became_peered":"2024-04-26T23:18:31.004670+0000","last_unstale":"2024-04-26T23:18:37.027917+0000","last_undegraded":"2024-04-26T23:18:37.027917+0000","last_fullsized":"2024-04-26T23:18:37.027917+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:34:02.747044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030347+0000","last_change":"2024-04-26T23:18:36.048181+0000","last_active":"2024-04-26T23:18:37.030347+0000","last_peered":"2024-04-26T23:18:37.030347+0000","last_clean":"2024-04-26T23:18:37.030347+0000","last_became_active":"2024-04-26T23:18:36.047762+0000","last_became_peered":"2024-04-26T23:18:36.047762+0000","last_unstale":"2024-04-26T23:18:37.030347+0000","last_undegraded":"2024-04-26T23:18:37.030347+0000","last_fullsized":"2024-04-26T23:18:37.030347+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:39:32.810976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.043710+0000","last_change":"2024-04-26T23:18:32.005125+0000","last_active":"2024-04-26T23:18:37.043710+0000","last_peered":"2024-04-26T23:18:37.043710+0000","last_clean":"2024-04-26T23:18:37.043710+0000","last_became_active":"2024-04-26T23:18:32.004826+0000","last_became_peered":"2024-04-26T23:18:32.004826+0000","last_unstale":"2024-04-26T23:18:37.043710+0000","last_undegraded":"2024-04-26T23:18:37.043710+0000","last_fullsized":"2024-04-26T23:18:37.043710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:53:16.817858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045815+0000","last_change":"2024-04-26T23:18:31.007674+0000","last_active":"2024-04-26T23:18:37.045815+0000","last_peered":"2024-04-26T23:18:37.045815+0000","last_clean":"2024-04-26T23:18:37.045815+0000","last_became_active":"2024-04-26T23:18:31.006787+0000","last_became_peered":"2024-04-26T23:18:31.006787+0000","last_unstale":"2024-04-26T23:18:37.045815+0000","last_undegraded":"2024-04-26T23:18:37.045815+0000","last_fullsized":"2024-04-26T23:18:37.045815+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:59:17.263468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044484+0000","last_change":"2024-04-26T23:18:36.046197+0000","last_active":"2024-04-26T23:18:37.044484+0000","last_peered":"2024-04-26T23:18:37.044484+0000","last_clean":"2024-04-26T23:18:37.044484+0000","last_became_active":"2024-04-26T23:18:36.045205+0000","last_became_peered":"2024-04-26T23:18:36.045205+0000","last_unstale":"2024-04-26T23:18:37.044484+0000","last_undegraded":"2024-04-26T23:18:37.044484+0000","last_fullsized":"2024-04-26T23:18:37.044484+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:39:32.810976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030371+0000","last_change":"2024-04-26T23:18:32.025774+0000","last_active":"2024-04-26T23:18:37.030371+0000","last_peered":"2024-04-26T23:18:37.030371+0000","last_clean":"2024-04-26T23:18:37.030371+0000","last_became_active":"2024-04-26T23:18:32.025487+0000","last_became_peered":"2024-04-26T23:18:32.025487+0000","last_unstale":"2024-04-26T23:18:37.030371+0000","last_undegraded":"2024-04-26T23:18:37.030371+0000","last_fullsized":"2024-04-26T23:18:37.030371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:04:13.811589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044628+0000","last_change":"2024-04-26T23:18:31.008744+0000","last_active":"2024-04-26T23:18:37.044628+0000","last_peered":"2024-04-26T23:18:37.044628+0000","last_clean":"2024-04-26T23:18:37.044628+0000","last_became_active":"2024-04-26T23:18:31.008449+0000","last_became_peered":"2024-04-26T23:18:31.008449+0000","last_unstale":"2024-04-26T23:18:37.044628+0000","last_undegraded":"2024-04-26T23:18:37.044628+0000","last_fullsized":"2024-04-26T23:18:37.044628+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:08.243628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045855+0000","last_change":"2024-04-26T23:18:18.862631+0000","last_active":"2024-04-26T23:18:37.045855+0000","last_peered":"2024-04-26T23:18:37.045855+0000","last_clean":"2024-04-26T23:18:37.045855+0000","last_became_active":"2024-04-26T23:18:18.861536+0000","last_became_peered":"2024-04-26T23:18:18.861536+0000","last_unstale":"2024-04-26T23:18:37.045855+0000","last_undegraded":"2024-04-26T23:18:37.045855+0000","last_fullsized":"2024-04-26T23:18:37.045855+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:17.808736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:17.808736+0000","last_clean_scrub_stamp":"2024-04-26T23:18:17.808736+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:59:26.359282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045887+0000","last_change":"2024-04-26T23:18:36.045014+0000","last_active":"2024-04-26T23:18:37.045887+0000","last_peered":"2024-04-26T23:18:37.045887+0000","last_clean":"2024-04-26T23:18:37.045887+0000","last_became_active":"2024-04-26T23:18:36.044467+0000","last_became_peered":"2024-04-26T23:18:36.044467+0000","last_unstale":"2024-04-26T23:18:37.045887+0000","last_undegraded":"2024-04-26T23:18:37.045887+0000","last_fullsized":"2024-04-26T23:18:37.045887+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:48:46.371866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046652+0000","last_change":"2024-04-26T23:18:32.030669+0000","last_active":"2024-04-26T23:18:37.046652+0000","last_peered":"2024-04-26T23:18:37.046652+0000","last_clean":"2024-04-26T23:18:37.046652+0000","last_became_active":"2024-04-26T23:18:32.030359+0000","last_became_peered":"2024-04-26T23:18:32.030359+0000","last_unstale":"2024-04-26T23:18:37.046652+0000","last_undegraded":"2024-04-26T23:18:37.046652+0000","last_fullsized":"2024-04-26T23:18:37.046652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:05.295882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.028346+0000","last_change":"2024-04-26T23:18:31.011343+0000","last_active":"2024-04-26T23:18:37.028346+0000","last_peered":"2024-04-26T23:18:37.028346+0000","last_clean":"2024-04-26T23:18:37.028346+0000","last_became_active":"2024-04-26T23:18:31.011011+0000","last_became_peered":"2024-04-26T23:18:31.011011+0000","last_unstale":"2024-04-26T23:18:37.028346+0000","last_undegraded":"2024-04-26T23:18:37.028346+0000","last_fullsized":"2024-04-26T23:18:37.028346+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:22.746946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045303+0000","last_change":"2024-04-26T23:18:36.045541+0000","last_active":"2024-04-26T23:18:37.045303+0000","last_peered":"2024-04-26T23:18:37.045303+0000","last_clean":"2024-04-26T23:18:37.045303+0000","last_became_active":"2024-04-26T23:18:36.045133+0000","last_became_peered":"2024-04-26T23:18:36.045133+0000","last_unstale":"2024-04-26T23:18:37.045303+0000","last_undegraded":"2024-04-26T23:18:37.045303+0000","last_fullsized":"2024-04-26T23:18:37.045303+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:45:26.386750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045276+0000","last_change":"2024-04-26T23:18:32.018614+0000","last_active":"2024-04-26T23:18:37.045276+0000","last_peered":"2024-04-26T23:18:37.045276+0000","last_clean":"2024-04-26T23:18:37.045276+0000","last_became_active":"2024-04-26T23:18:32.018088+0000","last_became_peered":"2024-04-26T23:18:32.018088+0000","last_unstale":"2024-04-26T23:18:37.045276+0000","last_undegraded":"2024-04-26T23:18:37.045276+0000","last_fullsized":"2024-04-26T23:18:37.045276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:26.533404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030223+0000","last_change":"2024-04-26T23:18:31.010377+0000","last_active":"2024-04-26T23:18:37.030223+0000","last_peered":"2024-04-26T23:18:37.030223+0000","last_clean":"2024-04-26T23:18:37.030223+0000","last_became_active":"2024-04-26T23:18:31.010084+0000","last_became_peered":"2024-04-26T23:18:31.010084+0000","last_unstale":"2024-04-26T23:18:37.030223+0000","last_undegraded":"2024-04-26T23:18:37.030223+0000","last_fullsized":"2024-04-26T23:18:37.030223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:08.243628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044284+0000","last_change":"2024-04-26T23:18:36.047754+0000","last_active":"2024-04-26T23:18:37.044284+0000","last_peered":"2024-04-26T23:18:37.044284+0000","last_clean":"2024-04-26T23:18:37.044284+0000","last_became_active":"2024-04-26T23:18:36.047105+0000","last_became_peered":"2024-04-26T23:18:36.047105+0000","last_unstale":"2024-04-26T23:18:37.044284+0000","last_undegraded":"2024-04-26T23:18:37.044284+0000","last_fullsized":"2024-04-26T23:18:37.044284+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:14.395241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.047247+0000","last_change":"2024-04-26T23:18:32.026649+0000","last_active":"2024-04-26T23:18:37.047247+0000","last_peered":"2024-04-26T23:18:37.047247+0000","last_clean":"2024-04-26T23:18:37.047247+0000","last_became_active":"2024-04-26T23:18:32.026014+0000","last_became_peered":"2024-04-26T23:18:32.026014+0000","last_unstale":"2024-04-26T23:18:37.047247+0000","last_undegraded":"2024-04-26T23:18:37.047247+0000","last_fullsized":"2024-04-26T23:18:37.047247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:08:47.417659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.028103+0000","last_change":"2024-04-26T23:18:31.006215+0000","last_active":"2024-04-26T23:18:37.028103+0000","last_peered":"2024-04-26T23:18:37.028103+0000","last_clean":"2024-04-26T23:18:37.028103+0000","last_became_active":"2024-04-26T23:18:31.005847+0000","last_became_peered":"2024-04-26T23:18:31.005847+0000","last_unstale":"2024-04-26T23:18:37.028103+0000","last_undegraded":"2024-04-26T23:18:37.028103+0000","last_fullsized":"2024-04-26T23:18:37.028103+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:44.844711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044109+0000","last_change":"2024-04-26T23:18:36.045812+0000","last_active":"2024-04-26T23:18:37.044109+0000","last_peered":"2024-04-26T23:18:37.044109+0000","last_clean":"2024-04-26T23:18:37.044109+0000","last_became_active":"2024-04-26T23:18:36.045445+0000","last_became_peered":"2024-04-26T23:18:36.045445+0000","last_unstale":"2024-04-26T23:18:37.044109+0000","last_undegraded":"2024-04-26T23:18:37.044109+0000","last_fullsized":"2024-04-26T23:18:37.044109+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:16:16.544972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046893+0000","last_change":"2024-04-26T23:18:32.025072+0000","last_active":"2024-04-26T23:18:37.046893+0000","last_peered":"2024-04-26T23:18:37.046893+0000","last_clean":"2024-04-26T23:18:37.046893+0000","last_became_active":"2024-04-26T23:18:32.024754+0000","last_became_peered":"2024-04-26T23:18:32.024754+0000","last_unstale":"2024-04-26T23:18:37.046893+0000","last_undegraded":"2024-04-26T23:18:37.046893+0000","last_fullsized":"2024-04-26T23:18:37.046893+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:45:22.362155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044833+0000","last_change":"2024-04-26T23:18:31.013340+0000","last_active":"2024-04-26T23:18:37.044833+0000","last_peered":"2024-04-26T23:18:37.044833+0000","last_clean":"2024-04-26T23:18:37.044833+0000","last_became_active":"2024-04-26T23:18:31.012824+0000","last_became_peered":"2024-04-26T23:18:31.012824+0000","last_unstale":"2024-04-26T23:18:37.044833+0000","last_undegraded":"2024-04-26T23:18:37.044833+0000","last_fullsized":"2024-04-26T23:18:37.044833+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:17.988227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044762+0000","last_change":"2024-04-26T23:18:36.041004+0000","last_active":"2024-04-26T23:18:37.044762+0000","last_peered":"2024-04-26T23:18:37.044762+0000","last_clean":"2024-04-26T23:18:37.044762+0000","last_became_active":"2024-04-26T23:18:36.040155+0000","last_became_peered":"2024-04-26T23:18:36.040155+0000","last_unstale":"2024-04-26T23:18:37.044762+0000","last_undegraded":"2024-04-26T23:18:37.044762+0000","last_fullsized":"2024-04-26T23:18:37.044762+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:59.856924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046692+0000","last_change":"2024-04-26T23:18:32.025214+0000","last_active":"2024-04-26T23:18:37.046692+0000","last_peered":"2024-04-26T23:18:37.046692+0000","last_clean":"2024-04-26T23:18:37.046692+0000","last_became_active":"2024-04-26T23:18:32.024952+0000","last_became_peered":"2024-04-26T23:18:32.024952+0000","last_unstale":"2024-04-26T23:18:37.046692+0000","last_undegraded":"2024-04-26T23:18:37.046692+0000","last_fullsized":"2024-04-26T23:18:37.046692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:39:01.950309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.027616+0000","last_change":"2024-04-26T23:18:31.001581+0000","last_active":"2024-04-26T23:18:37.027616+0000","last_peered":"2024-04-26T23:18:37.027616+0000","last_clean":"2024-04-26T23:18:37.027616+0000","last_became_active":"2024-04-26T23:18:31.001256+0000","last_became_peered":"2024-04-26T23:18:31.001256+0000","last_unstale":"2024-04-26T23:18:37.027616+0000","last_undegraded":"2024-04-26T23:18:37.027616+0000","last_fullsized":"2024-04-26T23:18:37.027616+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:13.713713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044283+0000","last_change":"2024-04-26T23:18:36.040673+0000","last_active":"2024-04-26T23:18:37.044283+0000","last_peered":"2024-04-26T23:18:37.044283+0000","last_clean":"2024-04-26T23:18:37.044283+0000","last_became_active":"2024-04-26T23:18:36.040246+0000","last_became_peered":"2024-04-26T23:18:36.040246+0000","last_unstale":"2024-04-26T23:18:37.044283+0000","last_undegraded":"2024-04-26T23:18:37.044283+0000","last_fullsized":"2024-04-26T23:18:37.044283+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:30.558000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044353+0000","last_change":"2024-04-26T23:18:32.013824+0000","last_active":"2024-04-26T23:18:37.044353+0000","last_peered":"2024-04-26T23:18:37.044353+0000","last_clean":"2024-04-26T23:18:37.044353+0000","last_became_active":"2024-04-26T23:18:32.012613+0000","last_became_peered":"2024-04-26T23:18:32.012613+0000","last_unstale":"2024-04-26T23:18:37.044353+0000","last_undegraded":"2024-04-26T23:18:37.044353+0000","last_fullsized":"2024-04-26T23:18:37.044353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:49:47.214737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044914+0000","last_change":"2024-04-26T23:18:31.011248+0000","last_active":"2024-04-26T23:18:37.044914+0000","last_peered":"2024-04-26T23:18:37.044914+0000","last_clean":"2024-04-26T23:18:37.044914+0000","last_became_active":"2024-04-26T23:18:31.010965+0000","last_became_peered":"2024-04-26T23:18:31.010965+0000","last_unstale":"2024-04-26T23:18:37.044914+0000","last_undegraded":"2024-04-26T23:18:37.044914+0000","last_fullsized":"2024-04-26T23:18:37.044914+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:13.713713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030962+0000","last_change":"2024-04-26T23:18:36.045330+0000","last_active":"2024-04-26T23:18:37.030962+0000","last_peered":"2024-04-26T23:18:37.030962+0000","last_clean":"2024-04-26T23:18:37.030962+0000","last_became_active":"2024-04-26T23:18:36.044903+0000","last_became_peered":"2024-04-26T23:18:36.044903+0000","last_unstale":"2024-04-26T23:18:37.030962+0000","last_undegraded":"2024-04-26T23:18:37.030962+0000","last_fullsized":"2024-04-26T23:18:37.030962+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:53:15.682467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.047083+0000","last_change":"2024-04-26T23:18:32.026417+0000","last_active":"2024-04-26T23:18:37.047083+0000","last_peered":"2024-04-26T23:18:37.047083+0000","last_clean":"2024-04-26T23:18:37.047083+0000","last_became_active":"2024-04-26T23:18:32.025839+0000","last_became_peered":"2024-04-26T23:18:32.025839+0000","last_unstale":"2024-04-26T23:18:37.047083+0000","last_undegraded":"2024-04-26T23:18:37.047083+0000","last_fullsized":"2024-04-26T23:18:37.047083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:39:28.786381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044878+0000","last_change":"2024-04-26T23:18:31.011956+0000","last_active":"2024-04-26T23:18:37.044878+0000","last_peered":"2024-04-26T23:18:37.044878+0000","last_clean":"2024-04-26T23:18:37.044878+0000","last_became_active":"2024-04-26T23:18:31.010493+0000","last_became_peered":"2024-04-26T23:18:31.010493+0000","last_unstale":"2024-04-26T23:18:37.044878+0000","last_undegraded":"2024-04-26T23:18:37.044878+0000","last_fullsized":"2024-04-26T23:18:37.044878+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:59:17.263468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030567+0000","last_change":"2024-04-26T23:18:36.047105+0000","last_active":"2024-04-26T23:18:37.030567+0000","last_peered":"2024-04-26T23:18:37.030567+0000","last_clean":"2024-04-26T23:18:37.030567+0000","last_became_active":"2024-04-26T23:18:36.046724+0000","last_became_peered":"2024-04-26T23:18:36.046724+0000","last_unstale":"2024-04-26T23:18:37.030567+0000","last_undegraded":"2024-04-26T23:18:37.030567+0000","last_fullsized":"2024-04-26T23:18:37.030567+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:46:07.146408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.043961+0000","last_change":"2024-04-26T23:18:32.015408+0000","last_active":"2024-04-26T23:18:37.043961+0000","last_peered":"2024-04-26T23:18:37.043961+0000","last_clean":"2024-04-26T23:18:37.043961+0000","last_became_active":"2024-04-26T23:18:32.015037+0000","last_became_peered":"2024-04-26T23:18:32.015037+0000","last_unstale":"2024-04-26T23:18:37.043961+0000","last_undegraded":"2024-04-26T23:18:37.043961+0000","last_fullsized":"2024-04-26T23:18:37.043961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:49.567077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044345+0000","last_change":"2024-04-26T23:18:31.010413+0000","last_active":"2024-04-26T23:18:37.044345+0000","last_peered":"2024-04-26T23:18:37.044345+0000","last_clean":"2024-04-26T23:18:37.044345+0000","last_became_active":"2024-04-26T23:18:31.010138+0000","last_became_peered":"2024-04-26T23:18:31.010138+0000","last_unstale":"2024-04-26T23:18:37.044345+0000","last_undegraded":"2024-04-26T23:18:37.044345+0000","last_fullsized":"2024-04-26T23:18:37.044345+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:15:53.470195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046231+0000","last_change":"2024-04-26T23:18:36.044782+0000","last_active":"2024-04-26T23:18:37.046231+0000","last_peered":"2024-04-26T23:18:37.046231+0000","last_clean":"2024-04-26T23:18:37.046231+0000","last_became_active":"2024-04-26T23:18:36.044157+0000","last_became_peered":"2024-04-26T23:18:36.044157+0000","last_unstale":"2024-04-26T23:18:37.046231+0000","last_undegraded":"2024-04-26T23:18:37.046231+0000","last_fullsized":"2024-04-26T23:18:37.046231+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:24:24.209562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046631+0000","last_change":"2024-04-26T23:18:32.023453+0000","last_active":"2024-04-26T23:18:37.046631+0000","last_peered":"2024-04-26T23:18:37.046631+0000","last_clean":"2024-04-26T23:18:37.046631+0000","last_became_active":"2024-04-26T23:18:32.023170+0000","last_became_peered":"2024-04-26T23:18:32.023170+0000","last_unstale":"2024-04-26T23:18:37.046631+0000","last_undegraded":"2024-04-26T23:18:37.046631+0000","last_fullsized":"2024-04-26T23:18:37.046631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:48:42.347270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:41.489012+0000","last_change":"2024-04-26T23:18:31.013165+0000","last_active":"2024-04-26T23:18:41.489012+0000","last_peered":"2024-04-26T23:18:41.489012+0000","last_clean":"2024-04-26T23:18:41.489012+0000","last_became_active":"2024-04-26T23:18:31.012790+0000","last_became_peered":"2024-04-26T23:18:31.012790+0000","last_unstale":"2024-04-26T23:18:41.489012+0000","last_undegraded":"2024-04-26T23:18:41.489012+0000","last_fullsized":"2024-04-26T23:18:41.489012+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:44.844711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029334+0000","last_change":"2024-04-26T23:18:36.047231+0000","last_active":"2024-04-26T23:18:37.029334+0000","last_peered":"2024-04-26T23:18:37.029334+0000","last_clean":"2024-04-26T23:18:37.029334+0000","last_became_active":"2024-04-26T23:18:36.046579+0000","last_became_peered":"2024-04-26T23:18:36.046579+0000","last_unstale":"2024-04-26T23:18:37.029334+0000","last_undegraded":"2024-04-26T23:18:37.029334+0000","last_fullsized":"2024-04-26T23:18:37.029334+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:14.395241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045390+0000","last_change":"2024-04-26T23:18:32.022720+0000","last_active":"2024-04-26T23:18:37.045390+0000","last_peered":"2024-04-26T23:18:37.045390+0000","last_clean":"2024-04-26T23:18:37.045390+0000","last_became_active":"2024-04-26T23:18:32.022450+0000","last_became_peered":"2024-04-26T23:18:32.022450+0000","last_unstale":"2024-04-26T23:18:37.045390+0000","last_undegraded":"2024-04-26T23:18:37.045390+0000","last_fullsized":"2024-04-26T23:18:37.045390+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:25:39.521124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029950+0000","last_change":"2024-04-26T23:18:31.008154+0000","last_active":"2024-04-26T23:18:37.029950+0000","last_peered":"2024-04-26T23:18:37.029950+0000","last_clean":"2024-04-26T23:18:37.029950+0000","last_became_active":"2024-04-26T23:18:31.007881+0000","last_became_peered":"2024-04-26T23:18:31.007881+0000","last_unstale":"2024-04-26T23:18:37.029950+0000","last_undegraded":"2024-04-26T23:18:37.029950+0000","last_fullsized":"2024-04-26T23:18:37.029950+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:20.581173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046058+0000","last_change":"2024-04-26T23:18:36.047010+0000","last_active":"2024-04-26T23:18:37.046058+0000","last_peered":"2024-04-26T23:18:37.046058+0000","last_clean":"2024-04-26T23:18:37.046058+0000","last_became_active":"2024-04-26T23:18:36.045988+0000","last_became_peered":"2024-04-26T23:18:36.045988+0000","last_unstale":"2024-04-26T23:18:37.046058+0000","last_undegraded":"2024-04-26T23:18:37.046058+0000","last_fullsized":"2024-04-26T23:18:37.046058+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:09.320477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044476+0000","last_change":"2024-04-26T23:18:32.019460+0000","last_active":"2024-04-26T23:18:37.044476+0000","last_peered":"2024-04-26T23:18:37.044476+0000","last_clean":"2024-04-26T23:18:37.044476+0000","last_became_active":"2024-04-26T23:18:32.019146+0000","last_became_peered":"2024-04-26T23:18:32.019146+0000","last_unstale":"2024-04-26T23:18:37.044476+0000","last_undegraded":"2024-04-26T23:18:37.044476+0000","last_fullsized":"2024-04-26T23:18:37.044476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:55:55.562630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045137+0000","last_change":"2024-04-26T23:18:31.010513+0000","last_active":"2024-04-26T23:18:37.045137+0000","last_peered":"2024-04-26T23:18:37.045137+0000","last_clean":"2024-04-26T23:18:37.045137+0000","last_became_active":"2024-04-26T23:18:31.010206+0000","last_became_peered":"2024-04-26T23:18:31.010206+0000","last_unstale":"2024-04-26T23:18:37.045137+0000","last_undegraded":"2024-04-26T23:18:37.045137+0000","last_fullsized":"2024-04-26T23:18:37.045137+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:17:03.522342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030875+0000","last_change":"2024-04-26T23:18:36.039670+0000","last_active":"2024-04-26T23:18:37.030875+0000","last_peered":"2024-04-26T23:18:37.030875+0000","last_clean":"2024-04-26T23:18:37.030875+0000","last_became_active":"2024-04-26T23:18:36.039247+0000","last_became_peered":"2024-04-26T23:18:36.039247+0000","last_unstale":"2024-04-26T23:18:37.030875+0000","last_undegraded":"2024-04-26T23:18:37.030875+0000","last_fullsized":"2024-04-26T23:18:37.030875+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:45:37.196730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045040+0000","last_change":"2024-04-26T23:18:32.029408+0000","last_active":"2024-04-26T23:18:37.045040+0000","last_peered":"2024-04-26T23:18:37.045040+0000","last_clean":"2024-04-26T23:18:37.045040+0000","last_became_active":"2024-04-26T23:18:32.029150+0000","last_became_peered":"2024-04-26T23:18:32.029150+0000","last_unstale":"2024-04-26T23:18:37.045040+0000","last_undegraded":"2024-04-26T23:18:37.045040+0000","last_fullsized":"2024-04-26T23:18:37.045040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:48:42.347270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046379+0000","last_change":"2024-04-26T23:18:31.013885+0000","last_active":"2024-04-26T23:18:37.046379+0000","last_peered":"2024-04-26T23:18:37.046379+0000","last_clean":"2024-04-26T23:18:37.046379+0000","last_became_active":"2024-04-26T23:18:31.013617+0000","last_became_peered":"2024-04-26T23:18:31.013617+0000","last_unstale":"2024-04-26T23:18:37.046379+0000","last_undegraded":"2024-04-26T23:18:37.046379+0000","last_fullsized":"2024-04-26T23:18:37.046379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:08.243628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029721+0000","last_change":"2024-04-26T23:18:36.049794+0000","last_active":"2024-04-26T23:18:37.029721+0000","last_peered":"2024-04-26T23:18:37.029721+0000","last_clean":"2024-04-26T23:18:37.029721+0000","last_became_active":"2024-04-26T23:18:36.044164+0000","last_became_peered":"2024-04-26T23:18:36.044164+0000","last_unstale":"2024-04-26T23:18:37.029721+0000","last_undegraded":"2024-04-26T23:18:37.029721+0000","last_fullsized":"2024-04-26T23:18:37.029721+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:41:38.190204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044157+0000","last_change":"2024-04-26T23:18:36.046931+0000","last_active":"2024-04-26T23:18:37.044157+0000","last_peered":"2024-04-26T23:18:37.044157+0000","last_clean":"2024-04-26T23:18:37.044157+0000","last_became_active":"2024-04-26T23:18:36.045786+0000","last_became_peered":"2024-04-26T23:18:36.045786+0000","last_unstale":"2024-04-26T23:18:37.044157+0000","last_undegraded":"2024-04-26T23:18:37.044157+0000","last_fullsized":"2024-04-26T23:18:37.044157+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:09.320477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.028930+0000","last_change":"2024-04-26T23:18:32.011300+0000","last_active":"2024-04-26T23:18:37.028930+0000","last_peered":"2024-04-26T23:18:37.028930+0000","last_clean":"2024-04-26T23:18:37.028930+0000","last_became_active":"2024-04-26T23:18:32.010998+0000","last_became_peered":"2024-04-26T23:18:32.010998+0000","last_unstale":"2024-04-26T23:18:37.028930+0000","last_undegraded":"2024-04-26T23:18:37.028930+0000","last_fullsized":"2024-04-26T23:18:37.028930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:57:02.596299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044365+0000","last_change":"2024-04-26T23:18:31.011198+0000","last_active":"2024-04-26T23:18:37.044365+0000","last_peered":"2024-04-26T23:18:37.044365+0000","last_clean":"2024-04-26T23:18:37.044365+0000","last_became_active":"2024-04-26T23:18:31.010908+0000","last_became_peered":"2024-04-26T23:18:31.010908+0000","last_unstale":"2024-04-26T23:18:37.044365+0000","last_undegraded":"2024-04-26T23:18:37.044365+0000","last_fullsized":"2024-04-26T23:18:37.044365+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:22.746946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046251+0000","last_change":"2024-04-26T23:18:36.042049+0000","last_active":"2024-04-26T23:18:37.046251+0000","last_peered":"2024-04-26T23:18:37.046251+0000","last_clean":"2024-04-26T23:18:37.046251+0000","last_became_active":"2024-04-26T23:18:36.041661+0000","last_became_peered":"2024-04-26T23:18:36.041661+0000","last_unstale":"2024-04-26T23:18:37.046251+0000","last_undegraded":"2024-04-26T23:18:37.046251+0000","last_fullsized":"2024-04-26T23:18:37.046251+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:09.320477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030801+0000","last_change":"2024-04-26T23:18:32.013285+0000","last_active":"2024-04-26T23:18:37.030801+0000","last_peered":"2024-04-26T23:18:37.030801+0000","last_clean":"2024-04-26T23:18:37.030801+0000","last_became_active":"2024-04-26T23:18:32.013013+0000","last_became_peered":"2024-04-26T23:18:32.013013+0000","last_unstale":"2024-04-26T23:18:37.030801+0000","last_undegraded":"2024-04-26T23:18:37.030801+0000","last_fullsized":"2024-04-26T23:18:37.030801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:54:25.468799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029792+0000","last_change":"2024-04-26T23:18:31.011691+0000","last_active":"2024-04-26T23:18:37.029792+0000","last_peered":"2024-04-26T23:18:37.029792+0000","last_clean":"2024-04-26T23:18:37.029792+0000","last_became_active":"2024-04-26T23:18:31.009521+0000","last_became_peered":"2024-04-26T23:18:31.009521+0000","last_unstale":"2024-04-26T23:18:37.029792+0000","last_undegraded":"2024-04-26T23:18:37.029792+0000","last_fullsized":"2024-04-26T23:18:37.029792+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:22.746946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045573+0000","last_change":"2024-04-26T23:18:31.010360+0000","last_active":"2024-04-26T23:18:37.045573+0000","last_peered":"2024-04-26T23:18:37.045573+0000","last_clean":"2024-04-26T23:18:37.045573+0000","last_became_active":"2024-04-26T23:18:31.010083+0000","last_became_peered":"2024-04-26T23:18:31.010083+0000","last_unstale":"2024-04-26T23:18:37.045573+0000","last_undegraded":"2024-04-26T23:18:37.045573+0000","last_fullsized":"2024-04-26T23:18:37.045573+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:34:02.747044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044258+0000","last_change":"2024-04-26T23:18:32.000871+0000","last_active":"2024-04-26T23:18:37.044258+0000","last_peered":"2024-04-26T23:18:37.044258+0000","last_clean":"2024-04-26T23:18:37.044258+0000","last_became_active":"2024-04-26T23:18:31.999865+0000","last_became_peered":"2024-04-26T23:18:31.999865+0000","last_unstale":"2024-04-26T23:18:37.044258+0000","last_undegraded":"2024-04-26T23:18:37.044258+0000","last_fullsized":"2024-04-26T23:18:37.044258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:57:02.596299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.028327+0000","last_change":"2024-04-26T23:18:36.040907+0000","last_active":"2024-04-26T23:18:37.028327+0000","last_peered":"2024-04-26T23:18:37.028327+0000","last_clean":"2024-04-26T23:18:37.028327+0000","last_became_active":"2024-04-26T23:18:36.040494+0000","last_became_peered":"2024-04-26T23:18:36.040494+0000","last_unstale":"2024-04-26T23:18:37.028327+0000","last_undegraded":"2024-04-26T23:18:37.028327+0000","last_fullsized":"2024-04-26T23:18:37.028327+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:17:08.552285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030078+0000","last_change":"2024-04-26T23:18:31.012855+0000","last_active":"2024-04-26T23:18:37.030078+0000","last_peered":"2024-04-26T23:18:37.030078+0000","last_clean":"2024-04-26T23:18:37.030078+0000","last_became_active":"2024-04-26T23:18:31.012510+0000","last_became_peered":"2024-04-26T23:18:31.012510+0000","last_unstale":"2024-04-26T23:18:37.030078+0000","last_undegraded":"2024-04-26T23:18:37.030078+0000","last_fullsized":"2024-04-26T23:18:37.030078+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:15:53.470195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045692+0000","last_change":"2024-04-26T23:18:32.002285+0000","last_active":"2024-04-26T23:18:37.045692+0000","last_peered":"2024-04-26T23:18:37.045692+0000","last_clean":"2024-04-26T23:18:37.045692+0000","last_became_active":"2024-04-26T23:18:32.001991+0000","last_became_peered":"2024-04-26T23:18:32.001991+0000","last_unstale":"2024-04-26T23:18:37.045692+0000","last_undegraded":"2024-04-26T23:18:37.045692+0000","last_fullsized":"2024-04-26T23:18:37.045692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:45.850060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030744+0000","last_change":"2024-04-26T23:18:36.038108+0000","last_active":"2024-04-26T23:18:37.030744+0000","last_peered":"2024-04-26T23:18:37.030744+0000","last_clean":"2024-04-26T23:18:37.030744+0000","last_became_active":"2024-04-26T23:18:36.037655+0000","last_became_peered":"2024-04-26T23:18:36.037655+0000","last_unstale":"2024-04-26T23:18:37.030744+0000","last_undegraded":"2024-04-26T23:18:37.030744+0000","last_fullsized":"2024-04-26T23:18:37.030744+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:02.473474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044156+0000","last_change":"2024-04-26T23:18:32.014058+0000","last_active":"2024-04-26T23:18:37.044156+0000","last_peered":"2024-04-26T23:18:37.044156+0000","last_clean":"2024-04-26T23:18:37.044156+0000","last_became_active":"2024-04-26T23:18:32.013500+0000","last_became_peered":"2024-04-26T23:18:32.013500+0000","last_unstale":"2024-04-26T23:18:37.044156+0000","last_undegraded":"2024-04-26T23:18:37.044156+0000","last_fullsized":"2024-04-26T23:18:37.044156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:54:25.468799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.027802+0000","last_change":"2024-04-26T23:18:32.018280+0000","last_active":"2024-04-26T23:18:37.027802+0000","last_peered":"2024-04-26T23:18:37.027802+0000","last_clean":"2024-04-26T23:18:37.027802+0000","last_became_active":"2024-04-26T23:18:32.017993+0000","last_became_peered":"2024-04-26T23:18:32.017993+0000","last_unstale":"2024-04-26T23:18:37.027802+0000","last_undegraded":"2024-04-26T23:18:37.027802+0000","last_fullsized":"2024-04-26T23:18:37.027802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:08:32.798896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044791+0000","last_change":"2024-04-26T23:18:32.022457+0000","last_active":"2024-04-26T23:18:37.044791+0000","last_peered":"2024-04-26T23:18:37.044791+0000","last_clean":"2024-04-26T23:18:37.044791+0000","last_became_active":"2024-04-26T23:18:32.021766+0000","last_became_peered":"2024-04-26T23:18:32.021766+0000","last_unstale":"2024-04-26T23:18:37.044791+0000","last_undegraded":"2024-04-26T23:18:37.044791+0000","last_fullsized":"2024-04-26T23:18:37.044791+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:48:51.877664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044133+0000","last_change":"2024-04-26T23:18:32.011541+0000","last_active":"2024-04-26T23:18:37.044133+0000","last_peered":"2024-04-26T23:18:37.044133+0000","last_clean":"2024-04-26T23:18:37.044133+0000","last_became_active":"2024-04-26T23:18:32.009252+0000","last_became_peered":"2024-04-26T23:18:32.009252+0000","last_unstale":"2024-04-26T23:18:37.044133+0000","last_undegraded":"2024-04-26T23:18:37.044133+0000","last_fullsized":"2024-04-26T23:18:37.044133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:54:25.468799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030659+0000","last_change":"2024-04-26T23:18:32.018905+0000","last_active":"2024-04-26T23:18:37.030659+0000","last_peered":"2024-04-26T23:18:37.030659+0000","last_clean":"2024-04-26T23:18:37.030659+0000","last_became_active":"2024-04-26T23:18:32.018586+0000","last_became_peered":"2024-04-26T23:18:32.018586+0000","last_unstale":"2024-04-26T23:18:37.030659+0000","last_undegraded":"2024-04-26T23:18:37.030659+0000","last_fullsized":"2024-04-26T23:18:37.030659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:37:21.828417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029798+0000","last_change":"2024-04-26T23:18:32.011775+0000","last_active":"2024-04-26T23:18:37.029798+0000","last_peered":"2024-04-26T23:18:37.029798+0000","last_clean":"2024-04-26T23:18:37.029798+0000","last_became_active":"2024-04-26T23:18:32.011443+0000","last_became_peered":"2024-04-26T23:18:32.011443+0000","last_unstale":"2024-04-26T23:18:37.029798+0000","last_undegraded":"2024-04-26T23:18:37.029798+0000","last_fullsized":"2024-04-26T23:18:37.029798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:49.567077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046666+0000","last_change":"2024-04-26T23:18:32.026884+0000","last_active":"2024-04-26T23:18:37.046666+0000","last_peered":"2024-04-26T23:18:37.046666+0000","last_clean":"2024-04-26T23:18:37.046666+0000","last_became_active":"2024-04-26T23:18:32.026579+0000","last_became_peered":"2024-04-26T23:18:32.026579+0000","last_unstale":"2024-04-26T23:18:37.046666+0000","last_undegraded":"2024-04-26T23:18:37.046666+0000","last_fullsized":"2024-04-26T23:18:37.046666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:55:55.562630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030747+0000","last_change":"2024-04-26T23:18:32.018063+0000","last_active":"2024-04-26T23:18:37.030747+0000","last_peered":"2024-04-26T23:18:37.030747+0000","last_clean":"2024-04-26T23:18:37.030747+0000","last_became_active":"2024-04-26T23:18:32.017430+0000","last_became_peered":"2024-04-26T23:18:32.017430+0000","last_unstale":"2024-04-26T23:18:37.030747+0000","last_undegraded":"2024-04-26T23:18:37.030747+0000","last_fullsized":"2024-04-26T23:18:37.030747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:48:51.877664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044241+0000","last_change":"2024-04-26T23:18:32.018383+0000","last_active":"2024-04-26T23:18:37.044241+0000","last_peered":"2024-04-26T23:18:37.044241+0000","last_clean":"2024-04-26T23:18:37.044241+0000","last_became_active":"2024-04-26T23:18:32.018101+0000","last_became_peered":"2024-04-26T23:18:32.018101+0000","last_unstale":"2024-04-26T23:18:37.044241+0000","last_undegraded":"2024-04-26T23:18:37.044241+0000","last_fullsized":"2024-04-26T23:18:37.044241+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:49:47.214737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029898+0000","last_change":"2024-04-26T23:18:32.020368+0000","last_active":"2024-04-26T23:18:37.029898+0000","last_peered":"2024-04-26T23:18:37.029898+0000","last_clean":"2024-04-26T23:18:37.029898+0000","last_became_active":"2024-04-26T23:18:32.019278+0000","last_became_peered":"2024-04-26T23:18:32.019278+0000","last_unstale":"2024-04-26T23:18:37.029898+0000","last_undegraded":"2024-04-26T23:18:37.029898+0000","last_fullsized":"2024-04-26T23:18:37.029898+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:51:42.870329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044697+0000","last_change":"2024-04-26T23:18:32.029085+0000","last_active":"2024-04-26T23:18:37.044697+0000","last_peered":"2024-04-26T23:18:37.044697+0000","last_clean":"2024-04-26T23:18:37.044697+0000","last_became_active":"2024-04-26T23:18:32.028769+0000","last_became_peered":"2024-04-26T23:18:32.028769+0000","last_unstale":"2024-04-26T23:18:37.044697+0000","last_undegraded":"2024-04-26T23:18:37.044697+0000","last_fullsized":"2024-04-26T23:18:37.044697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:55.832329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.043961+0000","last_change":"2024-04-26T23:18:32.014926+0000","last_active":"2024-04-26T23:18:37.043961+0000","last_peered":"2024-04-26T23:18:37.043961+0000","last_clean":"2024-04-26T23:18:37.043961+0000","last_became_active":"2024-04-26T23:18:32.014414+0000","last_became_peered":"2024-04-26T23:18:32.014414+0000","last_unstale":"2024-04-26T23:18:37.043961+0000","last_undegraded":"2024-04-26T23:18:37.043961+0000","last_fullsized":"2024-04-26T23:18:37.043961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:58:44.698418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045548+0000","last_change":"2024-04-26T23:18:32.019959+0000","last_active":"2024-04-26T23:18:37.045548+0000","last_peered":"2024-04-26T23:18:37.045548+0000","last_clean":"2024-04-26T23:18:37.045548+0000","last_became_active":"2024-04-26T23:18:32.019690+0000","last_became_peered":"2024-04-26T23:18:32.019690+0000","last_unstale":"2024-04-26T23:18:37.045548+0000","last_undegraded":"2024-04-26T23:18:37.045548+0000","last_fullsized":"2024-04-26T23:18:37.045548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:49.567077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.047090+0000","last_change":"2024-04-26T23:18:32.023444+0000","last_active":"2024-04-26T23:18:37.047090+0000","last_peered":"2024-04-26T23:18:37.047090+0000","last_clean":"2024-04-26T23:18:37.047090+0000","last_became_active":"2024-04-26T23:18:32.023211+0000","last_became_peered":"2024-04-26T23:18:32.023211+0000","last_unstale":"2024-04-26T23:18:37.047090+0000","last_undegraded":"2024-04-26T23:18:37.047090+0000","last_fullsized":"2024-04-26T23:18:37.047090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:57:19.300614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.027980+0000","last_change":"2024-04-26T23:18:32.011378+0000","last_active":"2024-04-26T23:18:37.027980+0000","last_peered":"2024-04-26T23:18:37.027980+0000","last_clean":"2024-04-26T23:18:37.027980+0000","last_became_active":"2024-04-26T23:18:32.011101+0000","last_became_peered":"2024-04-26T23:18:32.011101+0000","last_unstale":"2024-04-26T23:18:37.027980+0000","last_undegraded":"2024-04-26T23:18:37.027980+0000","last_fullsized":"2024-04-26T23:18:37.027980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:53:16.817858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045910+0000","last_change":"2024-04-26T23:18:32.028996+0000","last_active":"2024-04-26T23:18:37.045910+0000","last_peered":"2024-04-26T23:18:37.045910+0000","last_clean":"2024-04-26T23:18:37.045910+0000","last_became_active":"2024-04-26T23:18:32.028697+0000","last_became_peered":"2024-04-26T23:18:32.028697+0000","last_unstale":"2024-04-26T23:18:37.045910+0000","last_undegraded":"2024-04-26T23:18:37.045910+0000","last_fullsized":"2024-04-26T23:18:37.045910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:04:13.811589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044240+0000","last_change":"2024-04-26T23:18:32.018149+0000","last_active":"2024-04-26T23:18:37.044240+0000","last_peered":"2024-04-26T23:18:37.044240+0000","last_clean":"2024-04-26T23:18:37.044240+0000","last_became_active":"2024-04-26T23:18:32.017657+0000","last_became_peered":"2024-04-26T23:18:32.017657+0000","last_unstale":"2024-04-26T23:18:37.044240+0000","last_undegraded":"2024-04-26T23:18:37.044240+0000","last_fullsized":"2024-04-26T23:18:37.044240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:55:55.562630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029015+0000","last_change":"2024-04-26T23:18:32.000678+0000","last_active":"2024-04-26T23:18:37.029015+0000","last_peered":"2024-04-26T23:18:37.029015+0000","last_clean":"2024-04-26T23:18:37.029015+0000","last_became_active":"2024-04-26T23:18:32.000354+0000","last_became_peered":"2024-04-26T23:18:32.000354+0000","last_unstale":"2024-04-26T23:18:37.029015+0000","last_undegraded":"2024-04-26T23:18:37.029015+0000","last_fullsized":"2024-04-26T23:18:37.029015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:51:42.870329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.043891+0000","last_change":"2024-04-26T23:18:31.999586+0000","last_active":"2024-04-26T23:18:37.043891+0000","last_peered":"2024-04-26T23:18:37.043891+0000","last_clean":"2024-04-26T23:18:37.043891+0000","last_became_active":"2024-04-26T23:18:31.999185+0000","last_became_peered":"2024-04-26T23:18:31.999185+0000","last_unstale":"2024-04-26T23:18:37.043891+0000","last_undegraded":"2024-04-26T23:18:37.043891+0000","last_fullsized":"2024-04-26T23:18:37.043891+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:51:42.870329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029862+0000","last_change":"2024-04-26T23:18:32.024681+0000","last_active":"2024-04-26T23:18:37.029862+0000","last_peered":"2024-04-26T23:18:37.029862+0000","last_clean":"2024-04-26T23:18:37.029862+0000","last_became_active":"2024-04-26T23:18:32.024372+0000","last_became_peered":"2024-04-26T23:18:32.024372+0000","last_unstale":"2024-04-26T23:18:37.029862+0000","last_undegraded":"2024-04-26T23:18:37.029862+0000","last_fullsized":"2024-04-26T23:18:37.029862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:49.567077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045305+0000","last_change":"2024-04-26T23:18:32.011333+0000","last_active":"2024-04-26T23:18:37.045305+0000","last_peered":"2024-04-26T23:18:37.045305+0000","last_clean":"2024-04-26T23:18:37.045305+0000","last_became_active":"2024-04-26T23:18:32.011052+0000","last_became_peered":"2024-04-26T23:18:32.011052+0000","last_unstale":"2024-04-26T23:18:37.045305+0000","last_undegraded":"2024-04-26T23:18:37.045305+0000","last_fullsized":"2024-04-26T23:18:37.045305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:29.991591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046025+0000","last_change":"2024-04-26T23:18:32.021692+0000","last_active":"2024-04-26T23:18:37.046025+0000","last_peered":"2024-04-26T23:18:37.046025+0000","last_clean":"2024-04-26T23:18:37.046025+0000","last_became_active":"2024-04-26T23:18:32.021426+0000","last_became_peered":"2024-04-26T23:18:32.021426+0000","last_unstale":"2024-04-26T23:18:37.046025+0000","last_undegraded":"2024-04-26T23:18:37.046025+0000","last_fullsized":"2024-04-26T23:18:37.046025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:57:02.596299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046525+0000","last_change":"2024-04-26T23:18:32.020976+0000","last_active":"2024-04-26T23:18:37.046525+0000","last_peered":"2024-04-26T23:18:37.046525+0000","last_clean":"2024-04-26T23:18:37.046525+0000","last_became_active":"2024-04-26T23:18:32.020653+0000","last_became_peered":"2024-04-26T23:18:32.020653+0000","last_unstale":"2024-04-26T23:18:37.046525+0000","last_undegraded":"2024-04-26T23:18:37.046525+0000","last_fullsized":"2024-04-26T23:18:37.046525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:51:42.870329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.047206+0000","last_change":"2024-04-26T23:18:32.030726+0000","last_active":"2024-04-26T23:18:37.047206+0000","last_peered":"2024-04-26T23:18:37.047206+0000","last_clean":"2024-04-26T23:18:37.047206+0000","last_became_active":"2024-04-26T23:18:32.030451+0000","last_became_peered":"2024-04-26T23:18:32.030451+0000","last_unstale":"2024-04-26T23:18:37.047206+0000","last_undegraded":"2024-04-26T23:18:37.047206+0000","last_fullsized":"2024-04-26T23:18:37.047206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:26.533404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030676+0000","last_change":"2024-04-26T23:18:32.020062+0000","last_active":"2024-04-26T23:18:37.030676+0000","last_peered":"2024-04-26T23:18:37.030676+0000","last_clean":"2024-04-26T23:18:37.030676+0000","last_became_active":"2024-04-26T23:18:32.019651+0000","last_became_peered":"2024-04-26T23:18:32.019651+0000","last_unstale":"2024-04-26T23:18:37.030676+0000","last_undegraded":"2024-04-26T23:18:37.030676+0000","last_fullsized":"2024-04-26T23:18:37.030676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:17:04.527690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046652+0000","last_change":"2024-04-26T23:18:32.017808+0000","last_active":"2024-04-26T23:18:37.046652+0000","last_peered":"2024-04-26T23:18:37.046652+0000","last_clean":"2024-04-26T23:18:37.046652+0000","last_became_active":"2024-04-26T23:18:32.017162+0000","last_became_peered":"2024-04-26T23:18:32.017162+0000","last_unstale":"2024-04-26T23:18:37.046652+0000","last_undegraded":"2024-04-26T23:18:37.046652+0000","last_fullsized":"2024-04-26T23:18:37.046652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:48:51.877664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044589+0000","last_change":"2024-04-26T23:18:32.023064+0000","last_active":"2024-04-26T23:18:37.044589+0000","last_peered":"2024-04-26T23:18:37.044589+0000","last_clean":"2024-04-26T23:18:37.044589+0000","last_became_active":"2024-04-26T23:18:32.022771+0000","last_became_peered":"2024-04-26T23:18:32.022771+0000","last_unstale":"2024-04-26T23:18:37.044589+0000","last_undegraded":"2024-04-26T23:18:37.044589+0000","last_fullsized":"2024-04-26T23:18:37.044589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:37:21.828417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046963+0000","last_change":"2024-04-26T23:18:32.022130+0000","last_active":"2024-04-26T23:18:37.046963+0000","last_peered":"2024-04-26T23:18:37.046963+0000","last_clean":"2024-04-26T23:18:37.046963+0000","last_became_active":"2024-04-26T23:18:32.021842+0000","last_became_peered":"2024-04-26T23:18:32.021842+0000","last_unstale":"2024-04-26T23:18:37.046963+0000","last_undegraded":"2024-04-26T23:18:37.046963+0000","last_fullsized":"2024-04-26T23:18:37.046963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:10.370646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044765+0000","last_change":"2024-04-26T23:18:32.020676+0000","last_active":"2024-04-26T23:18:37.044765+0000","last_peered":"2024-04-26T23:18:37.044765+0000","last_clean":"2024-04-26T23:18:37.044765+0000","last_became_active":"2024-04-26T23:18:32.020391+0000","last_became_peered":"2024-04-26T23:18:32.020391+0000","last_unstale":"2024-04-26T23:18:37.044765+0000","last_undegraded":"2024-04-26T23:18:37.044765+0000","last_fullsized":"2024-04-26T23:18:37.044765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:05.295882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044140+0000","last_change":"2024-04-26T23:18:32.006201+0000","last_active":"2024-04-26T23:18:37.044140+0000","last_peered":"2024-04-26T23:18:37.044140+0000","last_clean":"2024-04-26T23:18:37.044140+0000","last_became_active":"2024-04-26T23:18:32.005906+0000","last_became_peered":"2024-04-26T23:18:32.005906+0000","last_unstale":"2024-04-26T23:18:37.044140+0000","last_undegraded":"2024-04-26T23:18:37.044140+0000","last_fullsized":"2024-04-26T23:18:37.044140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:59:18.268816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030075+0000","last_change":"2024-04-26T23:18:32.003546+0000","last_active":"2024-04-26T23:18:37.030075+0000","last_peered":"2024-04-26T23:18:37.030075+0000","last_clean":"2024-04-26T23:18:37.030075+0000","last_became_active":"2024-04-26T23:18:32.003204+0000","last_became_peered":"2024-04-26T23:18:32.003204+0000","last_unstale":"2024-04-26T23:18:37.030075+0000","last_undegraded":"2024-04-26T23:18:37.030075+0000","last_fullsized":"2024-04-26T23:18:37.030075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:37:21.828417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.047079+0000","last_change":"2024-04-26T23:18:32.026678+0000","last_active":"2024-04-26T23:18:37.047079+0000","last_peered":"2024-04-26T23:18:37.047079+0000","last_clean":"2024-04-26T23:18:37.047079+0000","last_became_active":"2024-04-26T23:18:32.026390+0000","last_became_peered":"2024-04-26T23:18:32.026390+0000","last_unstale":"2024-04-26T23:18:37.047079+0000","last_undegraded":"2024-04-26T23:18:37.047079+0000","last_fullsized":"2024-04-26T23:18:37.047079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:16:12.520377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542154,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":847872,"data_stored":690086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542156,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":262144,"data_stored":102978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542155,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":99226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542155,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":99262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31432,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340408,"statfs":{"total":96636764160,"available":96604577792,"internally_reserved":0,"allocated":262144,"data_stored":102978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":249856,"data_stored":99136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31432,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340408,"statfs":{"total":96636764160,"available":96604577792,"internally_reserved":0,"allocated":258048,"data_stored":99296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":856064,"data_stored":690142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:18:57.871 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T23:18:58.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.007+0000 7f75df16a640 1 Processor -- start 2024-04-26T23:18:58.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.007+0000 7f75df16a640 1 -- start start 2024-04-26T23:18:58.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.008+0000 7f75df16a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75d80cf5e0 0x7f75d80ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:58.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.008+0000 7f75df16a640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f75d80cbbd0 0x7f75d80cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:58.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.008+0000 7f75df16a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75d815b9e0 con 0x7f75d815b600 2024-04-26T23:18:58.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.008+0000 7f75df16a640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f75d80cb9b0 con 0x7f75d80cbbd0 2024-04-26T23:18:58.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.008+0000 7f75df16a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f75d80d0740 con 0x7f75d80cf5e0 2024-04-26T23:18:58.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.008+0000 7f75dd707640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f75d80cbbd0 0x7f75d80cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:58.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.008+0000 7f75d7fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75d80cf5e0 0x7f75d80ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:58.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.008+0000 7f75dcf06640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f75d815b600 0x7f75d8071f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.47:54154/0 (socket says 172.21.15.47:54154) 2024-04-26T23:18:58.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.008+0000 7f75dcf06640 1 -- 172.21.15.47:0/1122931307 learned_addr learned my addr 172.21.15.47:0/1122931307 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:18:58.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.009+0000 7f75dd707640 1 -- 172.21.15.47:0/1122931307 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75d80cf5e0 msgr2=0x7f75d80ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:58.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.009+0000 7f75dd707640 1 --2- 172.21.15.47:0/1122931307 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75d80cf5e0 0x7f75d80ce9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:58.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.009+0000 7f75dd707640 1 -- 172.21.15.47:0/1122931307 >> v1:172.21.15.203:6789/0 conn(0x7f75d815b600 legacy=0x7f75d8071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:58.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.009+0000 7f75dd707640 1 -- 172.21.15.47:0/1122931307 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d80cb360 con 0x7f75d80cbbd0 2024-04-26T23:18:58.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.009+0000 7f75dd707640 1 --2- 172.21.15.47:0/1122931307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75d80cbbd0 0x7f75d80cef10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f75d0009d90 tx=0x7f75d002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6e83bd195c13e0c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:58.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.010+0000 7f75d77fe640 1 -- 172.21.15.47:0/1122931307 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d0007740 con 0x7f75d80cbbd0 2024-04-26T23:18:58.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.010+0000 7f75d77fe640 1 -- 172.21.15.47:0/1122931307 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75d00078e0 con 0x7f75d80cbbd0 2024-04-26T23:18:58.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.010+0000 7f75d77fe640 1 -- 172.21.15.47:0/1122931307 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d0007cb0 con 0x7f75d80cbbd0 2024-04-26T23:18:58.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.010+0000 7f75df16a640 1 -- 172.21.15.47:0/1122931307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75d80cbbd0 msgr2=0x7f75d80cef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:58.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.010+0000 7f75df16a640 1 --2- 172.21.15.47:0/1122931307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75d80cbbd0 0x7f75d80cef10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f75d0009d90 tx=0x7f75d002a660 comp rx=0 tx=0).stop 2024-04-26T23:18:58.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.010+0000 7f75df16a640 1 -- 172.21.15.47:0/1122931307 shutdown_connections 2024-04-26T23:18:58.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.010+0000 7f75df16a640 1 --2- 172.21.15.47:0/1122931307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75d80cbbd0 0x7f75d80cef10 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:58.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.010+0000 7f75df16a640 1 --2- 172.21.15.47:0/1122931307 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75d80cf5e0 0x7f75d80ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:58.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.010+0000 7f75df16a640 1 -- 172.21.15.47:0/1122931307 >> 172.21.15.47:0/1122931307 conn(0x7f75d80d0be0 msgr2=0x7f75d80d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:58.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.010+0000 7f75df16a640 1 -- 172.21.15.47:0/1122931307 shutdown_connections 2024-04-26T23:18:58.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.010+0000 7f75df16a640 1 -- 172.21.15.47:0/1122931307 wait complete. 2024-04-26T23:18:58.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.011+0000 7f75df16a640 1 Processor -- start 2024-04-26T23:18:58.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.011+0000 7f75df16a640 1 -- start start 2024-04-26T23:18:58.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.012+0000 7f75df16a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75d80cbbd0 0x7f75d8146a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:58.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.012+0000 7f75df16a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75d80cf5e0 0x7f75d80b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:58.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.012+0000 7f75df16a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75d815b600 0x7f75d8142d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:58.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.012+0000 7f75df16a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f75d80d0470 con 0x7f75d80cbbd0 2024-04-26T23:18:58.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.012+0000 7f75df16a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f75d80d02f0 con 0x7f75d80cf5e0 2024-04-26T23:18:58.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.012+0000 7f75df16a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f75d806ddf0 con 0x7f75d815b600 2024-04-26T23:18:58.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.012+0000 7f75dcf06640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75d80cbbd0 0x7f75d8146a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:58.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.012+0000 7f75dcf06640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75d80cbbd0 0x7f75d8146a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58856/0 (socket says 172.21.15.47:58856) 2024-04-26T23:18:58.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.013+0000 7f75dcf06640 1 -- 172.21.15.47:0/2809021141 learned_addr learned my addr 172.21.15.47:0/2809021141 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:58.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.013+0000 7f75d7fff640 1 --2- 172.21.15.47:0/2809021141 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75d80cf5e0 0x7f75d80b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:58.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.013+0000 7f75dd707640 1 --2- 172.21.15.47:0/2809021141 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75d815b600 0x7f75d8142d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:58.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.013+0000 7f75dcf06640 1 -- 172.21.15.47:0/2809021141 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75d815b600 msgr2=0x7f75d8142d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:58.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.013+0000 7f75dcf06640 1 --2- 172.21.15.47:0/2809021141 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75d815b600 0x7f75d8142d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:58.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.013+0000 7f75dcf06640 1 -- 172.21.15.47:0/2809021141 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75d80cf5e0 msgr2=0x7f75d80b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:58.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.013+0000 7f75dcf06640 1 --2- 172.21.15.47:0/2809021141 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75d80cf5e0 0x7f75d80b54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:58.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.013+0000 7f75dcf06640 1 -- 172.21.15.47:0/2809021141 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d8146f40 con 0x7f75d80cbbd0 2024-04-26T23:18:58.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.013+0000 7f75dcf06640 1 --2- 172.21.15.47:0/2809021141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75d80cbbd0 0x7f75d8146a00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f75cc00c3f0 tx=0x7f75cc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:58.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.013+0000 7f75d57fa640 1 -- 172.21.15.47:0/2809021141 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75cc017070 con 0x7f75d80cbbd0 2024-04-26T23:18:58.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.013+0000 7f75d57fa640 1 -- 172.21.15.47:0/2809021141 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75cc005ce0 con 0x7f75d80cbbd0 2024-04-26T23:18:58.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.013+0000 7f75df16a640 1 -- 172.21.15.47:0/2809021141 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75d80b5fb0 con 0x7f75d80cbbd0 2024-04-26T23:18:58.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.013+0000 7f75d57fa640 1 -- 172.21.15.47:0/2809021141 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75cc0072f0 con 0x7f75d80cbbd0 2024-04-26T23:18:58.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.013+0000 7f75df16a640 1 -- 172.21.15.47:0/2809021141 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75d81520c0 con 0x7f75d80cbbd0 2024-04-26T23:18:58.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.014+0000 7f75df16a640 1 -- 172.21.15.47:0/2809021141 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75d80b64d0 con 0x7f75d80cbbd0 2024-04-26T23:18:58.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.015+0000 7f75d57fa640 1 -- 172.21.15.47:0/2809021141 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f75cc01a070 con 0x7f75d80cbbd0 2024-04-26T23:18:58.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.015+0000 7f75d57fa640 1 --2- 172.21.15.47:0/2809021141 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f75bc06c8f0 0x7f75bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:58.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.015+0000 7f75d57fa640 1 -- 172.21.15.47:0/2809021141 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f75cc08b210 con 0x7f75d80cbbd0 2024-04-26T23:18:58.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.015+0000 7f75d7fff640 1 --2- 172.21.15.47:0/2809021141 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f75bc06c8f0 0x7f75bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:58.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.016+0000 7f75d7fff640 1 --2- 172.21.15.47:0/2809021141 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f75bc06c8f0 0x7f75bc06edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f75d8072320 tx=0x7f75c4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:58.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.019+0000 7f75d57fa640 1 -- 172.21.15.47:0/2809021141 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f75cc059300 con 0x7f75d80cbbd0 2024-04-26T23:18:58.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.202+0000 7f75df16a640 1 -- 172.21.15.47:0/2809021141 --> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f75d80694f0 con 0x7f75bc06c8f0 2024-04-26T23:18:58.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.211+0000 7f75d57fa640 1 -- 172.21.15.47:0/2809021141 <== mgr.4099 v2:172.21.15.203:6832/992162732 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332529 (secure 0 0 0) 0x7f75d80694f0 con 0x7f75bc06c8f0 2024-04-26T23:18:58.217 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:18:58.219 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-26T23:18:58.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.215+0000 7f75df16a640 1 -- 172.21.15.47:0/2809021141 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f75bc06c8f0 msgr2=0x7f75bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:58.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.215+0000 7f75df16a640 1 --2- 172.21.15.47:0/2809021141 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f75bc06c8f0 0x7f75bc06edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f75d8072320 tx=0x7f75c4008040 comp rx=0 tx=0).stop 2024-04-26T23:18:58.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.215+0000 7f75df16a640 1 -- 172.21.15.47:0/2809021141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75d80cbbd0 msgr2=0x7f75d8146a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:58.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.215+0000 7f75df16a640 1 --2- 172.21.15.47:0/2809021141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75d80cbbd0 0x7f75d8146a00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f75cc00c3f0 tx=0x7f75cc00c8b0 comp rx=0 tx=0).stop 2024-04-26T23:18:58.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.215+0000 7f75df16a640 1 -- 172.21.15.47:0/2809021141 shutdown_connections 2024-04-26T23:18:58.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.215+0000 7f75df16a640 1 --2- 172.21.15.47:0/2809021141 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f75bc06c8f0 0x7f75bc06edb0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:58.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.215+0000 7f75df16a640 1 --2- 172.21.15.47:0/2809021141 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75d815b600 0x7f75d8142d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:58.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.215+0000 7f75df16a640 1 --2- 172.21.15.47:0/2809021141 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75d80cf5e0 0x7f75d80b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:58.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.215+0000 7f75df16a640 1 --2- 172.21.15.47:0/2809021141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75d80cbbd0 0x7f75d8146a00 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:58.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.215+0000 7f75df16a640 1 -- 172.21.15.47:0/2809021141 >> 172.21.15.47:0/2809021141 conn(0x7f75d80d0be0 msgr2=0x7f75d814e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:58.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.215+0000 7f75df16a640 1 -- 172.21.15.47:0/2809021141 shutdown_connections 2024-04-26T23:18:58.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.215+0000 7f75df16a640 1 -- 172.21.15.47:0/2809021141 wait complete. 2024-04-26T23:18:58.228 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2024-04-26T23:18:56.929714+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":231832,"kb_used_data":3176,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754742888,"statfs":{"total":773094113280,"available":772856717312,"internally_reserved":0,"allocated":3252224,"data_stored":1983104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":234016304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003411"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044475+0000","last_change":"2024-04-26T23:18:31.011895+0000","last_active":"2024-04-26T23:18:37.044475+0000","last_peered":"2024-04-26T23:18:37.044475+0000","last_clean":"2024-04-26T23:18:37.044475+0000","last_became_active":"2024-04-26T23:18:31.010976+0000","last_became_peered":"2024-04-26T23:18:31.010976+0000","last_unstale":"2024-04-26T23:18:37.044475+0000","last_undegraded":"2024-04-26T23:18:37.044475+0000","last_fullsized":"2024-04-26T23:18:37.044475+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:57:14.165871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030065+0000","last_change":"2024-04-26T23:18:32.020612+0000","last_active":"2024-04-26T23:18:37.030065+0000","last_peered":"2024-04-26T23:18:37.030065+0000","last_clean":"2024-04-26T23:18:37.030065+0000","last_became_active":"2024-04-26T23:18:32.018903+0000","last_became_peered":"2024-04-26T23:18:32.018903+0000","last_unstale":"2024-04-26T23:18:37.030065+0000","last_undegraded":"2024-04-26T23:18:37.030065+0000","last_fullsized":"2024-04-26T23:18:37.030065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:55:55.562630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029701+0000","last_change":"2024-04-26T23:18:36.051662+0000","last_active":"2024-04-26T23:18:37.029701+0000","last_peered":"2024-04-26T23:18:37.029701+0000","last_clean":"2024-04-26T23:18:37.029701+0000","last_became_active":"2024-04-26T23:18:36.051204+0000","last_became_peered":"2024-04-26T23:18:36.051204+0000","last_unstale":"2024-04-26T23:18:37.029701+0000","last_undegraded":"2024-04-26T23:18:37.029701+0000","last_fullsized":"2024-04-26T23:18:37.029701+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:24:24.209562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030279+0000","last_change":"2024-04-26T23:18:31.008954+0000","last_active":"2024-04-26T23:18:37.030279+0000","last_peered":"2024-04-26T23:18:37.030279+0000","last_clean":"2024-04-26T23:18:37.030279+0000","last_became_active":"2024-04-26T23:18:31.008597+0000","last_became_peered":"2024-04-26T23:18:31.008597+0000","last_unstale":"2024-04-26T23:18:37.030279+0000","last_undegraded":"2024-04-26T23:18:37.030279+0000","last_fullsized":"2024-04-26T23:18:37.030279+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:16:07.026710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044341+0000","last_change":"2024-04-26T23:18:32.019526+0000","last_active":"2024-04-26T23:18:37.044341+0000","last_peered":"2024-04-26T23:18:37.044341+0000","last_clean":"2024-04-26T23:18:37.044341+0000","last_became_active":"2024-04-26T23:18:32.019264+0000","last_became_peered":"2024-04-26T23:18:32.019264+0000","last_unstale":"2024-04-26T23:18:37.044341+0000","last_undegraded":"2024-04-26T23:18:37.044341+0000","last_fullsized":"2024-04-26T23:18:37.044341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:37:21.828417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044512+0000","last_change":"2024-04-26T23:18:36.041188+0000","last_active":"2024-04-26T23:18:37.044512+0000","last_peered":"2024-04-26T23:18:37.044512+0000","last_clean":"2024-04-26T23:18:37.044512+0000","last_became_active":"2024-04-26T23:18:36.040788+0000","last_became_peered":"2024-04-26T23:18:36.040788+0000","last_unstale":"2024-04-26T23:18:37.044512+0000","last_undegraded":"2024-04-26T23:18:37.044512+0000","last_fullsized":"2024-04-26T23:18:37.044512+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:49:51.239332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.047074+0000","last_change":"2024-04-26T23:18:31.011094+0000","last_active":"2024-04-26T23:18:37.047074+0000","last_peered":"2024-04-26T23:18:37.047074+0000","last_clean":"2024-04-26T23:18:37.047074+0000","last_became_active":"2024-04-26T23:18:31.010756+0000","last_became_peered":"2024-04-26T23:18:31.010756+0000","last_unstale":"2024-04-26T23:18:37.047074+0000","last_undegraded":"2024-04-26T23:18:37.047074+0000","last_fullsized":"2024-04-26T23:18:37.047074+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:28.986242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044870+0000","last_change":"2024-04-26T23:18:32.019790+0000","last_active":"2024-04-26T23:18:37.044870+0000","last_peered":"2024-04-26T23:18:37.044870+0000","last_clean":"2024-04-26T23:18:37.044870+0000","last_became_active":"2024-04-26T23:18:32.019535+0000","last_became_peered":"2024-04-26T23:18:32.019535+0000","last_unstale":"2024-04-26T23:18:37.044870+0000","last_undegraded":"2024-04-26T23:18:37.044870+0000","last_fullsized":"2024-04-26T23:18:37.044870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:10.370646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045799+0000","last_change":"2024-04-26T23:18:36.046954+0000","last_active":"2024-04-26T23:18:37.045799+0000","last_peered":"2024-04-26T23:18:37.045799+0000","last_clean":"2024-04-26T23:18:37.045799+0000","last_became_active":"2024-04-26T23:18:36.046588+0000","last_became_peered":"2024-04-26T23:18:36.046588+0000","last_unstale":"2024-04-26T23:18:37.045799+0000","last_undegraded":"2024-04-26T23:18:37.045799+0000","last_fullsized":"2024-04-26T23:18:37.045799+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:48:46.371866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046759+0000","last_change":"2024-04-26T23:18:31.011351+0000","last_active":"2024-04-26T23:18:37.046759+0000","last_peered":"2024-04-26T23:18:37.046759+0000","last_clean":"2024-04-26T23:18:37.046759+0000","last_became_active":"2024-04-26T23:18:31.010990+0000","last_became_peered":"2024-04-26T23:18:31.010990+0000","last_unstale":"2024-04-26T23:18:37.046759+0000","last_undegraded":"2024-04-26T23:18:37.046759+0000","last_fullsized":"2024-04-26T23:18:37.046759+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:08.243628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046200+0000","last_change":"2024-04-26T23:18:32.018999+0000","last_active":"2024-04-26T23:18:37.046200+0000","last_peered":"2024-04-26T23:18:37.046200+0000","last_clean":"2024-04-26T23:18:37.046200+0000","last_became_active":"2024-04-26T23:18:32.018713+0000","last_became_peered":"2024-04-26T23:18:32.018713+0000","last_unstale":"2024-04-26T23:18:37.046200+0000","last_undegraded":"2024-04-26T23:18:37.046200+0000","last_fullsized":"2024-04-26T23:18:37.046200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:39:01.950309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045585+0000","last_change":"2024-04-26T23:18:36.042244+0000","last_active":"2024-04-26T23:18:37.045585+0000","last_peered":"2024-04-26T23:18:37.045585+0000","last_clean":"2024-04-26T23:18:37.045585+0000","last_became_active":"2024-04-26T23:18:36.041833+0000","last_became_peered":"2024-04-26T23:18:36.041833+0000","last_unstale":"2024-04-26T23:18:37.045585+0000","last_undegraded":"2024-04-26T23:18:37.045585+0000","last_fullsized":"2024-04-26T23:18:37.045585+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:39:05.974905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045365+0000","last_change":"2024-04-26T23:18:32.017614+0000","last_active":"2024-04-26T23:18:37.045365+0000","last_peered":"2024-04-26T23:18:37.045365+0000","last_clean":"2024-04-26T23:18:37.045365+0000","last_became_active":"2024-04-26T23:18:32.017251+0000","last_became_peered":"2024-04-26T23:18:32.017251+0000","last_unstale":"2024-04-26T23:18:37.045365+0000","last_undegraded":"2024-04-26T23:18:37.045365+0000","last_fullsized":"2024-04-26T23:18:37.045365+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:37:21.828417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.047093+0000","last_change":"2024-04-26T23:18:31.011768+0000","last_active":"2024-04-26T23:18:37.047093+0000","last_peered":"2024-04-26T23:18:37.047093+0000","last_clean":"2024-04-26T23:18:37.047093+0000","last_became_active":"2024-04-26T23:18:31.011495+0000","last_became_peered":"2024-04-26T23:18:31.011495+0000","last_unstale":"2024-04-26T23:18:37.047093+0000","last_undegraded":"2024-04-26T23:18:37.047093+0000","last_fullsized":"2024-04-26T23:18:37.047093+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:25:38.515775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045600+0000","last_change":"2024-04-26T23:18:36.039079+0000","last_active":"2024-04-26T23:18:37.045600+0000","last_peered":"2024-04-26T23:18:37.045600+0000","last_clean":"2024-04-26T23:18:37.045600+0000","last_became_active":"2024-04-26T23:18:36.038665+0000","last_became_peered":"2024-04-26T23:18:36.038665+0000","last_unstale":"2024-04-26T23:18:37.045600+0000","last_undegraded":"2024-04-26T23:18:37.045600+0000","last_fullsized":"2024-04-26T23:18:37.045600+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:45:26.386750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029982+0000","last_change":"2024-04-26T23:18:32.012705+0000","last_active":"2024-04-26T23:18:37.029982+0000","last_peered":"2024-04-26T23:18:37.029982+0000","last_clean":"2024-04-26T23:18:37.029982+0000","last_became_active":"2024-04-26T23:18:32.012421+0000","last_became_peered":"2024-04-26T23:18:32.012421+0000","last_unstale":"2024-04-26T23:18:37.029982+0000","last_undegraded":"2024-04-26T23:18:37.029982+0000","last_fullsized":"2024-04-26T23:18:37.029982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:49:47.214737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030444+0000","last_change":"2024-04-26T23:18:31.011273+0000","last_active":"2024-04-26T23:18:37.030444+0000","last_peered":"2024-04-26T23:18:37.030444+0000","last_clean":"2024-04-26T23:18:37.030444+0000","last_became_active":"2024-04-26T23:18:31.010978+0000","last_became_peered":"2024-04-26T23:18:31.010978+0000","last_unstale":"2024-04-26T23:18:37.030444+0000","last_undegraded":"2024-04-26T23:18:37.030444+0000","last_fullsized":"2024-04-26T23:18:37.030444+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:13.713713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045147+0000","last_change":"2024-04-26T23:18:36.040606+0000","last_active":"2024-04-26T23:18:37.045147+0000","last_peered":"2024-04-26T23:18:37.045147+0000","last_clean":"2024-04-26T23:18:37.045147+0000","last_became_active":"2024-04-26T23:18:36.040058+0000","last_became_peered":"2024-04-26T23:18:36.040058+0000","last_unstale":"2024-04-26T23:18:37.045147+0000","last_undegraded":"2024-04-26T23:18:37.045147+0000","last_fullsized":"2024-04-26T23:18:37.045147+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:48:55.902259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030935+0000","last_change":"2024-04-26T23:18:32.015298+0000","last_active":"2024-04-26T23:18:37.030935+0000","last_peered":"2024-04-26T23:18:37.030935+0000","last_clean":"2024-04-26T23:18:37.030935+0000","last_became_active":"2024-04-26T23:18:32.014508+0000","last_became_peered":"2024-04-26T23:18:32.014508+0000","last_unstale":"2024-04-26T23:18:37.030935+0000","last_undegraded":"2024-04-26T23:18:37.030935+0000","last_fullsized":"2024-04-26T23:18:37.030935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:57:02.596299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029055+0000","last_change":"2024-04-26T23:18:31.008495+0000","last_active":"2024-04-26T23:18:37.029055+0000","last_peered":"2024-04-26T23:18:37.029055+0000","last_clean":"2024-04-26T23:18:37.029055+0000","last_became_active":"2024-04-26T23:18:31.008116+0000","last_became_peered":"2024-04-26T23:18:31.008116+0000","last_unstale":"2024-04-26T23:18:37.029055+0000","last_undegraded":"2024-04-26T23:18:37.029055+0000","last_fullsized":"2024-04-26T23:18:37.029055+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:34:02.747044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030423+0000","last_change":"2024-04-26T23:18:36.045871+0000","last_active":"2024-04-26T23:18:37.030423+0000","last_peered":"2024-04-26T23:18:37.030423+0000","last_clean":"2024-04-26T23:18:37.030423+0000","last_became_active":"2024-04-26T23:18:36.044888+0000","last_became_peered":"2024-04-26T23:18:36.044888+0000","last_unstale":"2024-04-26T23:18:37.030423+0000","last_undegraded":"2024-04-26T23:18:37.030423+0000","last_fullsized":"2024-04-26T23:18:37.030423+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:08:51.442254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045209+0000","last_change":"2024-04-26T23:18:32.015241+0000","last_active":"2024-04-26T23:18:37.045209+0000","last_peered":"2024-04-26T23:18:37.045209+0000","last_clean":"2024-04-26T23:18:37.045209+0000","last_became_active":"2024-04-26T23:18:32.013246+0000","last_became_peered":"2024-04-26T23:18:32.013246+0000","last_unstale":"2024-04-26T23:18:37.045209+0000","last_undegraded":"2024-04-26T23:18:37.045209+0000","last_fullsized":"2024-04-26T23:18:37.045209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:21.586522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045696+0000","last_change":"2024-04-26T23:18:31.015485+0000","last_active":"2024-04-26T23:18:37.045696+0000","last_peered":"2024-04-26T23:18:37.045696+0000","last_clean":"2024-04-26T23:18:37.045696+0000","last_became_active":"2024-04-26T23:18:31.015187+0000","last_became_peered":"2024-04-26T23:18:31.015187+0000","last_unstale":"2024-04-26T23:18:37.045696+0000","last_undegraded":"2024-04-26T23:18:37.045696+0000","last_fullsized":"2024-04-26T23:18:37.045696+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:53:15.812510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044493+0000","last_change":"2024-04-26T23:18:36.040493+0000","last_active":"2024-04-26T23:18:37.044493+0000","last_peered":"2024-04-26T23:18:37.044493+0000","last_clean":"2024-04-26T23:18:37.044493+0000","last_became_active":"2024-04-26T23:18:36.040040+0000","last_became_peered":"2024-04-26T23:18:36.040040+0000","last_unstale":"2024-04-26T23:18:37.044493+0000","last_undegraded":"2024-04-26T23:18:37.044493+0000","last_fullsized":"2024-04-26T23:18:37.044493+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:37:25.853012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044514+0000","last_change":"2024-04-26T23:18:32.001467+0000","last_active":"2024-04-26T23:18:37.044514+0000","last_peered":"2024-04-26T23:18:37.044514+0000","last_clean":"2024-04-26T23:18:37.044514+0000","last_became_active":"2024-04-26T23:18:32.001170+0000","last_became_peered":"2024-04-26T23:18:32.001170+0000","last_unstale":"2024-04-26T23:18:37.044514+0000","last_undegraded":"2024-04-26T23:18:37.044514+0000","last_fullsized":"2024-04-26T23:18:37.044514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:17:04.527690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046567+0000","last_change":"2024-04-26T23:18:31.014468+0000","last_active":"2024-04-26T23:18:37.046567+0000","last_peered":"2024-04-26T23:18:37.046567+0000","last_clean":"2024-04-26T23:18:37.046567+0000","last_became_active":"2024-04-26T23:18:31.014192+0000","last_became_peered":"2024-04-26T23:18:31.014192+0000","last_unstale":"2024-04-26T23:18:37.046567+0000","last_undegraded":"2024-04-26T23:18:37.046567+0000","last_fullsized":"2024-04-26T23:18:37.046567+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:20.581173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044988+0000","last_change":"2024-04-26T23:18:36.044805+0000","last_active":"2024-04-26T23:18:37.044988+0000","last_peered":"2024-04-26T23:18:37.044988+0000","last_clean":"2024-04-26T23:18:37.044988+0000","last_became_active":"2024-04-26T23:18:36.044409+0000","last_became_peered":"2024-04-26T23:18:36.044409+0000","last_unstale":"2024-04-26T23:18:37.044988+0000","last_undegraded":"2024-04-26T23:18:37.044988+0000","last_fullsized":"2024-04-26T23:18:37.044988+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:04:17.836184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030862+0000","last_change":"2024-04-26T23:18:32.018289+0000","last_active":"2024-04-26T23:18:37.030862+0000","last_peered":"2024-04-26T23:18:37.030862+0000","last_clean":"2024-04-26T23:18:37.030862+0000","last_became_active":"2024-04-26T23:18:32.017788+0000","last_became_peered":"2024-04-26T23:18:32.017788+0000","last_unstale":"2024-04-26T23:18:37.030862+0000","last_undegraded":"2024-04-26T23:18:37.030862+0000","last_fullsized":"2024-04-26T23:18:37.030862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:55.832329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030196+0000","last_change":"2024-04-26T23:18:31.009677+0000","last_active":"2024-04-26T23:18:37.030196+0000","last_peered":"2024-04-26T23:18:37.030196+0000","last_clean":"2024-04-26T23:18:37.030196+0000","last_became_active":"2024-04-26T23:18:31.009368+0000","last_became_peered":"2024-04-26T23:18:31.009368+0000","last_unstale":"2024-04-26T23:18:37.030196+0000","last_undegraded":"2024-04-26T23:18:37.030196+0000","last_fullsized":"2024-04-26T23:18:37.030196+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:28.986242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029507+0000","last_change":"2024-04-26T23:18:36.050766+0000","last_active":"2024-04-26T23:18:37.029507+0000","last_peered":"2024-04-26T23:18:37.029507+0000","last_clean":"2024-04-26T23:18:37.029507+0000","last_became_active":"2024-04-26T23:18:36.050091+0000","last_became_peered":"2024-04-26T23:18:36.050091+0000","last_unstale":"2024-04-26T23:18:37.029507+0000","last_undegraded":"2024-04-26T23:18:37.029507+0000","last_fullsized":"2024-04-26T23:18:37.029507+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:16:16.544972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029722+0000","last_change":"2024-04-26T23:18:32.007502+0000","last_active":"2024-04-26T23:18:37.029722+0000","last_peered":"2024-04-26T23:18:37.029722+0000","last_clean":"2024-04-26T23:18:37.029722+0000","last_became_active":"2024-04-26T23:18:32.006451+0000","last_became_peered":"2024-04-26T23:18:32.006451+0000","last_unstale":"2024-04-26T23:18:37.029722+0000","last_undegraded":"2024-04-26T23:18:37.029722+0000","last_fullsized":"2024-04-26T23:18:37.029722+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:55:55.562630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.047238+0000","last_change":"2024-04-26T23:18:31.012702+0000","last_active":"2024-04-26T23:18:37.047238+0000","last_peered":"2024-04-26T23:18:37.047238+0000","last_clean":"2024-04-26T23:18:37.047238+0000","last_became_active":"2024-04-26T23:18:31.012425+0000","last_became_peered":"2024-04-26T23:18:31.012425+0000","last_unstale":"2024-04-26T23:18:37.047238+0000","last_undegraded":"2024-04-26T23:18:37.047238+0000","last_fullsized":"2024-04-26T23:18:37.047238+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:08:31.793547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044060+0000","last_change":"2024-04-26T23:18:36.049119+0000","last_active":"2024-04-26T23:18:37.044060+0000","last_peered":"2024-04-26T23:18:37.044060+0000","last_clean":"2024-04-26T23:18:37.044060+0000","last_became_active":"2024-04-26T23:18:36.047621+0000","last_became_peered":"2024-04-26T23:18:36.047621+0000","last_unstale":"2024-04-26T23:18:37.044060+0000","last_undegraded":"2024-04-26T23:18:37.044060+0000","last_fullsized":"2024-04-26T23:18:37.044060+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:48:46.371866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046280+0000","last_change":"2024-04-26T23:18:32.021370+0000","last_active":"2024-04-26T23:18:37.046280+0000","last_peered":"2024-04-26T23:18:37.046280+0000","last_clean":"2024-04-26T23:18:37.046280+0000","last_became_active":"2024-04-26T23:18:32.020816+0000","last_became_peered":"2024-04-26T23:18:32.020816+0000","last_unstale":"2024-04-26T23:18:37.046280+0000","last_undegraded":"2024-04-26T23:18:37.046280+0000","last_fullsized":"2024-04-26T23:18:37.046280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:45:22.362155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029882+0000","last_change":"2024-04-26T23:18:31.007687+0000","last_active":"2024-04-26T23:18:37.029882+0000","last_peered":"2024-04-26T23:18:37.029882+0000","last_clean":"2024-04-26T23:18:37.029882+0000","last_became_active":"2024-04-26T23:18:31.006664+0000","last_became_peered":"2024-04-26T23:18:31.006664+0000","last_unstale":"2024-04-26T23:18:37.029882+0000","last_undegraded":"2024-04-26T23:18:37.029882+0000","last_fullsized":"2024-04-26T23:18:37.029882+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:44.844711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029232+0000","last_change":"2024-04-26T23:18:36.047590+0000","last_active":"2024-04-26T23:18:37.029232+0000","last_peered":"2024-04-26T23:18:37.029232+0000","last_clean":"2024-04-26T23:18:37.029232+0000","last_became_active":"2024-04-26T23:18:36.046796+0000","last_became_peered":"2024-04-26T23:18:36.046796+0000","last_unstale":"2024-04-26T23:18:37.029232+0000","last_undegraded":"2024-04-26T23:18:37.029232+0000","last_fullsized":"2024-04-26T23:18:37.029232+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:09.320477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046041+0000","last_change":"2024-04-26T23:18:32.002280+0000","last_active":"2024-04-26T23:18:37.046041+0000","last_peered":"2024-04-26T23:18:37.046041+0000","last_clean":"2024-04-26T23:18:37.046041+0000","last_became_active":"2024-04-26T23:18:32.002051+0000","last_became_peered":"2024-04-26T23:18:32.002051+0000","last_unstale":"2024-04-26T23:18:37.046041+0000","last_undegraded":"2024-04-26T23:18:37.046041+0000","last_fullsized":"2024-04-26T23:18:37.046041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:09.248977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044993+0000","last_change":"2024-04-26T23:18:31.011488+0000","last_active":"2024-04-26T23:18:37.044993+0000","last_peered":"2024-04-26T23:18:37.044993+0000","last_clean":"2024-04-26T23:18:37.044993+0000","last_became_active":"2024-04-26T23:18:31.010229+0000","last_became_peered":"2024-04-26T23:18:31.010229+0000","last_unstale":"2024-04-26T23:18:37.044993+0000","last_undegraded":"2024-04-26T23:18:37.044993+0000","last_fullsized":"2024-04-26T23:18:37.044993+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:25:38.515775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029825+0000","last_change":"2024-04-26T23:18:36.042997+0000","last_active":"2024-04-26T23:18:37.029825+0000","last_peered":"2024-04-26T23:18:37.029825+0000","last_clean":"2024-04-26T23:18:37.029825+0000","last_became_active":"2024-04-26T23:18:36.042510+0000","last_became_peered":"2024-04-26T23:18:36.042510+0000","last_unstale":"2024-04-26T23:18:37.029825+0000","last_undegraded":"2024-04-26T23:18:37.029825+0000","last_fullsized":"2024-04-26T23:18:37.029825+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:30.558000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.028944+0000","last_change":"2024-04-26T23:18:32.008615+0000","last_active":"2024-04-26T23:18:37.028944+0000","last_peered":"2024-04-26T23:18:37.028944+0000","last_clean":"2024-04-26T23:18:37.028944+0000","last_became_active":"2024-04-26T23:18:32.006670+0000","last_became_peered":"2024-04-26T23:18:32.006670+0000","last_unstale":"2024-04-26T23:18:37.028944+0000","last_undegraded":"2024-04-26T23:18:37.028944+0000","last_fullsized":"2024-04-26T23:18:37.028944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:54:25.468799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.027917+0000","last_change":"2024-04-26T23:18:31.005153+0000","last_active":"2024-04-26T23:18:37.027917+0000","last_peered":"2024-04-26T23:18:37.027917+0000","last_clean":"2024-04-26T23:18:37.027917+0000","last_became_active":"2024-04-26T23:18:31.004670+0000","last_became_peered":"2024-04-26T23:18:31.004670+0000","last_unstale":"2024-04-26T23:18:37.027917+0000","last_undegraded":"2024-04-26T23:18:37.027917+0000","last_fullsized":"2024-04-26T23:18:37.027917+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:34:02.747044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030347+0000","last_change":"2024-04-26T23:18:36.048181+0000","last_active":"2024-04-26T23:18:37.030347+0000","last_peered":"2024-04-26T23:18:37.030347+0000","last_clean":"2024-04-26T23:18:37.030347+0000","last_became_active":"2024-04-26T23:18:36.047762+0000","last_became_peered":"2024-04-26T23:18:36.047762+0000","last_unstale":"2024-04-26T23:18:37.030347+0000","last_undegraded":"2024-04-26T23:18:37.030347+0000","last_fullsized":"2024-04-26T23:18:37.030347+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:39:32.810976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.043710+0000","last_change":"2024-04-26T23:18:32.005125+0000","last_active":"2024-04-26T23:18:37.043710+0000","last_peered":"2024-04-26T23:18:37.043710+0000","last_clean":"2024-04-26T23:18:37.043710+0000","last_became_active":"2024-04-26T23:18:32.004826+0000","last_became_peered":"2024-04-26T23:18:32.004826+0000","last_unstale":"2024-04-26T23:18:37.043710+0000","last_undegraded":"2024-04-26T23:18:37.043710+0000","last_fullsized":"2024-04-26T23:18:37.043710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:53:16.817858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045815+0000","last_change":"2024-04-26T23:18:31.007674+0000","last_active":"2024-04-26T23:18:37.045815+0000","last_peered":"2024-04-26T23:18:37.045815+0000","last_clean":"2024-04-26T23:18:37.045815+0000","last_became_active":"2024-04-26T23:18:31.006787+0000","last_became_peered":"2024-04-26T23:18:31.006787+0000","last_unstale":"2024-04-26T23:18:37.045815+0000","last_undegraded":"2024-04-26T23:18:37.045815+0000","last_fullsized":"2024-04-26T23:18:37.045815+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:59:17.263468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044484+0000","last_change":"2024-04-26T23:18:36.046197+0000","last_active":"2024-04-26T23:18:37.044484+0000","last_peered":"2024-04-26T23:18:37.044484+0000","last_clean":"2024-04-26T23:18:37.044484+0000","last_became_active":"2024-04-26T23:18:36.045205+0000","last_became_peered":"2024-04-26T23:18:36.045205+0000","last_unstale":"2024-04-26T23:18:37.044484+0000","last_undegraded":"2024-04-26T23:18:37.044484+0000","last_fullsized":"2024-04-26T23:18:37.044484+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:39:32.810976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030371+0000","last_change":"2024-04-26T23:18:32.025774+0000","last_active":"2024-04-26T23:18:37.030371+0000","last_peered":"2024-04-26T23:18:37.030371+0000","last_clean":"2024-04-26T23:18:37.030371+0000","last_became_active":"2024-04-26T23:18:32.025487+0000","last_became_peered":"2024-04-26T23:18:32.025487+0000","last_unstale":"2024-04-26T23:18:37.030371+0000","last_undegraded":"2024-04-26T23:18:37.030371+0000","last_fullsized":"2024-04-26T23:18:37.030371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:04:13.811589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044628+0000","last_change":"2024-04-26T23:18:31.008744+0000","last_active":"2024-04-26T23:18:37.044628+0000","last_peered":"2024-04-26T23:18:37.044628+0000","last_clean":"2024-04-26T23:18:37.044628+0000","last_became_active":"2024-04-26T23:18:31.008449+0000","last_became_peered":"2024-04-26T23:18:31.008449+0000","last_unstale":"2024-04-26T23:18:37.044628+0000","last_undegraded":"2024-04-26T23:18:37.044628+0000","last_fullsized":"2024-04-26T23:18:37.044628+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:08.243628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045855+0000","last_change":"2024-04-26T23:18:18.862631+0000","last_active":"2024-04-26T23:18:37.045855+0000","last_peered":"2024-04-26T23:18:37.045855+0000","last_clean":"2024-04-26T23:18:37.045855+0000","last_became_active":"2024-04-26T23:18:18.861536+0000","last_became_peered":"2024-04-26T23:18:18.861536+0000","last_unstale":"2024-04-26T23:18:37.045855+0000","last_undegraded":"2024-04-26T23:18:37.045855+0000","last_fullsized":"2024-04-26T23:18:37.045855+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:17.808736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:17.808736+0000","last_clean_scrub_stamp":"2024-04-26T23:18:17.808736+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:59:26.359282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045887+0000","last_change":"2024-04-26T23:18:36.045014+0000","last_active":"2024-04-26T23:18:37.045887+0000","last_peered":"2024-04-26T23:18:37.045887+0000","last_clean":"2024-04-26T23:18:37.045887+0000","last_became_active":"2024-04-26T23:18:36.044467+0000","last_became_peered":"2024-04-26T23:18:36.044467+0000","last_unstale":"2024-04-26T23:18:37.045887+0000","last_undegraded":"2024-04-26T23:18:37.045887+0000","last_fullsized":"2024-04-26T23:18:37.045887+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:48:46.371866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046652+0000","last_change":"2024-04-26T23:18:32.030669+0000","last_active":"2024-04-26T23:18:37.046652+0000","last_peered":"2024-04-26T23:18:37.046652+0000","last_clean":"2024-04-26T23:18:37.046652+0000","last_became_active":"2024-04-26T23:18:32.030359+0000","last_became_peered":"2024-04-26T23:18:32.030359+0000","last_unstale":"2024-04-26T23:18:37.046652+0000","last_undegraded":"2024-04-26T23:18:37.046652+0000","last_fullsized":"2024-04-26T23:18:37.046652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:05.295882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.028346+0000","last_change":"2024-04-26T23:18:31.011343+0000","last_active":"2024-04-26T23:18:37.028346+0000","last_peered":"2024-04-26T23:18:37.028346+0000","last_clean":"2024-04-26T23:18:37.028346+0000","last_became_active":"2024-04-26T23:18:31.011011+0000","last_became_peered":"2024-04-26T23:18:31.011011+0000","last_unstale":"2024-04-26T23:18:37.028346+0000","last_undegraded":"2024-04-26T23:18:37.028346+0000","last_fullsized":"2024-04-26T23:18:37.028346+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:22.746946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045303+0000","last_change":"2024-04-26T23:18:36.045541+0000","last_active":"2024-04-26T23:18:37.045303+0000","last_peered":"2024-04-26T23:18:37.045303+0000","last_clean":"2024-04-26T23:18:37.045303+0000","last_became_active":"2024-04-26T23:18:36.045133+0000","last_became_peered":"2024-04-26T23:18:36.045133+0000","last_unstale":"2024-04-26T23:18:37.045303+0000","last_undegraded":"2024-04-26T23:18:37.045303+0000","last_fullsized":"2024-04-26T23:18:37.045303+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:45:26.386750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045276+0000","last_change":"2024-04-26T23:18:32.018614+0000","last_active":"2024-04-26T23:18:37.045276+0000","last_peered":"2024-04-26T23:18:37.045276+0000","last_clean":"2024-04-26T23:18:37.045276+0000","last_became_active":"2024-04-26T23:18:32.018088+0000","last_became_peered":"2024-04-26T23:18:32.018088+0000","last_unstale":"2024-04-26T23:18:37.045276+0000","last_undegraded":"2024-04-26T23:18:37.045276+0000","last_fullsized":"2024-04-26T23:18:37.045276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:26.533404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030223+0000","last_change":"2024-04-26T23:18:31.010377+0000","last_active":"2024-04-26T23:18:37.030223+0000","last_peered":"2024-04-26T23:18:37.030223+0000","last_clean":"2024-04-26T23:18:37.030223+0000","last_became_active":"2024-04-26T23:18:31.010084+0000","last_became_peered":"2024-04-26T23:18:31.010084+0000","last_unstale":"2024-04-26T23:18:37.030223+0000","last_undegraded":"2024-04-26T23:18:37.030223+0000","last_fullsized":"2024-04-26T23:18:37.030223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:08.243628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044284+0000","last_change":"2024-04-26T23:18:36.047754+0000","last_active":"2024-04-26T23:18:37.044284+0000","last_peered":"2024-04-26T23:18:37.044284+0000","last_clean":"2024-04-26T23:18:37.044284+0000","last_became_active":"2024-04-26T23:18:36.047105+0000","last_became_peered":"2024-04-26T23:18:36.047105+0000","last_unstale":"2024-04-26T23:18:37.044284+0000","last_undegraded":"2024-04-26T23:18:37.044284+0000","last_fullsized":"2024-04-26T23:18:37.044284+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:14.395241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.047247+0000","last_change":"2024-04-26T23:18:32.026649+0000","last_active":"2024-04-26T23:18:37.047247+0000","last_peered":"2024-04-26T23:18:37.047247+0000","last_clean":"2024-04-26T23:18:37.047247+0000","last_became_active":"2024-04-26T23:18:32.026014+0000","last_became_peered":"2024-04-26T23:18:32.026014+0000","last_unstale":"2024-04-26T23:18:37.047247+0000","last_undegraded":"2024-04-26T23:18:37.047247+0000","last_fullsized":"2024-04-26T23:18:37.047247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:08:47.417659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.028103+0000","last_change":"2024-04-26T23:18:31.006215+0000","last_active":"2024-04-26T23:18:37.028103+0000","last_peered":"2024-04-26T23:18:37.028103+0000","last_clean":"2024-04-26T23:18:37.028103+0000","last_became_active":"2024-04-26T23:18:31.005847+0000","last_became_peered":"2024-04-26T23:18:31.005847+0000","last_unstale":"2024-04-26T23:18:37.028103+0000","last_undegraded":"2024-04-26T23:18:37.028103+0000","last_fullsized":"2024-04-26T23:18:37.028103+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:44.844711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044109+0000","last_change":"2024-04-26T23:18:36.045812+0000","last_active":"2024-04-26T23:18:37.044109+0000","last_peered":"2024-04-26T23:18:37.044109+0000","last_clean":"2024-04-26T23:18:37.044109+0000","last_became_active":"2024-04-26T23:18:36.045445+0000","last_became_peered":"2024-04-26T23:18:36.045445+0000","last_unstale":"2024-04-26T23:18:37.044109+0000","last_undegraded":"2024-04-26T23:18:37.044109+0000","last_fullsized":"2024-04-26T23:18:37.044109+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:16:16.544972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046893+0000","last_change":"2024-04-26T23:18:32.025072+0000","last_active":"2024-04-26T23:18:37.046893+0000","last_peered":"2024-04-26T23:18:37.046893+0000","last_clean":"2024-04-26T23:18:37.046893+0000","last_became_active":"2024-04-26T23:18:32.024754+0000","last_became_peered":"2024-04-26T23:18:32.024754+0000","last_unstale":"2024-04-26T23:18:37.046893+0000","last_undegraded":"2024-04-26T23:18:37.046893+0000","last_fullsized":"2024-04-26T23:18:37.046893+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:45:22.362155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044833+0000","last_change":"2024-04-26T23:18:31.013340+0000","last_active":"2024-04-26T23:18:37.044833+0000","last_peered":"2024-04-26T23:18:37.044833+0000","last_clean":"2024-04-26T23:18:37.044833+0000","last_became_active":"2024-04-26T23:18:31.012824+0000","last_became_peered":"2024-04-26T23:18:31.012824+0000","last_unstale":"2024-04-26T23:18:37.044833+0000","last_undegraded":"2024-04-26T23:18:37.044833+0000","last_fullsized":"2024-04-26T23:18:37.044833+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:17.988227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044762+0000","last_change":"2024-04-26T23:18:36.041004+0000","last_active":"2024-04-26T23:18:37.044762+0000","last_peered":"2024-04-26T23:18:37.044762+0000","last_clean":"2024-04-26T23:18:37.044762+0000","last_became_active":"2024-04-26T23:18:36.040155+0000","last_became_peered":"2024-04-26T23:18:36.040155+0000","last_unstale":"2024-04-26T23:18:37.044762+0000","last_undegraded":"2024-04-26T23:18:37.044762+0000","last_fullsized":"2024-04-26T23:18:37.044762+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:59.856924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046692+0000","last_change":"2024-04-26T23:18:32.025214+0000","last_active":"2024-04-26T23:18:37.046692+0000","last_peered":"2024-04-26T23:18:37.046692+0000","last_clean":"2024-04-26T23:18:37.046692+0000","last_became_active":"2024-04-26T23:18:32.024952+0000","last_became_peered":"2024-04-26T23:18:32.024952+0000","last_unstale":"2024-04-26T23:18:37.046692+0000","last_undegraded":"2024-04-26T23:18:37.046692+0000","last_fullsized":"2024-04-26T23:18:37.046692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:39:01.950309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.027616+0000","last_change":"2024-04-26T23:18:31.001581+0000","last_active":"2024-04-26T23:18:37.027616+0000","last_peered":"2024-04-26T23:18:37.027616+0000","last_clean":"2024-04-26T23:18:37.027616+0000","last_became_active":"2024-04-26T23:18:31.001256+0000","last_became_peered":"2024-04-26T23:18:31.001256+0000","last_unstale":"2024-04-26T23:18:37.027616+0000","last_undegraded":"2024-04-26T23:18:37.027616+0000","last_fullsized":"2024-04-26T23:18:37.027616+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:13.713713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044283+0000","last_change":"2024-04-26T23:18:36.040673+0000","last_active":"2024-04-26T23:18:37.044283+0000","last_peered":"2024-04-26T23:18:37.044283+0000","last_clean":"2024-04-26T23:18:37.044283+0000","last_became_active":"2024-04-26T23:18:36.040246+0000","last_became_peered":"2024-04-26T23:18:36.040246+0000","last_unstale":"2024-04-26T23:18:37.044283+0000","last_undegraded":"2024-04-26T23:18:37.044283+0000","last_fullsized":"2024-04-26T23:18:37.044283+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:30.558000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044353+0000","last_change":"2024-04-26T23:18:32.013824+0000","last_active":"2024-04-26T23:18:37.044353+0000","last_peered":"2024-04-26T23:18:37.044353+0000","last_clean":"2024-04-26T23:18:37.044353+0000","last_became_active":"2024-04-26T23:18:32.012613+0000","last_became_peered":"2024-04-26T23:18:32.012613+0000","last_unstale":"2024-04-26T23:18:37.044353+0000","last_undegraded":"2024-04-26T23:18:37.044353+0000","last_fullsized":"2024-04-26T23:18:37.044353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:49:47.214737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044914+0000","last_change":"2024-04-26T23:18:31.011248+0000","last_active":"2024-04-26T23:18:37.044914+0000","last_peered":"2024-04-26T23:18:37.044914+0000","last_clean":"2024-04-26T23:18:37.044914+0000","last_became_active":"2024-04-26T23:18:31.010965+0000","last_became_peered":"2024-04-26T23:18:31.010965+0000","last_unstale":"2024-04-26T23:18:37.044914+0000","last_undegraded":"2024-04-26T23:18:37.044914+0000","last_fullsized":"2024-04-26T23:18:37.044914+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:13.713713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030962+0000","last_change":"2024-04-26T23:18:36.045330+0000","last_active":"2024-04-26T23:18:37.030962+0000","last_peered":"2024-04-26T23:18:37.030962+0000","last_clean":"2024-04-26T23:18:37.030962+0000","last_became_active":"2024-04-26T23:18:36.044903+0000","last_became_peered":"2024-04-26T23:18:36.044903+0000","last_unstale":"2024-04-26T23:18:37.030962+0000","last_undegraded":"2024-04-26T23:18:37.030962+0000","last_fullsized":"2024-04-26T23:18:37.030962+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:53:15.682467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.047083+0000","last_change":"2024-04-26T23:18:32.026417+0000","last_active":"2024-04-26T23:18:37.047083+0000","last_peered":"2024-04-26T23:18:37.047083+0000","last_clean":"2024-04-26T23:18:37.047083+0000","last_became_active":"2024-04-26T23:18:32.025839+0000","last_became_peered":"2024-04-26T23:18:32.025839+0000","last_unstale":"2024-04-26T23:18:37.047083+0000","last_undegraded":"2024-04-26T23:18:37.047083+0000","last_fullsized":"2024-04-26T23:18:37.047083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:39:28.786381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044878+0000","last_change":"2024-04-26T23:18:31.011956+0000","last_active":"2024-04-26T23:18:37.044878+0000","last_peered":"2024-04-26T23:18:37.044878+0000","last_clean":"2024-04-26T23:18:37.044878+0000","last_became_active":"2024-04-26T23:18:31.010493+0000","last_became_peered":"2024-04-26T23:18:31.010493+0000","last_unstale":"2024-04-26T23:18:37.044878+0000","last_undegraded":"2024-04-26T23:18:37.044878+0000","last_fullsized":"2024-04-26T23:18:37.044878+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:59:17.263468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030567+0000","last_change":"2024-04-26T23:18:36.047105+0000","last_active":"2024-04-26T23:18:37.030567+0000","last_peered":"2024-04-26T23:18:37.030567+0000","last_clean":"2024-04-26T23:18:37.030567+0000","last_became_active":"2024-04-26T23:18:36.046724+0000","last_became_peered":"2024-04-26T23:18:36.046724+0000","last_unstale":"2024-04-26T23:18:37.030567+0000","last_undegraded":"2024-04-26T23:18:37.030567+0000","last_fullsized":"2024-04-26T23:18:37.030567+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:46:07.146408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.043961+0000","last_change":"2024-04-26T23:18:32.015408+0000","last_active":"2024-04-26T23:18:37.043961+0000","last_peered":"2024-04-26T23:18:37.043961+0000","last_clean":"2024-04-26T23:18:37.043961+0000","last_became_active":"2024-04-26T23:18:32.015037+0000","last_became_peered":"2024-04-26T23:18:32.015037+0000","last_unstale":"2024-04-26T23:18:37.043961+0000","last_undegraded":"2024-04-26T23:18:37.043961+0000","last_fullsized":"2024-04-26T23:18:37.043961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:49.567077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044345+0000","last_change":"2024-04-26T23:18:31.010413+0000","last_active":"2024-04-26T23:18:37.044345+0000","last_peered":"2024-04-26T23:18:37.044345+0000","last_clean":"2024-04-26T23:18:37.044345+0000","last_became_active":"2024-04-26T23:18:31.010138+0000","last_became_peered":"2024-04-26T23:18:31.010138+0000","last_unstale":"2024-04-26T23:18:37.044345+0000","last_undegraded":"2024-04-26T23:18:37.044345+0000","last_fullsized":"2024-04-26T23:18:37.044345+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:15:53.470195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046231+0000","last_change":"2024-04-26T23:18:36.044782+0000","last_active":"2024-04-26T23:18:37.046231+0000","last_peered":"2024-04-26T23:18:37.046231+0000","last_clean":"2024-04-26T23:18:37.046231+0000","last_became_active":"2024-04-26T23:18:36.044157+0000","last_became_peered":"2024-04-26T23:18:36.044157+0000","last_unstale":"2024-04-26T23:18:37.046231+0000","last_undegraded":"2024-04-26T23:18:37.046231+0000","last_fullsized":"2024-04-26T23:18:37.046231+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:24:24.209562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046631+0000","last_change":"2024-04-26T23:18:32.023453+0000","last_active":"2024-04-26T23:18:37.046631+0000","last_peered":"2024-04-26T23:18:37.046631+0000","last_clean":"2024-04-26T23:18:37.046631+0000","last_became_active":"2024-04-26T23:18:32.023170+0000","last_became_peered":"2024-04-26T23:18:32.023170+0000","last_unstale":"2024-04-26T23:18:37.046631+0000","last_undegraded":"2024-04-26T23:18:37.046631+0000","last_fullsized":"2024-04-26T23:18:37.046631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:48:42.347270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:41.489012+0000","last_change":"2024-04-26T23:18:31.013165+0000","last_active":"2024-04-26T23:18:41.489012+0000","last_peered":"2024-04-26T23:18:41.489012+0000","last_clean":"2024-04-26T23:18:41.489012+0000","last_became_active":"2024-04-26T23:18:31.012790+0000","last_became_peered":"2024-04-26T23:18:31.012790+0000","last_unstale":"2024-04-26T23:18:41.489012+0000","last_undegraded":"2024-04-26T23:18:41.489012+0000","last_fullsized":"2024-04-26T23:18:41.489012+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:44.844711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029334+0000","last_change":"2024-04-26T23:18:36.047231+0000","last_active":"2024-04-26T23:18:37.029334+0000","last_peered":"2024-04-26T23:18:37.029334+0000","last_clean":"2024-04-26T23:18:37.029334+0000","last_became_active":"2024-04-26T23:18:36.046579+0000","last_became_peered":"2024-04-26T23:18:36.046579+0000","last_unstale":"2024-04-26T23:18:37.029334+0000","last_undegraded":"2024-04-26T23:18:37.029334+0000","last_fullsized":"2024-04-26T23:18:37.029334+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:14.395241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045390+0000","last_change":"2024-04-26T23:18:32.022720+0000","last_active":"2024-04-26T23:18:37.045390+0000","last_peered":"2024-04-26T23:18:37.045390+0000","last_clean":"2024-04-26T23:18:37.045390+0000","last_became_active":"2024-04-26T23:18:32.022450+0000","last_became_peered":"2024-04-26T23:18:32.022450+0000","last_unstale":"2024-04-26T23:18:37.045390+0000","last_undegraded":"2024-04-26T23:18:37.045390+0000","last_fullsized":"2024-04-26T23:18:37.045390+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:25:39.521124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029950+0000","last_change":"2024-04-26T23:18:31.008154+0000","last_active":"2024-04-26T23:18:37.029950+0000","last_peered":"2024-04-26T23:18:37.029950+0000","last_clean":"2024-04-26T23:18:37.029950+0000","last_became_active":"2024-04-26T23:18:31.007881+0000","last_became_peered":"2024-04-26T23:18:31.007881+0000","last_unstale":"2024-04-26T23:18:37.029950+0000","last_undegraded":"2024-04-26T23:18:37.029950+0000","last_fullsized":"2024-04-26T23:18:37.029950+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:20.581173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046058+0000","last_change":"2024-04-26T23:18:36.047010+0000","last_active":"2024-04-26T23:18:37.046058+0000","last_peered":"2024-04-26T23:18:37.046058+0000","last_clean":"2024-04-26T23:18:37.046058+0000","last_became_active":"2024-04-26T23:18:36.045988+0000","last_became_peered":"2024-04-26T23:18:36.045988+0000","last_unstale":"2024-04-26T23:18:37.046058+0000","last_undegraded":"2024-04-26T23:18:37.046058+0000","last_fullsized":"2024-04-26T23:18:37.046058+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:09.320477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044476+0000","last_change":"2024-04-26T23:18:32.019460+0000","last_active":"2024-04-26T23:18:37.044476+0000","last_peered":"2024-04-26T23:18:37.044476+0000","last_clean":"2024-04-26T23:18:37.044476+0000","last_became_active":"2024-04-26T23:18:32.019146+0000","last_became_peered":"2024-04-26T23:18:32.019146+0000","last_unstale":"2024-04-26T23:18:37.044476+0000","last_undegraded":"2024-04-26T23:18:37.044476+0000","last_fullsized":"2024-04-26T23:18:37.044476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:55:55.562630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045137+0000","last_change":"2024-04-26T23:18:31.010513+0000","last_active":"2024-04-26T23:18:37.045137+0000","last_peered":"2024-04-26T23:18:37.045137+0000","last_clean":"2024-04-26T23:18:37.045137+0000","last_became_active":"2024-04-26T23:18:31.010206+0000","last_became_peered":"2024-04-26T23:18:31.010206+0000","last_unstale":"2024-04-26T23:18:37.045137+0000","last_undegraded":"2024-04-26T23:18:37.045137+0000","last_fullsized":"2024-04-26T23:18:37.045137+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:17:03.522342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030875+0000","last_change":"2024-04-26T23:18:36.039670+0000","last_active":"2024-04-26T23:18:37.030875+0000","last_peered":"2024-04-26T23:18:37.030875+0000","last_clean":"2024-04-26T23:18:37.030875+0000","last_became_active":"2024-04-26T23:18:36.039247+0000","last_became_peered":"2024-04-26T23:18:36.039247+0000","last_unstale":"2024-04-26T23:18:37.030875+0000","last_undegraded":"2024-04-26T23:18:37.030875+0000","last_fullsized":"2024-04-26T23:18:37.030875+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:45:37.196730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045040+0000","last_change":"2024-04-26T23:18:32.029408+0000","last_active":"2024-04-26T23:18:37.045040+0000","last_peered":"2024-04-26T23:18:37.045040+0000","last_clean":"2024-04-26T23:18:37.045040+0000","last_became_active":"2024-04-26T23:18:32.029150+0000","last_became_peered":"2024-04-26T23:18:32.029150+0000","last_unstale":"2024-04-26T23:18:37.045040+0000","last_undegraded":"2024-04-26T23:18:37.045040+0000","last_fullsized":"2024-04-26T23:18:37.045040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:48:42.347270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046379+0000","last_change":"2024-04-26T23:18:31.013885+0000","last_active":"2024-04-26T23:18:37.046379+0000","last_peered":"2024-04-26T23:18:37.046379+0000","last_clean":"2024-04-26T23:18:37.046379+0000","last_became_active":"2024-04-26T23:18:31.013617+0000","last_became_peered":"2024-04-26T23:18:31.013617+0000","last_unstale":"2024-04-26T23:18:37.046379+0000","last_undegraded":"2024-04-26T23:18:37.046379+0000","last_fullsized":"2024-04-26T23:18:37.046379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:08.243628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029721+0000","last_change":"2024-04-26T23:18:36.049794+0000","last_active":"2024-04-26T23:18:37.029721+0000","last_peered":"2024-04-26T23:18:37.029721+0000","last_clean":"2024-04-26T23:18:37.029721+0000","last_became_active":"2024-04-26T23:18:36.044164+0000","last_became_peered":"2024-04-26T23:18:36.044164+0000","last_unstale":"2024-04-26T23:18:37.029721+0000","last_undegraded":"2024-04-26T23:18:37.029721+0000","last_fullsized":"2024-04-26T23:18:37.029721+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:41:38.190204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044157+0000","last_change":"2024-04-26T23:18:36.046931+0000","last_active":"2024-04-26T23:18:37.044157+0000","last_peered":"2024-04-26T23:18:37.044157+0000","last_clean":"2024-04-26T23:18:37.044157+0000","last_became_active":"2024-04-26T23:18:36.045786+0000","last_became_peered":"2024-04-26T23:18:36.045786+0000","last_unstale":"2024-04-26T23:18:37.044157+0000","last_undegraded":"2024-04-26T23:18:37.044157+0000","last_fullsized":"2024-04-26T23:18:37.044157+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:09.320477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.028930+0000","last_change":"2024-04-26T23:18:32.011300+0000","last_active":"2024-04-26T23:18:37.028930+0000","last_peered":"2024-04-26T23:18:37.028930+0000","last_clean":"2024-04-26T23:18:37.028930+0000","last_became_active":"2024-04-26T23:18:32.010998+0000","last_became_peered":"2024-04-26T23:18:32.010998+0000","last_unstale":"2024-04-26T23:18:37.028930+0000","last_undegraded":"2024-04-26T23:18:37.028930+0000","last_fullsized":"2024-04-26T23:18:37.028930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:57:02.596299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044365+0000","last_change":"2024-04-26T23:18:31.011198+0000","last_active":"2024-04-26T23:18:37.044365+0000","last_peered":"2024-04-26T23:18:37.044365+0000","last_clean":"2024-04-26T23:18:37.044365+0000","last_became_active":"2024-04-26T23:18:31.010908+0000","last_became_peered":"2024-04-26T23:18:31.010908+0000","last_unstale":"2024-04-26T23:18:37.044365+0000","last_undegraded":"2024-04-26T23:18:37.044365+0000","last_fullsized":"2024-04-26T23:18:37.044365+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:22.746946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046251+0000","last_change":"2024-04-26T23:18:36.042049+0000","last_active":"2024-04-26T23:18:37.046251+0000","last_peered":"2024-04-26T23:18:37.046251+0000","last_clean":"2024-04-26T23:18:37.046251+0000","last_became_active":"2024-04-26T23:18:36.041661+0000","last_became_peered":"2024-04-26T23:18:36.041661+0000","last_unstale":"2024-04-26T23:18:37.046251+0000","last_undegraded":"2024-04-26T23:18:37.046251+0000","last_fullsized":"2024-04-26T23:18:37.046251+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:09.320477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030801+0000","last_change":"2024-04-26T23:18:32.013285+0000","last_active":"2024-04-26T23:18:37.030801+0000","last_peered":"2024-04-26T23:18:37.030801+0000","last_clean":"2024-04-26T23:18:37.030801+0000","last_became_active":"2024-04-26T23:18:32.013013+0000","last_became_peered":"2024-04-26T23:18:32.013013+0000","last_unstale":"2024-04-26T23:18:37.030801+0000","last_undegraded":"2024-04-26T23:18:37.030801+0000","last_fullsized":"2024-04-26T23:18:37.030801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:54:25.468799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029792+0000","last_change":"2024-04-26T23:18:31.011691+0000","last_active":"2024-04-26T23:18:37.029792+0000","last_peered":"2024-04-26T23:18:37.029792+0000","last_clean":"2024-04-26T23:18:37.029792+0000","last_became_active":"2024-04-26T23:18:31.009521+0000","last_became_peered":"2024-04-26T23:18:31.009521+0000","last_unstale":"2024-04-26T23:18:37.029792+0000","last_undegraded":"2024-04-26T23:18:37.029792+0000","last_fullsized":"2024-04-26T23:18:37.029792+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:22.746946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045573+0000","last_change":"2024-04-26T23:18:31.010360+0000","last_active":"2024-04-26T23:18:37.045573+0000","last_peered":"2024-04-26T23:18:37.045573+0000","last_clean":"2024-04-26T23:18:37.045573+0000","last_became_active":"2024-04-26T23:18:31.010083+0000","last_became_peered":"2024-04-26T23:18:31.010083+0000","last_unstale":"2024-04-26T23:18:37.045573+0000","last_undegraded":"2024-04-26T23:18:37.045573+0000","last_fullsized":"2024-04-26T23:18:37.045573+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:34:02.747044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044258+0000","last_change":"2024-04-26T23:18:32.000871+0000","last_active":"2024-04-26T23:18:37.044258+0000","last_peered":"2024-04-26T23:18:37.044258+0000","last_clean":"2024-04-26T23:18:37.044258+0000","last_became_active":"2024-04-26T23:18:31.999865+0000","last_became_peered":"2024-04-26T23:18:31.999865+0000","last_unstale":"2024-04-26T23:18:37.044258+0000","last_undegraded":"2024-04-26T23:18:37.044258+0000","last_fullsized":"2024-04-26T23:18:37.044258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:57:02.596299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.028327+0000","last_change":"2024-04-26T23:18:36.040907+0000","last_active":"2024-04-26T23:18:37.028327+0000","last_peered":"2024-04-26T23:18:37.028327+0000","last_clean":"2024-04-26T23:18:37.028327+0000","last_became_active":"2024-04-26T23:18:36.040494+0000","last_became_peered":"2024-04-26T23:18:36.040494+0000","last_unstale":"2024-04-26T23:18:37.028327+0000","last_undegraded":"2024-04-26T23:18:37.028327+0000","last_fullsized":"2024-04-26T23:18:37.028327+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:17:08.552285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030078+0000","last_change":"2024-04-26T23:18:31.012855+0000","last_active":"2024-04-26T23:18:37.030078+0000","last_peered":"2024-04-26T23:18:37.030078+0000","last_clean":"2024-04-26T23:18:37.030078+0000","last_became_active":"2024-04-26T23:18:31.012510+0000","last_became_peered":"2024-04-26T23:18:31.012510+0000","last_unstale":"2024-04-26T23:18:37.030078+0000","last_undegraded":"2024-04-26T23:18:37.030078+0000","last_fullsized":"2024-04-26T23:18:37.030078+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:29.979716+0000","last_clean_scrub_stamp":"2024-04-26T23:18:29.979716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:15:53.470195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045692+0000","last_change":"2024-04-26T23:18:32.002285+0000","last_active":"2024-04-26T23:18:37.045692+0000","last_peered":"2024-04-26T23:18:37.045692+0000","last_clean":"2024-04-26T23:18:37.045692+0000","last_became_active":"2024-04-26T23:18:32.001991+0000","last_became_peered":"2024-04-26T23:18:32.001991+0000","last_unstale":"2024-04-26T23:18:37.045692+0000","last_undegraded":"2024-04-26T23:18:37.045692+0000","last_fullsized":"2024-04-26T23:18:37.045692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:45.850060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030744+0000","last_change":"2024-04-26T23:18:36.038108+0000","last_active":"2024-04-26T23:18:37.030744+0000","last_peered":"2024-04-26T23:18:37.030744+0000","last_clean":"2024-04-26T23:18:37.030744+0000","last_became_active":"2024-04-26T23:18:36.037655+0000","last_became_peered":"2024-04-26T23:18:36.037655+0000","last_unstale":"2024-04-26T23:18:37.030744+0000","last_undegraded":"2024-04-26T23:18:37.030744+0000","last_fullsized":"2024-04-26T23:18:37.030744+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:35.009660+0000","last_clean_scrub_stamp":"2024-04-26T23:18:35.009660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:02.473474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044156+0000","last_change":"2024-04-26T23:18:32.014058+0000","last_active":"2024-04-26T23:18:37.044156+0000","last_peered":"2024-04-26T23:18:37.044156+0000","last_clean":"2024-04-26T23:18:37.044156+0000","last_became_active":"2024-04-26T23:18:32.013500+0000","last_became_peered":"2024-04-26T23:18:32.013500+0000","last_unstale":"2024-04-26T23:18:37.044156+0000","last_undegraded":"2024-04-26T23:18:37.044156+0000","last_fullsized":"2024-04-26T23:18:37.044156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:54:25.468799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.027802+0000","last_change":"2024-04-26T23:18:32.018280+0000","last_active":"2024-04-26T23:18:37.027802+0000","last_peered":"2024-04-26T23:18:37.027802+0000","last_clean":"2024-04-26T23:18:37.027802+0000","last_became_active":"2024-04-26T23:18:32.017993+0000","last_became_peered":"2024-04-26T23:18:32.017993+0000","last_unstale":"2024-04-26T23:18:37.027802+0000","last_undegraded":"2024-04-26T23:18:37.027802+0000","last_fullsized":"2024-04-26T23:18:37.027802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:08:32.798896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044791+0000","last_change":"2024-04-26T23:18:32.022457+0000","last_active":"2024-04-26T23:18:37.044791+0000","last_peered":"2024-04-26T23:18:37.044791+0000","last_clean":"2024-04-26T23:18:37.044791+0000","last_became_active":"2024-04-26T23:18:32.021766+0000","last_became_peered":"2024-04-26T23:18:32.021766+0000","last_unstale":"2024-04-26T23:18:37.044791+0000","last_undegraded":"2024-04-26T23:18:37.044791+0000","last_fullsized":"2024-04-26T23:18:37.044791+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:48:51.877664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044133+0000","last_change":"2024-04-26T23:18:32.011541+0000","last_active":"2024-04-26T23:18:37.044133+0000","last_peered":"2024-04-26T23:18:37.044133+0000","last_clean":"2024-04-26T23:18:37.044133+0000","last_became_active":"2024-04-26T23:18:32.009252+0000","last_became_peered":"2024-04-26T23:18:32.009252+0000","last_unstale":"2024-04-26T23:18:37.044133+0000","last_undegraded":"2024-04-26T23:18:37.044133+0000","last_fullsized":"2024-04-26T23:18:37.044133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:54:25.468799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030659+0000","last_change":"2024-04-26T23:18:32.018905+0000","last_active":"2024-04-26T23:18:37.030659+0000","last_peered":"2024-04-26T23:18:37.030659+0000","last_clean":"2024-04-26T23:18:37.030659+0000","last_became_active":"2024-04-26T23:18:32.018586+0000","last_became_peered":"2024-04-26T23:18:32.018586+0000","last_unstale":"2024-04-26T23:18:37.030659+0000","last_undegraded":"2024-04-26T23:18:37.030659+0000","last_fullsized":"2024-04-26T23:18:37.030659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:37:21.828417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029798+0000","last_change":"2024-04-26T23:18:32.011775+0000","last_active":"2024-04-26T23:18:37.029798+0000","last_peered":"2024-04-26T23:18:37.029798+0000","last_clean":"2024-04-26T23:18:37.029798+0000","last_became_active":"2024-04-26T23:18:32.011443+0000","last_became_peered":"2024-04-26T23:18:32.011443+0000","last_unstale":"2024-04-26T23:18:37.029798+0000","last_undegraded":"2024-04-26T23:18:37.029798+0000","last_fullsized":"2024-04-26T23:18:37.029798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:49.567077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046666+0000","last_change":"2024-04-26T23:18:32.026884+0000","last_active":"2024-04-26T23:18:37.046666+0000","last_peered":"2024-04-26T23:18:37.046666+0000","last_clean":"2024-04-26T23:18:37.046666+0000","last_became_active":"2024-04-26T23:18:32.026579+0000","last_became_peered":"2024-04-26T23:18:32.026579+0000","last_unstale":"2024-04-26T23:18:37.046666+0000","last_undegraded":"2024-04-26T23:18:37.046666+0000","last_fullsized":"2024-04-26T23:18:37.046666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:55:55.562630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030747+0000","last_change":"2024-04-26T23:18:32.018063+0000","last_active":"2024-04-26T23:18:37.030747+0000","last_peered":"2024-04-26T23:18:37.030747+0000","last_clean":"2024-04-26T23:18:37.030747+0000","last_became_active":"2024-04-26T23:18:32.017430+0000","last_became_peered":"2024-04-26T23:18:32.017430+0000","last_unstale":"2024-04-26T23:18:37.030747+0000","last_undegraded":"2024-04-26T23:18:37.030747+0000","last_fullsized":"2024-04-26T23:18:37.030747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:48:51.877664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044241+0000","last_change":"2024-04-26T23:18:32.018383+0000","last_active":"2024-04-26T23:18:37.044241+0000","last_peered":"2024-04-26T23:18:37.044241+0000","last_clean":"2024-04-26T23:18:37.044241+0000","last_became_active":"2024-04-26T23:18:32.018101+0000","last_became_peered":"2024-04-26T23:18:32.018101+0000","last_unstale":"2024-04-26T23:18:37.044241+0000","last_undegraded":"2024-04-26T23:18:37.044241+0000","last_fullsized":"2024-04-26T23:18:37.044241+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:49:47.214737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029898+0000","last_change":"2024-04-26T23:18:32.020368+0000","last_active":"2024-04-26T23:18:37.029898+0000","last_peered":"2024-04-26T23:18:37.029898+0000","last_clean":"2024-04-26T23:18:37.029898+0000","last_became_active":"2024-04-26T23:18:32.019278+0000","last_became_peered":"2024-04-26T23:18:32.019278+0000","last_unstale":"2024-04-26T23:18:37.029898+0000","last_undegraded":"2024-04-26T23:18:37.029898+0000","last_fullsized":"2024-04-26T23:18:37.029898+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:51:42.870329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044697+0000","last_change":"2024-04-26T23:18:32.029085+0000","last_active":"2024-04-26T23:18:37.044697+0000","last_peered":"2024-04-26T23:18:37.044697+0000","last_clean":"2024-04-26T23:18:37.044697+0000","last_became_active":"2024-04-26T23:18:32.028769+0000","last_became_peered":"2024-04-26T23:18:32.028769+0000","last_unstale":"2024-04-26T23:18:37.044697+0000","last_undegraded":"2024-04-26T23:18:37.044697+0000","last_fullsized":"2024-04-26T23:18:37.044697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:55.832329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.043961+0000","last_change":"2024-04-26T23:18:32.014926+0000","last_active":"2024-04-26T23:18:37.043961+0000","last_peered":"2024-04-26T23:18:37.043961+0000","last_clean":"2024-04-26T23:18:37.043961+0000","last_became_active":"2024-04-26T23:18:32.014414+0000","last_became_peered":"2024-04-26T23:18:32.014414+0000","last_unstale":"2024-04-26T23:18:37.043961+0000","last_undegraded":"2024-04-26T23:18:37.043961+0000","last_fullsized":"2024-04-26T23:18:37.043961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:58:44.698418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045548+0000","last_change":"2024-04-26T23:18:32.019959+0000","last_active":"2024-04-26T23:18:37.045548+0000","last_peered":"2024-04-26T23:18:37.045548+0000","last_clean":"2024-04-26T23:18:37.045548+0000","last_became_active":"2024-04-26T23:18:32.019690+0000","last_became_peered":"2024-04-26T23:18:32.019690+0000","last_unstale":"2024-04-26T23:18:37.045548+0000","last_undegraded":"2024-04-26T23:18:37.045548+0000","last_fullsized":"2024-04-26T23:18:37.045548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:49.567077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.047090+0000","last_change":"2024-04-26T23:18:32.023444+0000","last_active":"2024-04-26T23:18:37.047090+0000","last_peered":"2024-04-26T23:18:37.047090+0000","last_clean":"2024-04-26T23:18:37.047090+0000","last_became_active":"2024-04-26T23:18:32.023211+0000","last_became_peered":"2024-04-26T23:18:32.023211+0000","last_unstale":"2024-04-26T23:18:37.047090+0000","last_undegraded":"2024-04-26T23:18:37.047090+0000","last_fullsized":"2024-04-26T23:18:37.047090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:57:19.300614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.027980+0000","last_change":"2024-04-26T23:18:32.011378+0000","last_active":"2024-04-26T23:18:37.027980+0000","last_peered":"2024-04-26T23:18:37.027980+0000","last_clean":"2024-04-26T23:18:37.027980+0000","last_became_active":"2024-04-26T23:18:32.011101+0000","last_became_peered":"2024-04-26T23:18:32.011101+0000","last_unstale":"2024-04-26T23:18:37.027980+0000","last_undegraded":"2024-04-26T23:18:37.027980+0000","last_fullsized":"2024-04-26T23:18:37.027980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:53:16.817858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045910+0000","last_change":"2024-04-26T23:18:32.028996+0000","last_active":"2024-04-26T23:18:37.045910+0000","last_peered":"2024-04-26T23:18:37.045910+0000","last_clean":"2024-04-26T23:18:37.045910+0000","last_became_active":"2024-04-26T23:18:32.028697+0000","last_became_peered":"2024-04-26T23:18:32.028697+0000","last_unstale":"2024-04-26T23:18:37.045910+0000","last_undegraded":"2024-04-26T23:18:37.045910+0000","last_fullsized":"2024-04-26T23:18:37.045910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:04:13.811589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044240+0000","last_change":"2024-04-26T23:18:32.018149+0000","last_active":"2024-04-26T23:18:37.044240+0000","last_peered":"2024-04-26T23:18:37.044240+0000","last_clean":"2024-04-26T23:18:37.044240+0000","last_became_active":"2024-04-26T23:18:32.017657+0000","last_became_peered":"2024-04-26T23:18:32.017657+0000","last_unstale":"2024-04-26T23:18:37.044240+0000","last_undegraded":"2024-04-26T23:18:37.044240+0000","last_fullsized":"2024-04-26T23:18:37.044240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:55:55.562630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029015+0000","last_change":"2024-04-26T23:18:32.000678+0000","last_active":"2024-04-26T23:18:37.029015+0000","last_peered":"2024-04-26T23:18:37.029015+0000","last_clean":"2024-04-26T23:18:37.029015+0000","last_became_active":"2024-04-26T23:18:32.000354+0000","last_became_peered":"2024-04-26T23:18:32.000354+0000","last_unstale":"2024-04-26T23:18:37.029015+0000","last_undegraded":"2024-04-26T23:18:37.029015+0000","last_fullsized":"2024-04-26T23:18:37.029015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:51:42.870329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.043891+0000","last_change":"2024-04-26T23:18:31.999586+0000","last_active":"2024-04-26T23:18:37.043891+0000","last_peered":"2024-04-26T23:18:37.043891+0000","last_clean":"2024-04-26T23:18:37.043891+0000","last_became_active":"2024-04-26T23:18:31.999185+0000","last_became_peered":"2024-04-26T23:18:31.999185+0000","last_unstale":"2024-04-26T23:18:37.043891+0000","last_undegraded":"2024-04-26T23:18:37.043891+0000","last_fullsized":"2024-04-26T23:18:37.043891+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:51:42.870329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.029862+0000","last_change":"2024-04-26T23:18:32.024681+0000","last_active":"2024-04-26T23:18:37.029862+0000","last_peered":"2024-04-26T23:18:37.029862+0000","last_clean":"2024-04-26T23:18:37.029862+0000","last_became_active":"2024-04-26T23:18:32.024372+0000","last_became_peered":"2024-04-26T23:18:32.024372+0000","last_unstale":"2024-04-26T23:18:37.029862+0000","last_undegraded":"2024-04-26T23:18:37.029862+0000","last_fullsized":"2024-04-26T23:18:37.029862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:49.567077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.045305+0000","last_change":"2024-04-26T23:18:32.011333+0000","last_active":"2024-04-26T23:18:37.045305+0000","last_peered":"2024-04-26T23:18:37.045305+0000","last_clean":"2024-04-26T23:18:37.045305+0000","last_became_active":"2024-04-26T23:18:32.011052+0000","last_became_peered":"2024-04-26T23:18:32.011052+0000","last_unstale":"2024-04-26T23:18:37.045305+0000","last_undegraded":"2024-04-26T23:18:37.045305+0000","last_fullsized":"2024-04-26T23:18:37.045305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:29.991591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046025+0000","last_change":"2024-04-26T23:18:32.021692+0000","last_active":"2024-04-26T23:18:37.046025+0000","last_peered":"2024-04-26T23:18:37.046025+0000","last_clean":"2024-04-26T23:18:37.046025+0000","last_became_active":"2024-04-26T23:18:32.021426+0000","last_became_peered":"2024-04-26T23:18:32.021426+0000","last_unstale":"2024-04-26T23:18:37.046025+0000","last_undegraded":"2024-04-26T23:18:37.046025+0000","last_fullsized":"2024-04-26T23:18:37.046025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:57:02.596299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046525+0000","last_change":"2024-04-26T23:18:32.020976+0000","last_active":"2024-04-26T23:18:37.046525+0000","last_peered":"2024-04-26T23:18:37.046525+0000","last_clean":"2024-04-26T23:18:37.046525+0000","last_became_active":"2024-04-26T23:18:32.020653+0000","last_became_peered":"2024-04-26T23:18:32.020653+0000","last_unstale":"2024-04-26T23:18:37.046525+0000","last_undegraded":"2024-04-26T23:18:37.046525+0000","last_fullsized":"2024-04-26T23:18:37.046525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:51:42.870329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.047206+0000","last_change":"2024-04-26T23:18:32.030726+0000","last_active":"2024-04-26T23:18:37.047206+0000","last_peered":"2024-04-26T23:18:37.047206+0000","last_clean":"2024-04-26T23:18:37.047206+0000","last_became_active":"2024-04-26T23:18:32.030451+0000","last_became_peered":"2024-04-26T23:18:32.030451+0000","last_unstale":"2024-04-26T23:18:37.047206+0000","last_undegraded":"2024-04-26T23:18:37.047206+0000","last_fullsized":"2024-04-26T23:18:37.047206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:27:26.533404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030676+0000","last_change":"2024-04-26T23:18:32.020062+0000","last_active":"2024-04-26T23:18:37.030676+0000","last_peered":"2024-04-26T23:18:37.030676+0000","last_clean":"2024-04-26T23:18:37.030676+0000","last_became_active":"2024-04-26T23:18:32.019651+0000","last_became_peered":"2024-04-26T23:18:32.019651+0000","last_unstale":"2024-04-26T23:18:37.030676+0000","last_undegraded":"2024-04-26T23:18:37.030676+0000","last_fullsized":"2024-04-26T23:18:37.030676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:17:04.527690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046652+0000","last_change":"2024-04-26T23:18:32.017808+0000","last_active":"2024-04-26T23:18:37.046652+0000","last_peered":"2024-04-26T23:18:37.046652+0000","last_clean":"2024-04-26T23:18:37.046652+0000","last_became_active":"2024-04-26T23:18:32.017162+0000","last_became_peered":"2024-04-26T23:18:32.017162+0000","last_unstale":"2024-04-26T23:18:37.046652+0000","last_undegraded":"2024-04-26T23:18:37.046652+0000","last_fullsized":"2024-04-26T23:18:37.046652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:48:51.877664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044589+0000","last_change":"2024-04-26T23:18:32.023064+0000","last_active":"2024-04-26T23:18:37.044589+0000","last_peered":"2024-04-26T23:18:37.044589+0000","last_clean":"2024-04-26T23:18:37.044589+0000","last_became_active":"2024-04-26T23:18:32.022771+0000","last_became_peered":"2024-04-26T23:18:32.022771+0000","last_unstale":"2024-04-26T23:18:37.044589+0000","last_undegraded":"2024-04-26T23:18:37.044589+0000","last_fullsized":"2024-04-26T23:18:37.044589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:37:21.828417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.046963+0000","last_change":"2024-04-26T23:18:32.022130+0000","last_active":"2024-04-26T23:18:37.046963+0000","last_peered":"2024-04-26T23:18:37.046963+0000","last_clean":"2024-04-26T23:18:37.046963+0000","last_became_active":"2024-04-26T23:18:32.021842+0000","last_became_peered":"2024-04-26T23:18:32.021842+0000","last_unstale":"2024-04-26T23:18:37.046963+0000","last_undegraded":"2024-04-26T23:18:37.046963+0000","last_fullsized":"2024-04-26T23:18:37.046963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:10.370646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044765+0000","last_change":"2024-04-26T23:18:32.020676+0000","last_active":"2024-04-26T23:18:37.044765+0000","last_peered":"2024-04-26T23:18:37.044765+0000","last_clean":"2024-04-26T23:18:37.044765+0000","last_became_active":"2024-04-26T23:18:32.020391+0000","last_became_peered":"2024-04-26T23:18:32.020391+0000","last_unstale":"2024-04-26T23:18:37.044765+0000","last_undegraded":"2024-04-26T23:18:37.044765+0000","last_fullsized":"2024-04-26T23:18:37.044765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:05.295882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.044140+0000","last_change":"2024-04-26T23:18:32.006201+0000","last_active":"2024-04-26T23:18:37.044140+0000","last_peered":"2024-04-26T23:18:37.044140+0000","last_clean":"2024-04-26T23:18:37.044140+0000","last_became_active":"2024-04-26T23:18:32.005906+0000","last_became_peered":"2024-04-26T23:18:32.005906+0000","last_unstale":"2024-04-26T23:18:37.044140+0000","last_undegraded":"2024-04-26T23:18:37.044140+0000","last_fullsized":"2024-04-26T23:18:37.044140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:59:18.268816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.030075+0000","last_change":"2024-04-26T23:18:32.003546+0000","last_active":"2024-04-26T23:18:37.030075+0000","last_peered":"2024-04-26T23:18:37.030075+0000","last_clean":"2024-04-26T23:18:37.030075+0000","last_became_active":"2024-04-26T23:18:32.003204+0000","last_became_peered":"2024-04-26T23:18:32.003204+0000","last_unstale":"2024-04-26T23:18:37.030075+0000","last_undegraded":"2024-04-26T23:18:37.030075+0000","last_fullsized":"2024-04-26T23:18:37.030075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:37:21.828417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T23:18:37.047079+0000","last_change":"2024-04-26T23:18:32.026678+0000","last_active":"2024-04-26T23:18:37.047079+0000","last_peered":"2024-04-26T23:18:37.047079+0000","last_clean":"2024-04-26T23:18:37.047079+0000","last_became_active":"2024-04-26T23:18:32.026390+0000","last_became_peered":"2024-04-26T23:18:32.026390+0000","last_unstale":"2024-04-26T23:18:37.047079+0000","last_undegraded":"2024-04-26T23:18:37.047079+0000","last_fullsized":"2024-04-26T23:18:37.047079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:30.985065+0000","last_clean_scrub_stamp":"2024-04-26T23:18:30.985065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:16:12.520377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542154,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":847872,"data_stored":690086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542156,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":262144,"data_stored":102978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542155,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":99226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542155,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":99262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31432,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340408,"statfs":{"total":96636764160,"available":96604577792,"internally_reserved":0,"allocated":262144,"data_stored":102978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":249856,"data_stored":99136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31432,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340408,"statfs":{"total":96636764160,"available":96604577792,"internally_reserved":0,"allocated":258048,"data_stored":99296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":856064,"data_stored":690142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:18:58.232 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-26T23:18:58.233 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-26T23:18:58.233 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-26T23:18:58.233 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T23:18:58.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.366+0000 7fa2391da640 1 Processor -- start 2024-04-26T23:18:58.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.366+0000 7fa2391da640 1 -- start start 2024-04-26T23:18:58.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.367+0000 7fa2391da640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2340d06f0 0x7fa2340d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:58.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.367+0000 7fa2391da640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2340d0b70 con 0x7fa23415f3a0 2024-04-26T23:18:58.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.367+0000 7fa2391da640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2340d0e20 con 0x7fa2340b2850 2024-04-26T23:18:58.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.367+0000 7fa2391da640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa2340b1d70 con 0x7fa2340d06f0 2024-04-26T23:18:58.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.367+0000 7fa232d9d640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa23415f3a0 0x7fa2340d1770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:54642/0 (socket says 172.21.15.47:54642) 2024-04-26T23:18:58.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.367+0000 7fa232d9d640 1 -- 172.21.15.47:0/3122170601 learned_addr learned my addr 172.21.15.47:0/3122170601 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:18:58.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.367+0000 7fa23259c640 1 --2- 172.21.15.47:0/3122170601 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2340d06f0 0x7fa2340d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:58.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.367+0000 7fa231d9b640 1 -- 172.21.15.47:0/3122170601 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3632493617 0 0) 0x7fa2340d0b70 con 0x7fa23415f3a0 2024-04-26T23:18:58.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.367+0000 7fa231d9b640 1 -- 172.21.15.47:0/3122170601 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa21c003520 con 0x7fa23415f3a0 2024-04-26T23:18:58.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.367+0000 7fa231d9b640 1 -- 172.21.15.47:0/3122170601 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3226129753 0 0) 0x7fa21c003520 con 0x7fa23415f3a0 2024-04-26T23:18:58.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.367+0000 7fa231d9b640 1 -- 172.21.15.47:0/3122170601 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2340d06f0 msgr2=0x7fa2340d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:58.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.367+0000 7fa231d9b640 1 --2- 172.21.15.47:0/3122170601 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2340d06f0 0x7fa2340d1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:58.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.367+0000 7fa231d9b640 1 -- 172.21.15.47:0/3122170601 >> v1:172.21.15.203:6789/0 conn(0x7fa2340b2850 legacy=0x7fa2340b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:58.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.367+0000 7fa231d9b640 1 -- 172.21.15.47:0/3122170601 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2340b2050 con 0x7fa23415f3a0 2024-04-26T23:18:58.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.367+0000 7fa231d9b640 1 -- 172.21.15.47:0/3122170601 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7fa224001670 con 0x7fa23415f3a0 2024-04-26T23:18:58.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.367+0000 7fa231d9b640 1 -- 172.21.15.47:0/3122170601 >> v1:172.21.15.47:6789/0 conn(0x7fa23415f3a0 legacy=0x7fa2340d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:58.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.367+0000 7fa231d9b640 1 --2- 172.21.15.47:0/3122170601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa21c0039e0 0x7fa21c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:58.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.367+0000 7fa231d9b640 1 --2- 172.21.15.47:0/3122170601 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2340d06f0 0x7fa21c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:58.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.368+0000 7fa231d9b640 1 --2- 172.21.15.47:0/3122170601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa21c004d40 0x7fa21c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:58.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.368+0000 7fa231d9b640 1 -- 172.21.15.47:0/3122170601 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa2340b1d70 con 0x7fa21c004d40 2024-04-26T23:18:58.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.368+0000 7fa231d9b640 1 -- 172.21.15.47:0/3122170601 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa21c00ba70 con 0x7fa21c0039e0 2024-04-26T23:18:58.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.368+0000 7fa231d9b640 1 -- 172.21.15.47:0/3122170601 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa21c00bc70 con 0x7fa2340d06f0 2024-04-26T23:18:58.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.368+0000 7fa23359e640 1 --2- 172.21.15.47:0/3122170601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa21c004d40 0x7fa21c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:58.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.368+0000 7fa232d9d640 1 --2- 172.21.15.47:0/3122170601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa21c0039e0 0x7fa21c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:58.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.368+0000 7fa23259c640 1 --2- 172.21.15.47:0/3122170601 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2340d06f0 0x7fa21c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:58.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.368+0000 7fa23359e640 1 -- 172.21.15.47:0/3122170601 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2340d06f0 msgr2=0x7fa21c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:58.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.368+0000 7fa23359e640 1 --2- 172.21.15.47:0/3122170601 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2340d06f0 0x7fa21c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:58.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.368+0000 7fa23359e640 1 -- 172.21.15.47:0/3122170601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa21c0039e0 msgr2=0x7fa21c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:58.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.368+0000 7fa23359e640 1 --2- 172.21.15.47:0/3122170601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa21c0039e0 0x7fa21c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:58.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.368+0000 7fa23359e640 1 -- 172.21.15.47:0/3122170601 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa21c003520 con 0x7fa21c004d40 2024-04-26T23:18:58.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.369+0000 7fa23359e640 1 --2- 172.21.15.47:0/3122170601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa21c004d40 0x7fa21c0051c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa22c001e20 tx=0x7fa21c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89334b556862df7e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:58.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.369+0000 7fa231d9b640 1 -- 172.21.15.47:0/3122170601 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22c001a70 con 0x7fa21c004d40 2024-04-26T23:18:58.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.369+0000 7fa231d9b640 1 -- 172.21.15.47:0/3122170601 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa22c02ae40 con 0x7fa21c004d40 2024-04-26T23:18:58.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.369+0000 7fa231d9b640 1 -- 172.21.15.47:0/3122170601 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22c003bb0 con 0x7fa21c004d40 2024-04-26T23:18:58.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.369+0000 7fa2391da640 1 -- 172.21.15.47:0/3122170601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa21c004d40 msgr2=0x7fa21c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:58.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.369+0000 7fa2391da640 1 --2- 172.21.15.47:0/3122170601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa21c004d40 0x7fa21c0051c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa22c001e20 tx=0x7fa21c005830 comp rx=0 tx=0).stop 2024-04-26T23:18:58.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.369+0000 7fa23359e640 1 -- 172.21.15.47:0/3122170601 reap_dead start 2024-04-26T23:18:58.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.369+0000 7fa2391da640 1 -- 172.21.15.47:0/3122170601 shutdown_connections 2024-04-26T23:18:58.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.369+0000 7fa2391da640 1 -- 172.21.15.47:0/3122170601 >> 172.21.15.47:0/3122170601 conn(0x7fa2340d4f90 msgr2=0x7fa2340d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:58.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.369+0000 7fa2391da640 1 -- 172.21.15.47:0/3122170601 shutdown_connections 2024-04-26T23:18:58.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.369+0000 7fa2391da640 1 -- 172.21.15.47:0/3122170601 wait complete. 2024-04-26T23:18:58.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.372+0000 7fa2391da640 1 Processor -- start 2024-04-26T23:18:58.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.372+0000 7fa2391da640 1 -- start start 2024-04-26T23:18:58.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.373+0000 7fa2391da640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2341589b0 0x7fa2340cc270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:58.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.373+0000 7fa2391da640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2340cca70 0x7fa23406ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:58.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.373+0000 7fa2391da640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa23406d3c0 0x7fa2340caa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:58.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.373+0000 7fa2391da640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa23415f900 con 0x7fa2340cca70 2024-04-26T23:18:58.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.373+0000 7fa2391da640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa23415f780 con 0x7fa2341589b0 2024-04-26T23:18:58.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.373+0000 7fa2391da640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa2340d1e80 con 0x7fa23406d3c0 2024-04-26T23:18:58.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.373+0000 7fa23259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2340cca70 0x7fa23406ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:58.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.373+0000 7fa23259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2340cca70 0x7fa23406ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58878/0 (socket says 172.21.15.47:58878) 2024-04-26T23:18:58.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.373+0000 7fa23259c640 1 -- 172.21.15.47:0/3494446125 learned_addr learned my addr 172.21.15.47:0/3494446125 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:58.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.373+0000 7fa23359e640 1 --2- 172.21.15.47:0/3494446125 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa23406d3c0 0x7fa2340caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:58.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.373+0000 7fa232d9d640 1 --2- 172.21.15.47:0/3494446125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2341589b0 0x7fa2340cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:58.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.373+0000 7fa23259c640 1 -- 172.21.15.47:0/3494446125 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa23406d3c0 msgr2=0x7fa2340caa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:58.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.373+0000 7fa23259c640 1 --2- 172.21.15.47:0/3494446125 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa23406d3c0 0x7fa2340caa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:58.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.373+0000 7fa23259c640 1 -- 172.21.15.47:0/3494446125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2341589b0 msgr2=0x7fa2340cc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:58.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.373+0000 7fa23259c640 1 --2- 172.21.15.47:0/3494446125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2341589b0 0x7fa2340cc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:58.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.373+0000 7fa23259c640 1 -- 172.21.15.47:0/3494446125 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2340cc7b0 con 0x7fa2340cca70 2024-04-26T23:18:58.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.373+0000 7fa232d9d640 1 --2- 172.21.15.47:0/3494446125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2341589b0 0x7fa2340cc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:18:58.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.373+0000 7fa23259c640 1 --2- 172.21.15.47:0/3494446125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2340cca70 0x7fa23406ce80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa228011960 tx=0x7fa228011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:58.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.374+0000 7fa21b7fe640 1 -- 172.21.15.47:0/3494446125 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22800faf0 con 0x7fa2340cca70 2024-04-26T23:18:58.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.374+0000 7fa21b7fe640 1 -- 172.21.15.47:0/3494446125 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa22800fc90 con 0x7fa2340cca70 2024-04-26T23:18:58.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.374+0000 7fa2391da640 1 -- 172.21.15.47:0/3494446125 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2340b7b80 con 0x7fa2340cca70 2024-04-26T23:18:58.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.374+0000 7fa21b7fe640 1 -- 172.21.15.47:0/3494446125 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa228014440 con 0x7fa2340cca70 2024-04-26T23:18:58.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.374+0000 7fa2391da640 1 -- 172.21.15.47:0/3494446125 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa234160d10 con 0x7fa2340cca70 2024-04-26T23:18:58.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.375+0000 7fa2391da640 1 -- 172.21.15.47:0/3494446125 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2340694f0 con 0x7fa2340cca70 2024-04-26T23:18:58.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.375+0000 7fa21b7fe640 1 -- 172.21.15.47:0/3494446125 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa228012040 con 0x7fa2340cca70 2024-04-26T23:18:58.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.376+0000 7fa21b7fe640 1 --2- 172.21.15.47:0/3494446125 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa22006c920 0x7fa22006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:58.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.376+0000 7fa21b7fe640 1 -- 172.21.15.47:0/3494446125 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fa22808c5a0 con 0x7fa2340cca70 2024-04-26T23:18:58.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.376+0000 7fa232d9d640 1 --2- 172.21.15.47:0/3494446125 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa22006c920 0x7fa22006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:58.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.376+0000 7fa232d9d640 1 --2- 172.21.15.47:0/3494446125 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa22006c920 0x7fa22006ede0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa224001eb0 tx=0x7fa224009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:58.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.380+0000 7fa21b7fe640 1 -- 172.21.15.47:0/3494446125 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa228019030 con 0x7fa2340cca70 2024-04-26T23:18:58.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.604+0000 7fa2391da640 1 -- 172.21.15.47:0/3494446125 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fa23406de90 con 0x7fa2340cca70 2024-04-26T23:18:58.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.604+0000 7fa21b7fe640 1 -- 172.21.15.47:0/3494446125 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fa22805a040 con 0x7fa2340cca70 2024-04-26T23:18:58.606 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:18:58.606 INFO:teuthology.orchestra.run.smithi047.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-26T23:18:58.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.606+0000 7fa2391da640 1 -- 172.21.15.47:0/3494446125 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa22006c920 msgr2=0x7fa22006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:58.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.606+0000 7fa2391da640 1 --2- 172.21.15.47:0/3494446125 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa22006c920 0x7fa22006ede0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa224001eb0 tx=0x7fa224009040 comp rx=0 tx=0).stop 2024-04-26T23:18:58.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.606+0000 7fa2391da640 1 -- 172.21.15.47:0/3494446125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2340cca70 msgr2=0x7fa23406ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:58.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.606+0000 7fa2391da640 1 --2- 172.21.15.47:0/3494446125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2340cca70 0x7fa23406ce80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa228011960 tx=0x7fa228011e20 comp rx=0 tx=0).stop 2024-04-26T23:18:58.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.606+0000 7fa2391da640 1 -- 172.21.15.47:0/3494446125 shutdown_connections 2024-04-26T23:18:58.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.606+0000 7fa2391da640 1 --2- 172.21.15.47:0/3494446125 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa22006c920 0x7fa22006ede0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:58.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.606+0000 7fa2391da640 1 --2- 172.21.15.47:0/3494446125 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa23406d3c0 0x7fa2340caa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:58.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.606+0000 7fa2391da640 1 --2- 172.21.15.47:0/3494446125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2340cca70 0x7fa23406ce80 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:58.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.606+0000 7fa2391da640 1 --2- 172.21.15.47:0/3494446125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2341589b0 0x7fa2340cc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:58.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.606+0000 7fa2391da640 1 -- 172.21.15.47:0/3494446125 >> 172.21.15.47:0/3494446125 conn(0x7fa2340d4f90 msgr2=0x7fa23415e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:58.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.606+0000 7fa2391da640 1 -- 172.21.15.47:0/3494446125 shutdown_connections 2024-04-26T23:18:58.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.606+0000 7fa2391da640 1 -- 172.21.15.47:0/3494446125 wait complete. 2024-04-26T23:18:58.619 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-26T23:18:58.620 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:18:58.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.719+0000 7fa12ba6d640 1 Processor -- start 2024-04-26T23:18:58.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.719+0000 7fa12ba6d640 1 -- start start 2024-04-26T23:18:58.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.720+0000 7fa12ba6d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa12415d100 0x7fa124160ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:58.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.720+0000 7fa12ba6d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa12406c950 0x7fa1240aed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:58.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.720+0000 7fa12ba6d640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1240b0990 con 0x7fa124070930 2024-04-26T23:18:58.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.720+0000 7fa12ba6d640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fa1240af2d0 con 0x7fa12415d100 2024-04-26T23:18:58.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.720+0000 7fa12ba6d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fa1240b0c70 con 0x7fa12406c950 2024-04-26T23:18:58.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.720+0000 7fa129809640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa12415d100 0x7fa124160ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:58.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.720+0000 7fa129809640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa12415d100 0x7fa124160ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58880/0 (socket says 172.21.15.47:58880) 2024-04-26T23:18:58.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.720+0000 7fa129809640 1 -- 172.21.15.47:0/3736572785 learned_addr learned my addr 172.21.15.47:0/3736572785 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:58.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.721+0000 7fa12a00a640 1 --2- 172.21.15.47:0/3736572785 >> v2:172.21.15.203:3300/0 conn(0x7fa12406c950 0x7fa1240aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:58.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.721+0000 7fa129809640 1 -- 172.21.15.47:0/3736572785 >> v2:172.21.15.203:3300/0 conn(0x7fa12406c950 msgr2=0x7fa1240aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:58.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.721+0000 7fa129809640 1 --2- 172.21.15.47:0/3736572785 >> v2:172.21.15.203:3300/0 conn(0x7fa12406c950 0x7fa1240aed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:58.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.721+0000 7fa129809640 1 -- 172.21.15.47:0/3736572785 >> v1:172.21.15.47:6789/0 conn(0x7fa124070930 legacy=0x7fa12406c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:58.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.721+0000 7fa129809640 1 -- 172.21.15.47:0/3736572785 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1240b0f50 con 0x7fa12415d100 2024-04-26T23:18:58.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.721+0000 7fa12a00a640 1 --2- 172.21.15.47:0/3736572785 >> v2:172.21.15.203:3300/0 conn(0x7fa12406c950 0x7fa1240aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:18:58.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.721+0000 7fa128807640 1 -- 172.21.15.47:0/3736572785 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2261465377 0 0) 0x7fa1240b0990 con 0x7fa124070930 2024-04-26T23:18:58.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.722+0000 7fa129809640 1 --2- 172.21.15.47:0/3736572785 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa12415d100 0x7fa124160ba0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa1240d0da0 tx=0x7fa11402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df7fa17cea8b1ae0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:58.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.722+0000 7fa128807640 1 -- 172.21.15.47:0/3736572785 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa114037070 con 0x7fa12415d100 2024-04-26T23:18:58.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.722+0000 7fa128807640 1 -- 172.21.15.47:0/3736572785 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa114032440 con 0x7fa12415d100 2024-04-26T23:18:58.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.722+0000 7fa128807640 1 -- 172.21.15.47:0/3736572785 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa114032810 con 0x7fa12415d100 2024-04-26T23:18:58.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.722+0000 7fa12ba6d640 1 -- 172.21.15.47:0/3736572785 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa12415d100 msgr2=0x7fa124160ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:58.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.722+0000 7fa12ba6d640 1 --2- 172.21.15.47:0/3736572785 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa12415d100 0x7fa124160ba0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa1240d0da0 tx=0x7fa11402a270 comp rx=0 tx=0).stop 2024-04-26T23:18:58.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.722+0000 7fa12ba6d640 1 -- 172.21.15.47:0/3736572785 shutdown_connections 2024-04-26T23:18:58.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.722+0000 7fa12ba6d640 1 --2- 172.21.15.47:0/3736572785 >> v2:172.21.15.203:3300/0 conn(0x7fa12406c950 0x7fa1240aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:58.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.722+0000 7fa12ba6d640 1 --2- 172.21.15.47:0/3736572785 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa12415d100 0x7fa124160ba0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:58.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.722+0000 7fa12ba6d640 1 -- 172.21.15.47:0/3736572785 >> 172.21.15.47:0/3736572785 conn(0x7fa1240b39b0 msgr2=0x7fa1240b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:58.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.723+0000 7fa12ba6d640 1 -- 172.21.15.47:0/3736572785 shutdown_connections 2024-04-26T23:18:58.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.723+0000 7fa12ba6d640 1 -- 172.21.15.47:0/3736572785 wait complete. 2024-04-26T23:18:58.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.724+0000 7fa12ba6d640 1 Processor -- start 2024-04-26T23:18:58.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.724+0000 7fa12ba6d640 1 -- start start 2024-04-26T23:18:58.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.724+0000 7fa12ba6d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa12406c950 0x7fa124153850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:58.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.724+0000 7fa12ba6d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa124070930 0x7fa124159270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:58.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.724+0000 7fa12ba6d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa12415d100 0x7fa1240b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:58.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.724+0000 7fa12ba6d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa124073160 con 0x7fa124070930 2024-04-26T23:18:58.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.724+0000 7fa12ba6d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa1240b1510 con 0x7fa12406c950 2024-04-26T23:18:58.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.724+0000 7fa12ba6d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa1240732e0 con 0x7fa12415d100 2024-04-26T23:18:58.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.725+0000 7fa129008640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa124070930 0x7fa124159270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:58.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.725+0000 7fa129008640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa124070930 0x7fa124159270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58884/0 (socket says 172.21.15.47:58884) 2024-04-26T23:18:58.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.725+0000 7fa129008640 1 -- 172.21.15.47:0/2844502458 learned_addr learned my addr 172.21.15.47:0/2844502458 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:58.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.725+0000 7fa129809640 1 --2- 172.21.15.47:0/2844502458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa12406c950 0x7fa124153850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:58.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.725+0000 7fa12a00a640 1 --2- 172.21.15.47:0/2844502458 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa12415d100 0x7fa1240b54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:58.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.725+0000 7fa129008640 1 -- 172.21.15.47:0/2844502458 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa12415d100 msgr2=0x7fa1240b54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:58.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.725+0000 7fa129008640 1 --2- 172.21.15.47:0/2844502458 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa12415d100 0x7fa1240b54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:58.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.725+0000 7fa129008640 1 -- 172.21.15.47:0/2844502458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa12406c950 msgr2=0x7fa124153850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:58.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.725+0000 7fa129008640 1 --2- 172.21.15.47:0/2844502458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa12406c950 0x7fa124153850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:58.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.725+0000 7fa129008640 1 -- 172.21.15.47:0/2844502458 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa12415be90 con 0x7fa124070930 2024-04-26T23:18:58.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.725+0000 7fa129008640 1 --2- 172.21.15.47:0/2844502458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa124070930 0x7fa124159270 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa12000c4b0 tx=0x7fa12000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:58.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.725+0000 7fa11a7fc640 1 -- 172.21.15.47:0/2844502458 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa120017070 con 0x7fa124070930 2024-04-26T23:18:58.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.725+0000 7fa11a7fc640 1 -- 172.21.15.47:0/2844502458 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa120005480 con 0x7fa124070930 2024-04-26T23:18:58.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.725+0000 7fa12ba6d640 1 -- 172.21.15.47:0/2844502458 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa12415c170 con 0x7fa124070930 2024-04-26T23:18:58.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.725+0000 7fa11a7fc640 1 -- 172.21.15.47:0/2844502458 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa120003830 con 0x7fa124070930 2024-04-26T23:18:58.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.725+0000 7fa12ba6d640 1 -- 172.21.15.47:0/2844502458 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa12415c730 con 0x7fa124070930 2024-04-26T23:18:58.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.727+0000 7fa12ba6d640 1 -- 172.21.15.47:0/2844502458 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1240694f0 con 0x7fa124070930 2024-04-26T23:18:58.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.727+0000 7fa11a7fc640 1 -- 172.21.15.47:0/2844502458 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa12001a070 con 0x7fa124070930 2024-04-26T23:18:58.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.727+0000 7fa11a7fc640 1 --2- 172.21.15.47:0/2844502458 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa10806c8f0 0x7fa10806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:58.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.727+0000 7fa11a7fc640 1 -- 172.21.15.47:0/2844502458 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fa12008b260 con 0x7fa124070930 2024-04-26T23:18:58.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.728+0000 7fa129809640 1 --2- 172.21.15.47:0/2844502458 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa10806c8f0 0x7fa10806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:58.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.728+0000 7fa129809640 1 --2- 172.21.15.47:0/2844502458 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa10806c8f0 0x7fa10806edb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa114034000 tx=0x7fa11402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:58.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.732+0000 7fa11a7fc640 1 -- 172.21.15.47:0/2844502458 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa120059350 con 0x7fa124070930 2024-04-26T23:18:58.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.919+0000 7fa12ba6d640 1 -- 172.21.15.47:0/2844502458 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa12406cd70 con 0x7fa124070930 2024-04-26T23:18:58.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.920+0000 7fa11a7fc640 1 -- 172.21.15.47:0/2844502458 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18478 (secure 0 0 0) 0x7fa120058c50 con 0x7fa124070930 2024-04-26T23:18:58.922 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:18:58.923 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":24,"fsid":"399006d3-4db1-43e6-ba81-a79a276eabfd","created":"2024-04-26T23:18:06.731211+0000","modified":"2024-04-26T23:18:36.402132+0000","last_up_change":"2024-04-26T23:18:17.808736+0000","last_in_change":"2024-04-26T23:18:08.945859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:18:16.937492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:18:29.033575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:18:30.318499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:18:34.014536+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ee091567-fea0-4a14-bf4e-0e16c20bb2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6819","nonce":326520779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6821","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6823","nonce":326520779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6829","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6831","nonce":326520779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6825","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6827","nonce":326520779}]},"public_addr":"172.21.15.47:6819/326520779","cluster_addr":"172.21.15.47:6823/326520779","heartbeat_back_addr":"172.21.15.47:6831/326520779","heartbeat_front_addr":"172.21.15.47:6827/326520779","state":["exists","up"]},{"osd":1,"uuid":"c14440a0-bba5-4931-a6db-b736c08e869b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6809","nonce":1912610931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6811","nonce":1912610931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6815","nonce":1912610931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6813","nonce":1912610931}]},"public_addr":"172.21.15.47:6809/1912610931","cluster_addr":"172.21.15.47:6811/1912610931","heartbeat_back_addr":"172.21.15.47:6815/1912610931","heartbeat_front_addr":"172.21.15.47:6813/1912610931","state":["exists","up"]},{"osd":2,"uuid":"3940f8de-c9b2-4cd3-a911-9c301729eb30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6801","nonce":2569869305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6803","nonce":2569869305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6807","nonce":2569869305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6805","nonce":2569869305}]},"public_addr":"172.21.15.47:6801/2569869305","cluster_addr":"172.21.15.47:6803/2569869305","heartbeat_back_addr":"172.21.15.47:6807/2569869305","heartbeat_front_addr":"172.21.15.47:6805/2569869305","state":["exists","up"]},{"osd":3,"uuid":"1286a67e-8958-426a-8c2e-bc85498c619c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6818","nonce":2791411683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6822","nonce":2791411683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6830","nonce":2791411683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6826","nonce":2791411683}]},"public_addr":"172.21.15.47:6818/2791411683","cluster_addr":"172.21.15.47:6822/2791411683","heartbeat_back_addr":"172.21.15.47:6830/2791411683","heartbeat_front_addr":"172.21.15.47:6826/2791411683","state":["exists","up"]},{"osd":4,"uuid":"d8ee2c20-e5b7-46c2-9999-b234070126cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6809","nonce":441313104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6811","nonce":441313104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6815","nonce":441313104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6813","nonce":441313104}]},"public_addr":"172.21.15.203:6809/441313104","cluster_addr":"172.21.15.203:6811/441313104","heartbeat_back_addr":"172.21.15.203:6815/441313104","heartbeat_front_addr":"172.21.15.203:6813/441313104","state":["exists","up"]},{"osd":5,"uuid":"b453a8f0-f324-4840-9ae0-d67bdadb9f8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6801","nonce":56838444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6803","nonce":56838444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6807","nonce":56838444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6805","nonce":56838444}]},"public_addr":"172.21.15.203:6801/56838444","cluster_addr":"172.21.15.203:6803/56838444","heartbeat_back_addr":"172.21.15.203:6807/56838444","heartbeat_front_addr":"172.21.15.203:6805/56838444","state":["exists","up"]},{"osd":6,"uuid":"a6d31951-2d4b-4e8d-a510-6283b01dc158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6825","nonce":2738517080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6827","nonce":2738517080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6831","nonce":2738517080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6829","nonce":2738517080}]},"public_addr":"172.21.15.203:6825/2738517080","cluster_addr":"172.21.15.203:6827/2738517080","heartbeat_back_addr":"172.21.15.203:6831/2738517080","heartbeat_front_addr":"172.21.15.203:6829/2738517080","state":["exists","up"]},{"osd":7,"uuid":"7119b8c0-fe96-402d-9e75-7b509d9d44d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6817","nonce":1315565058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6819","nonce":1315565058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6823","nonce":1315565058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6821","nonce":1315565058}]},"public_addr":"172.21.15.203:6817/1315565058","cluster_addr":"172.21.15.203:6819/1315565058","heartbeat_back_addr":"172.21.15.203:6823/1315565058","heartbeat_front_addr":"172.21.15.203:6821/1315565058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.121921+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.137494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.084814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.111593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.072430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.125066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.097867+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.057061+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:18:58.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.923+0000 7fa12ba6d640 1 -- 172.21.15.47:0/2844502458 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa10806c8f0 msgr2=0x7fa10806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:58.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.923+0000 7fa12ba6d640 1 --2- 172.21.15.47:0/2844502458 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa10806c8f0 0x7fa10806edb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa114034000 tx=0x7fa11402e000 comp rx=0 tx=0).stop 2024-04-26T23:18:58.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.923+0000 7fa12ba6d640 1 -- 172.21.15.47:0/2844502458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa124070930 msgr2=0x7fa124159270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:58.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.923+0000 7fa12ba6d640 1 --2- 172.21.15.47:0/2844502458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa124070930 0x7fa124159270 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa12000c4b0 tx=0x7fa12000c970 comp rx=0 tx=0).stop 2024-04-26T23:18:58.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.923+0000 7fa12ba6d640 1 -- 172.21.15.47:0/2844502458 shutdown_connections 2024-04-26T23:18:58.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.923+0000 7fa12ba6d640 1 --2- 172.21.15.47:0/2844502458 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa10806c8f0 0x7fa10806edb0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:58.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.923+0000 7fa12ba6d640 1 --2- 172.21.15.47:0/2844502458 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa12415d100 0x7fa1240b54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:58.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.923+0000 7fa12ba6d640 1 --2- 172.21.15.47:0/2844502458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa124070930 0x7fa124159270 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:58.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.923+0000 7fa12ba6d640 1 --2- 172.21.15.47:0/2844502458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa12406c950 0x7fa124153850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:58.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.923+0000 7fa12ba6d640 1 -- 172.21.15.47:0/2844502458 >> 172.21.15.47:0/2844502458 conn(0x7fa1240b39b0 msgr2=0x7fa1240d0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:58.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.923+0000 7fa12ba6d640 1 -- 172.21.15.47:0/2844502458 shutdown_connections 2024-04-26T23:18:58.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:58.923+0000 7fa12ba6d640 1 -- 172.21.15.47:0/2844502458 wait complete. 2024-04-26T23:18:58.936 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:18:16.937492+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:18:29.033575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:18:30.318499+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T23:18:34.014536+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T23:18:58.936 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:18:59.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.039+0000 7f518dec5640 1 Processor -- start 2024-04-26T23:18:59.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.039+0000 7f518dec5640 1 -- start start 2024-04-26T23:18:59.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.039+0000 7f518dec5640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f51880b40d0 0x7f51881625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:59.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.039+0000 7f518dec5640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5188162b30 0x7f51880b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:59.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.039+0000 7f518dec5640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51880d0a10 con 0x7f51880d1a00 2024-04-26T23:18:59.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.039+0000 7f518dec5640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f51880d0cf0 con 0x7f5188162b30 2024-04-26T23:18:59.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.039+0000 7f518dec5640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f51880d0fd0 con 0x7f51880b40d0 2024-04-26T23:18:59.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.040+0000 7f5187fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5188162b30 0x7f51880b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:59.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.040+0000 7f5187fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5188162b30 0x7f51880b4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58890/0 (socket says 172.21.15.47:58890) 2024-04-26T23:18:59.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.040+0000 7f51877fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f51880d1a00 0x7f51880d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:54652/0 (socket says 172.21.15.47:54652) 2024-04-26T23:18:59.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.040+0000 7f5187fff640 1 -- 172.21.15.47:0/1129297875 learned_addr learned my addr 172.21.15.47:0/1129297875 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:59.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.040+0000 7f5186ffd640 1 --2- 172.21.15.47:0/1129297875 >> v2:172.21.15.203:3300/0 conn(0x7f51880b40d0 0x7f51881625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:59.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.040+0000 7f5187fff640 1 -- 172.21.15.47:0/1129297875 >> v2:172.21.15.203:3300/0 conn(0x7f51880b40d0 msgr2=0x7f51881625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:59.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.040+0000 7f5187fff640 1 --2- 172.21.15.47:0/1129297875 >> v2:172.21.15.203:3300/0 conn(0x7f51880b40d0 0x7f51881625f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:59.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.040+0000 7f5187fff640 1 -- 172.21.15.47:0/1129297875 >> v1:172.21.15.47:6789/0 conn(0x7f51880d1a00 legacy=0x7f51880d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:59.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.040+0000 7f5187fff640 1 -- 172.21.15.47:0/1129297875 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5188162f80 con 0x7f5188162b30 2024-04-26T23:18:59.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.040+0000 7f51867fc640 1 -- 172.21.15.47:0/1129297875 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 971276290 0 0) 0x7f51880d0a10 con 0x7f51880d1a00 2024-04-26T23:18:59.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.041+0000 7f5187fff640 1 --2- 172.21.15.47:0/1129297875 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5188162b30 0x7f51880b4b00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f518808d380 tx=0x7f518002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5bc321ff6103891 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:59.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.041+0000 7f51867fc640 1 -- 172.21.15.47:0/1129297875 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5180036070 con 0x7f5188162b30 2024-04-26T23:18:59.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.041+0000 7f51867fc640 1 -- 172.21.15.47:0/1129297875 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5180031600 con 0x7f5188162b30 2024-04-26T23:18:59.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.041+0000 7f51867fc640 1 -- 172.21.15.47:0/1129297875 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51800319d0 con 0x7f5188162b30 2024-04-26T23:18:59.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.041+0000 7f518dec5640 1 -- 172.21.15.47:0/1129297875 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5188162b30 msgr2=0x7f51880b4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:59.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.041+0000 7f518dec5640 1 --2- 172.21.15.47:0/1129297875 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5188162b30 0x7f51880b4b00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f518808d380 tx=0x7f518002a6a0 comp rx=0 tx=0).stop 2024-04-26T23:18:59.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.041+0000 7f518dec5640 1 -- 172.21.15.47:0/1129297875 shutdown_connections 2024-04-26T23:18:59.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.041+0000 7f518dec5640 1 --2- 172.21.15.47:0/1129297875 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5188162b30 0x7f51880b4b00 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:59.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.041+0000 7f518dec5640 1 --2- 172.21.15.47:0/1129297875 >> v2:172.21.15.203:3300/0 conn(0x7f51880b40d0 0x7f51881625f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:59.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.041+0000 7f518dec5640 1 -- 172.21.15.47:0/1129297875 >> 172.21.15.47:0/1129297875 conn(0x7f51880d4fd0 msgr2=0x7f51880d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:59.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.041+0000 7f518dec5640 1 -- 172.21.15.47:0/1129297875 shutdown_connections 2024-04-26T23:18:59.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.041+0000 7f518dec5640 1 -- 172.21.15.47:0/1129297875 wait complete. 2024-04-26T23:18:59.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.042+0000 7f518dec5640 1 Processor -- start 2024-04-26T23:18:59.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.043+0000 7f518dec5640 1 -- start start 2024-04-26T23:18:59.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.043+0000 7f518dec5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51880b40d0 0x7f518815dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:59.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.043+0000 7f518dec5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51880d1a00 0x7f518815af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:59.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.043+0000 7f518dec5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5188162b30 0x7f51880b2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:59.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.043+0000 7f518dec5640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f51880d2670 con 0x7f5188162b30 2024-04-26T23:18:59.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.043+0000 7f518dec5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f51880d24f0 con 0x7f51880b40d0 2024-04-26T23:18:59.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.043+0000 7f518dec5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f51880b5040 con 0x7f51880d1a00 2024-04-26T23:18:59.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.044+0000 7f5187fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5188162b30 0x7f51880b2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:59.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.044+0000 7f5187fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5188162b30 0x7f51880b2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58894/0 (socket says 172.21.15.47:58894) 2024-04-26T23:18:59.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.044+0000 7f5187fff640 1 -- 172.21.15.47:0/2539035878 learned_addr learned my addr 172.21.15.47:0/2539035878 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:59.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.044+0000 7f51877fe640 1 --2- 172.21.15.47:0/2539035878 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51880b40d0 0x7f518815dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:59.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.044+0000 7f5186ffd640 1 --2- 172.21.15.47:0/2539035878 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51880d1a00 0x7f518815af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:59.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.044+0000 7f5187fff640 1 -- 172.21.15.47:0/2539035878 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51880d1a00 msgr2=0x7f518815af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:59.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.044+0000 7f5187fff640 1 --2- 172.21.15.47:0/2539035878 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51880d1a00 0x7f518815af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:59.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.044+0000 7f5187fff640 1 -- 172.21.15.47:0/2539035878 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51880b40d0 msgr2=0x7f518815dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:59.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.044+0000 7f5187fff640 1 --2- 172.21.15.47:0/2539035878 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51880b40d0 0x7f518815dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:59.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.044+0000 7f5187fff640 1 -- 172.21.15.47:0/2539035878 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f518815e020 con 0x7f5188162b30 2024-04-26T23:18:59.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.044+0000 7f5186ffd640 1 --2- 172.21.15.47:0/2539035878 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51880d1a00 0x7f518815af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:18:59.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.044+0000 7f51877fe640 1 --2- 172.21.15.47:0/2539035878 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51880b40d0 0x7f518815dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:18:59.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.044+0000 7f5187fff640 1 --2- 172.21.15.47:0/2539035878 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5188162b30 0x7f51880b2cb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5180009f40 tx=0x7f5180004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:59.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.044+0000 7f516ffff640 1 -- 172.21.15.47:0/2539035878 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5180036050 con 0x7f5188162b30 2024-04-26T23:18:59.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.044+0000 7f516ffff640 1 -- 172.21.15.47:0/2539035878 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5180042070 con 0x7f5188162b30 2024-04-26T23:18:59.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.044+0000 7f518dec5640 1 -- 172.21.15.47:0/2539035878 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51880cc410 con 0x7f5188162b30 2024-04-26T23:18:59.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.044+0000 7f516ffff640 1 -- 172.21.15.47:0/2539035878 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5180009510 con 0x7f5188162b30 2024-04-26T23:18:59.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.044+0000 7f518dec5640 1 -- 172.21.15.47:0/2539035878 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51880cc9d0 con 0x7f5188162b30 2024-04-26T23:18:59.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.045+0000 7f518dec5640 1 -- 172.21.15.47:0/2539035878 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51880694f0 con 0x7f5188162b30 2024-04-26T23:18:59.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.046+0000 7f516ffff640 1 -- 172.21.15.47:0/2539035878 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5180007bd0 con 0x7f5188162b30 2024-04-26T23:18:59.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.046+0000 7f516ffff640 1 --2- 172.21.15.47:0/2539035878 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f517406c8f0 0x7f517406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:59.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.046+0000 7f516ffff640 1 -- 172.21.15.47:0/2539035878 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f51800acc70 con 0x7f5188162b30 2024-04-26T23:18:59.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.047+0000 7f51877fe640 1 --2- 172.21.15.47:0/2539035878 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f517406c8f0 0x7f517406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:59.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.047+0000 7f51877fe640 1 --2- 172.21.15.47:0/2539035878 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f517406c8f0 0x7f517406edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5178000f60 tx=0x7f5178009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:59.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.050+0000 7f516ffff640 1 -- 172.21.15.47:0/2539035878 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5180045360 con 0x7f5188162b30 2024-04-26T23:18:59.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.240+0000 7f518dec5640 1 -- 172.21.15.47:0/2539035878 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f51880d0a30 con 0x7f5188162b30 2024-04-26T23:18:59.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.240+0000 7f516ffff640 1 -- 172.21.15.47:0/2539035878 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f518007adf0 con 0x7f5188162b30 2024-04-26T23:18:59.242 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-26T23:18:59.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.242+0000 7f518dec5640 1 -- 172.21.15.47:0/2539035878 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f517406c8f0 msgr2=0x7f517406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:59.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.242+0000 7f518dec5640 1 --2- 172.21.15.47:0/2539035878 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f517406c8f0 0x7f517406edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5178000f60 tx=0x7f5178009040 comp rx=0 tx=0).stop 2024-04-26T23:18:59.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.242+0000 7f518dec5640 1 -- 172.21.15.47:0/2539035878 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5188162b30 msgr2=0x7f51880b2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:59.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.242+0000 7f518dec5640 1 --2- 172.21.15.47:0/2539035878 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5188162b30 0x7f51880b2cb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5180009f40 tx=0x7f5180004a40 comp rx=0 tx=0).stop 2024-04-26T23:18:59.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.243+0000 7f518dec5640 1 -- 172.21.15.47:0/2539035878 shutdown_connections 2024-04-26T23:18:59.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.243+0000 7f518dec5640 1 --2- 172.21.15.47:0/2539035878 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f517406c8f0 0x7f517406edb0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:59.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.243+0000 7f518dec5640 1 --2- 172.21.15.47:0/2539035878 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5188162b30 0x7f51880b2cb0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:59.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.243+0000 7f518dec5640 1 --2- 172.21.15.47:0/2539035878 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51880d1a00 0x7f518815af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:59.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.243+0000 7f518dec5640 1 --2- 172.21.15.47:0/2539035878 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51880b40d0 0x7f518815dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:59.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.243+0000 7f518dec5640 1 -- 172.21.15.47:0/2539035878 >> 172.21.15.47:0/2539035878 conn(0x7f51880d4fd0 msgr2=0x7f51880d1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:59.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.243+0000 7f518dec5640 1 -- 172.21.15.47:0/2539035878 shutdown_connections 2024-04-26T23:18:59.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.243+0000 7f518dec5640 1 -- 172.21.15.47:0/2539035878 wait complete. 2024-04-26T23:18:59.255 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:18:59.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.358+0000 7f3a45407640 1 Processor -- start 2024-04-26T23:18:59.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.358+0000 7f3a45407640 1 -- start start 2024-04-26T23:18:59.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.358+0000 7f3a45407640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3a400d1a00 0x7f3a400d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:59.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.359+0000 7f3a45407640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3a400b40d0 0x7f3a400b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:59.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.359+0000 7f3a45407640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a400d0a10 con 0x7f3a400d2320 2024-04-26T23:18:59.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.359+0000 7f3a45407640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f3a400d0cf0 con 0x7f3a400d1a00 2024-04-26T23:18:59.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.359+0000 7f3a45407640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f3a400d0fd0 con 0x7f3a400b40d0 2024-04-26T23:18:59.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.359+0000 7f3a3effd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3a400d1a00 0x7f3a400d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:59.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.359+0000 7f3a37fff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f3a400d2320 0x7f3a401625f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:54656/0 (socket says 172.21.15.47:54656) 2024-04-26T23:18:59.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.359+0000 7f3a37fff640 1 -- 172.21.15.47:0/238253579 learned_addr learned my addr 172.21.15.47:0/238253579 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:18:59.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.359+0000 7f3a3f7fe640 1 --2- 172.21.15.47:0/238253579 >> v2:172.21.15.203:3300/0 conn(0x7f3a400b40d0 0x7f3a400b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:59.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.359+0000 7f3a3effd640 1 -- 172.21.15.47:0/238253579 >> v2:172.21.15.203:3300/0 conn(0x7f3a400b40d0 msgr2=0x7f3a400b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:59.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.359+0000 7f3a3effd640 1 --2- 172.21.15.47:0/238253579 >> v2:172.21.15.203:3300/0 conn(0x7f3a400b40d0 0x7f3a400b4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:59.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.359+0000 7f3a3effd640 1 -- 172.21.15.47:0/238253579 >> v1:172.21.15.47:6789/0 conn(0x7f3a400d2320 legacy=0x7f3a401625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:59.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.359+0000 7f3a3effd640 1 -- 172.21.15.47:0/238253579 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a400d40a0 con 0x7f3a400d1a00 2024-04-26T23:18:59.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.359+0000 7f3a3e7fc640 1 -- 172.21.15.47:0/238253579 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4163007687 0 0) 0x7f3a400d0a10 con 0x7f3a400d2320 2024-04-26T23:18:59.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.360+0000 7f3a3effd640 1 --2- 172.21.15.47:0/238253579 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a400d1a00 0x7f3a400d1de0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3a400b69a0 tx=0x7f3a3002a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bc567e18b3ad702 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:59.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.360+0000 7f3a3e7fc640 1 -- 172.21.15.47:0/238253579 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a30037070 con 0x7f3a400d1a00 2024-04-26T23:18:59.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.360+0000 7f3a3e7fc640 1 -- 172.21.15.47:0/238253579 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a3002add0 con 0x7f3a400d1a00 2024-04-26T23:18:59.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.360+0000 7f3a3e7fc640 1 -- 172.21.15.47:0/238253579 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a30032660 con 0x7f3a400d1a00 2024-04-26T23:18:59.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.360+0000 7f3a45407640 1 -- 172.21.15.47:0/238253579 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a400d1a00 msgr2=0x7f3a400d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:59.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.360+0000 7f3a45407640 1 --2- 172.21.15.47:0/238253579 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a400d1a00 0x7f3a400d1de0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3a400b69a0 tx=0x7f3a3002a080 comp rx=0 tx=0).stop 2024-04-26T23:18:59.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.360+0000 7f3a45407640 1 -- 172.21.15.47:0/238253579 shutdown_connections 2024-04-26T23:18:59.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.360+0000 7f3a45407640 1 --2- 172.21.15.47:0/238253579 >> v2:172.21.15.203:3300/0 conn(0x7f3a400b40d0 0x7f3a400b4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:59.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.360+0000 7f3a45407640 1 --2- 172.21.15.47:0/238253579 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a400d1a00 0x7f3a400d1de0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:59.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.360+0000 7f3a45407640 1 -- 172.21.15.47:0/238253579 >> 172.21.15.47:0/238253579 conn(0x7f3a400d4fd0 msgr2=0x7f3a400d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:59.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.361+0000 7f3a45407640 1 -- 172.21.15.47:0/238253579 shutdown_connections 2024-04-26T23:18:59.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.361+0000 7f3a45407640 1 -- 172.21.15.47:0/238253579 wait complete. 2024-04-26T23:18:59.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.362+0000 7f3a45407640 1 Processor -- start 2024-04-26T23:18:59.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.362+0000 7f3a45407640 1 -- start start 2024-04-26T23:18:59.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.362+0000 7f3a45407640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a400b40d0 0x7f3a400ba970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:59.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.362+0000 7f3a45407640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3a400d1a00 0x7f3a400cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:59.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.362+0000 7f3a45407640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a400d2320 0x7f3a40151000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:59.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.362+0000 7f3a45407640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3a400b5040 con 0x7f3a400b40d0 2024-04-26T23:18:59.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.362+0000 7f3a45407640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3a40162f50 con 0x7f3a400d2320 2024-04-26T23:18:59.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.362+0000 7f3a45407640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3a400b51c0 con 0x7f3a400d1a00 2024-04-26T23:18:59.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.362+0000 7f3a3effd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a400b40d0 0x7f3a400ba970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:59.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.363+0000 7f3a3effd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a400b40d0 0x7f3a400ba970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58908/0 (socket says 172.21.15.47:58908) 2024-04-26T23:18:59.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.363+0000 7f3a3effd640 1 -- 172.21.15.47:0/1077061011 learned_addr learned my addr 172.21.15.47:0/1077061011 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:59.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.363+0000 7f3a3f7fe640 1 --2- 172.21.15.47:0/1077061011 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a400d2320 0x7f3a40151000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:59.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.363+0000 7f3a37fff640 1 --2- 172.21.15.47:0/1077061011 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3a400d1a00 0x7f3a400cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:59.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.363+0000 7f3a3effd640 1 -- 172.21.15.47:0/1077061011 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3a400d1a00 msgr2=0x7f3a400cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:59.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.363+0000 7f3a3effd640 1 --2- 172.21.15.47:0/1077061011 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3a400d1a00 0x7f3a400cb9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:59.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.363+0000 7f3a3effd640 1 -- 172.21.15.47:0/1077061011 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a400d2320 msgr2=0x7f3a40151000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:59.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.363+0000 7f3a3effd640 1 --2- 172.21.15.47:0/1077061011 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a400d2320 0x7f3a40151000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:59.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.363+0000 7f3a3effd640 1 -- 172.21.15.47:0/1077061011 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a401f5ad0 con 0x7f3a400b40d0 2024-04-26T23:18:59.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.363+0000 7f3a3f7fe640 1 --2- 172.21.15.47:0/1077061011 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a400d2320 0x7f3a40151000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:18:59.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.363+0000 7f3a3effd640 1 --2- 172.21.15.47:0/1077061011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a400b40d0 0x7f3a400ba970 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3a30000c00 tx=0x7f3a300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:59.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.363+0000 7f3a377fe640 1 -- 172.21.15.47:0/1077061011 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a30037040 con 0x7f3a400b40d0 2024-04-26T23:18:59.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.363+0000 7f3a377fe640 1 -- 172.21.15.47:0/1077061011 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a30042070 con 0x7f3a400b40d0 2024-04-26T23:18:59.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.363+0000 7f3a45407640 1 -- 172.21.15.47:0/1077061011 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a400c88d0 con 0x7f3a400b40d0 2024-04-26T23:18:59.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.363+0000 7f3a377fe640 1 -- 172.21.15.47:0/1077061011 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3003b890 con 0x7f3a400b40d0 2024-04-26T23:18:59.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.363+0000 7f3a45407640 1 -- 172.21.15.47:0/1077061011 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a400c8bb0 con 0x7f3a400b40d0 2024-04-26T23:18:59.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.364+0000 7f3a45407640 1 -- 172.21.15.47:0/1077061011 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a400694f0 con 0x7f3a400b40d0 2024-04-26T23:18:59.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.365+0000 7f3a377fe640 1 -- 172.21.15.47:0/1077061011 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3a3002ac50 con 0x7f3a400b40d0 2024-04-26T23:18:59.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.365+0000 7f3a377fe640 1 --2- 172.21.15.47:0/1077061011 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3a2006c8f0 0x7f3a2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:59.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.365+0000 7f3a377fe640 1 -- 172.21.15.47:0/1077061011 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f3a300ac4f0 con 0x7f3a400b40d0 2024-04-26T23:18:59.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.365+0000 7f3a37fff640 1 --2- 172.21.15.47:0/1077061011 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3a2006c8f0 0x7f3a2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:59.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.366+0000 7f3a37fff640 1 --2- 172.21.15.47:0/1077061011 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3a2006c8f0 0x7f3a2006edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3a28000f60 tx=0x7f3a28009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:59.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.369+0000 7f3a377fe640 1 -- 172.21.15.47:0/1077061011 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3a3007a670 con 0x7f3a400b40d0 2024-04-26T23:18:59.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.554+0000 7f3a45407640 1 -- 172.21.15.47:0/1077061011 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3a400d31b0 con 0x7f3a400b40d0 2024-04-26T23:18:59.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.554+0000 7f3a377fe640 1 -- 172.21.15.47:0/1077061011 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f3a30079f70 con 0x7f3a400b40d0 2024-04-26T23:18:59.556 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-26T23:18:59.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.556+0000 7f3a45407640 1 -- 172.21.15.47:0/1077061011 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3a2006c8f0 msgr2=0x7f3a2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:59.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.556+0000 7f3a45407640 1 --2- 172.21.15.47:0/1077061011 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3a2006c8f0 0x7f3a2006edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3a28000f60 tx=0x7f3a28009040 comp rx=0 tx=0).stop 2024-04-26T23:18:59.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.556+0000 7f3a45407640 1 -- 172.21.15.47:0/1077061011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a400b40d0 msgr2=0x7f3a400ba970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:59.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.556+0000 7f3a45407640 1 --2- 172.21.15.47:0/1077061011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a400b40d0 0x7f3a400ba970 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3a30000c00 tx=0x7f3a300046c0 comp rx=0 tx=0).stop 2024-04-26T23:18:59.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.556+0000 7f3a45407640 1 -- 172.21.15.47:0/1077061011 shutdown_connections 2024-04-26T23:18:59.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.556+0000 7f3a45407640 1 --2- 172.21.15.47:0/1077061011 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3a2006c8f0 0x7f3a2006edb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:59.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.556+0000 7f3a45407640 1 --2- 172.21.15.47:0/1077061011 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a400d2320 0x7f3a40151000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:59.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.556+0000 7f3a45407640 1 --2- 172.21.15.47:0/1077061011 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3a400d1a00 0x7f3a400cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:59.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.556+0000 7f3a45407640 1 --2- 172.21.15.47:0/1077061011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a400b40d0 0x7f3a400ba970 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:59.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.556+0000 7f3a45407640 1 -- 172.21.15.47:0/1077061011 >> 172.21.15.47:0/1077061011 conn(0x7f3a400d4fd0 msgr2=0x7f3a4006cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:59.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.556+0000 7f3a45407640 1 -- 172.21.15.47:0/1077061011 shutdown_connections 2024-04-26T23:18:59.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.556+0000 7f3a45407640 1 -- 172.21.15.47:0/1077061011 wait complete. 2024-04-26T23:18:59.569 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:18:59.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.671+0000 7ff8925b7640 1 Processor -- start 2024-04-26T23:18:59.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.671+0000 7ff8925b7640 1 -- start start 2024-04-26T23:18:59.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.671+0000 7ff8925b7640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff88c15f170 0x7ff88c0d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:59.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.672+0000 7ff8925b7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff88c0d3bc0 0x7ff88c0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:59.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.672+0000 7ff8925b7640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff88c0d1040 con 0x7ff88c15f550 2024-04-26T23:18:59.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.672+0000 7ff8925b7640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7ff88c0c9430 con 0x7ff88c15f170 2024-04-26T23:18:59.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.672+0000 7ff8925b7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7ff88c0c9680 con 0x7ff88c0d3bc0 2024-04-26T23:18:59.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.672+0000 7ff88bfff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff88c15f170 0x7ff88c0d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:59.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.672+0000 7ff88bfff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff88c15f170 0x7ff88c0d5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58914/0 (socket says 172.21.15.47:58914) 2024-04-26T23:18:59.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.672+0000 7ff88bfff640 1 -- 172.21.15.47:0/2819553581 learned_addr learned my addr 172.21.15.47:0/2819553581 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:59.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.672+0000 7ff890b54640 1 --2- 172.21.15.47:0/2819553581 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff88c0d3bc0 0x7ff88c0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:59.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.672+0000 7ff88bfff640 1 -- 172.21.15.47:0/2819553581 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff88c0d3bc0 msgr2=0x7ff88c0d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:59.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.672+0000 7ff88bfff640 1 --2- 172.21.15.47:0/2819553581 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff88c0d3bc0 0x7ff88c0d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:59.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.672+0000 7ff88bfff640 1 -- 172.21.15.47:0/2819553581 >> v1:172.21.15.203:6789/0 conn(0x7ff88c15f550 legacy=0x7ff88c0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:59.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.672+0000 7ff88bfff640 1 -- 172.21.15.47:0/2819553581 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff88c0c9960 con 0x7ff88c15f170 2024-04-26T23:18:59.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.672+0000 7ff890b54640 1 --2- 172.21.15.47:0/2819553581 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff88c0d3bc0 0x7ff88c0d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:18:59.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.673+0000 7ff88bfff640 1 --2- 172.21.15.47:0/2819553581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff88c15f170 0x7ff88c0d5a80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff87c0098a0 tx=0x7ff87c02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1762c370cc8fab4e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:59.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.673+0000 7ff88affd640 1 -- 172.21.15.47:0/2819553581 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87c037070 con 0x7ff88c15f170 2024-04-26T23:18:59.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.673+0000 7ff88affd640 1 -- 172.21.15.47:0/2819553581 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff87c032440 con 0x7ff88c15f170 2024-04-26T23:18:59.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.673+0000 7ff88affd640 1 -- 172.21.15.47:0/2819553581 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87c032810 con 0x7ff88c15f170 2024-04-26T23:18:59.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.673+0000 7ff8925b7640 1 -- 172.21.15.47:0/2819553581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff88c15f170 msgr2=0x7ff88c0d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:59.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.673+0000 7ff8925b7640 1 --2- 172.21.15.47:0/2819553581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff88c15f170 0x7ff88c0d5a80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff87c0098a0 tx=0x7ff87c02a250 comp rx=0 tx=0).stop 2024-04-26T23:18:59.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.673+0000 7ff8925b7640 1 -- 172.21.15.47:0/2819553581 shutdown_connections 2024-04-26T23:18:59.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.673+0000 7ff8925b7640 1 --2- 172.21.15.47:0/2819553581 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff88c0d3bc0 0x7ff88c0d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:59.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.673+0000 7ff8925b7640 1 --2- 172.21.15.47:0/2819553581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff88c15f170 0x7ff88c0d5a80 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:59.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.673+0000 7ff8925b7640 1 -- 172.21.15.47:0/2819553581 >> 172.21.15.47:0/2819553581 conn(0x7ff88c0d4f60 msgr2=0x7ff88c0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:59.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.673+0000 7ff8925b7640 1 -- 172.21.15.47:0/2819553581 shutdown_connections 2024-04-26T23:18:59.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.674+0000 7ff8925b7640 1 -- 172.21.15.47:0/2819553581 wait complete. 2024-04-26T23:18:59.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.675+0000 7ff8925b7640 1 Processor -- start 2024-04-26T23:18:59.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.675+0000 7ff8925b7640 1 -- start start 2024-04-26T23:18:59.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.675+0000 7ff8925b7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff88c0d3bc0 0x7ff88c0a2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:59.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.675+0000 7ff8925b7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff88c15f170 0x7ff88c156960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:59.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.676+0000 7ff8925b7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff88c0ce640 0x7ff88c15d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:59.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.676+0000 7ff8925b7640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff88c0ca5c0 con 0x7ff88c0ce640 2024-04-26T23:18:59.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.676+0000 7ff8925b7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff88c0ca440 con 0x7ff88c15f170 2024-04-26T23:18:59.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.676+0000 7ff8925b7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7ff88c0ca740 con 0x7ff88c0d3bc0 2024-04-26T23:18:59.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.676+0000 7ff890b54640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff88c0ce640 0x7ff88c15d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:59.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.676+0000 7ff890b54640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff88c0ce640 0x7ff88c15d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58928/0 (socket says 172.21.15.47:58928) 2024-04-26T23:18:59.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.676+0000 7ff890b54640 1 -- 172.21.15.47:0/4188453933 learned_addr learned my addr 172.21.15.47:0/4188453933 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:59.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.676+0000 7ff88b7fe640 1 --2- 172.21.15.47:0/4188453933 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff88c15f170 0x7ff88c156960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:59.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.676+0000 7ff88bfff640 1 --2- 172.21.15.47:0/4188453933 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff88c0d3bc0 0x7ff88c0a2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:59.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.676+0000 7ff890b54640 1 -- 172.21.15.47:0/4188453933 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff88c0d3bc0 msgr2=0x7ff88c0a2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:59.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.676+0000 7ff890b54640 1 --2- 172.21.15.47:0/4188453933 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff88c0d3bc0 0x7ff88c0a2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:59.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.676+0000 7ff890b54640 1 -- 172.21.15.47:0/4188453933 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff88c15f170 msgr2=0x7ff88c156960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:59.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.676+0000 7ff890b54640 1 --2- 172.21.15.47:0/4188453933 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff88c15f170 0x7ff88c156960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:59.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.676+0000 7ff890b54640 1 -- 172.21.15.47:0/4188453933 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff88c0cdb80 con 0x7ff88c0ce640 2024-04-26T23:18:59.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.676+0000 7ff890b54640 1 --2- 172.21.15.47:0/4188453933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff88c0ce640 0x7ff88c15d2a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff88400cef0 tx=0x7ff88400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:59.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.676+0000 7ff888ff9640 1 -- 172.21.15.47:0/4188453933 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff88401a070 con 0x7ff88c0ce640 2024-04-26T23:18:59.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.676+0000 7ff888ff9640 1 -- 172.21.15.47:0/4188453933 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff88400d860 con 0x7ff88c0ce640 2024-04-26T23:18:59.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.676+0000 7ff8925b7640 1 -- 172.21.15.47:0/4188453933 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff88c0d2630 con 0x7ff88c0ce640 2024-04-26T23:18:59.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.676+0000 7ff888ff9640 1 -- 172.21.15.47:0/4188453933 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff88400dc50 con 0x7ff88c0ce640 2024-04-26T23:18:59.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.676+0000 7ff8925b7640 1 -- 172.21.15.47:0/4188453933 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff88c0ce120 con 0x7ff88c0ce640 2024-04-26T23:18:59.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.678+0000 7ff8925b7640 1 -- 172.21.15.47:0/4188453933 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff88c0694f0 con 0x7ff88c0ce640 2024-04-26T23:18:59.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.678+0000 7ff888ff9640 1 -- 172.21.15.47:0/4188453933 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff88401d070 con 0x7ff88c0ce640 2024-04-26T23:18:59.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.678+0000 7ff888ff9640 1 --2- 172.21.15.47:0/4188453933 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7ff87806c8f0 0x7ff87806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:59.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.678+0000 7ff888ff9640 1 -- 172.21.15.47:0/4188453933 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7ff88408c670 con 0x7ff88c0ce640 2024-04-26T23:18:59.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.679+0000 7ff88bfff640 1 --2- 172.21.15.47:0/4188453933 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7ff87806c8f0 0x7ff87806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:59.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.679+0000 7ff88bfff640 1 --2- 172.21.15.47:0/4188453933 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7ff87806c8f0 0x7ff87806edb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff87c034000 tx=0x7ff87c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:59.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.683+0000 7ff888ff9640 1 -- 172.21.15.47:0/4188453933 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff88405a1d0 con 0x7ff88c0ce640 2024-04-26T23:18:59.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.866+0000 7ff8925b7640 1 -- 172.21.15.47:0/4188453933 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff88c15ea90 con 0x7ff88c0ce640 2024-04-26T23:18:59.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.866+0000 7ff888ff9640 1 -- 172.21.15.47:0/4188453933 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7ff88405e4d0 con 0x7ff88c0ce640 2024-04-26T23:18:59.869 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-26T23:18:59.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.868+0000 7ff8925b7640 1 -- 172.21.15.47:0/4188453933 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7ff87806c8f0 msgr2=0x7ff87806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:59.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.868+0000 7ff8925b7640 1 --2- 172.21.15.47:0/4188453933 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7ff87806c8f0 0x7ff87806edb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff87c034000 tx=0x7ff87c02e000 comp rx=0 tx=0).stop 2024-04-26T23:18:59.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.868+0000 7ff8925b7640 1 -- 172.21.15.47:0/4188453933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff88c0ce640 msgr2=0x7ff88c15d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:59.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.868+0000 7ff8925b7640 1 --2- 172.21.15.47:0/4188453933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff88c0ce640 0x7ff88c15d2a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff88400cef0 tx=0x7ff88400a710 comp rx=0 tx=0).stop 2024-04-26T23:18:59.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.868+0000 7ff8925b7640 1 -- 172.21.15.47:0/4188453933 shutdown_connections 2024-04-26T23:18:59.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.868+0000 7ff8925b7640 1 --2- 172.21.15.47:0/4188453933 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7ff87806c8f0 0x7ff87806edb0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:59.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.868+0000 7ff8925b7640 1 --2- 172.21.15.47:0/4188453933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff88c0ce640 0x7ff88c15d2a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:59.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.868+0000 7ff8925b7640 1 --2- 172.21.15.47:0/4188453933 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff88c15f170 0x7ff88c156960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:59.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.868+0000 7ff8925b7640 1 --2- 172.21.15.47:0/4188453933 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff88c0d3bc0 0x7ff88c0a2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:59.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.869+0000 7ff8925b7640 1 -- 172.21.15.47:0/4188453933 >> 172.21.15.47:0/4188453933 conn(0x7ff88c0d4f60 msgr2=0x7ff88c06d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:59.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.869+0000 7ff8925b7640 1 -- 172.21.15.47:0/4188453933 shutdown_connections 2024-04-26T23:18:59.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.869+0000 7ff8925b7640 1 -- 172.21.15.47:0/4188453933 wait complete. 2024-04-26T23:18:59.881 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T23:18:59.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.984+0000 7fd5cf645640 1 Processor -- start 2024-04-26T23:18:59.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.984+0000 7fd5cf645640 1 -- start start 2024-04-26T23:18:59.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.985+0000 7fd5cf645640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd5c815f170 0x7fd5c80d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:59.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.985+0000 7fd5cf645640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd5c815f550 0x7fd5c80d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:59.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.985+0000 7fd5cf645640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5c80d1040 con 0x7fd5c80d34b0 2024-04-26T23:18:59.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.985+0000 7fd5cf645640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fd5c80c9430 con 0x7fd5c815f170 2024-04-26T23:18:59.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.985+0000 7fd5cf645640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fd5c80c9680 con 0x7fd5c815f550 2024-04-26T23:18:59.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.985+0000 7fd5cd3e1640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd5c815f170 0x7fd5c80d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:59.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.985+0000 7fd5cdbe2640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd5c80d34b0 0x7fd5c80cfc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:48134/0 (socket says 172.21.15.47:48134) 2024-04-26T23:18:59.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.985+0000 7fd5cdbe2640 1 -- 172.21.15.47:0/3410078899 learned_addr learned my addr 172.21.15.47:0/3410078899 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:18:59.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.985+0000 7fd5cd3e1640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd5c815f170 0x7fd5c80d5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54616/0 (socket says 172.21.15.47:54616) 2024-04-26T23:18:59.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.985+0000 7fd5ccbe0640 1 --2- 172.21.15.47:0/3410078899 >> v2:172.21.15.203:3300/0 conn(0x7fd5c815f550 0x7fd5c80d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:59.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.985+0000 7fd5cd3e1640 1 -- 172.21.15.47:0/3410078899 >> v2:172.21.15.203:3300/0 conn(0x7fd5c815f550 msgr2=0x7fd5c80d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:18:59.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.985+0000 7fd5cd3e1640 1 --2- 172.21.15.47:0/3410078899 >> v2:172.21.15.203:3300/0 conn(0x7fd5c815f550 0x7fd5c80d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:59.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.985+0000 7fd5cd3e1640 1 -- 172.21.15.47:0/3410078899 >> v1:172.21.15.47:6789/0 conn(0x7fd5c80d34b0 legacy=0x7fd5c80cfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:59.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.985+0000 7fd5cd3e1640 1 -- 172.21.15.47:0/3410078899 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5c80c9960 con 0x7fd5c815f170 2024-04-26T23:18:59.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.985+0000 7fd5c7fff640 1 -- 172.21.15.47:0/3410078899 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3190962214 0 0) 0x7fd5c0001110 con 0x7fd5c80d34b0 2024-04-26T23:18:59.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.986+0000 7fd5cd3e1640 1 --2- 172.21.15.47:0/3410078899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5c815f170 0x7fd5c80d5a80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8009d90 tx=0x7fd5b802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ad11f26c3655a07 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:59.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.986+0000 7fd5c7fff640 1 -- 172.21.15.47:0/3410078899 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8007740 con 0x7fd5c815f170 2024-04-26T23:18:59.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.986+0000 7fd5c7fff640 1 -- 172.21.15.47:0/3410078899 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5b80078e0 con 0x7fd5c815f170 2024-04-26T23:18:59.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.986+0000 7fd5c7fff640 1 -- 172.21.15.47:0/3410078899 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8007cb0 con 0x7fd5c815f170 2024-04-26T23:18:59.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.987+0000 7fd5cf645640 1 -- 172.21.15.47:0/3410078899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5c815f170 msgr2=0x7fd5c80d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:59.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.987+0000 7fd5cf645640 1 --2- 172.21.15.47:0/3410078899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5c815f170 0x7fd5c80d5a80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8009d90 tx=0x7fd5b802a6a0 comp rx=0 tx=0).stop 2024-04-26T23:18:59.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.987+0000 7fd5cf645640 1 -- 172.21.15.47:0/3410078899 shutdown_connections 2024-04-26T23:18:59.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.987+0000 7fd5cf645640 1 --2- 172.21.15.47:0/3410078899 >> v2:172.21.15.203:3300/0 conn(0x7fd5c815f550 0x7fd5c80d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:59.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.987+0000 7fd5cf645640 1 --2- 172.21.15.47:0/3410078899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5c815f170 0x7fd5c80d5a80 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:59.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.987+0000 7fd5cf645640 1 -- 172.21.15.47:0/3410078899 >> 172.21.15.47:0/3410078899 conn(0x7fd5c80d4f60 msgr2=0x7fd5c80d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:18:59.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.987+0000 7fd5cf645640 1 -- 172.21.15.47:0/3410078899 shutdown_connections 2024-04-26T23:18:59.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.987+0000 7fd5cf645640 1 -- 172.21.15.47:0/3410078899 wait complete. 2024-04-26T23:18:59.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.988+0000 7fd5cf645640 1 Processor -- start 2024-04-26T23:18:59.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.988+0000 7fd5cf645640 1 -- start start 2024-04-26T23:18:59.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.989+0000 7fd5cf645640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5c80d34b0 0x7fd5c815b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:59.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.989+0000 7fd5cf645640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5c815f170 0x7fd5c806cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:59.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.989+0000 7fd5cf645640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5c806d3e0 0x7fd5c8159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:59.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.989+0000 7fd5cf645640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c80cb5d0 con 0x7fd5c815f170 2024-04-26T23:18:59.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.989+0000 7fd5cf645640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c80cb450 con 0x7fd5c806d3e0 2024-04-26T23:18:59.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.989+0000 7fd5cf645640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd5c80cb750 con 0x7fd5c80d34b0 2024-04-26T23:18:59.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.989+0000 7fd5ccbe0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5c815f170 0x7fd5c806cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:59.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.989+0000 7fd5ccbe0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5c815f170 0x7fd5c806cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54632/0 (socket says 172.21.15.47:54632) 2024-04-26T23:18:59.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.989+0000 7fd5ccbe0640 1 -- 172.21.15.47:0/3555690118 learned_addr learned my addr 172.21.15.47:0/3555690118 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:18:59.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.989+0000 7fd5cd3e1640 1 --2- 172.21.15.47:0/3555690118 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5c80d34b0 0x7fd5c815b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:59.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.989+0000 7fd5cdbe2640 1 --2- 172.21.15.47:0/3555690118 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5c806d3e0 0x7fd5c8159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:59.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.989+0000 7fd5ccbe0640 1 -- 172.21.15.47:0/3555690118 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5c80d34b0 msgr2=0x7fd5c815b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:59.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.989+0000 7fd5ccbe0640 1 --2- 172.21.15.47:0/3555690118 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5c80d34b0 0x7fd5c815b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:59.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.989+0000 7fd5ccbe0640 1 -- 172.21.15.47:0/3555690118 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5c806d3e0 msgr2=0x7fd5c8159bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:18:59.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.989+0000 7fd5ccbe0640 1 --2- 172.21.15.47:0/3555690118 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5c806d3e0 0x7fd5c8159bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:18:59.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.989+0000 7fd5ccbe0640 1 -- 172.21.15.47:0/3555690118 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5c815a920 con 0x7fd5c815f170 2024-04-26T23:18:59.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.990+0000 7fd5ccbe0640 1 --2- 172.21.15.47:0/3555690118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5c815f170 0x7fd5c806cea0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc00e8a0 tx=0x7fd5bc00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:59.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.990+0000 7fd5c5ffb640 1 -- 172.21.15.47:0/3555690118 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc017070 con 0x7fd5c815f170 2024-04-26T23:18:59.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.990+0000 7fd5c5ffb640 1 -- 172.21.15.47:0/3555690118 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5bc00ee90 con 0x7fd5c815f170 2024-04-26T23:18:59.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.990+0000 7fd5c5ffb640 1 -- 172.21.15.47:0/3555690118 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc00cae0 con 0x7fd5c815f170 2024-04-26T23:18:59.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.990+0000 7fd5cf645640 1 -- 172.21.15.47:0/3555690118 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5c815ac00 con 0x7fd5c815f170 2024-04-26T23:18:59.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.990+0000 7fd5cf645640 1 -- 172.21.15.47:0/3555690118 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5c80cd0e0 con 0x7fd5c815f170 2024-04-26T23:18:59.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.991+0000 7fd5cf645640 1 -- 172.21.15.47:0/3555690118 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5c80ccc30 con 0x7fd5c815f170 2024-04-26T23:18:59.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.991+0000 7fd5c5ffb640 1 -- 172.21.15.47:0/3555690118 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd5bc01a070 con 0x7fd5c815f170 2024-04-26T23:18:59.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.992+0000 7fd5c5ffb640 1 --2- 172.21.15.47:0/3555690118 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd5b406c8f0 0x7fd5b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:18:59.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.992+0000 7fd5c5ffb640 1 -- 172.21.15.47:0/3555690118 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd5bc08c030 con 0x7fd5c815f170 2024-04-26T23:18:59.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.992+0000 7fd5cd3e1640 1 --2- 172.21.15.47:0/3555690118 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd5b406c8f0 0x7fd5b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:18:59.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.992+0000 7fd5cd3e1640 1 --2- 172.21.15.47:0/3555690118 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd5b406c8f0 0x7fd5b406edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8034000 tx=0x7fd5b802f000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:18:59.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:18:59.996+0000 7fd5c5ffb640 1 -- 172.21.15.47:0/3555690118 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd5bc059a30 con 0x7fd5c815f170 2024-04-26T23:19:00.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.187+0000 7fd5cf645640 1 -- 172.21.15.47:0/3555690118 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fd5c80d3b30 con 0x7fd5c815f170 2024-04-26T23:19:00.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.188+0000 7fd5c5ffb640 1 -- 172.21.15.47:0/3555690118 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7fd5c80d3b30 con 0x7fd5c815f170 2024-04-26T23:19:00.189 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-26T23:19:00.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.189+0000 7fd5cf645640 1 -- 172.21.15.47:0/3555690118 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd5b406c8f0 msgr2=0x7fd5b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:00.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.189+0000 7fd5cf645640 1 --2- 172.21.15.47:0/3555690118 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd5b406c8f0 0x7fd5b406edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8034000 tx=0x7fd5b802f000 comp rx=0 tx=0).stop 2024-04-26T23:19:00.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.190+0000 7fd5cf645640 1 -- 172.21.15.47:0/3555690118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5c815f170 msgr2=0x7fd5c806cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:00.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.190+0000 7fd5cf645640 1 --2- 172.21.15.47:0/3555690118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5c815f170 0x7fd5c806cea0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc00e8a0 tx=0x7fd5bc00ed60 comp rx=0 tx=0).stop 2024-04-26T23:19:00.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.190+0000 7fd5cf645640 1 -- 172.21.15.47:0/3555690118 shutdown_connections 2024-04-26T23:19:00.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.190+0000 7fd5cf645640 1 --2- 172.21.15.47:0/3555690118 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd5b406c8f0 0x7fd5b406edb0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:00.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.190+0000 7fd5cf645640 1 --2- 172.21.15.47:0/3555690118 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5c806d3e0 0x7fd5c8159bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:00.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.190+0000 7fd5cf645640 1 --2- 172.21.15.47:0/3555690118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5c815f170 0x7fd5c806cea0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:00.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.190+0000 7fd5cf645640 1 --2- 172.21.15.47:0/3555690118 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5c80d34b0 0x7fd5c815b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:00.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.190+0000 7fd5cf645640 1 -- 172.21.15.47:0/3555690118 >> 172.21.15.47:0/3555690118 conn(0x7fd5c80d4f60 msgr2=0x7fd5c80b9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:00.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.190+0000 7fd5cf645640 1 -- 172.21.15.47:0/3555690118 shutdown_connections 2024-04-26T23:19:00.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.190+0000 7fd5cf645640 1 -- 172.21.15.47:0/3555690118 wait complete. 2024-04-26T23:19:00.202 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:19:00.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.301+0000 7f1d54ed1640 1 Processor -- start 2024-04-26T23:19:00.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.301+0000 7f1d54ed1640 1 -- start start 2024-04-26T23:19:00.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.302+0000 7f1d54ed1640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1d500d3bc0 0x7f1d500d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:00.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.302+0000 7f1d54ed1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d500b5f20 0x7f1d500b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:00.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.302+0000 7f1d54ed1640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d500b7e50 con 0x7f1d50162880 2024-04-26T23:19:00.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.302+0000 7f1d54ed1640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f1d500d0580 con 0x7f1d500d3bc0 2024-04-26T23:19:00.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.302+0000 7f1d54ed1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f1d500d1800 con 0x7f1d500b5f20 2024-04-26T23:19:00.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.302+0000 7f1d4ed9d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d500b5f20 0x7f1d500b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:00.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.302+0000 7f1d4dd9b640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1d500d3bc0 0x7f1d500d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:00.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.302+0000 7f1d4dd9b640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1d500d3bc0 0x7f1d500d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.47:47964/0 (socket says 172.21.15.47:47964) 2024-04-26T23:19:00.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.302+0000 7f1d4dd9b640 1 -- 172.21.15.47:0/3446333333 learned_addr learned my addr 172.21.15.47:0/3446333333 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:00.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.302+0000 7f1d4ed9d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d500b5f20 0x7f1d500b8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.47:52252/0 (socket says 172.21.15.47:52252) 2024-04-26T23:19:00.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.303+0000 7f1d4dd9b640 1 -- 172.21.15.47:0/3446333333 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d500b5f20 msgr2=0x7f1d500b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:00.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.303+0000 7f1d4dd9b640 1 --2- 172.21.15.47:0/3446333333 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d500b5f20 0x7f1d500b8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:00.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.303+0000 7f1d4dd9b640 1 -- 172.21.15.47:0/3446333333 >> v1:172.21.15.203:6789/0 conn(0x7f1d50162880 legacy=0x7f1d500d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:00.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.303+0000 7f1d4dd9b640 1 -- 172.21.15.47:0/3446333333 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d500d07d0 con 0x7f1d500d3bc0 2024-04-26T23:19:00.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.303+0000 7f1d4d59a640 1 -- 172.21.15.47:0/3446333333 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2212868460 0 0) 0x7f1d500b7e50 con 0x7f1d50162880 2024-04-26T23:19:00.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.303+0000 7f1d4ed9d640 1 --2- 172.21.15.47:0/3446333333 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d500b5f20 0x7f1d500b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:19:00.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.303+0000 7f1d4dd9b640 1 --2- 172.21.15.47:0/3446333333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d500d3bc0 0x7f1d500d2cb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1d38009870 tx=0x7f1d3802a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b68f6ec72b28e49f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:00.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.304+0000 7f1d4d59a640 1 -- 172.21.15.47:0/3446333333 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d38037070 con 0x7f1d500d3bc0 2024-04-26T23:19:00.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.304+0000 7f1d4d59a640 1 -- 172.21.15.47:0/3446333333 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d38032440 con 0x7f1d500d3bc0 2024-04-26T23:19:00.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.304+0000 7f1d4d59a640 1 -- 172.21.15.47:0/3446333333 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d38032810 con 0x7f1d500d3bc0 2024-04-26T23:19:00.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.304+0000 7f1d54ed1640 1 -- 172.21.15.47:0/3446333333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d500d3bc0 msgr2=0x7f1d500d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:00.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.304+0000 7f1d54ed1640 1 --2- 172.21.15.47:0/3446333333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d500d3bc0 0x7f1d500d2cb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1d38009870 tx=0x7f1d3802a250 comp rx=0 tx=0).stop 2024-04-26T23:19:00.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.304+0000 7f1d54ed1640 1 -- 172.21.15.47:0/3446333333 shutdown_connections 2024-04-26T23:19:00.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.304+0000 7f1d54ed1640 1 --2- 172.21.15.47:0/3446333333 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d500b5f20 0x7f1d500b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:00.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.304+0000 7f1d54ed1640 1 --2- 172.21.15.47:0/3446333333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d500d3bc0 0x7f1d500d2cb0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:00.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.304+0000 7f1d54ed1640 1 -- 172.21.15.47:0/3446333333 >> 172.21.15.47:0/3446333333 conn(0x7f1d500d4f60 msgr2=0x7f1d500d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:00.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.304+0000 7f1d54ed1640 1 -- 172.21.15.47:0/3446333333 shutdown_connections 2024-04-26T23:19:00.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.304+0000 7f1d54ed1640 1 -- 172.21.15.47:0/3446333333 wait complete. 2024-04-26T23:19:00.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.306+0000 7f1d54ed1640 1 Processor -- start 2024-04-26T23:19:00.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.306+0000 7f1d54ed1640 1 -- start start 2024-04-26T23:19:00.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.306+0000 7f1d54ed1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d500b5f20 0x7f1d50159640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:00.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.306+0000 7f1d54ed1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d500d3bc0 0x7f1d500ceba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:00.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.306+0000 7f1d54ed1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d50162880 0x7f1d500b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:00.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.306+0000 7f1d54ed1640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1d50162c60 con 0x7f1d500b5f20 2024-04-26T23:19:00.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.306+0000 7f1d54ed1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1d500d0d90 con 0x7f1d500d3bc0 2024-04-26T23:19:00.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.306+0000 7f1d54ed1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f1d50162de0 con 0x7f1d50162880 2024-04-26T23:19:00.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.306+0000 7f1d4e59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d500b5f20 0x7f1d50159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:00.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.307+0000 7f1d4e59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d500b5f20 0x7f1d50159640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54634/0 (socket says 172.21.15.47:54634) 2024-04-26T23:19:00.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.307+0000 7f1d4e59c640 1 -- 172.21.15.47:0/2493849568 learned_addr learned my addr 172.21.15.47:0/2493849568 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:00.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.307+0000 7f1d4dd9b640 1 --2- 172.21.15.47:0/2493849568 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d500d3bc0 0x7f1d500ceba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:00.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.307+0000 7f1d4ed9d640 1 --2- 172.21.15.47:0/2493849568 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d50162880 0x7f1d500b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:00.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.307+0000 7f1d4e59c640 1 -- 172.21.15.47:0/2493849568 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d50162880 msgr2=0x7f1d500b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:00.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.307+0000 7f1d4e59c640 1 --2- 172.21.15.47:0/2493849568 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d50162880 0x7f1d500b69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:00.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.307+0000 7f1d4e59c640 1 -- 172.21.15.47:0/2493849568 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d500d3bc0 msgr2=0x7f1d500ceba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:00.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.307+0000 7f1d4e59c640 1 --2- 172.21.15.47:0/2493849568 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d500d3bc0 0x7f1d500ceba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:00.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.307+0000 7f1d4e59c640 1 -- 172.21.15.47:0/2493849568 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d500ccff0 con 0x7f1d500b5f20 2024-04-26T23:19:00.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.307+0000 7f1d4e59c640 1 --2- 172.21.15.47:0/2493849568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d500b5f20 0x7f1d50159640 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1d4000c4b0 tx=0x7f1d4000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:00.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.307+0000 7f1d46ffd640 1 -- 172.21.15.47:0/2493849568 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d40017070 con 0x7f1d500b5f20 2024-04-26T23:19:00.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.307+0000 7f1d46ffd640 1 -- 172.21.15.47:0/2493849568 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d400035f0 con 0x7f1d500b5f20 2024-04-26T23:19:00.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.307+0000 7f1d54ed1640 1 -- 172.21.15.47:0/2493849568 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d500cd2d0 con 0x7f1d500b5f20 2024-04-26T23:19:00.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.307+0000 7f1d46ffd640 1 -- 172.21.15.47:0/2493849568 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d400039e0 con 0x7f1d500b5f20 2024-04-26T23:19:00.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.307+0000 7f1d54ed1640 1 -- 172.21.15.47:0/2493849568 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d500cd890 con 0x7f1d500b5f20 2024-04-26T23:19:00.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.308+0000 7f1d54ed1640 1 -- 172.21.15.47:0/2493849568 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d500694f0 con 0x7f1d500b5f20 2024-04-26T23:19:00.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.309+0000 7f1d46ffd640 1 -- 172.21.15.47:0/2493849568 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1d4001a070 con 0x7f1d500b5f20 2024-04-26T23:19:00.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.309+0000 7f1d46ffd640 1 --2- 172.21.15.47:0/2493849568 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1d3006c8f0 0x7f1d3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:00.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.309+0000 7f1d46ffd640 1 -- 172.21.15.47:0/2493849568 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1d4008b270 con 0x7f1d500b5f20 2024-04-26T23:19:00.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.309+0000 7f1d4dd9b640 1 --2- 172.21.15.47:0/2493849568 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1d3006c8f0 0x7f1d3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:00.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.310+0000 7f1d4dd9b640 1 --2- 172.21.15.47:0/2493849568 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1d3006c8f0 0x7f1d3006edb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1d38034000 tx=0x7f1d3802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:00.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.314+0000 7f1d46ffd640 1 -- 172.21.15.47:0/2493849568 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1d40058c70 con 0x7f1d500b5f20 2024-04-26T23:19:00.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.541+0000 7f1d54ed1640 1 -- 172.21.15.47:0/2493849568 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d500b78b0 con 0x7f1d500b5f20 2024-04-26T23:19:00.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.542+0000 7f1d46ffd640 1 -- 172.21.15.47:0/2493849568 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5180 (secure 0 0 0) 0x7f1d4005cf70 con 0x7f1d500b5f20 2024-04-26T23:19:00.544 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:19:00.544 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/1553696401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1553696401},{"type":"v1","addr":"172.21.15.47:6834","nonce":1553696401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/3792767564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3792767564},{"type":"v1","addr":"172.21.15.203:6836","nonce":3792767564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4088100766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":4088100766},{"type":"v1","addr":"172.21.15.47:6835","nonce":4088100766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:18:31.367292+0000","modified":"2024-04-26T23:18:53.430267+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/997115011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":997115011},{"type":"v1","addr":"172.21.15.203:6837","nonce":997115011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4282,"qdb_cluster":[4282]},"id":1}]} 2024-04-26T23:19:00.544 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 9 2024-04-26T23:19:00.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.544+0000 7f1d54ed1640 1 -- 172.21.15.47:0/2493849568 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1d3006c8f0 msgr2=0x7f1d3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:00.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.544+0000 7f1d54ed1640 1 --2- 172.21.15.47:0/2493849568 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1d3006c8f0 0x7f1d3006edb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1d38034000 tx=0x7f1d3802e000 comp rx=0 tx=0).stop 2024-04-26T23:19:00.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.544+0000 7f1d54ed1640 1 -- 172.21.15.47:0/2493849568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d500b5f20 msgr2=0x7f1d50159640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:00.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.544+0000 7f1d54ed1640 1 --2- 172.21.15.47:0/2493849568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d500b5f20 0x7f1d50159640 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1d4000c4b0 tx=0x7f1d4000c8b0 comp rx=0 tx=0).stop 2024-04-26T23:19:00.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.544+0000 7f1d54ed1640 1 -- 172.21.15.47:0/2493849568 shutdown_connections 2024-04-26T23:19:00.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.544+0000 7f1d54ed1640 1 --2- 172.21.15.47:0/2493849568 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1d3006c8f0 0x7f1d3006edb0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:00.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.544+0000 7f1d54ed1640 1 --2- 172.21.15.47:0/2493849568 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d50162880 0x7f1d500b69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:00.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.544+0000 7f1d54ed1640 1 --2- 172.21.15.47:0/2493849568 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d500d3bc0 0x7f1d500ceba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:00.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.544+0000 7f1d54ed1640 1 --2- 172.21.15.47:0/2493849568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d500b5f20 0x7f1d50159640 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:00.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.544+0000 7f1d54ed1640 1 -- 172.21.15.47:0/2493849568 >> 172.21.15.47:0/2493849568 conn(0x7f1d500d4f60 msgr2=0x7f1d500d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:00.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.544+0000 7f1d54ed1640 1 -- 172.21.15.47:0/2493849568 shutdown_connections 2024-04-26T23:19:00.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.544+0000 7f1d54ed1640 1 -- 172.21.15.47:0/2493849568 wait complete. 2024-04-26T23:19:00.557 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:19:00.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.656+0000 7f0bf9afc640 1 Processor -- start 2024-04-26T23:19:00.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.656+0000 7f0bf9afc640 1 -- start start 2024-04-26T23:19:00.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.656+0000 7f0bf9afc640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0bf415f5a0 0x7f0bf40d5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:00.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.656+0000 7f0bf9afc640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0bf40d2830 0x7f0bf4162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:00.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.656+0000 7f0bf9afc640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bf40d1a70 con 0x7f0bf40d3c20 2024-04-26T23:19:00.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.656+0000 7f0bf9afc640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f0bf4163d30 con 0x7f0bf40d2830 2024-04-26T23:19:00.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.656+0000 7f0bf9afc640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f0bf40d1d50 con 0x7f0bf415f5a0 2024-04-26T23:19:00.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.657+0000 7f0bf2ffd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0bf40d2830 0x7f0bf4162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:00.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.657+0000 7f0bf2ffd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0bf40d2830 0x7f0bf4162f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54636/0 (socket says 172.21.15.47:54636) 2024-04-26T23:19:00.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.657+0000 7f0bf2ffd640 1 -- 172.21.15.47:0/1414857092 learned_addr learned my addr 172.21.15.47:0/1414857092 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:00.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.657+0000 7f0bf37fe640 1 --2- 172.21.15.47:0/1414857092 >> v2:172.21.15.203:3300/0 conn(0x7f0bf415f5a0 0x7f0bf40d5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:00.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.657+0000 7f0bf2ffd640 1 -- 172.21.15.47:0/1414857092 >> v2:172.21.15.203:3300/0 conn(0x7f0bf415f5a0 msgr2=0x7f0bf40d5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:00.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.657+0000 7f0bf2ffd640 1 --2- 172.21.15.47:0/1414857092 >> v2:172.21.15.203:3300/0 conn(0x7f0bf415f5a0 0x7f0bf40d5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:00.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.657+0000 7f0bf2ffd640 1 -- 172.21.15.47:0/1414857092 >> v1:172.21.15.203:6789/0 conn(0x7f0bf40d3c20 legacy=0x7f0bf415f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:00.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.657+0000 7f0bf2ffd640 1 -- 172.21.15.47:0/1414857092 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bf40d2030 con 0x7f0bf40d2830 2024-04-26T23:19:00.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.657+0000 7f0bf37fe640 1 --2- 172.21.15.47:0/1414857092 >> v2:172.21.15.203:3300/0 conn(0x7f0bf415f5a0 0x7f0bf40d5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:19:00.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.658+0000 7f0bf2ffd640 1 --2- 172.21.15.47:0/1414857092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bf40d2830 0x7f0bf4162f00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0bf408d2f0 tx=0x7f0bdc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9bc83a027dd0033 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:00.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.658+0000 7f0bf27fc640 1 -- 172.21.15.47:0/1414857092 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc037070 con 0x7f0bf40d2830 2024-04-26T23:19:00.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.658+0000 7f0bf27fc640 1 -- 172.21.15.47:0/1414857092 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bdc032440 con 0x7f0bf40d2830 2024-04-26T23:19:00.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.658+0000 7f0bf27fc640 1 -- 172.21.15.47:0/1414857092 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc032810 con 0x7f0bf40d2830 2024-04-26T23:19:00.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.658+0000 7f0bf9afc640 1 -- 172.21.15.47:0/1414857092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bf40d2830 msgr2=0x7f0bf4162f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:00.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.658+0000 7f0bf9afc640 1 --2- 172.21.15.47:0/1414857092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bf40d2830 0x7f0bf4162f00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0bf408d2f0 tx=0x7f0bdc02a1e0 comp rx=0 tx=0).stop 2024-04-26T23:19:00.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.658+0000 7f0bf9afc640 1 -- 172.21.15.47:0/1414857092 shutdown_connections 2024-04-26T23:19:00.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.658+0000 7f0bf9afc640 1 --2- 172.21.15.47:0/1414857092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bf40d2830 0x7f0bf4162f00 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:00.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.658+0000 7f0bf9afc640 1 --2- 172.21.15.47:0/1414857092 >> v2:172.21.15.203:3300/0 conn(0x7f0bf415f5a0 0x7f0bf40d5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:00.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.658+0000 7f0bf9afc640 1 -- 172.21.15.47:0/1414857092 >> 172.21.15.47:0/1414857092 conn(0x7f0bf40d5190 msgr2=0x7f0bf40d55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:00.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.658+0000 7f0bf9afc640 1 -- 172.21.15.47:0/1414857092 shutdown_connections 2024-04-26T23:19:00.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.658+0000 7f0bf9afc640 1 -- 172.21.15.47:0/1414857092 wait complete. 2024-04-26T23:19:00.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.659+0000 7f0bf9afc640 1 Processor -- start 2024-04-26T23:19:00.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.660+0000 7f0bf9afc640 1 -- start start 2024-04-26T23:19:00.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.660+0000 7f0bf9afc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0bf40d2830 0x7f0bf40cb450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:00.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.660+0000 7f0bf9afc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0bf40d3c20 0x7f0bf41f7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:00.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.660+0000 7f0bf9afc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bf41f8310 0x7f0bf40ccc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:00.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.660+0000 7f0bf9afc640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf4163440 con 0x7f0bf41f8310 2024-04-26T23:19:00.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.660+0000 7f0bf9afc640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf40d4190 con 0x7f0bf40d3c20 2024-04-26T23:19:00.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.660+0000 7f0bf9afc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f0bf41635c0 con 0x7f0bf40d2830 2024-04-26T23:19:00.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.661+0000 7f0bf3fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bf41f8310 0x7f0bf40ccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:00.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.661+0000 7f0bf3fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bf41f8310 0x7f0bf40ccc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54638/0 (socket says 172.21.15.47:54638) 2024-04-26T23:19:00.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.661+0000 7f0bf3fff640 1 -- 172.21.15.47:0/129149398 learned_addr learned my addr 172.21.15.47:0/129149398 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:00.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.661+0000 7f0bf2ffd640 1 --2- 172.21.15.47:0/129149398 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0bf40d3c20 0x7f0bf41f7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:00.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.661+0000 7f0bf37fe640 1 --2- 172.21.15.47:0/129149398 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0bf40d2830 0x7f0bf40cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:00.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.661+0000 7f0bf3fff640 1 -- 172.21.15.47:0/129149398 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0bf40d2830 msgr2=0x7f0bf40cb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:00.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.661+0000 7f0bf3fff640 1 --2- 172.21.15.47:0/129149398 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0bf40d2830 0x7f0bf40cb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:00.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.661+0000 7f0bf3fff640 1 -- 172.21.15.47:0/129149398 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0bf40d3c20 msgr2=0x7f0bf41f7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:00.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.661+0000 7f0bf3fff640 1 --2- 172.21.15.47:0/129149398 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0bf40d3c20 0x7f0bf41f7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:00.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.661+0000 7f0bf3fff640 1 -- 172.21.15.47:0/129149398 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bf41f8790 con 0x7f0bf41f8310 2024-04-26T23:19:00.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.661+0000 7f0bf37fe640 1 --2- 172.21.15.47:0/129149398 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0bf40d2830 0x7f0bf40cb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:19:00.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.661+0000 7f0bf3fff640 1 --2- 172.21.15.47:0/129149398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bf41f8310 0x7f0bf40ccc70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0bec00c710 tx=0x7f0bec00cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:00.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.661+0000 7f0bebfff640 1 -- 172.21.15.47:0/129149398 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec016020 con 0x7f0bf41f8310 2024-04-26T23:19:00.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.661+0000 7f0bebfff640 1 -- 172.21.15.47:0/129149398 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bec006d50 con 0x7f0bf41f8310 2024-04-26T23:19:00.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.661+0000 7f0bebfff640 1 -- 172.21.15.47:0/129149398 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec003bb0 con 0x7f0bf41f8310 2024-04-26T23:19:00.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.661+0000 7f0bf9afc640 1 -- 172.21.15.47:0/129149398 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bf40cdbd0 con 0x7f0bf41f8310 2024-04-26T23:19:00.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.661+0000 7f0bf9afc640 1 -- 172.21.15.47:0/129149398 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bf414e250 con 0x7f0bf41f8310 2024-04-26T23:19:00.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.662+0000 7f0bf9afc640 1 -- 172.21.15.47:0/129149398 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bf40694f0 con 0x7f0bf41f8310 2024-04-26T23:19:00.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.663+0000 7f0bebfff640 1 -- 172.21.15.47:0/129149398 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0bec00a850 con 0x7f0bf41f8310 2024-04-26T23:19:00.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.663+0000 7f0bebfff640 1 --2- 172.21.15.47:0/129149398 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f0bd406c8f0 0x7f0bd406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:00.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.663+0000 7f0bebfff640 1 -- 172.21.15.47:0/129149398 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f0bec08bad0 con 0x7f0bf41f8310 2024-04-26T23:19:00.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.663+0000 7f0bf37fe640 1 --2- 172.21.15.47:0/129149398 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f0bd406c8f0 0x7f0bd406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:00.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.664+0000 7f0bf37fe640 1 --2- 172.21.15.47:0/129149398 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f0bd406c8f0 0x7f0bd406edb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0be4006fd0 tx=0x7f0be4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:00.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.668+0000 7f0bebfff640 1 -- 172.21.15.47:0/129149398 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0bec020020 con 0x7f0bf41f8310 2024-04-26T23:19:00.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.856+0000 7f0bf9afc640 1 -- 172.21.15.47:0/129149398 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0bf4068d30 con 0x7f0bf41f8310 2024-04-26T23:19:00.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.857+0000 7f0bebfff640 1 -- 172.21.15.47:0/129149398 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18478 (secure 0 0 0) 0x7f0bec019310 con 0x7f0bf41f8310 2024-04-26T23:19:00.859 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:19:00.859 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":24,"fsid":"399006d3-4db1-43e6-ba81-a79a276eabfd","created":"2024-04-26T23:18:06.731211+0000","modified":"2024-04-26T23:18:36.402132+0000","last_up_change":"2024-04-26T23:18:17.808736+0000","last_in_change":"2024-04-26T23:18:08.945859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:18:16.937492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:18:29.033575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:18:30.318499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:18:34.014536+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ee091567-fea0-4a14-bf4e-0e16c20bb2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6819","nonce":326520779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6821","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6823","nonce":326520779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6829","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6831","nonce":326520779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6825","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6827","nonce":326520779}]},"public_addr":"172.21.15.47:6819/326520779","cluster_addr":"172.21.15.47:6823/326520779","heartbeat_back_addr":"172.21.15.47:6831/326520779","heartbeat_front_addr":"172.21.15.47:6827/326520779","state":["exists","up"]},{"osd":1,"uuid":"c14440a0-bba5-4931-a6db-b736c08e869b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6809","nonce":1912610931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6811","nonce":1912610931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6815","nonce":1912610931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6813","nonce":1912610931}]},"public_addr":"172.21.15.47:6809/1912610931","cluster_addr":"172.21.15.47:6811/1912610931","heartbeat_back_addr":"172.21.15.47:6815/1912610931","heartbeat_front_addr":"172.21.15.47:6813/1912610931","state":["exists","up"]},{"osd":2,"uuid":"3940f8de-c9b2-4cd3-a911-9c301729eb30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6801","nonce":2569869305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6803","nonce":2569869305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6807","nonce":2569869305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6805","nonce":2569869305}]},"public_addr":"172.21.15.47:6801/2569869305","cluster_addr":"172.21.15.47:6803/2569869305","heartbeat_back_addr":"172.21.15.47:6807/2569869305","heartbeat_front_addr":"172.21.15.47:6805/2569869305","state":["exists","up"]},{"osd":3,"uuid":"1286a67e-8958-426a-8c2e-bc85498c619c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6818","nonce":2791411683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6822","nonce":2791411683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6830","nonce":2791411683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6826","nonce":2791411683}]},"public_addr":"172.21.15.47:6818/2791411683","cluster_addr":"172.21.15.47:6822/2791411683","heartbeat_back_addr":"172.21.15.47:6830/2791411683","heartbeat_front_addr":"172.21.15.47:6826/2791411683","state":["exists","up"]},{"osd":4,"uuid":"d8ee2c20-e5b7-46c2-9999-b234070126cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6809","nonce":441313104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6811","nonce":441313104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6815","nonce":441313104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6813","nonce":441313104}]},"public_addr":"172.21.15.203:6809/441313104","cluster_addr":"172.21.15.203:6811/441313104","heartbeat_back_addr":"172.21.15.203:6815/441313104","heartbeat_front_addr":"172.21.15.203:6813/441313104","state":["exists","up"]},{"osd":5,"uuid":"b453a8f0-f324-4840-9ae0-d67bdadb9f8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6801","nonce":56838444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6803","nonce":56838444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6807","nonce":56838444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6805","nonce":56838444}]},"public_addr":"172.21.15.203:6801/56838444","cluster_addr":"172.21.15.203:6803/56838444","heartbeat_back_addr":"172.21.15.203:6807/56838444","heartbeat_front_addr":"172.21.15.203:6805/56838444","state":["exists","up"]},{"osd":6,"uuid":"a6d31951-2d4b-4e8d-a510-6283b01dc158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6825","nonce":2738517080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6827","nonce":2738517080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6831","nonce":2738517080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6829","nonce":2738517080}]},"public_addr":"172.21.15.203:6825/2738517080","cluster_addr":"172.21.15.203:6827/2738517080","heartbeat_back_addr":"172.21.15.203:6831/2738517080","heartbeat_front_addr":"172.21.15.203:6829/2738517080","state":["exists","up"]},{"osd":7,"uuid":"7119b8c0-fe96-402d-9e75-7b509d9d44d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6817","nonce":1315565058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6819","nonce":1315565058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6823","nonce":1315565058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6821","nonce":1315565058}]},"public_addr":"172.21.15.203:6817/1315565058","cluster_addr":"172.21.15.203:6819/1315565058","heartbeat_back_addr":"172.21.15.203:6823/1315565058","heartbeat_front_addr":"172.21.15.203:6821/1315565058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.121921+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.137494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.084814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.111593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.072430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.125066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.097867+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.057061+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:19:00.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.859+0000 7f0bf9afc640 1 -- 172.21.15.47:0/129149398 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f0bd406c8f0 msgr2=0x7f0bd406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:00.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.859+0000 7f0bf9afc640 1 --2- 172.21.15.47:0/129149398 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f0bd406c8f0 0x7f0bd406edb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0be4006fd0 tx=0x7f0be4008040 comp rx=0 tx=0).stop 2024-04-26T23:19:00.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.859+0000 7f0bf9afc640 1 -- 172.21.15.47:0/129149398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bf41f8310 msgr2=0x7f0bf40ccc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:00.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.859+0000 7f0bf9afc640 1 --2- 172.21.15.47:0/129149398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bf41f8310 0x7f0bf40ccc70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0bec00c710 tx=0x7f0bec00cbd0 comp rx=0 tx=0).stop 2024-04-26T23:19:00.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.859+0000 7f0bf9afc640 1 -- 172.21.15.47:0/129149398 shutdown_connections 2024-04-26T23:19:00.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.859+0000 7f0bf9afc640 1 --2- 172.21.15.47:0/129149398 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f0bd406c8f0 0x7f0bd406edb0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:00.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.859+0000 7f0bf9afc640 1 --2- 172.21.15.47:0/129149398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bf41f8310 0x7f0bf40ccc70 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:00.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.859+0000 7f0bf9afc640 1 --2- 172.21.15.47:0/129149398 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0bf40d3c20 0x7f0bf41f7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:00.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.859+0000 7f0bf9afc640 1 --2- 172.21.15.47:0/129149398 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0bf40d2830 0x7f0bf40cb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:00.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.859+0000 7f0bf9afc640 1 -- 172.21.15.47:0/129149398 >> 172.21.15.47:0/129149398 conn(0x7f0bf40d5190 msgr2=0x7f0bf406ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:00.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.859+0000 7f0bf9afc640 1 -- 172.21.15.47:0/129149398 shutdown_connections 2024-04-26T23:19:00.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.859+0000 7f0bf9afc640 1 -- 172.21.15.47:0/129149398 wait complete. 2024-04-26T23:19:00.872 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:18:16.937492+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:18:29.033575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:18:30.318499+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T23:18:34.014536+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T23:19:00.872 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:19:00.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.975+0000 7f40153af640 1 Processor -- start 2024-04-26T23:19:00.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.975+0000 7f40153af640 1 -- start start 2024-04-26T23:19:00.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.975+0000 7f40153af640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f40100d1a00 0x7f40100d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:00.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.975+0000 7f40153af640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f40100d2320 0x7f40101625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:00.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.975+0000 7f40153af640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40100b40d0 0x7f4010162b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:00.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.975+0000 7f40153af640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f4010163070 con 0x7f40100d1a00 2024-04-26T23:19:00.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.975+0000 7f40153af640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f40100d0a10 con 0x7f40100d2320 2024-04-26T23:19:00.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.975+0000 7f40153af640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f40100d0cf0 con 0x7f40100b40d0 2024-04-26T23:19:00.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.976+0000 7f400effd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f40100d1a00 0x7f40100d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:00.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.976+0000 7f400effd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f40100d1a00 0x7f40100d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54644/0 (socket says 172.21.15.47:54644) 2024-04-26T23:19:00.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.976+0000 7f400effd640 1 -- 172.21.15.47:0/3035708632 learned_addr learned my addr 172.21.15.47:0/3035708632 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:00.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.976+0000 7f4007fff640 1 --2- 172.21.15.47:0/3035708632 >> v2:172.21.15.203:3300/0 conn(0x7f40100d2320 0x7f40101625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:00.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.976+0000 7f400f7fe640 1 --2- 172.21.15.47:0/3035708632 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40100b40d0 0x7f4010162b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:00.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.976+0000 7f400effd640 1 -- 172.21.15.47:0/3035708632 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40100b40d0 msgr2=0x7f4010162b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:00.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.976+0000 7f400effd640 1 --2- 172.21.15.47:0/3035708632 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40100b40d0 0x7f4010162b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:00.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.976+0000 7f400effd640 1 -- 172.21.15.47:0/3035708632 >> v2:172.21.15.203:3300/0 conn(0x7f40100d2320 msgr2=0x7f40101625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:00.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.976+0000 7f400effd640 1 --2- 172.21.15.47:0/3035708632 >> v2:172.21.15.203:3300/0 conn(0x7f40100d2320 0x7f40101625f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:00.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.976+0000 7f400effd640 1 -- 172.21.15.47:0/3035708632 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40100d0fd0 con 0x7f40100d1a00 2024-04-26T23:19:00.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.977+0000 7f400effd640 1 --2- 172.21.15.47:0/3035708632 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40100d1a00 0x7f40100d1de0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f401008d380 tx=0x7f400002a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ae5cddcdec5f620 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:00.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.977+0000 7f400e7fc640 1 -- 172.21.15.47:0/3035708632 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4000037070 con 0x7f40100d1a00 2024-04-26T23:19:00.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.977+0000 7f400e7fc640 1 -- 172.21.15.47:0/3035708632 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f400002ae00 con 0x7f40100d1a00 2024-04-26T23:19:00.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.977+0000 7f400e7fc640 1 -- 172.21.15.47:0/3035708632 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4000032660 con 0x7f40100d1a00 2024-04-26T23:19:00.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.977+0000 7f40153af640 1 -- 172.21.15.47:0/3035708632 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40100d1a00 msgr2=0x7f40100d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:00.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.977+0000 7f40153af640 1 --2- 172.21.15.47:0/3035708632 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40100d1a00 0x7f40100d1de0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f401008d380 tx=0x7f400002a0b0 comp rx=0 tx=0).stop 2024-04-26T23:19:00.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.977+0000 7f40153af640 1 -- 172.21.15.47:0/3035708632 shutdown_connections 2024-04-26T23:19:00.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.977+0000 7f40153af640 1 --2- 172.21.15.47:0/3035708632 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40100b40d0 0x7f4010162b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:00.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.977+0000 7f40153af640 1 --2- 172.21.15.47:0/3035708632 >> v2:172.21.15.203:3300/0 conn(0x7f40100d2320 0x7f40101625f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:00.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.977+0000 7f40153af640 1 --2- 172.21.15.47:0/3035708632 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40100d1a00 0x7f40100d1de0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:00.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.977+0000 7f40153af640 1 -- 172.21.15.47:0/3035708632 >> 172.21.15.47:0/3035708632 conn(0x7f40100d4fd0 msgr2=0x7f40100d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:00.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.978+0000 7f40153af640 1 -- 172.21.15.47:0/3035708632 shutdown_connections 2024-04-26T23:19:00.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.978+0000 7f40153af640 1 -- 172.21.15.47:0/3035708632 wait complete. 2024-04-26T23:19:00.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.979+0000 7f40153af640 1 Processor -- start 2024-04-26T23:19:00.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.979+0000 7f40153af640 1 -- start start 2024-04-26T23:19:00.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.979+0000 7f40153af640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40100b40d0 0x7f401015b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:00.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.979+0000 7f40153af640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40100d1a00 0x7f401015d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:00.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.979+0000 7f40153af640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40100d2320 0x7f40100cca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:00.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.979+0000 7f40153af640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f40100d3050 con 0x7f40100d1a00 2024-04-26T23:19:00.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.979+0000 7f40153af640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f40100d2ed0 con 0x7f40100d2320 2024-04-26T23:19:00.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.979+0000 7f40153af640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f40100d31d0 con 0x7f40100b40d0 2024-04-26T23:19:00.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.979+0000 7f4007fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40100d1a00 0x7f401015d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:00.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.980+0000 7f4007fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40100d1a00 0x7f401015d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54646/0 (socket says 172.21.15.47:54646) 2024-04-26T23:19:00.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.980+0000 7f4007fff640 1 -- 172.21.15.47:0/1992351530 learned_addr learned my addr 172.21.15.47:0/1992351530 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:00.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.980+0000 7f400f7fe640 1 --2- 172.21.15.47:0/1992351530 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40100d2320 0x7f40100cca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:00.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.980+0000 7f400effd640 1 --2- 172.21.15.47:0/1992351530 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40100b40d0 0x7f401015b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:00.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.980+0000 7f4007fff640 1 -- 172.21.15.47:0/1992351530 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40100b40d0 msgr2=0x7f401015b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:00.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.980+0000 7f4007fff640 1 --2- 172.21.15.47:0/1992351530 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40100b40d0 0x7f401015b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:00.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.980+0000 7f4007fff640 1 -- 172.21.15.47:0/1992351530 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40100d2320 msgr2=0x7f40100cca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:00.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.980+0000 7f4007fff640 1 --2- 172.21.15.47:0/1992351530 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40100d2320 0x7f40100cca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:00.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.980+0000 7f4007fff640 1 -- 172.21.15.47:0/1992351530 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40100cd570 con 0x7f40100d1a00 2024-04-26T23:19:00.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.980+0000 7f4007fff640 1 --2- 172.21.15.47:0/1992351530 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40100d1a00 0x7f401015d730 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3ff800d620 tx=0x7f3ff800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:00.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.980+0000 7f40077fe640 1 -- 172.21.15.47:0/1992351530 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff800b620 con 0x7f40100d1a00 2024-04-26T23:19:00.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.980+0000 7f40077fe640 1 -- 172.21.15.47:0/1992351530 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ff8019070 con 0x7f40100d1a00 2024-04-26T23:19:00.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.980+0000 7f40153af640 1 -- 172.21.15.47:0/1992351530 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40100b7e40 con 0x7f40100d1a00 2024-04-26T23:19:00.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.980+0000 7f40077fe640 1 -- 172.21.15.47:0/1992351530 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff800ba00 con 0x7f40100d1a00 2024-04-26T23:19:00.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.980+0000 7f40153af640 1 -- 172.21.15.47:0/1992351530 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40100b8400 con 0x7f40100d1a00 2024-04-26T23:19:00.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.981+0000 7f40153af640 1 -- 172.21.15.47:0/1992351530 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40100694f0 con 0x7f40100d1a00 2024-04-26T23:19:00.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.982+0000 7f40077fe640 1 -- 172.21.15.47:0/1992351530 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3ff800bba0 con 0x7f40100d1a00 2024-04-26T23:19:00.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.982+0000 7f40077fe640 1 --2- 172.21.15.47:0/1992351530 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3ff006c8f0 0x7f3ff006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:00.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.982+0000 7f40077fe640 1 -- 172.21.15.47:0/1992351530 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f3ff808c0c0 con 0x7f40100d1a00 2024-04-26T23:19:00.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.982+0000 7f400effd640 1 --2- 172.21.15.47:0/1992351530 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3ff006c8f0 0x7f3ff006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:00.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.983+0000 7f400effd640 1 --2- 172.21.15.47:0/1992351530 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3ff006c8f0 0x7f3ff006edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4000034000 tx=0x7f400002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:00.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:00.987+0000 7f40077fe640 1 -- 172.21.15.47:0/1992351530 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3ff8059bf0 con 0x7f40100d1a00 2024-04-26T23:19:01.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.174+0000 7f40153af640 1 -- 172.21.15.47:0/1992351530 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4010162a20 con 0x7f40100d1a00 2024-04-26T23:19:01.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.175+0000 7f40077fe640 1 -- 172.21.15.47:0/1992351530 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f3ff805def0 con 0x7f40100d1a00 2024-04-26T23:19:01.177 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-26T23:19:01.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.177+0000 7f40153af640 1 -- 172.21.15.47:0/1992351530 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3ff006c8f0 msgr2=0x7f3ff006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:01.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.177+0000 7f40153af640 1 --2- 172.21.15.47:0/1992351530 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3ff006c8f0 0x7f3ff006edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4000034000 tx=0x7f400002e000 comp rx=0 tx=0).stop 2024-04-26T23:19:01.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.177+0000 7f40153af640 1 -- 172.21.15.47:0/1992351530 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40100d1a00 msgr2=0x7f401015d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:01.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.177+0000 7f40153af640 1 --2- 172.21.15.47:0/1992351530 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40100d1a00 0x7f401015d730 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3ff800d620 tx=0x7f3ff800dae0 comp rx=0 tx=0).stop 2024-04-26T23:19:01.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.177+0000 7f40153af640 1 -- 172.21.15.47:0/1992351530 shutdown_connections 2024-04-26T23:19:01.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.177+0000 7f40153af640 1 --2- 172.21.15.47:0/1992351530 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3ff006c8f0 0x7f3ff006edb0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:01.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.177+0000 7f40153af640 1 --2- 172.21.15.47:0/1992351530 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40100d2320 0x7f40100cca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:01.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.177+0000 7f40153af640 1 --2- 172.21.15.47:0/1992351530 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40100d1a00 0x7f401015d730 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:01.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.177+0000 7f40153af640 1 --2- 172.21.15.47:0/1992351530 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40100b40d0 0x7f401015b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:01.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.177+0000 7f40153af640 1 -- 172.21.15.47:0/1992351530 >> 172.21.15.47:0/1992351530 conn(0x7f40100d4fd0 msgr2=0x7f40100cfe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:01.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.177+0000 7f40153af640 1 -- 172.21.15.47:0/1992351530 shutdown_connections 2024-04-26T23:19:01.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.177+0000 7f40153af640 1 -- 172.21.15.47:0/1992351530 wait complete. 2024-04-26T23:19:01.189 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:19:01.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.292+0000 7f92492bd640 1 Processor -- start 2024-04-26T23:19:01.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.292+0000 7f92492bd640 1 -- start start 2024-04-26T23:19:01.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.293+0000 7f92492bd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f924415f550 0x7f92440d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:01.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.293+0000 7f92492bd640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92440d1040 con 0x7f92440d3bc0 2024-04-26T23:19:01.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.293+0000 7f92492bd640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92440c9680 con 0x7f924415f170 2024-04-26T23:19:01.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.293+0000 7f92492bd640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f92440c9430 con 0x7f924415f550 2024-04-26T23:19:01.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.293+0000 7f92437fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f92440d3bc0 0x7f92440ca440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:48148/0 (socket says 172.21.15.47:48148) 2024-04-26T23:19:01.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.293+0000 7f92437fe640 1 -- 172.21.15.47:0/1982733192 learned_addr learned my addr 172.21.15.47:0/1982733192 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:01.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.293+0000 7f92427fc640 1 --2- 172.21.15.47:0/1982733192 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f924415f550 0x7f92440d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:01.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.293+0000 7f9241ffb640 1 -- 172.21.15.47:0/1982733192 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 199451398 0 0) 0x7f923c001490 con 0x7f92440d3bc0 2024-04-26T23:19:01.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.293+0000 7f9241ffb640 1 -- 172.21.15.47:0/1982733192 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f922c003520 con 0x7f92440d3bc0 2024-04-26T23:19:01.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.293+0000 7f9241ffb640 1 -- 172.21.15.47:0/1982733192 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1131297673 0 0) 0x7f92440c9680 con 0x7f924415f170 2024-04-26T23:19:01.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.293+0000 7f9241ffb640 1 -- 172.21.15.47:0/1982733192 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f923c001490 con 0x7f924415f170 2024-04-26T23:19:01.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.293+0000 7f9241ffb640 1 -- 172.21.15.47:0/1982733192 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1806318507 0 0) 0x7f922c003520 con 0x7f92440d3bc0 2024-04-26T23:19:01.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.293+0000 7f92427fc640 1 -- 172.21.15.47:0/1982733192 >> v1:172.21.15.203:6789/0 conn(0x7f924415f170 legacy=0x7f92440d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:01.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.293+0000 7f92427fc640 1 -- 172.21.15.47:0/1982733192 >> v1:172.21.15.47:6789/0 conn(0x7f92440d3bc0 legacy=0x7f92440ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:01.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.293+0000 7f92427fc640 1 -- 172.21.15.47:0/1982733192 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92440c9960 con 0x7f924415f550 2024-04-26T23:19:01.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.294+0000 7f92427fc640 1 --2- 172.21.15.47:0/1982733192 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f924415f550 0x7f92440d5a80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f92380097d0 tx=0x7f923802a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfd43e224f29138e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:01.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.294+0000 7f9241ffb640 1 -- 172.21.15.47:0/1982733192 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9238037070 con 0x7f924415f550 2024-04-26T23:19:01.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.294+0000 7f9241ffb640 1 -- 172.21.15.47:0/1982733192 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f923802ac30 con 0x7f924415f550 2024-04-26T23:19:01.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.294+0000 7f9241ffb640 1 -- 172.21.15.47:0/1982733192 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923c001b40 con 0x7f924415f550 2024-04-26T23:19:01.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.295+0000 7f92492bd640 1 -- 172.21.15.47:0/1982733192 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f924415f550 msgr2=0x7f92440d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:01.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.295+0000 7f92492bd640 1 --2- 172.21.15.47:0/1982733192 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f924415f550 0x7f92440d5a80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f92380097d0 tx=0x7f923802a1a0 comp rx=0 tx=0).stop 2024-04-26T23:19:01.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.295+0000 7f92492bd640 1 -- 172.21.15.47:0/1982733192 shutdown_connections 2024-04-26T23:19:01.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.295+0000 7f92492bd640 1 --2- 172.21.15.47:0/1982733192 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f924415f550 0x7f92440d5a80 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:01.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.295+0000 7f92492bd640 1 -- 172.21.15.47:0/1982733192 >> 172.21.15.47:0/1982733192 conn(0x7f92440d4f60 msgr2=0x7f92440d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:01.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.295+0000 7f92492bd640 1 -- 172.21.15.47:0/1982733192 shutdown_connections 2024-04-26T23:19:01.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.295+0000 7f92492bd640 1 -- 172.21.15.47:0/1982733192 wait complete. 2024-04-26T23:19:01.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.296+0000 7f92492bd640 1 Processor -- start 2024-04-26T23:19:01.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.296+0000 7f92492bd640 1 -- start start 2024-04-26T23:19:01.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.297+0000 7f92492bd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f92440d3bc0 0x7f9244150d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:01.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.297+0000 7f92492bd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f924415f170 0x7f92440a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:01.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.297+0000 7f92492bd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92440a24d0 0x7f92440b9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:01.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.297+0000 7f92492bd640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f92440d2e30 con 0x7f924415f170 2024-04-26T23:19:01.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.297+0000 7f92492bd640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f92440d2cb0 con 0x7f92440a24d0 2024-04-26T23:19:01.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.297+0000 7f92492bd640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f92440d2fb0 con 0x7f92440d3bc0 2024-04-26T23:19:01.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.297+0000 7f92427fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f924415f170 0x7f92440a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:01.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.297+0000 7f92427fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f924415f170 0x7f92440a1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54658/0 (socket says 172.21.15.47:54658) 2024-04-26T23:19:01.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.297+0000 7f92427fc640 1 -- 172.21.15.47:0/1650364334 learned_addr learned my addr 172.21.15.47:0/1650364334 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:01.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.297+0000 7f9242ffd640 1 --2- 172.21.15.47:0/1650364334 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f92440d3bc0 0x7f9244150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:01.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.297+0000 7f92437fe640 1 --2- 172.21.15.47:0/1650364334 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92440a24d0 0x7f92440b9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:01.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.298+0000 7f92427fc640 1 -- 172.21.15.47:0/1650364334 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f92440d3bc0 msgr2=0x7f9244150d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:01.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.298+0000 7f92427fc640 1 --2- 172.21.15.47:0/1650364334 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f92440d3bc0 0x7f9244150d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:01.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.298+0000 7f92427fc640 1 -- 172.21.15.47:0/1650364334 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92440a24d0 msgr2=0x7f92440b9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:01.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.298+0000 7f92427fc640 1 --2- 172.21.15.47:0/1650364334 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92440a24d0 0x7f92440b9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:01.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.298+0000 7f92427fc640 1 -- 172.21.15.47:0/1650364334 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92440baa50 con 0x7f924415f170 2024-04-26T23:19:01.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.298+0000 7f92427fc640 1 --2- 172.21.15.47:0/1650364334 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f924415f170 0x7f92440a1f90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9238035040 tx=0x7f92380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:01.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.298+0000 7f922b7fe640 1 -- 172.21.15.47:0/1650364334 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9238041020 con 0x7f924415f170 2024-04-26T23:19:01.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.298+0000 7f922b7fe640 1 -- 172.21.15.47:0/1650364334 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f923803d640 con 0x7f924415f170 2024-04-26T23:19:01.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.298+0000 7f92492bd640 1 -- 172.21.15.47:0/1650364334 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9244151580 con 0x7f924415f170 2024-04-26T23:19:01.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.298+0000 7f922b7fe640 1 -- 172.21.15.47:0/1650364334 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9238037040 con 0x7f924415f170 2024-04-26T23:19:01.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.298+0000 7f92492bd640 1 -- 172.21.15.47:0/1650364334 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9244151b40 con 0x7f924415f170 2024-04-26T23:19:01.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.299+0000 7f92492bd640 1 -- 172.21.15.47:0/1650364334 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92440694f0 con 0x7f924415f170 2024-04-26T23:19:01.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.299+0000 7f922b7fe640 1 -- 172.21.15.47:0/1650364334 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9238044020 con 0x7f924415f170 2024-04-26T23:19:01.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.300+0000 7f922b7fe640 1 --2- 172.21.15.47:0/1650364334 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f923006c8f0 0x7f923006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:01.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.300+0000 7f922b7fe640 1 -- 172.21.15.47:0/1650364334 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f92380ac390 con 0x7f924415f170 2024-04-26T23:19:01.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.300+0000 7f9242ffd640 1 --2- 172.21.15.47:0/1650364334 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f923006c8f0 0x7f923006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:01.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.301+0000 7f9242ffd640 1 --2- 172.21.15.47:0/1650364334 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f923006c8f0 0x7f923006edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f92340010e0 tx=0x7f9234009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:01.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.304+0000 7f922b7fe640 1 -- 172.21.15.47:0/1650364334 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f923804b030 con 0x7f924415f170 2024-04-26T23:19:01.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.487+0000 7f92492bd640 1 -- 172.21.15.47:0/1650364334 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f924408eff0 con 0x7f924415f170 2024-04-26T23:19:01.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.488+0000 7f922b7fe640 1 -- 172.21.15.47:0/1650364334 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f9238079d00 con 0x7f924415f170 2024-04-26T23:19:01.490 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-26T23:19:01.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.490+0000 7f92492bd640 1 -- 172.21.15.47:0/1650364334 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f923006c8f0 msgr2=0x7f923006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:01.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.490+0000 7f92492bd640 1 --2- 172.21.15.47:0/1650364334 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f923006c8f0 0x7f923006edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f92340010e0 tx=0x7f9234009040 comp rx=0 tx=0).stop 2024-04-26T23:19:01.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.490+0000 7f92492bd640 1 -- 172.21.15.47:0/1650364334 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f924415f170 msgr2=0x7f92440a1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:01.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.490+0000 7f92492bd640 1 --2- 172.21.15.47:0/1650364334 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f924415f170 0x7f92440a1f90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9238035040 tx=0x7f92380046c0 comp rx=0 tx=0).stop 2024-04-26T23:19:01.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.490+0000 7f92492bd640 1 -- 172.21.15.47:0/1650364334 shutdown_connections 2024-04-26T23:19:01.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.490+0000 7f92492bd640 1 --2- 172.21.15.47:0/1650364334 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f923006c8f0 0x7f923006edb0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:01.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.490+0000 7f92492bd640 1 --2- 172.21.15.47:0/1650364334 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92440a24d0 0x7f92440b9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:01.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.490+0000 7f92492bd640 1 --2- 172.21.15.47:0/1650364334 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f924415f170 0x7f92440a1f90 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:01.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.490+0000 7f92492bd640 1 --2- 172.21.15.47:0/1650364334 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f92440d3bc0 0x7f9244150d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:01.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.490+0000 7f92492bd640 1 -- 172.21.15.47:0/1650364334 >> 172.21.15.47:0/1650364334 conn(0x7f92440d4f60 msgr2=0x7f92440d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:01.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.490+0000 7f92492bd640 1 -- 172.21.15.47:0/1650364334 shutdown_connections 2024-04-26T23:19:01.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.490+0000 7f92492bd640 1 -- 172.21.15.47:0/1650364334 wait complete. 2024-04-26T23:19:01.502 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:19:01.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.604+0000 7f2741452640 1 Processor -- start 2024-04-26T23:19:01.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.604+0000 7f2741452640 1 -- start start 2024-04-26T23:19:01.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.605+0000 7f2741452640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f273c162a20 0x7f273c0d1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:01.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.605+0000 7f2741452640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f273c0ca360 0x7f273c0d20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:01.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.605+0000 7f2741452640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f273c0c8bc0 con 0x7f273c0d1c40 2024-04-26T23:19:01.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.605+0000 7f2741452640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f273c0d07a0 con 0x7f273c162a20 2024-04-26T23:19:01.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.605+0000 7f2741452640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f273c0d09f0 con 0x7f273c0ca360 2024-04-26T23:19:01.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.605+0000 7f273affd640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f273c162a20 0x7f273c0d1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:01.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.605+0000 7f273b7fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f273c0ca360 0x7f273c0d20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:01.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.605+0000 7f273affd640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f273c162a20 0x7f273c0d1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.47:48008/0 (socket says 172.21.15.47:48008) 2024-04-26T23:19:01.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.605+0000 7f273affd640 1 -- 172.21.15.47:0/1826355338 learned_addr learned my addr 172.21.15.47:0/1826355338 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:01.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.605+0000 7f273affd640 1 -- 172.21.15.47:0/1826355338 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f273c0ca360 msgr2=0x7f273c0d20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:01.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.605+0000 7f273affd640 1 --2- 172.21.15.47:0/1826355338 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f273c0ca360 0x7f273c0d20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:01.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.605+0000 7f273affd640 1 -- 172.21.15.47:0/1826355338 >> v1:172.21.15.203:6789/0 conn(0x7f273c0d1c40 legacy=0x7f273c0d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:01.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.605+0000 7f273affd640 1 -- 172.21.15.47:0/1826355338 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f273c0d0cd0 con 0x7f273c162a20 2024-04-26T23:19:01.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.606+0000 7f273affd640 1 --2- 172.21.15.47:0/1826355338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f273c162a20 0x7f273c0d1700 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f272c009870 tx=0x7f272c02a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf3d14802928d210 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:01.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.606+0000 7f2739ffb640 1 -- 172.21.15.47:0/1826355338 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272c037070 con 0x7f273c162a20 2024-04-26T23:19:01.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.606+0000 7f2739ffb640 1 -- 172.21.15.47:0/1826355338 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f272c032440 con 0x7f273c162a20 2024-04-26T23:19:01.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.606+0000 7f2739ffb640 1 -- 172.21.15.47:0/1826355338 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272c032810 con 0x7f273c162a20 2024-04-26T23:19:01.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.607+0000 7f2741452640 1 -- 172.21.15.47:0/1826355338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f273c162a20 msgr2=0x7f273c0d1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:01.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.607+0000 7f2741452640 1 --2- 172.21.15.47:0/1826355338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f273c162a20 0x7f273c0d1700 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f272c009870 tx=0x7f272c02a1c0 comp rx=0 tx=0).stop 2024-04-26T23:19:01.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.607+0000 7f2741452640 1 -- 172.21.15.47:0/1826355338 shutdown_connections 2024-04-26T23:19:01.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.607+0000 7f2741452640 1 --2- 172.21.15.47:0/1826355338 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f273c0ca360 0x7f273c0d20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:01.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.607+0000 7f2741452640 1 --2- 172.21.15.47:0/1826355338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f273c162a20 0x7f273c0d1700 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:01.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.607+0000 7f2741452640 1 -- 172.21.15.47:0/1826355338 >> 172.21.15.47:0/1826355338 conn(0x7f273c0d4fd0 msgr2=0x7f273c0d53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:01.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.607+0000 7f2741452640 1 -- 172.21.15.47:0/1826355338 shutdown_connections 2024-04-26T23:19:01.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.607+0000 7f2741452640 1 -- 172.21.15.47:0/1826355338 wait complete. 2024-04-26T23:19:01.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.608+0000 7f2741452640 1 Processor -- start 2024-04-26T23:19:01.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.608+0000 7f2741452640 1 -- start start 2024-04-26T23:19:01.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.609+0000 7f2741452640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f273c0ca360 0x7f273c1513a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:01.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.609+0000 7f2741452640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f273c0d1c40 0x7f273c15b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:01.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.609+0000 7f2741452640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f273c162a20 0x7f273c15bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:01.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.609+0000 7f2741452640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f273c0cbd90 con 0x7f273c0ca360 2024-04-26T23:19:01.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.609+0000 7f2741452640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f273c0d0fb0 con 0x7f273c0d1c40 2024-04-26T23:19:01.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.609+0000 7f2741452640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f273c162e00 con 0x7f273c162a20 2024-04-26T23:19:01.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.609+0000 7f273affd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f273c0ca360 0x7f273c1513a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:01.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.609+0000 7f273affd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f273c0ca360 0x7f273c1513a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54668/0 (socket says 172.21.15.47:54668) 2024-04-26T23:19:01.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.609+0000 7f273affd640 1 -- 172.21.15.47:0/741510744 learned_addr learned my addr 172.21.15.47:0/741510744 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:01.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.609+0000 7f273b7fe640 1 --2- 172.21.15.47:0/741510744 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f273c162a20 0x7f273c15bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:01.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.609+0000 7f273a7fc640 1 --2- 172.21.15.47:0/741510744 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f273c0d1c40 0x7f273c15b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:01.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.609+0000 7f273affd640 1 -- 172.21.15.47:0/741510744 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f273c162a20 msgr2=0x7f273c15bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:01.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.609+0000 7f273affd640 1 --2- 172.21.15.47:0/741510744 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f273c162a20 0x7f273c15bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:01.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.609+0000 7f273affd640 1 -- 172.21.15.47:0/741510744 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f273c0d1c40 msgr2=0x7f273c15b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:01.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.609+0000 7f273affd640 1 --2- 172.21.15.47:0/741510744 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f273c0d1c40 0x7f273c15b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:01.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.609+0000 7f273affd640 1 -- 172.21.15.47:0/741510744 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f273c0b4810 con 0x7f273c0ca360 2024-04-26T23:19:01.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.610+0000 7f273affd640 1 --2- 172.21.15.47:0/741510744 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f273c0ca360 0x7f273c1513a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f272c000c00 tx=0x7f272c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:01.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.610+0000 7f27237fe640 1 -- 172.21.15.47:0/741510744 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272c037040 con 0x7f273c0ca360 2024-04-26T23:19:01.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.610+0000 7f27237fe640 1 -- 172.21.15.47:0/741510744 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f272c042070 con 0x7f273c0ca360 2024-04-26T23:19:01.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.610+0000 7f2741452640 1 -- 172.21.15.47:0/741510744 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f273c0b4af0 con 0x7f273c0ca360 2024-04-26T23:19:01.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.610+0000 7f27237fe640 1 -- 172.21.15.47:0/741510744 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272c030880 con 0x7f273c0ca360 2024-04-26T23:19:01.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.610+0000 7f2741452640 1 -- 172.21.15.47:0/741510744 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f273c0b50b0 con 0x7f273c0ca360 2024-04-26T23:19:01.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.611+0000 7f2741452640 1 -- 172.21.15.47:0/741510744 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f273c0694f0 con 0x7f273c0ca360 2024-04-26T23:19:01.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.611+0000 7f27237fe640 1 -- 172.21.15.47:0/741510744 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f272c02ad10 con 0x7f273c0ca360 2024-04-26T23:19:01.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.612+0000 7f27237fe640 1 --2- 172.21.15.47:0/741510744 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f272806c8f0 0x7f272806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:01.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.612+0000 7f27237fe640 1 -- 172.21.15.47:0/741510744 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f272c0ac340 con 0x7f273c0ca360 2024-04-26T23:19:01.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.612+0000 7f273a7fc640 1 --2- 172.21.15.47:0/741510744 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f272806c8f0 0x7f272806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:01.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.612+0000 7f273a7fc640 1 --2- 172.21.15.47:0/741510744 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f272806c8f0 0x7f272806edb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f273c0cb9b0 tx=0x7f2730009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:01.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.616+0000 7f27237fe640 1 -- 172.21.15.47:0/741510744 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f272c079d60 con 0x7f273c0ca360 2024-04-26T23:19:01.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.802+0000 7f2741452640 1 -- 172.21.15.47:0/741510744 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f273c069cc0 con 0x7f273c0ca360 2024-04-26T23:19:01.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.802+0000 7f27237fe640 1 -- 172.21.15.47:0/741510744 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7f272c07e060 con 0x7f273c0ca360 2024-04-26T23:19:01.804 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-26T23:19:01.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.804+0000 7f2741452640 1 -- 172.21.15.47:0/741510744 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f272806c8f0 msgr2=0x7f272806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:01.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.804+0000 7f2741452640 1 --2- 172.21.15.47:0/741510744 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f272806c8f0 0x7f272806edb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f273c0cb9b0 tx=0x7f2730009040 comp rx=0 tx=0).stop 2024-04-26T23:19:01.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.804+0000 7f2741452640 1 -- 172.21.15.47:0/741510744 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f273c0ca360 msgr2=0x7f273c1513a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:01.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.804+0000 7f2741452640 1 --2- 172.21.15.47:0/741510744 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f273c0ca360 0x7f273c1513a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f272c000c00 tx=0x7f272c0046c0 comp rx=0 tx=0).stop 2024-04-26T23:19:01.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.804+0000 7f2741452640 1 -- 172.21.15.47:0/741510744 shutdown_connections 2024-04-26T23:19:01.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.804+0000 7f2741452640 1 --2- 172.21.15.47:0/741510744 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f272806c8f0 0x7f272806edb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:01.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.804+0000 7f2741452640 1 --2- 172.21.15.47:0/741510744 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f273c162a20 0x7f273c15bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:01.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.804+0000 7f2741452640 1 --2- 172.21.15.47:0/741510744 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f273c0d1c40 0x7f273c15b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:01.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.804+0000 7f2741452640 1 --2- 172.21.15.47:0/741510744 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f273c0ca360 0x7f273c1513a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:01.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.804+0000 7f2741452640 1 -- 172.21.15.47:0/741510744 >> 172.21.15.47:0/741510744 conn(0x7f273c0d4fd0 msgr2=0x7f273c1617f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:01.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.804+0000 7f2741452640 1 -- 172.21.15.47:0/741510744 shutdown_connections 2024-04-26T23:19:01.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.804+0000 7f2741452640 1 -- 172.21.15.47:0/741510744 wait complete. 2024-04-26T23:19:01.816 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T23:19:01.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.918+0000 7f90e4589640 1 Processor -- start 2024-04-26T23:19:01.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.918+0000 7f90e4589640 1 -- start start 2024-04-26T23:19:01.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.919+0000 7f90e4589640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f90dc15f170 0x7f90dc0d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:01.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.919+0000 7f90e4589640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f90dc15f550 0x7f90dc0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:01.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.919+0000 7f90e4589640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f90dc0d34b0 0x7f90dc0d3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:01.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.919+0000 7f90e4589640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f90dc0d3e60 con 0x7f90dc15f550 2024-04-26T23:19:01.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.919+0000 7f90e4589640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f90dc0c9430 con 0x7f90dc15f170 2024-04-26T23:19:01.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.919+0000 7f90e4589640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f90dc0d1040 con 0x7f90dc0d34b0 2024-04-26T23:19:01.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.919+0000 7f90e1b24640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f90dc15f550 0x7f90dc0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:01.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.919+0000 7f90e1b24640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f90dc15f550 0x7f90dc0d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54682/0 (socket says 172.21.15.47:54682) 2024-04-26T23:19:01.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.919+0000 7f90e1b24640 1 -- 172.21.15.47:0/1137424603 learned_addr learned my addr 172.21.15.47:0/1137424603 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:01.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.919+0000 7f90e2b26640 1 --2- 172.21.15.47:0/1137424603 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f90dc0d34b0 0x7f90dc0d3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:01.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.919+0000 7f90e2325640 1 --2- 172.21.15.47:0/1137424603 >> v2:172.21.15.203:3300/0 conn(0x7f90dc15f170 0x7f90dc0d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:01.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.919+0000 7f90e1b24640 1 -- 172.21.15.47:0/1137424603 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f90dc0d34b0 msgr2=0x7f90dc0d3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:01.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.919+0000 7f90e1b24640 1 --2- 172.21.15.47:0/1137424603 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f90dc0d34b0 0x7f90dc0d3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:01.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.919+0000 7f90e1b24640 1 -- 172.21.15.47:0/1137424603 >> v2:172.21.15.203:3300/0 conn(0x7f90dc15f170 msgr2=0x7f90dc0d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:01.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.919+0000 7f90e1b24640 1 --2- 172.21.15.47:0/1137424603 >> v2:172.21.15.203:3300/0 conn(0x7f90dc15f170 0x7f90dc0d5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:01.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.919+0000 7f90e1b24640 1 -- 172.21.15.47:0/1137424603 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90dc0c9680 con 0x7f90dc15f550 2024-04-26T23:19:01.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.919+0000 7f90e2b26640 1 --2- 172.21.15.47:0/1137424603 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f90dc0d34b0 0x7f90dc0d3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:19:01.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.919+0000 7f90e2325640 1 --2- 172.21.15.47:0/1137424603 >> v2:172.21.15.203:3300/0 conn(0x7f90dc15f170 0x7f90dc0d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:19:01.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.920+0000 7f90e1b24640 1 --2- 172.21.15.47:0/1137424603 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90dc15f550 0x7f90dc0d2cb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f90cc009870 tx=0x7f90cc02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ad197502108352 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:01.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.920+0000 7f90e1323640 1 -- 172.21.15.47:0/1137424603 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90cc037070 con 0x7f90dc15f550 2024-04-26T23:19:01.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.920+0000 7f90e1323640 1 -- 172.21.15.47:0/1137424603 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90cc032440 con 0x7f90dc15f550 2024-04-26T23:19:01.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.920+0000 7f90e1323640 1 -- 172.21.15.47:0/1137424603 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90cc032810 con 0x7f90dc15f550 2024-04-26T23:19:01.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.921+0000 7f90e4589640 1 -- 172.21.15.47:0/1137424603 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90dc15f550 msgr2=0x7f90dc0d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:01.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.921+0000 7f90e4589640 1 --2- 172.21.15.47:0/1137424603 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90dc15f550 0x7f90dc0d2cb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f90cc009870 tx=0x7f90cc02a210 comp rx=0 tx=0).stop 2024-04-26T23:19:01.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.921+0000 7f90e4589640 1 -- 172.21.15.47:0/1137424603 shutdown_connections 2024-04-26T23:19:01.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.921+0000 7f90e4589640 1 --2- 172.21.15.47:0/1137424603 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90dc15f550 0x7f90dc0d2cb0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:01.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.921+0000 7f90e4589640 1 --2- 172.21.15.47:0/1137424603 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f90dc0d34b0 0x7f90dc0d3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:01.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.921+0000 7f90e4589640 1 --2- 172.21.15.47:0/1137424603 >> v2:172.21.15.203:3300/0 conn(0x7f90dc15f170 0x7f90dc0d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:01.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.921+0000 7f90e4589640 1 -- 172.21.15.47:0/1137424603 >> 172.21.15.47:0/1137424603 conn(0x7f90dc0d4f60 msgr2=0x7f90dc0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:01.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.921+0000 7f90e4589640 1 -- 172.21.15.47:0/1137424603 shutdown_connections 2024-04-26T23:19:01.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.921+0000 7f90e4589640 1 -- 172.21.15.47:0/1137424603 wait complete. 2024-04-26T23:19:01.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.922+0000 7f90e4589640 1 Processor -- start 2024-04-26T23:19:01.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.922+0000 7f90e4589640 1 -- start start 2024-04-26T23:19:01.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.923+0000 7f90e4589640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90dc0d34b0 0x7f90dc1544a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:01.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.923+0000 7f90e4589640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f90dc15f170 0x7f90dc0ce3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:01.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.923+0000 7f90e4589640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90dc0b1c30 0x7f90dc156f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:01.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.923+0000 7f90e4589640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f90dc0cfc50 con 0x7f90dc0b1c30 2024-04-26T23:19:01.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.923+0000 7f90e4589640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f90dc0c9c40 con 0x7f90dc0d34b0 2024-04-26T23:19:01.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.923+0000 7f90e4589640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f90dc0cfdd0 con 0x7f90dc15f170 2024-04-26T23:19:01.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.923+0000 7f90e2b26640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90dc0b1c30 0x7f90dc156f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:01.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.923+0000 7f90e2b26640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90dc0b1c30 0x7f90dc156f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54688/0 (socket says 172.21.15.47:54688) 2024-04-26T23:19:01.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.923+0000 7f90e2325640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90dc0d34b0 0x7f90dc1544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:01.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.923+0000 7f90e2b26640 1 -- 172.21.15.47:0/190569470 learned_addr learned my addr 172.21.15.47:0/190569470 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:01.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.923+0000 7f90e1b24640 1 --2- 172.21.15.47:0/190569470 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f90dc15f170 0x7f90dc0ce3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:01.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.923+0000 7f90e2b26640 1 -- 172.21.15.47:0/190569470 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f90dc15f170 msgr2=0x7f90dc0ce3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:01.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.923+0000 7f90e2b26640 1 --2- 172.21.15.47:0/190569470 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f90dc15f170 0x7f90dc0ce3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:01.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.923+0000 7f90e2b26640 1 -- 172.21.15.47:0/190569470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90dc0d34b0 msgr2=0x7f90dc1544a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:01.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.923+0000 7f90e2b26640 1 --2- 172.21.15.47:0/190569470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90dc0d34b0 0x7f90dc1544a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:01.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.923+0000 7f90e2b26640 1 -- 172.21.15.47:0/190569470 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90dc1549e0 con 0x7f90dc0b1c30 2024-04-26T23:19:01.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.924+0000 7f90e2b26640 1 --2- 172.21.15.47:0/190569470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90dc0b1c30 0x7f90dc156f60 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f90d4007e10 tx=0x7f90d400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:01.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.924+0000 7f90d2ffd640 1 -- 172.21.15.47:0/190569470 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d401a070 con 0x7f90dc0b1c30 2024-04-26T23:19:01.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.924+0000 7f90d2ffd640 1 -- 172.21.15.47:0/190569470 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90d4003b90 con 0x7f90dc0b1c30 2024-04-26T23:19:01.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.924+0000 7f90e4589640 1 -- 172.21.15.47:0/190569470 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90dc0a28c0 con 0x7f90dc0b1c30 2024-04-26T23:19:01.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.924+0000 7f90d2ffd640 1 -- 172.21.15.47:0/190569470 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d4004030 con 0x7f90dc0b1c30 2024-04-26T23:19:01.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.924+0000 7f90e4589640 1 -- 172.21.15.47:0/190569470 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90dc151a50 con 0x7f90dc0b1c30 2024-04-26T23:19:01.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.925+0000 7f90e4589640 1 -- 172.21.15.47:0/190569470 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90dc0694f0 con 0x7f90dc0b1c30 2024-04-26T23:19:01.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.925+0000 7f90d2ffd640 1 -- 172.21.15.47:0/190569470 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f90d401d070 con 0x7f90dc0b1c30 2024-04-26T23:19:01.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.926+0000 7f90d2ffd640 1 --2- 172.21.15.47:0/190569470 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f90c006c920 0x7f90c006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:01.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.926+0000 7f90d2ffd640 1 -- 172.21.15.47:0/190569470 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f90d408bfb0 con 0x7f90dc0b1c30 2024-04-26T23:19:01.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.926+0000 7f90e2325640 1 --2- 172.21.15.47:0/190569470 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f90c006c920 0x7f90c006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:01.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.926+0000 7f90e2325640 1 --2- 172.21.15.47:0/190569470 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f90c006c920 0x7f90c006ede0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f90d8006fd0 tx=0x7f90d8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:01.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:01.930+0000 7f90d2ffd640 1 -- 172.21.15.47:0/190569470 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f90d4010070 con 0x7f90dc0b1c30 2024-04-26T23:19:02.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.114+0000 7f90e4589640 1 -- 172.21.15.47:0/190569470 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f90dc0baaf0 con 0x7f90dc0b1c30 2024-04-26T23:19:02.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.114+0000 7f90d2ffd640 1 -- 172.21.15.47:0/190569470 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7f90d4059ae0 con 0x7f90dc0b1c30 2024-04-26T23:19:02.116 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-26T23:19:02.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.116+0000 7f90e4589640 1 -- 172.21.15.47:0/190569470 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f90c006c920 msgr2=0x7f90c006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:02.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.116+0000 7f90e4589640 1 --2- 172.21.15.47:0/190569470 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f90c006c920 0x7f90c006ede0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f90d8006fd0 tx=0x7f90d8008040 comp rx=0 tx=0).stop 2024-04-26T23:19:02.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.116+0000 7f90e4589640 1 -- 172.21.15.47:0/190569470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90dc0b1c30 msgr2=0x7f90dc156f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:02.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.116+0000 7f90e4589640 1 --2- 172.21.15.47:0/190569470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90dc0b1c30 0x7f90dc156f60 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f90d4007e10 tx=0x7f90d400d4a0 comp rx=0 tx=0).stop 2024-04-26T23:19:02.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.116+0000 7f90e4589640 1 -- 172.21.15.47:0/190569470 shutdown_connections 2024-04-26T23:19:02.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.116+0000 7f90e4589640 1 --2- 172.21.15.47:0/190569470 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f90c006c920 0x7f90c006ede0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:02.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.116+0000 7f90e4589640 1 --2- 172.21.15.47:0/190569470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90dc0b1c30 0x7f90dc156f60 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:02.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.116+0000 7f90e4589640 1 --2- 172.21.15.47:0/190569470 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f90dc15f170 0x7f90dc0ce3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:02.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.116+0000 7f90e4589640 1 --2- 172.21.15.47:0/190569470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90dc0d34b0 0x7f90dc1544a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:02.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.116+0000 7f90e4589640 1 -- 172.21.15.47:0/190569470 >> 172.21.15.47:0/190569470 conn(0x7f90dc0d4f60 msgr2=0x7f90dc06d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:02.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.116+0000 7f90e4589640 1 -- 172.21.15.47:0/190569470 shutdown_connections 2024-04-26T23:19:02.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.116+0000 7f90e4589640 1 -- 172.21.15.47:0/190569470 wait complete. 2024-04-26T23:19:02.129 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:19:02.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.228+0000 7fb1a979d640 1 Processor -- start 2024-04-26T23:19:02.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.228+0000 7fb1a979d640 1 -- start start 2024-04-26T23:19:02.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.229+0000 7fb1a979d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb1a40b5f20 0x7fb1a40d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:02.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.229+0000 7fb1a979d640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1a40b7e50 con 0x7fb1a4162880 2024-04-26T23:19:02.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.229+0000 7fb1a979d640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1a40d1800 con 0x7fb1a40d3bc0 2024-04-26T23:19:02.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.229+0000 7fb1a979d640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fb1a40d0580 con 0x7fb1a40b5f20 2024-04-26T23:19:02.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.229+0000 7fb1a2ffd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb1a4162880 0x7fb1a40d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:48156/0 (socket says 172.21.15.47:48156) 2024-04-26T23:19:02.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.229+0000 7fb1a2ffd640 1 -- 172.21.15.47:0/577285247 learned_addr learned my addr 172.21.15.47:0/577285247 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:02.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.229+0000 7fb1a37fe640 1 --2- 172.21.15.47:0/577285247 >> v2:172.21.15.47:3300/0 conn(0x7fb1a40b5f20 0x7fb1a40d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:02.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.229+0000 7fb1a27fc640 1 -- 172.21.15.47:0/577285247 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 428390346 0 0) 0x7fb1a40b7e50 con 0x7fb1a4162880 2024-04-26T23:19:02.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.229+0000 7fb1a27fc640 1 -- 172.21.15.47:0/577285247 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb190003520 con 0x7fb1a4162880 2024-04-26T23:19:02.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.229+0000 7fb1a37fe640 1 -- 172.21.15.47:0/577285247 >> v1:172.21.15.203:6789/0 conn(0x7fb1a40d3bc0 legacy=0x7fb1a40b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:02.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.229+0000 7fb1a37fe640 1 -- 172.21.15.47:0/577285247 >> v1:172.21.15.47:6789/0 conn(0x7fb1a4162880 legacy=0x7fb1a40d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:02.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.229+0000 7fb1a37fe640 1 -- 172.21.15.47:0/577285247 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1a40d07d0 con 0x7fb1a40b5f20 2024-04-26T23:19:02.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.230+0000 7fb1a37fe640 1 --2- 172.21.15.47:0/577285247 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1a40b5f20 0x7fb1a40d2cb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb19c009d90 tx=0x7fb19c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81a3142d6c792c86 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:02.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.230+0000 7fb1a27fc640 1 -- 172.21.15.47:0/577285247 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19c007740 con 0x7fb1a40b5f20 2024-04-26T23:19:02.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.230+0000 7fb1a27fc640 1 -- 172.21.15.47:0/577285247 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb19c0078e0 con 0x7fb1a40b5f20 2024-04-26T23:19:02.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.230+0000 7fb1a27fc640 1 -- 172.21.15.47:0/577285247 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19c007cb0 con 0x7fb1a40b5f20 2024-04-26T23:19:02.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.231+0000 7fb1a979d640 1 -- 172.21.15.47:0/577285247 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1a40b5f20 msgr2=0x7fb1a40d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:02.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.231+0000 7fb1a979d640 1 --2- 172.21.15.47:0/577285247 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1a40b5f20 0x7fb1a40d2cb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb19c009d90 tx=0x7fb19c02a6a0 comp rx=0 tx=0).stop 2024-04-26T23:19:02.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.231+0000 7fb1a979d640 1 -- 172.21.15.47:0/577285247 shutdown_connections 2024-04-26T23:19:02.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.231+0000 7fb1a979d640 1 --2- 172.21.15.47:0/577285247 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1a40b5f20 0x7fb1a40d2cb0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:02.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.231+0000 7fb1a979d640 1 -- 172.21.15.47:0/577285247 >> 172.21.15.47:0/577285247 conn(0x7fb1a40d4f60 msgr2=0x7fb1a40d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:02.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.231+0000 7fb1a979d640 1 -- 172.21.15.47:0/577285247 shutdown_connections 2024-04-26T23:19:02.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.231+0000 7fb1a979d640 1 -- 172.21.15.47:0/577285247 wait complete. 2024-04-26T23:19:02.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.232+0000 7fb1a979d640 1 Processor -- start 2024-04-26T23:19:02.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.232+0000 7fb1a979d640 1 -- start start 2024-04-26T23:19:02.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.233+0000 7fb1a979d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1a40b5f20 0x7fb1a40ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:02.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.233+0000 7fb1a979d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1a40d3bc0 0x7fb1a40cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:02.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.233+0000 7fb1a979d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1a4162880 0x7fb1a4153350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:02.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.233+0000 7fb1a979d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a4162c60 con 0x7fb1a40b5f20 2024-04-26T23:19:02.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.233+0000 7fb1a979d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a40d0d90 con 0x7fb1a4162880 2024-04-26T23:19:02.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.233+0000 7fb1a979d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb1a4162de0 con 0x7fb1a40d3bc0 2024-04-26T23:19:02.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.233+0000 7fb1a2ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1a40b5f20 0x7fb1a40ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:02.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.233+0000 7fb1a2ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1a40b5f20 0x7fb1a40ce3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54710/0 (socket says 172.21.15.47:54710) 2024-04-26T23:19:02.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.233+0000 7fb1a2ffd640 1 -- 172.21.15.47:0/3176986954 learned_addr learned my addr 172.21.15.47:0/3176986954 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:02.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.233+0000 7fb1a2ffd640 1 -- 172.21.15.47:0/3176986954 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1a40d3bc0 msgr2=0x7fb1a40cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:02.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.233+0000 7fb1a2ffd640 1 --2- 172.21.15.47:0/3176986954 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1a40d3bc0 0x7fb1a40cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:02.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.233+0000 7fb1a2ffd640 1 -- 172.21.15.47:0/3176986954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1a4162880 msgr2=0x7fb1a4153350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:02.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.233+0000 7fb1a37fe640 1 --2- 172.21.15.47:0/3176986954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1a4162880 0x7fb1a4153350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:02.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.233+0000 7fb1a2ffd640 1 --2- 172.21.15.47:0/3176986954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1a4162880 0x7fb1a4153350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:02.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.233+0000 7fb1a2ffd640 1 -- 172.21.15.47:0/3176986954 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1a40ccb30 con 0x7fb1a40b5f20 2024-04-26T23:19:02.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.233+0000 7fb1a2ffd640 1 --2- 172.21.15.47:0/3176986954 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1a40b5f20 0x7fb1a40ce3b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb19400c3c0 tx=0x7fb19400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:02.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.233+0000 7fb19b7fe640 1 -- 172.21.15.47:0/3176986954 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb194017070 con 0x7fb1a40b5f20 2024-04-26T23:19:02.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.233+0000 7fb19b7fe640 1 -- 172.21.15.47:0/3176986954 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb194005ce0 con 0x7fb1a40b5f20 2024-04-26T23:19:02.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.233+0000 7fb1a979d640 1 -- 172.21.15.47:0/3176986954 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1a40ccdf0 con 0x7fb1a40b5f20 2024-04-26T23:19:02.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.233+0000 7fb19b7fe640 1 -- 172.21.15.47:0/3176986954 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1940072f0 con 0x7fb1a40b5f20 2024-04-26T23:19:02.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.233+0000 7fb1a979d640 1 -- 172.21.15.47:0/3176986954 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1a40cd3b0 con 0x7fb1a40b5f20 2024-04-26T23:19:02.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.235+0000 7fb1a979d640 1 -- 172.21.15.47:0/3176986954 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1a40694f0 con 0x7fb1a40b5f20 2024-04-26T23:19:02.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.235+0000 7fb19b7fe640 1 -- 172.21.15.47:0/3176986954 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb19401a070 con 0x7fb1a40b5f20 2024-04-26T23:19:02.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.235+0000 7fb19b7fe640 1 --2- 172.21.15.47:0/3176986954 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb18406c8f0 0x7fb18406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:02.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.235+0000 7fb19b7fe640 1 -- 172.21.15.47:0/3176986954 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb19408b1b0 con 0x7fb1a40b5f20 2024-04-26T23:19:02.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.236+0000 7fb19bfff640 1 --2- 172.21.15.47:0/3176986954 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb18406c8f0 0x7fb18406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:02.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.236+0000 7fb19bfff640 1 --2- 172.21.15.47:0/3176986954 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb18406c8f0 0x7fb18406edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb1a406d4a0 tx=0x7fb18c0079b0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:02.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.240+0000 7fb19b7fe640 1 -- 172.21.15.47:0/3176986954 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb1940593f0 con 0x7fb1a40b5f20 2024-04-26T23:19:02.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.462+0000 7fb1a979d640 1 -- 172.21.15.47:0/3176986954 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb1a40cfd90 con 0x7fb1a40b5f20 2024-04-26T23:19:02.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.463+0000 7fb19b7fe640 1 -- 172.21.15.47:0/3176986954 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5180 (secure 0 0 0) 0x7fb194058cf0 con 0x7fb1a40b5f20 2024-04-26T23:19:02.465 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:19:02.465 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/1553696401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1553696401},{"type":"v1","addr":"172.21.15.47:6834","nonce":1553696401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/3792767564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3792767564},{"type":"v1","addr":"172.21.15.203:6836","nonce":3792767564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4088100766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":4088100766},{"type":"v1","addr":"172.21.15.47:6835","nonce":4088100766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:18:31.367292+0000","modified":"2024-04-26T23:18:53.430267+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/997115011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":997115011},{"type":"v1","addr":"172.21.15.203:6837","nonce":997115011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4282,"qdb_cluster":[4282]},"id":1}]} 2024-04-26T23:19:02.465 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 9 2024-04-26T23:19:02.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.465+0000 7fb1a979d640 1 -- 172.21.15.47:0/3176986954 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb18406c8f0 msgr2=0x7fb18406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:02.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.465+0000 7fb1a979d640 1 --2- 172.21.15.47:0/3176986954 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb18406c8f0 0x7fb18406edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb1a406d4a0 tx=0x7fb18c0079b0 comp rx=0 tx=0).stop 2024-04-26T23:19:02.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.465+0000 7fb1a979d640 1 -- 172.21.15.47:0/3176986954 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1a40b5f20 msgr2=0x7fb1a40ce3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:02.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.465+0000 7fb1a979d640 1 --2- 172.21.15.47:0/3176986954 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1a40b5f20 0x7fb1a40ce3b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb19400c3c0 tx=0x7fb19400c880 comp rx=0 tx=0).stop 2024-04-26T23:19:02.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.465+0000 7fb1a979d640 1 -- 172.21.15.47:0/3176986954 shutdown_connections 2024-04-26T23:19:02.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.465+0000 7fb1a979d640 1 --2- 172.21.15.47:0/3176986954 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb18406c8f0 0x7fb18406edb0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:02.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.465+0000 7fb1a979d640 1 --2- 172.21.15.47:0/3176986954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1a4162880 0x7fb1a4153350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:02.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.465+0000 7fb1a979d640 1 --2- 172.21.15.47:0/3176986954 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1a40d3bc0 0x7fb1a40cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:02.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.465+0000 7fb1a979d640 1 --2- 172.21.15.47:0/3176986954 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1a40b5f20 0x7fb1a40ce3b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:02.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.465+0000 7fb1a979d640 1 -- 172.21.15.47:0/3176986954 >> 172.21.15.47:0/3176986954 conn(0x7fb1a40d4f60 msgr2=0x7fb1a40d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:02.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.465+0000 7fb1a979d640 1 -- 172.21.15.47:0/3176986954 shutdown_connections 2024-04-26T23:19:02.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.465+0000 7fb1a979d640 1 -- 172.21.15.47:0/3176986954 wait complete. 2024-04-26T23:19:02.478 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:19:02.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.577+0000 7fa2509e9640 1 Processor -- start 2024-04-26T23:19:02.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.577+0000 7fa2509e9640 1 -- start start 2024-04-26T23:19:02.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.578+0000 7fa2509e9640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa24c15f3a0 0x7fa24c0d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:02.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.578+0000 7fa2509e9640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa24c0d0e20 con 0x7fa24c0d2330 2024-04-26T23:19:02.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.578+0000 7fa2509e9640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa24c0b1d70 con 0x7fa24c0d1770 2024-04-26T23:19:02.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.578+0000 7fa2509e9640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fa24c0b2050 con 0x7fa24c15f3a0 2024-04-26T23:19:02.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.578+0000 7fa24ad9d640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa24c0d2330 0x7fa24c0b2850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:48160/0 (socket says 172.21.15.47:48160) 2024-04-26T23:19:02.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.578+0000 7fa24ad9d640 1 -- 172.21.15.47:0/3195817548 learned_addr learned my addr 172.21.15.47:0/3195817548 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:02.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.578+0000 7fa24a59c640 1 --2- 172.21.15.47:0/3195817548 >> v2:172.21.15.203:3300/0 conn(0x7fa24c15f3a0 0x7fa24c0d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:02.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.578+0000 7fa249d9b640 1 -- 172.21.15.47:0/3195817548 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2020049564 0 0) 0x7fa24c0d0e20 con 0x7fa24c0d2330 2024-04-26T23:19:02.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.578+0000 7fa249d9b640 1 -- 172.21.15.47:0/3195817548 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa238003520 con 0x7fa24c0d2330 2024-04-26T23:19:02.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.578+0000 7fa249d9b640 1 -- 172.21.15.47:0/3195817548 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 994382518 0 0) 0x7fa238003520 con 0x7fa24c0d2330 2024-04-26T23:19:02.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.578+0000 7fa249d9b640 1 -- 172.21.15.47:0/3195817548 >> v2:172.21.15.203:3300/0 conn(0x7fa24c15f3a0 msgr2=0x7fa24c0d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:02.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.578+0000 7fa249d9b640 1 --2- 172.21.15.47:0/3195817548 >> v2:172.21.15.203:3300/0 conn(0x7fa24c15f3a0 0x7fa24c0d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:02.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.578+0000 7fa249d9b640 1 -- 172.21.15.47:0/3195817548 >> v1:172.21.15.203:6789/0 conn(0x7fa24c0d1770 legacy=0x7fa24c0d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:02.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.578+0000 7fa249d9b640 1 -- 172.21.15.47:0/3195817548 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa24c0b2330 con 0x7fa24c0d2330 2024-04-26T23:19:02.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.578+0000 7fa249d9b640 1 -- 172.21.15.47:0/3195817548 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7fa244001a10 con 0x7fa24c0d2330 2024-04-26T23:19:02.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.578+0000 7fa249d9b640 1 -- 172.21.15.47:0/3195817548 >> v1:172.21.15.47:6789/0 conn(0x7fa24c0d2330 legacy=0x7fa24c0b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:02.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.578+0000 7fa249d9b640 1 --2- 172.21.15.47:0/3195817548 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa238003a00 0x7fa238003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:02.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.578+0000 7fa249d9b640 1 --2- 172.21.15.47:0/3195817548 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa238004800 0x7fa238004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:02.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.578+0000 7fa249d9b640 1 --2- 172.21.15.47:0/3195817548 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2380051c0 0x7fa23800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:02.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.578+0000 7fa249d9b640 1 -- 172.21.15.47:0/3195817548 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa24c0b2050 con 0x7fa238003a00 2024-04-26T23:19:02.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.578+0000 7fa249d9b640 1 -- 172.21.15.47:0/3195817548 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa23800c0c0 con 0x7fa2380051c0 2024-04-26T23:19:02.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.578+0000 7fa249d9b640 1 -- 172.21.15.47:0/3195817548 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa23800c2c0 con 0x7fa238004800 2024-04-26T23:19:02.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.579+0000 7fa24a59c640 1 --2- 172.21.15.47:0/3195817548 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa238003a00 0x7fa238003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:02.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.579+0000 7fa243fff640 1 --2- 172.21.15.47:0/3195817548 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa238004800 0x7fa238004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:02.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.579+0000 7fa24ad9d640 1 --2- 172.21.15.47:0/3195817548 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2380051c0 0x7fa23800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:02.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.579+0000 7fa24a59c640 1 -- 172.21.15.47:0/3195817548 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa238004800 msgr2=0x7fa238004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:02.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.579+0000 7fa24a59c640 1 --2- 172.21.15.47:0/3195817548 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa238004800 0x7fa238004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:02.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.579+0000 7fa24a59c640 1 -- 172.21.15.47:0/3195817548 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2380051c0 msgr2=0x7fa23800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:02.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.579+0000 7fa24a59c640 1 --2- 172.21.15.47:0/3195817548 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2380051c0 0x7fa23800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:02.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.579+0000 7fa24ad9d640 1 -- 172.21.15.47:0/3195817548 reap_dead start 2024-04-26T23:19:02.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.579+0000 7fa24a59c640 1 -- 172.21.15.47:0/3195817548 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa238003520 con 0x7fa238003a00 2024-04-26T23:19:02.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.579+0000 7fa243fff640 1 --2- 172.21.15.47:0/3195817548 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa238004800 0x7fa238004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:19:02.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.580+0000 7fa24a59c640 1 --2- 172.21.15.47:0/3195817548 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa238003a00 0x7fa238003e20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa23c009ac0 tx=0x7fa238005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23ab5902fe14ebc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:02.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.580+0000 7fa249d9b640 1 -- 172.21.15.47:0/3195817548 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23c036070 con 0x7fa238003a00 2024-04-26T23:19:02.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.580+0000 7fa249d9b640 1 -- 172.21.15.47:0/3195817548 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa23c031400 con 0x7fa238003a00 2024-04-26T23:19:02.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.580+0000 7fa249d9b640 1 -- 172.21.15.47:0/3195817548 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23c009690 con 0x7fa238003a00 2024-04-26T23:19:02.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.580+0000 7fa2509e9640 1 -- 172.21.15.47:0/3195817548 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa238003a00 msgr2=0x7fa238003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:02.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.580+0000 7fa2509e9640 1 --2- 172.21.15.47:0/3195817548 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa238003a00 0x7fa238003e20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa23c009ac0 tx=0x7fa238005830 comp rx=0 tx=0).stop 2024-04-26T23:19:02.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.580+0000 7fa2509e9640 1 -- 172.21.15.47:0/3195817548 shutdown_connections 2024-04-26T23:19:02.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.580+0000 7fa2509e9640 1 --2- 172.21.15.47:0/3195817548 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa238003a00 0x7fa238003e20 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:02.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.580+0000 7fa2509e9640 1 -- 172.21.15.47:0/3195817548 >> 172.21.15.47:0/3195817548 conn(0x7fa24c0d4f90 msgr2=0x7fa24c0d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:02.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.580+0000 7fa2509e9640 1 -- 172.21.15.47:0/3195817548 shutdown_connections 2024-04-26T23:19:02.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.580+0000 7fa2509e9640 1 -- 172.21.15.47:0/3195817548 wait complete. 2024-04-26T23:19:02.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.582+0000 7fa2509e9640 1 Processor -- start 2024-04-26T23:19:02.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.582+0000 7fa2509e9640 1 -- start start 2024-04-26T23:19:02.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.582+0000 7fa2509e9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa238003a00 0x7fa24c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:02.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.582+0000 7fa2509e9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa24c15a6a0 0x7fa24c1f8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:02.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.582+0000 7fa2509e9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa24c0d3100 0x7fa24c0d3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:02.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.582+0000 7fa2509e9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa24c0d0870 con 0x7fa24c0d3100 2024-04-26T23:19:02.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.582+0000 7fa2509e9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa24c0d06f0 con 0x7fa24c15a6a0 2024-04-26T23:19:02.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.582+0000 7fa2509e9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa24c0d09f0 con 0x7fa238003a00 2024-04-26T23:19:02.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.582+0000 7fa24ad9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa24c0d3100 0x7fa24c0d3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:02.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.582+0000 7fa24ad9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa24c0d3100 0x7fa24c0d3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54738/0 (socket says 172.21.15.47:54738) 2024-04-26T23:19:02.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.582+0000 7fa24ad9d640 1 -- 172.21.15.47:0/2170375355 learned_addr learned my addr 172.21.15.47:0/2170375355 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:02.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.583+0000 7fa24a59c640 1 --2- 172.21.15.47:0/2170375355 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa238003a00 0x7fa24c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:02.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.583+0000 7fa243fff640 1 --2- 172.21.15.47:0/2170375355 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa24c15a6a0 0x7fa24c1f8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:02.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.583+0000 7fa24ad9d640 1 -- 172.21.15.47:0/2170375355 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa238003a00 msgr2=0x7fa24c0b8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:02.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.583+0000 7fa24ad9d640 1 --2- 172.21.15.47:0/2170375355 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa238003a00 0x7fa24c0b8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:02.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.583+0000 7fa24ad9d640 1 -- 172.21.15.47:0/2170375355 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa24c15a6a0 msgr2=0x7fa24c1f8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:02.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.583+0000 7fa24ad9d640 1 --2- 172.21.15.47:0/2170375355 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa24c15a6a0 0x7fa24c1f8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:02.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.583+0000 7fa24ad9d640 1 -- 172.21.15.47:0/2170375355 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa24c1f9280 con 0x7fa24c0d3100 2024-04-26T23:19:02.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.583+0000 7fa24ad9d640 1 --2- 172.21.15.47:0/2170375355 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa24c0d3100 0x7fa24c0d3580 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa244002230 tx=0x7fa244007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:02.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.583+0000 7fa242ffd640 1 -- 172.21.15.47:0/2170375355 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24400ab30 con 0x7fa24c0d3100 2024-04-26T23:19:02.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.583+0000 7fa242ffd640 1 -- 172.21.15.47:0/2170375355 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa244019070 con 0x7fa24c0d3100 2024-04-26T23:19:02.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.583+0000 7fa2509e9640 1 -- 172.21.15.47:0/2170375355 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa24c0b9080 con 0x7fa24c0d3100 2024-04-26T23:19:02.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.583+0000 7fa242ffd640 1 -- 172.21.15.47:0/2170375355 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2440133f0 con 0x7fa24c0d3100 2024-04-26T23:19:02.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.583+0000 7fa2509e9640 1 -- 172.21.15.47:0/2170375355 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa24c0ca590 con 0x7fa24c0d3100 2024-04-26T23:19:02.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.584+0000 7fa2509e9640 1 -- 172.21.15.47:0/2170375355 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa24c0d3ac0 con 0x7fa24c0d3100 2024-04-26T23:19:02.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.585+0000 7fa242ffd640 1 -- 172.21.15.47:0/2170375355 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa244013590 con 0x7fa24c0d3100 2024-04-26T23:19:02.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.585+0000 7fa242ffd640 1 --2- 172.21.15.47:0/2170375355 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa22c06c920 0x7fa22c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:02.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.585+0000 7fa242ffd640 1 -- 172.21.15.47:0/2170375355 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fa24408c3f0 con 0x7fa24c0d3100 2024-04-26T23:19:02.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.585+0000 7fa24a59c640 1 --2- 172.21.15.47:0/2170375355 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa22c06c920 0x7fa22c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:02.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.586+0000 7fa24a59c640 1 --2- 172.21.15.47:0/2170375355 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa22c06c920 0x7fa22c06ede0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa23c033000 tx=0x7fa23c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:02.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.589+0000 7fa242ffd640 1 -- 172.21.15.47:0/2170375355 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa24c1f9280 con 0x7fa24c0d3100 2024-04-26T23:19:02.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.777+0000 7fa2509e9640 1 -- 172.21.15.47:0/2170375355 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa24c0694f0 con 0x7fa24c0d3100 2024-04-26T23:19:02.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.779+0000 7fa242ffd640 1 -- 172.21.15.47:0/2170375355 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18478 (secure 0 0 0) 0x7fa24405a4e0 con 0x7fa24c0d3100 2024-04-26T23:19:02.780 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:19:02.781 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":24,"fsid":"399006d3-4db1-43e6-ba81-a79a276eabfd","created":"2024-04-26T23:18:06.731211+0000","modified":"2024-04-26T23:18:36.402132+0000","last_up_change":"2024-04-26T23:18:17.808736+0000","last_in_change":"2024-04-26T23:18:08.945859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:18:16.937492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:18:29.033575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:18:30.318499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:18:34.014536+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ee091567-fea0-4a14-bf4e-0e16c20bb2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6819","nonce":326520779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6821","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6823","nonce":326520779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6829","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6831","nonce":326520779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6825","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6827","nonce":326520779}]},"public_addr":"172.21.15.47:6819/326520779","cluster_addr":"172.21.15.47:6823/326520779","heartbeat_back_addr":"172.21.15.47:6831/326520779","heartbeat_front_addr":"172.21.15.47:6827/326520779","state":["exists","up"]},{"osd":1,"uuid":"c14440a0-bba5-4931-a6db-b736c08e869b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6809","nonce":1912610931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6811","nonce":1912610931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6815","nonce":1912610931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6813","nonce":1912610931}]},"public_addr":"172.21.15.47:6809/1912610931","cluster_addr":"172.21.15.47:6811/1912610931","heartbeat_back_addr":"172.21.15.47:6815/1912610931","heartbeat_front_addr":"172.21.15.47:6813/1912610931","state":["exists","up"]},{"osd":2,"uuid":"3940f8de-c9b2-4cd3-a911-9c301729eb30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6801","nonce":2569869305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6803","nonce":2569869305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6807","nonce":2569869305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6805","nonce":2569869305}]},"public_addr":"172.21.15.47:6801/2569869305","cluster_addr":"172.21.15.47:6803/2569869305","heartbeat_back_addr":"172.21.15.47:6807/2569869305","heartbeat_front_addr":"172.21.15.47:6805/2569869305","state":["exists","up"]},{"osd":3,"uuid":"1286a67e-8958-426a-8c2e-bc85498c619c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6818","nonce":2791411683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6822","nonce":2791411683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6830","nonce":2791411683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6826","nonce":2791411683}]},"public_addr":"172.21.15.47:6818/2791411683","cluster_addr":"172.21.15.47:6822/2791411683","heartbeat_back_addr":"172.21.15.47:6830/2791411683","heartbeat_front_addr":"172.21.15.47:6826/2791411683","state":["exists","up"]},{"osd":4,"uuid":"d8ee2c20-e5b7-46c2-9999-b234070126cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6809","nonce":441313104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6811","nonce":441313104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6815","nonce":441313104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6813","nonce":441313104}]},"public_addr":"172.21.15.203:6809/441313104","cluster_addr":"172.21.15.203:6811/441313104","heartbeat_back_addr":"172.21.15.203:6815/441313104","heartbeat_front_addr":"172.21.15.203:6813/441313104","state":["exists","up"]},{"osd":5,"uuid":"b453a8f0-f324-4840-9ae0-d67bdadb9f8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6801","nonce":56838444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6803","nonce":56838444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6807","nonce":56838444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6805","nonce":56838444}]},"public_addr":"172.21.15.203:6801/56838444","cluster_addr":"172.21.15.203:6803/56838444","heartbeat_back_addr":"172.21.15.203:6807/56838444","heartbeat_front_addr":"172.21.15.203:6805/56838444","state":["exists","up"]},{"osd":6,"uuid":"a6d31951-2d4b-4e8d-a510-6283b01dc158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6825","nonce":2738517080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6827","nonce":2738517080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6831","nonce":2738517080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6829","nonce":2738517080}]},"public_addr":"172.21.15.203:6825/2738517080","cluster_addr":"172.21.15.203:6827/2738517080","heartbeat_back_addr":"172.21.15.203:6831/2738517080","heartbeat_front_addr":"172.21.15.203:6829/2738517080","state":["exists","up"]},{"osd":7,"uuid":"7119b8c0-fe96-402d-9e75-7b509d9d44d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6817","nonce":1315565058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6819","nonce":1315565058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6823","nonce":1315565058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6821","nonce":1315565058}]},"public_addr":"172.21.15.203:6817/1315565058","cluster_addr":"172.21.15.203:6819/1315565058","heartbeat_back_addr":"172.21.15.203:6823/1315565058","heartbeat_front_addr":"172.21.15.203:6821/1315565058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.121921+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.137494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.084814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.111593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.072430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.125066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.097867+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.057061+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:19:02.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.781+0000 7fa2509e9640 1 -- 172.21.15.47:0/2170375355 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa22c06c920 msgr2=0x7fa22c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:02.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.781+0000 7fa2509e9640 1 --2- 172.21.15.47:0/2170375355 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa22c06c920 0x7fa22c06ede0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa23c033000 tx=0x7fa23c008030 comp rx=0 tx=0).stop 2024-04-26T23:19:02.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.781+0000 7fa2509e9640 1 -- 172.21.15.47:0/2170375355 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa24c0d3100 msgr2=0x7fa24c0d3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:02.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.781+0000 7fa2509e9640 1 --2- 172.21.15.47:0/2170375355 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa24c0d3100 0x7fa24c0d3580 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa244002230 tx=0x7fa244007bd0 comp rx=0 tx=0).stop 2024-04-26T23:19:02.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.781+0000 7fa2509e9640 1 -- 172.21.15.47:0/2170375355 shutdown_connections 2024-04-26T23:19:02.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.781+0000 7fa2509e9640 1 --2- 172.21.15.47:0/2170375355 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa22c06c920 0x7fa22c06ede0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:02.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.781+0000 7fa2509e9640 1 --2- 172.21.15.47:0/2170375355 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa24c0d3100 0x7fa24c0d3580 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:02.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.781+0000 7fa2509e9640 1 --2- 172.21.15.47:0/2170375355 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa24c15a6a0 0x7fa24c1f8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:02.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.781+0000 7fa2509e9640 1 --2- 172.21.15.47:0/2170375355 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa238003a00 0x7fa24c0b8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:02.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.781+0000 7fa2509e9640 1 -- 172.21.15.47:0/2170375355 >> 172.21.15.47:0/2170375355 conn(0x7fa24c0d4f90 msgr2=0x7fa24c0b2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:02.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.781+0000 7fa2509e9640 1 -- 172.21.15.47:0/2170375355 shutdown_connections 2024-04-26T23:19:02.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.781+0000 7fa2509e9640 1 -- 172.21.15.47:0/2170375355 wait complete. 2024-04-26T23:19:02.794 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:19:02.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.892+0000 7f6561104640 1 Processor -- start 2024-04-26T23:19:02.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.892+0000 7f6561104640 1 -- start start 2024-04-26T23:19:02.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.893+0000 7f6561104640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f655c0cffa0 0x7f655c0d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:02.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.893+0000 7f6561104640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f655c0d0a90 0x7f655c0b2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:02.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.893+0000 7f6561104640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f655c0d0f20 con 0x7f655c1608f0 2024-04-26T23:19:02.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.893+0000 7f6561104640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f655c0cea30 con 0x7f655c0d0a90 2024-04-26T23:19:02.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.893+0000 7f6561104640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f655c0cde20 con 0x7f655c0cffa0 2024-04-26T23:19:02.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.894+0000 7f6553fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f655c0cffa0 0x7f655c0d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:02.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.894+0000 7f655b59e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f655c0d0a90 0x7f655c0b2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:02.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.894+0000 7f6553fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f655c0cffa0 0x7f655c0d1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.47:52380/0 (socket says 172.21.15.47:52380) 2024-04-26T23:19:02.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.894+0000 7f6553fff640 1 -- 172.21.15.47:0/3594792039 learned_addr learned my addr 172.21.15.47:0/3594792039 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:02.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.894+0000 7f655b59e640 1 --2- 172.21.15.47:0/3594792039 >> v2:172.21.15.203:3300/0 conn(0x7f655c0d0a90 0x7f655c0b2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.47:48100/0 (socket says 172.21.15.47:48100) 2024-04-26T23:19:02.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.894+0000 7f655ad9d640 1 --1- 172.21.15.47:0/3594792039 >> v1:172.21.15.203:6789/0 conn(0x7f655c1608f0 0x7f655c0cf890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.47:47214/0 (socket says 172.21.15.47:47214) 2024-04-26T23:19:02.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.894+0000 7f6553fff640 1 -- 172.21.15.47:0/3594792039 >> v2:172.21.15.203:3300/0 conn(0x7f655c0d0a90 msgr2=0x7f655c0b2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:02.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.894+0000 7f6553fff640 1 --2- 172.21.15.47:0/3594792039 >> v2:172.21.15.203:3300/0 conn(0x7f655c0d0a90 0x7f655c0b2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:02.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.894+0000 7f6553fff640 1 -- 172.21.15.47:0/3594792039 >> v1:172.21.15.203:6789/0 conn(0x7f655c1608f0 legacy=0x7f655c0cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:02.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.894+0000 7f6553fff640 1 -- 172.21.15.47:0/3594792039 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f655c0ce100 con 0x7f655c0cffa0 2024-04-26T23:19:02.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.895+0000 7f6553fff640 1 --2- 172.21.15.47:0/3594792039 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f655c0cffa0 0x7f655c0d1bf0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6544009870 tx=0x7f654402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bba01eeb2c1e4039 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:02.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.895+0000 7f655a59c640 1 -- 172.21.15.47:0/3594792039 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6544037070 con 0x7f655c0cffa0 2024-04-26T23:19:02.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.895+0000 7f655a59c640 1 -- 172.21.15.47:0/3594792039 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6544032440 con 0x7f655c0cffa0 2024-04-26T23:19:02.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.895+0000 7f655a59c640 1 -- 172.21.15.47:0/3594792039 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6544032810 con 0x7f655c0cffa0 2024-04-26T23:19:02.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.895+0000 7f6561104640 1 -- 172.21.15.47:0/3594792039 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f655c0cffa0 msgr2=0x7f655c0d1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:02.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.895+0000 7f6561104640 1 --2- 172.21.15.47:0/3594792039 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f655c0cffa0 0x7f655c0d1bf0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6544009870 tx=0x7f654402a270 comp rx=0 tx=0).stop 2024-04-26T23:19:02.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.895+0000 7f6561104640 1 -- 172.21.15.47:0/3594792039 shutdown_connections 2024-04-26T23:19:02.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.895+0000 7f6561104640 1 --2- 172.21.15.47:0/3594792039 >> v2:172.21.15.203:3300/0 conn(0x7f655c0d0a90 0x7f655c0b2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:02.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.895+0000 7f6561104640 1 --2- 172.21.15.47:0/3594792039 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f655c0cffa0 0x7f655c0d1bf0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:02.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.895+0000 7f6561104640 1 -- 172.21.15.47:0/3594792039 >> 172.21.15.47:0/3594792039 conn(0x7f655c0d2ec0 msgr2=0x7f655c0d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:02.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.895+0000 7f6561104640 1 -- 172.21.15.47:0/3594792039 shutdown_connections 2024-04-26T23:19:02.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.895+0000 7f6561104640 1 -- 172.21.15.47:0/3594792039 wait complete. 2024-04-26T23:19:02.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.897+0000 7f6561104640 1 Processor -- start 2024-04-26T23:19:02.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.897+0000 7f6561104640 1 -- start start 2024-04-26T23:19:02.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.897+0000 7f6561104640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f655c0cffa0 0x7f655c155f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:02.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.897+0000 7f6561104640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f655c0d0a90 0x7f655c0c8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:02.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.897+0000 7f6561104640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f655c1608f0 0x7f655c1f3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:02.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.897+0000 7f6561104640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f655c0ceec0 con 0x7f655c1608f0 2024-04-26T23:19:02.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.897+0000 7f6561104640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f655c0ce6c0 con 0x7f655c0cffa0 2024-04-26T23:19:02.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.897+0000 7f6561104640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f655c0cda40 con 0x7f655c0d0a90 2024-04-26T23:19:02.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.898+0000 7f655b59e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f655c1608f0 0x7f655c1f3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:02.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.898+0000 7f655b59e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f655c1608f0 0x7f655c1f3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54746/0 (socket says 172.21.15.47:54746) 2024-04-26T23:19:02.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.898+0000 7f655b59e640 1 -- 172.21.15.47:0/1198768177 learned_addr learned my addr 172.21.15.47:0/1198768177 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:02.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.898+0000 7f6553fff640 1 --2- 172.21.15.47:0/1198768177 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f655c0d0a90 0x7f655c0c8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:02.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.898+0000 7f655ad9d640 1 --2- 172.21.15.47:0/1198768177 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f655c0cffa0 0x7f655c155f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:02.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.898+0000 7f655b59e640 1 -- 172.21.15.47:0/1198768177 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f655c0d0a90 msgr2=0x7f655c0c8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:02.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.898+0000 7f655b59e640 1 --2- 172.21.15.47:0/1198768177 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f655c0d0a90 0x7f655c0c8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:02.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.898+0000 7f655b59e640 1 -- 172.21.15.47:0/1198768177 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f655c0cffa0 msgr2=0x7f655c155f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:02.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.898+0000 7f655b59e640 1 --2- 172.21.15.47:0/1198768177 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f655c0cffa0 0x7f655c155f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:02.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.898+0000 7f655b59e640 1 -- 172.21.15.47:0/1198768177 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f655c0ccdd0 con 0x7f655c1608f0 2024-04-26T23:19:02.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.898+0000 7f655b59e640 1 --2- 172.21.15.47:0/1198768177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f655c1608f0 0x7f655c1f3bd0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f655400efc0 tx=0x7f655400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:02.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.898+0000 7f65537fe640 1 -- 172.21.15.47:0/1198768177 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6554018020 con 0x7f655c1608f0 2024-04-26T23:19:02.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.898+0000 7f65537fe640 1 -- 172.21.15.47:0/1198768177 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6554014480 con 0x7f655c1608f0 2024-04-26T23:19:02.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.898+0000 7f6561104640 1 -- 172.21.15.47:0/1198768177 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f655c0c84d0 con 0x7f655c1608f0 2024-04-26T23:19:02.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.898+0000 7f65537fe640 1 -- 172.21.15.47:0/1198768177 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6554014870 con 0x7f655c1608f0 2024-04-26T23:19:02.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.898+0000 7f6561104640 1 -- 172.21.15.47:0/1198768177 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f655c1f4110 con 0x7f655c1608f0 2024-04-26T23:19:02.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.899+0000 7f6561104640 1 -- 172.21.15.47:0/1198768177 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f655c0d1bf0 con 0x7f655c1608f0 2024-04-26T23:19:02.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.900+0000 7f65537fe640 1 -- 172.21.15.47:0/1198768177 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6554005d10 con 0x7f655c1608f0 2024-04-26T23:19:02.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.900+0000 7f65537fe640 1 --2- 172.21.15.47:0/1198768177 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f653c06c8f0 0x7f653c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:02.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.900+0000 7f65537fe640 1 -- 172.21.15.47:0/1198768177 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f655408c710 con 0x7f655c1608f0 2024-04-26T23:19:02.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.900+0000 7f655ad9d640 1 --2- 172.21.15.47:0/1198768177 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f653c06c8f0 0x7f653c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:02.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.901+0000 7f655ad9d640 1 --2- 172.21.15.47:0/1198768177 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f653c06c8f0 0x7f653c06edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f655c1e7ad0 tx=0x7f654c007e80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:02.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:02.905+0000 7f65537fe640 1 -- 172.21.15.47:0/1198768177 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f655c0d1bf0 con 0x7f655c1608f0 2024-04-26T23:19:03.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.134+0000 7f6561104640 1 -- 172.21.15.47:0/1198768177 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f655c0c6600 con 0x7f655c1608f0 2024-04-26T23:19:03.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.134+0000 7f65537fe640 1 -- 172.21.15.47:0/1198768177 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5180 (secure 0 0 0) 0x7f6554022070 con 0x7f655c1608f0 2024-04-26T23:19:03.136 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:19:03.136 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/1553696401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1553696401},{"type":"v1","addr":"172.21.15.47:6834","nonce":1553696401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/3792767564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3792767564},{"type":"v1","addr":"172.21.15.203:6836","nonce":3792767564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4088100766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":4088100766},{"type":"v1","addr":"172.21.15.47:6835","nonce":4088100766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:18:31.367292+0000","modified":"2024-04-26T23:18:53.430267+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/997115011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":997115011},{"type":"v1","addr":"172.21.15.203:6837","nonce":997115011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4282,"qdb_cluster":[4282]},"id":1}]} 2024-04-26T23:19:03.137 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 9 2024-04-26T23:19:03.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.137+0000 7f6561104640 1 -- 172.21.15.47:0/1198768177 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f653c06c8f0 msgr2=0x7f653c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:03.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.137+0000 7f6561104640 1 --2- 172.21.15.47:0/1198768177 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f653c06c8f0 0x7f653c06edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f655c1e7ad0 tx=0x7f654c007e80 comp rx=0 tx=0).stop 2024-04-26T23:19:03.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.137+0000 7f6561104640 1 -- 172.21.15.47:0/1198768177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f655c1608f0 msgr2=0x7f655c1f3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:03.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.137+0000 7f6561104640 1 --2- 172.21.15.47:0/1198768177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f655c1608f0 0x7f655c1f3bd0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f655400efc0 tx=0x7f655400c490 comp rx=0 tx=0).stop 2024-04-26T23:19:03.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.137+0000 7f6561104640 1 -- 172.21.15.47:0/1198768177 shutdown_connections 2024-04-26T23:19:03.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.137+0000 7f6561104640 1 --2- 172.21.15.47:0/1198768177 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f653c06c8f0 0x7f653c06edb0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:03.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.137+0000 7f6561104640 1 --2- 172.21.15.47:0/1198768177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f655c1608f0 0x7f655c1f3bd0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:03.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.137+0000 7f6561104640 1 --2- 172.21.15.47:0/1198768177 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f655c0d0a90 0x7f655c0c8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:03.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.137+0000 7f6561104640 1 --2- 172.21.15.47:0/1198768177 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f655c0cffa0 0x7f655c155f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:03.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.137+0000 7f6561104640 1 -- 172.21.15.47:0/1198768177 >> 172.21.15.47:0/1198768177 conn(0x7f655c0d2ec0 msgr2=0x7f655c0b6180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:03.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.137+0000 7f6561104640 1 -- 172.21.15.47:0/1198768177 shutdown_connections 2024-04-26T23:19:03.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.137+0000 7f6561104640 1 -- 172.21.15.47:0/1198768177 wait complete. 2024-04-26T23:19:03.553 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T23:18:31.367292+0000', 'modified': '2024-04-26T23:18:53.430267+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4282}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4282': {'gid': 4282, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.203:6837/997115011', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6835', 'nonce': 997115011}, {'type': 'v1', 'addr': '172.21.15.203:6837', 'nonce': 997115011}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4282, 'qdb_cluster': [4282]} 2024-04-26T23:19:03.553 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T23:19:03.553 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-26T23:19:03.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.655+0000 7feeffa72640 1 Processor -- start 2024-04-26T23:19:03.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.655+0000 7feeffa72640 1 -- start start 2024-04-26T23:19:03.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.656+0000 7feeffa72640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feef80b3ff0 0x7feef80b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:03.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.656+0000 7feeffa72640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feef80d2d70 con 0x7feef80d3b80 2024-04-26T23:19:03.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.656+0000 7feeffa72640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feef80d3050 con 0x7feef815f540 2024-04-26T23:19:03.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.656+0000 7feeffa72640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7feef815f920 con 0x7feef80b3ff0 2024-04-26T23:19:03.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.656+0000 7feefd00d640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7feef80d3b80 0x7feef80b4aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:48164/0 (socket says 172.21.15.47:48164) 2024-04-26T23:19:03.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.656+0000 7feefd00d640 1 -- 172.21.15.47:0/258806787 learned_addr learned my addr 172.21.15.47:0/258806787 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:03.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.656+0000 7feefe00f640 1 --2- 172.21.15.47:0/258806787 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feef80b3ff0 0x7feef80b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:03.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.656+0000 7feefc80c640 1 -- 172.21.15.47:0/258806787 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1734204380 0 0) 0x7feef80d2d70 con 0x7feef80d3b80 2024-04-26T23:19:03.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.656+0000 7feefc80c640 1 -- 172.21.15.47:0/258806787 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feee0003520 con 0x7feef80d3b80 2024-04-26T23:19:03.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.657+0000 7feefc80c640 1 -- 172.21.15.47:0/258806787 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1225978776 0 0) 0x7feef80d3050 con 0x7feef815f540 2024-04-26T23:19:03.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.657+0000 7feefc80c640 1 -- 172.21.15.47:0/258806787 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feef80d2d70 con 0x7feef815f540 2024-04-26T23:19:03.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.657+0000 7feefc80c640 1 -- 172.21.15.47:0/258806787 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4278769795 0 0) 0x7feee0003520 con 0x7feef80d3b80 2024-04-26T23:19:03.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.657+0000 7feefe00f640 1 -- 172.21.15.47:0/258806787 >> v1:172.21.15.203:6789/0 conn(0x7feef815f540 legacy=0x7feef80d3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:03.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.657+0000 7feefe00f640 1 -- 172.21.15.47:0/258806787 >> v1:172.21.15.47:6789/0 conn(0x7feef80d3b80 legacy=0x7feef80b4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:03.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.657+0000 7feefe00f640 1 -- 172.21.15.47:0/258806787 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feef80d08e0 con 0x7feef80b3ff0 2024-04-26T23:19:03.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.658+0000 7feefe00f640 1 --2- 172.21.15.47:0/258806787 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feef80b3ff0 0x7feef80b51b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7feee8009dc0 tx=0x7feee802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b62ebf37955f749b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:03.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.658+0000 7feefc80c640 1 -- 172.21.15.47:0/258806787 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeec001670 con 0x7feef80b3ff0 2024-04-26T23:19:03.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.658+0000 7feefc80c640 1 -- 172.21.15.47:0/258806787 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feee8007540 con 0x7feef80b3ff0 2024-04-26T23:19:03.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.658+0000 7feefc80c640 1 -- 172.21.15.47:0/258806787 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee80078f0 con 0x7feef80b3ff0 2024-04-26T23:19:03.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.658+0000 7feeffa72640 1 -- 172.21.15.47:0/258806787 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feef80b3ff0 msgr2=0x7feef80b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:03.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.658+0000 7feeffa72640 1 --2- 172.21.15.47:0/258806787 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feef80b3ff0 0x7feef80b51b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7feee8009dc0 tx=0x7feee802a660 comp rx=0 tx=0).stop 2024-04-26T23:19:03.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.658+0000 7feeffa72640 1 -- 172.21.15.47:0/258806787 shutdown_connections 2024-04-26T23:19:03.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.658+0000 7feeffa72640 1 --2- 172.21.15.47:0/258806787 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feef80b3ff0 0x7feef80b51b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:03.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.658+0000 7feeffa72640 1 -- 172.21.15.47:0/258806787 >> 172.21.15.47:0/258806787 conn(0x7feef80d4f90 msgr2=0x7feef80d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:03.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.658+0000 7feeffa72640 1 -- 172.21.15.47:0/258806787 shutdown_connections 2024-04-26T23:19:03.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.658+0000 7feeffa72640 1 -- 172.21.15.47:0/258806787 wait complete. 2024-04-26T23:19:03.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.659+0000 7feeffa72640 1 Processor -- start 2024-04-26T23:19:03.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.660+0000 7feeffa72640 1 -- start start 2024-04-26T23:19:03.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.660+0000 7feeffa72640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feef80b3ff0 0x7feef80a1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:03.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.660+0000 7feeffa72640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feef80d3b80 0x7feef80a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:03.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.660+0000 7feeffa72640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feef815f540 0x7feef80b8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:03.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.660+0000 7feeffa72640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7feef8068e50 con 0x7feef80d3b80 2024-04-26T23:19:03.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.660+0000 7feeffa72640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7feef80d0f30 con 0x7feef80b3ff0 2024-04-26T23:19:03.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.660+0000 7feeffa72640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7feef80d2780 con 0x7feef815f540 2024-04-26T23:19:03.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.660+0000 7feefd00d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feef80d3b80 0x7feef80a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:03.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.660+0000 7feefd00d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feef80d3b80 0x7feef80a17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54752/0 (socket says 172.21.15.47:54752) 2024-04-26T23:19:03.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.660+0000 7feefd00d640 1 -- 172.21.15.47:0/3620338671 learned_addr learned my addr 172.21.15.47:0/3620338671 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:03.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.661+0000 7feefd80e640 1 --2- 172.21.15.47:0/3620338671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feef80b3ff0 0x7feef80a1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:03.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.661+0000 7feefe00f640 1 --2- 172.21.15.47:0/3620338671 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feef815f540 0x7feef80b8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:03.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.661+0000 7feefd00d640 1 -- 172.21.15.47:0/3620338671 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feef815f540 msgr2=0x7feef80b8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:03.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.661+0000 7feefd00d640 1 --2- 172.21.15.47:0/3620338671 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feef815f540 0x7feef80b8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:03.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.661+0000 7feefd00d640 1 -- 172.21.15.47:0/3620338671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feef80b3ff0 msgr2=0x7feef80a1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:03.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.661+0000 7feefd00d640 1 --2- 172.21.15.47:0/3620338671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feef80b3ff0 0x7feef80a1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:03.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.661+0000 7feefd00d640 1 -- 172.21.15.47:0/3620338671 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feef809ee70 con 0x7feef80d3b80 2024-04-26T23:19:03.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.661+0000 7feefd80e640 1 --2- 172.21.15.47:0/3620338671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feef80b3ff0 0x7feef80a1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:19:03.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.661+0000 7feefd00d640 1 --2- 172.21.15.47:0/3620338671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feef80d3b80 0x7feef80a17d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7feeec00c770 tx=0x7feeec00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:03.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.661+0000 7feef67fc640 1 -- 172.21.15.47:0/3620338671 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeec017070 con 0x7feef80d3b80 2024-04-26T23:19:03.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.661+0000 7feef67fc640 1 -- 172.21.15.47:0/3620338671 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeec005ad0 con 0x7feef80d3b80 2024-04-26T23:19:03.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.661+0000 7feeffa72640 1 -- 172.21.15.47:0/3620338671 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feef809f150 con 0x7feef80d3b80 2024-04-26T23:19:03.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.661+0000 7feef67fc640 1 -- 172.21.15.47:0/3620338671 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeec007070 con 0x7feef80d3b80 2024-04-26T23:19:03.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.661+0000 7feeffa72640 1 -- 172.21.15.47:0/3620338671 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feef809f710 con 0x7feef80d3b80 2024-04-26T23:19:03.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.663+0000 7feef67fc640 1 -- 172.21.15.47:0/3620338671 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feeec01a070 con 0x7feef80d3b80 2024-04-26T23:19:03.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.663+0000 7feef67fc640 1 --2- 172.21.15.47:0/3620338671 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7feee406c8f0 0x7feee406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:03.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.663+0000 7feef67fc640 1 -- 172.21.15.47:0/3620338671 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7feeec08b280 con 0x7feef80d3b80 2024-04-26T23:19:03.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.663+0000 7feefd80e640 1 --2- 172.21.15.47:0/3620338671 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7feee406c8f0 0x7feee406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:03.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.664+0000 7feefd80e640 1 --2- 172.21.15.47:0/3620338671 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7feee406c8f0 0x7feee406edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7feef815ea10 tx=0x7feef0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:03.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.664+0000 7fef0278f740 1 Processor -- start 2024-04-26T23:19:03.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.664+0000 7fef0278f740 1 -- start start 2024-04-26T23:19:03.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.665+0000 7fef0278f740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559e1744c440 0x559e1744c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:03.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.665+0000 7fef0278f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e17469ce0 0x559e1746a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:03.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.665+0000 7fef0278f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559e17422120 0x559e174683a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:03.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.665+0000 7fef0278f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x559e17386740 con 0x559e17469ce0 2024-04-26T23:19:03.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.665+0000 7fef0278f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x559e173223a0 con 0x559e17422120 2024-04-26T23:19:03.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.665+0000 7fef0278f740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x559e17382700 con 0x559e1744c440 2024-04-26T23:19:03.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.665+0000 7feefd00d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e17469ce0 0x559e1746a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:03.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.665+0000 7feefd00d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e17469ce0 0x559e1746a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54768/0 (socket says 172.21.15.47:54768) 2024-04-26T23:19:03.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.665+0000 7feefd00d640 1 -- 172.21.15.47:0/445903446 learned_addr learned my addr 172.21.15.47:0/445903446 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:03.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.665+0000 7feefd80e640 1 --2- 172.21.15.47:0/445903446 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559e1744c440 0x559e1744c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:03.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.665+0000 7feefe00f640 1 --2- 172.21.15.47:0/445903446 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559e17422120 0x559e174683a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:03.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.665+0000 7feefd00d640 1 -- 172.21.15.47:0/445903446 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559e1744c440 msgr2=0x559e1744c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:03.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.665+0000 7feefd00d640 1 --2- 172.21.15.47:0/445903446 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559e1744c440 0x559e1744c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:03.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.665+0000 7feefd00d640 1 -- 172.21.15.47:0/445903446 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559e17422120 msgr2=0x559e174683a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:03.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.665+0000 7feefd00d640 1 --2- 172.21.15.47:0/445903446 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559e17422120 0x559e174683a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:03.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.665+0000 7feefd00d640 1 -- 172.21.15.47:0/445903446 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e1733e750 con 0x559e17469ce0 2024-04-26T23:19:03.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.665+0000 7feefe00f640 1 --2- 172.21.15.47:0/445903446 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559e17422120 0x559e174683a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:19:03.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.665+0000 7feefd00d640 1 --2- 172.21.15.47:0/445903446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e17469ce0 0x559e1746a230 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7feeec00cfd0 tx=0x7feeec05d170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab41122a4d1d7afe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:03.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.666+0000 7feedf7fe640 1 -- 172.21.15.47:0/445903446 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeec082e60 con 0x559e17469ce0 2024-04-26T23:19:03.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.666+0000 7feedf7fe640 1 -- 172.21.15.47:0/445903446 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeec08b0c0 con 0x559e17469ce0 2024-04-26T23:19:03.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.666+0000 7feedf7fe640 1 -- 172.21.15.47:0/445903446 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeec055420 con 0x559e17469ce0 2024-04-26T23:19:03.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.666+0000 7fef0278f740 1 -- 172.21.15.47:0/445903446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e17469ce0 msgr2=0x559e1746a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:03.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.666+0000 7fef0278f740 1 --2- 172.21.15.47:0/445903446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e17469ce0 0x559e1746a230 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7feeec00cfd0 tx=0x7feeec05d170 comp rx=0 tx=0).stop 2024-04-26T23:19:03.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.666+0000 7fef0278f740 1 -- 172.21.15.47:0/445903446 shutdown_connections 2024-04-26T23:19:03.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.666+0000 7fef0278f740 1 --2- 172.21.15.47:0/445903446 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559e17422120 0x559e174683a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:03.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.666+0000 7fef0278f740 1 --2- 172.21.15.47:0/445903446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e17469ce0 0x559e1746a230 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:03.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.666+0000 7fef0278f740 1 --2- 172.21.15.47:0/445903446 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559e1744c440 0x559e1744c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:03.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.666+0000 7fef0278f740 1 -- 172.21.15.47:0/445903446 >> 172.21.15.47:0/445903446 conn(0x559e1741ea50 msgr2=0x559e17467250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:03.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.666+0000 7fef0278f740 1 -- 172.21.15.47:0/445903446 shutdown_connections 2024-04-26T23:19:03.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.666+0000 7fef0278f740 1 -- 172.21.15.47:0/445903446 wait complete. 2024-04-26T23:19:03.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.667+0000 7fef0278f740 1 -- start start 2024-04-26T23:19:03.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.667+0000 7fef0278f740 1 Processor -- start 2024-04-26T23:19:03.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.668+0000 7fef0278f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e1744c440 0x559e1744fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:03.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.668+0000 7fef0278f740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559e17469ce0 0x559e1745fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:03.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.668+0000 7fef0278f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559e17478a10 0x559e1745ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:03.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.668+0000 7fef0278f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x559e17422ec0 con 0x559e1744c440 2024-04-26T23:19:03.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.668+0000 7fef0278f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x559e1730dfe0 con 0x559e17478a10 2024-04-26T23:19:03.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.668+0000 7fef0278f740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x559e1741c980 con 0x559e17469ce0 2024-04-26T23:19:03.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.668+0000 7fef0278f740 10 client.0 _ll_register_callbacks cb 0x559e17324150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:19:03.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.668+0000 7feefd80e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e1744c440 0x559e1744fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:03.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.668+0000 7feefd80e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e1744c440 0x559e1744fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54776/0 (socket says 172.21.15.47:54776) 2024-04-26T23:19:03.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.668+0000 7feefd80e640 1 -- 172.21.15.47:0/2538879092 learned_addr learned my addr 172.21.15.47:0/2538879092 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:03.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.668+0000 7feefd00d640 1 --2- 172.21.15.47:0/2538879092 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559e17469ce0 0x559e1745fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:03.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.668+0000 7feefe00f640 1 --2- 172.21.15.47:0/2538879092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559e17478a10 0x559e1745ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:03.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.668+0000 7feefd80e640 1 -- 172.21.15.47:0/2538879092 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559e17469ce0 msgr2=0x559e1745fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:03.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.668+0000 7feefd80e640 1 --2- 172.21.15.47:0/2538879092 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559e17469ce0 0x559e1745fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:03.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.668+0000 7feefd80e640 1 -- 172.21.15.47:0/2538879092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559e17478a10 msgr2=0x559e1745ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:03.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.668+0000 7feefd80e640 1 --2- 172.21.15.47:0/2538879092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559e17478a10 0x559e1745ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:03.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.668+0000 7feefd80e640 1 -- 172.21.15.47:0/2538879092 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e170982d0 con 0x559e1744c440 2024-04-26T23:19:03.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.668+0000 7feefd80e640 1 --2- 172.21.15.47:0/2538879092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e1744c440 0x559e1744fe60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7feef000de80 tx=0x7feef0000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6776765f86d5dc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:03.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.668+0000 7feedcff9640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-26T23:19:03.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.669+0000 7feedcff9640 1 -- 172.21.15.47:0/2538879092 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef0024020 con 0x559e1744c440 2024-04-26T23:19:03.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.669+0000 7feedcff9640 1 -- 172.21.15.47:0/2538879092 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7feef0020600 con 0x559e1744c440 2024-04-26T23:19:03.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.669+0000 7fef0278f740 1 -- 172.21.15.47:0/2538879092 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559e17394350 con 0x559e1744c440 2024-04-26T23:19:03.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.669+0000 7feedcff9640 1 -- 172.21.15.47:0/2538879092 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7feef002e050 con 0x559e1744c440 2024-04-26T23:19:03.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.669+0000 7fef0278f740 10 client.4818 fetch_fsmap learned FSMap version 9 2024-04-26T23:19:03.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.669+0000 7fef0278f740 1 -- 172.21.15.47:0/2538879092 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x559e171626b0 con 0x559e1744c440 2024-04-26T23:19:03.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.669+0000 7feedcff9640 1 -- 172.21.15.47:0/2538879092 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x559e17394350 con 0x559e1744c440 2024-04-26T23:19:03.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.669+0000 7feedcff9640 1 -- 172.21.15.47:0/2538879092 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7feef0031020 con 0x559e1744c440 2024-04-26T23:19:03.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.669+0000 7fef0278f740 10 client.4818 fetch_fsmap finished waiting for FSMap version 9 2024-04-26T23:19:03.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.669+0000 7fef0278f740 10 client.4818 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T23:19:03.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.669+0000 7fef0278f740 20 client.4818 populate_metadata read hostname 'smithi047' 2024-04-26T23:19:03.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.669+0000 7fef0278f740 1 --2- 172.21.15.47:0/2538879092 >> [v2:172.21.15.203:6835/997115011,v1:172.21.15.203:6837/997115011] conn(0x559e17477d20 0x559e1745e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:03.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.669+0000 7fef0278f740 4 client.4818 mds_command: new command op to 4282 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:19:03.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.669+0000 7fef0278f740 1 -- 172.21.15.47:0/2538879092 --> [v2:172.21.15.203:6835/997115011,v1:172.21.15.203:6837/997115011] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e1738e1c0 con 0x559e17477d20 2024-04-26T23:19:03.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.669+0000 7feefd00d640 1 --2- 172.21.15.47:0/2538879092 >> [v2:172.21.15.203:6835/997115011,v1:172.21.15.203:6837/997115011] conn(0x559e17477d20 0x559e1745e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:03.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.670+0000 7feefd00d640 1 --2- 172.21.15.47:0/2538879092 >> [v2:172.21.15.203:6835/997115011,v1:172.21.15.203:6837/997115011] conn(0x559e17477d20 0x559e1745e170 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee4c7ace04f8ff0 server_cookie=323ce6ae2fa66f20 in_seq=0 out_seq=0 2024-04-26T23:19:03.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.670+0000 7feedcff9640 10 client.4818 ms_handle_connect on v2:172.21.15.203:6835/997115011 2024-04-26T23:19:03.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.670+0000 7feedcff9640 1 -- 172.21.15.47:0/2538879092 <== mds.0 v2:172.21.15.203:6835/997115011 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559e1738e1c0 con 0x559e17477d20 2024-04-26T23:19:03.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.671+0000 7feedcff9640 10 client.4818 handle_command_reply: tid=0 2024-04-26T23:19:03.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.671+0000 7fef0278f740 1 client.4818 shutdown 2024-04-26T23:19:03.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.671+0000 7fef0278f740 1 -- 172.21.15.47:0/2538879092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e1744c440 msgr2=0x559e1744fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:03.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.671+0000 7fef0278f740 1 --2- 172.21.15.47:0/2538879092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e1744c440 0x559e1744fe60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7feef000de80 tx=0x7feef0000ed0 comp rx=0 tx=0).stop 2024-04-26T23:19:03.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.671+0000 7fef0278f740 1 -- 172.21.15.47:0/2538879092 shutdown_connections 2024-04-26T23:19:03.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.671+0000 7fef0278f740 1 --2- 172.21.15.47:0/2538879092 >> [v2:172.21.15.203:6835/997115011,v1:172.21.15.203:6837/997115011] conn(0x559e17477d20 0x559e1745e170 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:03.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.671+0000 7fef0278f740 1 --2- 172.21.15.47:0/2538879092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559e17478a10 0x559e1745ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:03.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.671+0000 7fef0278f740 1 --2- 172.21.15.47:0/2538879092 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559e17469ce0 0x559e1745fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:03.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.671+0000 7feedcff9640 0 client.4818 ms_handle_reset on v2:172.21.15.203:6835/997115011 2024-04-26T23:19:03.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.671+0000 7fef0278f740 1 --2- 172.21.15.47:0/2538879092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e1744c440 0x559e1744fe60 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:03.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.671+0000 7fef0278f740 1 -- 172.21.15.47:0/2538879092 >> 172.21.15.47:0/2538879092 conn(0x559e17422120 msgr2=0x559e1746b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:03.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.671+0000 7fef0278f740 1 -- 172.21.15.47:0/2538879092 shutdown_connections 2024-04-26T23:19:03.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.671+0000 7fef0278f740 1 -- 172.21.15.47:0/2538879092 wait complete. 2024-04-26T23:19:03.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.671+0000 7fef0278f740 20 client.4818 trim_cache size 0 max 16384 2024-04-26T23:19:03.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.689+0000 7fef0278f740 1 Processor -- start 2024-04-26T23:19:03.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.689+0000 7fef0278f740 1 -- start start 2024-04-26T23:19:03.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.690+0000 7fef0278f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559e1744c440 0x559e1741b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:03.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.690+0000 7fef0278f740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559e17469ce0 0x559e1744a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:03.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.690+0000 7fef0278f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e17478a10 0x559e17478fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:03.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.690+0000 7fef0278f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7feef8201cd0 con 0x559e17478a10 2024-04-26T23:19:03.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.690+0000 7fef0278f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7feef8201030 con 0x559e1744c440 2024-04-26T23:19:03.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.690+0000 7fef0278f740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7feef8201680 con 0x559e17469ce0 2024-04-26T23:19:03.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.690+0000 7feefe00f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e17478a10 0x559e17478fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:03.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.690+0000 7feefe00f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e17478a10 0x559e17478fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54792/0 (socket says 172.21.15.47:54792) 2024-04-26T23:19:03.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.690+0000 7feefe00f640 1 -- 172.21.15.47:0/3241874165 learned_addr learned my addr 172.21.15.47:0/3241874165 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:03.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.690+0000 7feefd80e640 1 --2- 172.21.15.47:0/3241874165 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559e1744c440 0x559e1741b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:03.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.690+0000 7feefd00d640 1 --2- 172.21.15.47:0/3241874165 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559e17469ce0 0x559e1744a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:03.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.691+0000 7feefe00f640 1 -- 172.21.15.47:0/3241874165 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559e17469ce0 msgr2=0x559e1744a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:03.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.691+0000 7feefe00f640 1 --2- 172.21.15.47:0/3241874165 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559e17469ce0 0x559e1744a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:03.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.691+0000 7feefe00f640 1 -- 172.21.15.47:0/3241874165 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559e1744c440 msgr2=0x559e1741b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:03.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.691+0000 7feefe00f640 1 --2- 172.21.15.47:0/3241874165 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559e1744c440 0x559e1741b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:03.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.691+0000 7feefe00f640 1 -- 172.21.15.47:0/3241874165 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e1741df10 con 0x559e17478a10 2024-04-26T23:19:03.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.691+0000 7feefd80e640 1 --2- 172.21.15.47:0/3241874165 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559e1744c440 0x559e1741b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:19:03.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.691+0000 7feefe00f640 1 --2- 172.21.15.47:0/3241874165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e17478a10 0x559e17478fe0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7feee802a6a0 tx=0x7feee802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85c7edbd7a75060c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:03.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.691+0000 7feedeffd640 1 -- 172.21.15.47:0/3241874165 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee8003990 con 0x559e17478a10 2024-04-26T23:19:03.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.691+0000 7feedeffd640 1 -- 172.21.15.47:0/3241874165 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feee8003b30 con 0x559e17478a10 2024-04-26T23:19:03.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.691+0000 7feedeffd640 1 -- 172.21.15.47:0/3241874165 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee8004040 con 0x559e17478a10 2024-04-26T23:19:03.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.691+0000 7fef0278f740 1 -- 172.21.15.47:0/3241874165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e17478a10 msgr2=0x559e17478fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:03.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.691+0000 7fef0278f740 1 --2- 172.21.15.47:0/3241874165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e17478a10 0x559e17478fe0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7feee802a6a0 tx=0x7feee802e000 comp rx=0 tx=0).stop 2024-04-26T23:19:03.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.691+0000 7fef0278f740 1 -- 172.21.15.47:0/3241874165 shutdown_connections 2024-04-26T23:19:03.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.691+0000 7fef0278f740 1 --2- 172.21.15.47:0/3241874165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e17478a10 0x559e17478fe0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:03.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.691+0000 7fef0278f740 1 --2- 172.21.15.47:0/3241874165 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559e17469ce0 0x559e1744a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:03.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.691+0000 7fef0278f740 1 --2- 172.21.15.47:0/3241874165 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559e1744c440 0x559e1741b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:03.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.691+0000 7fef0278f740 1 -- 172.21.15.47:0/3241874165 >> 172.21.15.47:0/3241874165 conn(0x559e17422120 msgr2=0x559e17465a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:03.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.692+0000 7fef0278f740 1 -- 172.21.15.47:0/3241874165 shutdown_connections 2024-04-26T23:19:03.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.692+0000 7fef0278f740 1 -- 172.21.15.47:0/3241874165 wait complete. 2024-04-26T23:19:03.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.692+0000 7fef0278f740 1 -- start start 2024-04-26T23:19:03.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.692+0000 7fef0278f740 1 Processor -- start 2024-04-26T23:19:03.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.693+0000 7fef0278f740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559e1744c440 0x559e1747d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:03.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.693+0000 7fef0278f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559e17469ce0 0x559e1741b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:03.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.693+0000 7fef0278f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e17478a10 0x559e1741bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:03.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.693+0000 7fef0278f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x559e17477ba0 con 0x559e17478a10 2024-04-26T23:19:03.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.693+0000 7fef0278f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x559e17467470 con 0x559e17469ce0 2024-04-26T23:19:03.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.693+0000 7fef0278f740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x559e17478190 con 0x559e1744c440 2024-04-26T23:19:03.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.693+0000 7fef0278f740 10 client.0 _ll_register_callbacks cb 0x7feef0001dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:19:03.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.693+0000 7feefe00f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e17478a10 0x559e1741bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:03.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.693+0000 7feefe00f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e17478a10 0x559e1741bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54802/0 (socket says 172.21.15.47:54802) 2024-04-26T23:19:03.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.693+0000 7feefe00f640 1 -- 172.21.15.47:0/965011871 learned_addr learned my addr 172.21.15.47:0/965011871 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:03.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.693+0000 7feefd80e640 1 --2- 172.21.15.47:0/965011871 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559e1744c440 0x559e1747d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:03.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.693+0000 7feefd00d640 1 --2- 172.21.15.47:0/965011871 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559e17469ce0 0x559e1741b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:03.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.693+0000 7feefe00f640 1 -- 172.21.15.47:0/965011871 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559e1744c440 msgr2=0x559e1747d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:03.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.693+0000 7feefe00f640 1 --2- 172.21.15.47:0/965011871 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559e1744c440 0x559e1747d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:03.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.693+0000 7feefe00f640 1 -- 172.21.15.47:0/965011871 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559e17469ce0 msgr2=0x559e1741b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:03.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.693+0000 7feefe00f640 1 --2- 172.21.15.47:0/965011871 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559e17469ce0 0x559e1741b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:03.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.693+0000 7feefe00f640 1 -- 172.21.15.47:0/965011871 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e1747e600 con 0x559e17478a10 2024-04-26T23:19:03.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.694+0000 7feefe00f640 1 --2- 172.21.15.47:0/965011871 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e17478a10 0x559e1741bbb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7feee8009790 tx=0x7feee8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d078feccad5fb2f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:03.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.694+0000 7feedcff9640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-26T23:19:03.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.694+0000 7feedcff9640 1 -- 172.21.15.47:0/965011871 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee8003990 con 0x559e17478a10 2024-04-26T23:19:03.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.694+0000 7feedcff9640 1 -- 172.21.15.47:0/965011871 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7feee8043070 con 0x559e17478a10 2024-04-26T23:19:03.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.694+0000 7fef0278f740 1 -- 172.21.15.47:0/965011871 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feee8003b30 con 0x559e17478a10 2024-04-26T23:19:03.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.694+0000 7feedcff9640 1 -- 172.21.15.47:0/965011871 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7feee8057050 con 0x559e17478a10 2024-04-26T23:19:03.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.694+0000 7fef0278f740 10 client.4824 fetch_fsmap learned FSMap version 9 2024-04-26T23:19:03.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.694+0000 7fef0278f740 1 -- 172.21.15.47:0/965011871 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x559e174e2000 con 0x559e17478a10 2024-04-26T23:19:03.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.694+0000 7feedcff9640 1 -- 172.21.15.47:0/965011871 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feee8003b30 con 0x559e17478a10 2024-04-26T23:19:03.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.694+0000 7feedcff9640 1 -- 172.21.15.47:0/965011871 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7feee805a020 con 0x559e17478a10 2024-04-26T23:19:03.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.694+0000 7fef0278f740 10 client.4824 fetch_fsmap finished waiting for FSMap version 9 2024-04-26T23:19:03.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.694+0000 7fef0278f740 10 client.4824 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T23:19:03.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.694+0000 7fef0278f740 20 client.4824 populate_metadata read hostname 'smithi047' 2024-04-26T23:19:03.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.694+0000 7fef0278f740 1 --2- 172.21.15.47:0/965011871 >> [v2:172.21.15.203:6835/997115011,v1:172.21.15.203:6837/997115011] conn(0x559e174eb110 0x559e174ed500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:03.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.694+0000 7fef0278f740 4 client.4824 mds_command: new command op to 4282 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T23:19:03.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.694+0000 7fef0278f740 1 -- 172.21.15.47:0/965011871 --> [v2:172.21.15.203:6835/997115011,v1:172.21.15.203:6837/997115011] -- command(tid 0: {"prefix": "status"}) -- 0x559e173099a0 con 0x559e174eb110 2024-04-26T23:19:03.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.694+0000 7feefd80e640 1 --2- 172.21.15.47:0/965011871 >> [v2:172.21.15.203:6835/997115011,v1:172.21.15.203:6837/997115011] conn(0x559e174eb110 0x559e174ed500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:03.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.695+0000 7feefd80e640 1 --2- 172.21.15.47:0/965011871 >> [v2:172.21.15.203:6835/997115011,v1:172.21.15.203:6837/997115011] conn(0x559e174eb110 0x559e174ed500 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c687a312cbd12ca server_cookie=adabe23e2877df41 in_seq=0 out_seq=0 2024-04-26T23:19:03.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.695+0000 7feedcff9640 10 client.4824 ms_handle_connect on v2:172.21.15.203:6835/997115011 2024-04-26T23:19:03.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.695+0000 7feedcff9640 1 -- 172.21.15.47:0/965011871 <== mds.0 v2:172.21.15.203:6835/997115011 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x559e173099a0 con 0x559e174eb110 2024-04-26T23:19:03.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.695+0000 7feedcff9640 10 client.4824 handle_command_reply: tid=0 2024-04-26T23:19:03.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.695+0000 7fef0278f740 1 client.4824 shutdown 2024-04-26T23:19:03.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.695+0000 7fef0278f740 1 -- 172.21.15.47:0/965011871 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e17478a10 msgr2=0x559e1741bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:03.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.695+0000 7fef0278f740 1 --2- 172.21.15.47:0/965011871 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e17478a10 0x559e1741bbb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7feee8009790 tx=0x7feee8004ab0 comp rx=0 tx=0).stop 2024-04-26T23:19:03.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.695+0000 7fef0278f740 1 -- 172.21.15.47:0/965011871 shutdown_connections 2024-04-26T23:19:03.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.695+0000 7fef0278f740 1 --2- 172.21.15.47:0/965011871 >> [v2:172.21.15.203:6835/997115011,v1:172.21.15.203:6837/997115011] conn(0x559e174eb110 0x559e174ed500 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:03.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.695+0000 7fef0278f740 1 --2- 172.21.15.47:0/965011871 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e17478a10 0x559e1741bbb0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:03.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.695+0000 7fef0278f740 1 --2- 172.21.15.47:0/965011871 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559e17469ce0 0x559e1741b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:03.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.695+0000 7fef0278f740 1 --2- 172.21.15.47:0/965011871 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559e1744c440 0x559e1747d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:03.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.695+0000 7feedcff9640 0 client.4824 ms_handle_reset on v2:172.21.15.203:6835/997115011 2024-04-26T23:19:03.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.695+0000 7fef0278f740 1 -- 172.21.15.47:0/965011871 >> 172.21.15.47:0/965011871 conn(0x559e17422120 msgr2=0x559e17469130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:03.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.695+0000 7fef0278f740 1 -- 172.21.15.47:0/965011871 shutdown_connections 2024-04-26T23:19:03.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.695+0000 7fef0278f740 1 -- 172.21.15.47:0/965011871 wait complete. 2024-04-26T23:19:03.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.695+0000 7fef0278f740 20 client.4824 trim_cache size 0 max 16384 2024-04-26T23:19:03.706 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-26T23:19:03.707 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "399006d3-4db1-43e6-ba81-a79a276eabfd", 2024-04-26T23:19:03.707 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 0, 2024-04-26T23:19:03.707 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 4282, 2024-04-26T23:19:03.707 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-04-26T23:19:03.707 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-04-26T23:19:03.707 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-04-26T23:19:03.707 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 31.704663799999999, 2024-04-26T23:19:03.707 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 9, 2024-04-26T23:19:03.707 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 24, 2024-04-26T23:19:03.707 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 24, 2024-04-26T23:19:03.707 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 39.170882796000001 2024-04-26T23:19:03.707 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-26T23:19:03.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.696+0000 7feeffa72640 1 -- 172.21.15.47:0/3620338671 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7feee406c8f0 msgr2=0x7feee406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:03.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.696+0000 7feeffa72640 1 --2- 172.21.15.47:0/3620338671 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7feee406c8f0 0x7feee406edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7feef815ea10 tx=0x7feef0009040 comp rx=0 tx=0).stop 2024-04-26T23:19:03.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.696+0000 7feeffa72640 1 -- 172.21.15.47:0/3620338671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feef80d3b80 msgr2=0x7feef80a17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:03.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.696+0000 7feeffa72640 1 --2- 172.21.15.47:0/3620338671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feef80d3b80 0x7feef80a17d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7feeec00c770 tx=0x7feeec00cc30 comp rx=0 tx=0).stop 2024-04-26T23:19:03.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.696+0000 7feeffa72640 1 -- 172.21.15.47:0/3620338671 shutdown_connections 2024-04-26T23:19:03.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.696+0000 7feeffa72640 1 --2- 172.21.15.47:0/3620338671 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7feee406c8f0 0x7feee406edb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:03.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.696+0000 7feeffa72640 1 --2- 172.21.15.47:0/3620338671 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feef815f540 0x7feef80b8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:03.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.696+0000 7feeffa72640 1 --2- 172.21.15.47:0/3620338671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feef80d3b80 0x7feef80a17d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:03.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.696+0000 7feeffa72640 1 --2- 172.21.15.47:0/3620338671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feef80b3ff0 0x7feef80a1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:03.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.696+0000 7feeffa72640 1 -- 172.21.15.47:0/3620338671 >> 172.21.15.47:0/3620338671 conn(0x7feef80d4f90 msgr2=0x7feef80b2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:03.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.696+0000 7feeffa72640 1 -- 172.21.15.47:0/3620338671 shutdown_connections 2024-04-26T23:19:03.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:03.696+0000 7feeffa72640 1 -- 172.21.15.47:0/3620338671 wait complete. 2024-04-26T23:19:03.711 INFO:teuthology.run_tasks:Running task sequential... 2024-04-26T23:19:03.733 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-26T23:19:03.733 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-26T23:19:03.734 INFO:teuthology.task.print:Enabling mgr modules 2024-04-26T23:19:03.734 INFO:teuthology.run_tasks:Running task kclient... 2024-04-26T23:19:03.783 INFO:tasks.kclient:Mounting kernel clients... 2024-04-26T23:19:03.783 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-26T23:19:03.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:03.783 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-04-26T23:19:03.800 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-26T23:19:03.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:03.801 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T23:19:03.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:03.865 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link delete ceph-brx 2024-04-26T23:19:03.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:03.957 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-04-26T23:19:03.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:03.973 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link delete ceph-brx 2024-04-26T23:19:04.041 INFO:teuthology.orchestra.run.smithi047.stderr:Cannot find device "ceph-brx" 2024-04-26T23:19:04.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T23:19:04.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:04.043 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list 2024-04-26T23:19:04.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:04.061 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link delete ceph-brx 2024-04-26T23:19:04.131 INFO:teuthology.orchestra.run.smithi203.stderr:Cannot find device "ceph-brx" 2024-04-26T23:19:04.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T23:19:04.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:04.133 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list 2024-04-26T23:19:04.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:04.192 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link delete ceph-brx 2024-04-26T23:19:04.261 INFO:teuthology.orchestra.run.smithi203.stderr:Cannot find device "ceph-brx" 2024-04-26T23:19:04.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T23:19:04.263 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-04-26T23:19:04.263 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T23:19:04.264 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T23:19:04.264 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T23:19:04.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.365+0000 7f25185e7640 1 Processor -- start 2024-04-26T23:19:04.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.365+0000 7f25185e7640 1 -- start start 2024-04-26T23:19:04.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.366+0000 7f25185e7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f251006f790 0x7f251006fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:04.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.366+0000 7f25185e7640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2510072c50 0x7f25100730c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:04.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.366+0000 7f25185e7640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2510156840 con 0x7f2510156460 2024-04-26T23:19:04.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.366+0000 7f25185e7640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f25100cbf20 con 0x7f2510072c50 2024-04-26T23:19:04.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.366+0000 7f25185e7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f25100cc200 con 0x7f251006f790 2024-04-26T23:19:04.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.366+0000 7f2516b84640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2510072c50 0x7f25100730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:04.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.366+0000 7f2516b84640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2510072c50 0x7f25100730c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54818/0 (socket says 172.21.15.47:54818) 2024-04-26T23:19:04.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.366+0000 7f2516b84640 1 -- 172.21.15.47:0/1285253310 learned_addr learned my addr 172.21.15.47:0/1285253310 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:04.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.367+0000 7f2516b84640 1 -- 172.21.15.47:0/1285253310 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f251006f790 msgr2=0x7f251006fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:04.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.367+0000 7f2516b84640 1 --2- 172.21.15.47:0/1285253310 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f251006f790 0x7f251006fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:04.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.367+0000 7f2516b84640 1 -- 172.21.15.47:0/1285253310 >> v1:172.21.15.203:6789/0 conn(0x7f2510156460 legacy=0x7f25100cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:04.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.367+0000 7f2516b84640 1 -- 172.21.15.47:0/1285253310 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25100cc4e0 con 0x7f2510072c50 2024-04-26T23:19:04.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.367+0000 7f2516b84640 1 --2- 172.21.15.47:0/1285253310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2510072c50 0x7f25100730c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f250800bb10 tx=0x7f250802c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15b6bf8acada260 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:04.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.368+0000 7f2515381640 1 -- 172.21.15.47:0/1285253310 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2508032790 con 0x7f2510072c50 2024-04-26T23:19:04.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.368+0000 7f2515381640 1 -- 172.21.15.47:0/1285253310 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2508038070 con 0x7f2510072c50 2024-04-26T23:19:04.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.368+0000 7f2515381640 1 -- 172.21.15.47:0/1285253310 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2508032b50 con 0x7f2510072c50 2024-04-26T23:19:04.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.368+0000 7f25185e7640 1 -- 172.21.15.47:0/1285253310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2510072c50 msgr2=0x7f25100730c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:04.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.368+0000 7f25185e7640 1 --2- 172.21.15.47:0/1285253310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2510072c50 0x7f25100730c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f250800bb10 tx=0x7f250802c8d0 comp rx=0 tx=0).stop 2024-04-26T23:19:04.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.368+0000 7f25185e7640 1 -- 172.21.15.47:0/1285253310 shutdown_connections 2024-04-26T23:19:04.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.368+0000 7f25185e7640 1 --2- 172.21.15.47:0/1285253310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2510072c50 0x7f25100730c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:04.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.368+0000 7f25185e7640 1 --2- 172.21.15.47:0/1285253310 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f251006f790 0x7f251006fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:04.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.368+0000 7f25185e7640 1 -- 172.21.15.47:0/1285253310 >> 172.21.15.47:0/1285253310 conn(0x7f25100d0be0 msgr2=0x7f25100d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:04.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.368+0000 7f25185e7640 1 -- 172.21.15.47:0/1285253310 shutdown_connections 2024-04-26T23:19:04.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.368+0000 7f25185e7640 1 -- 172.21.15.47:0/1285253310 wait complete. 2024-04-26T23:19:04.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.369+0000 7f25185e7640 1 Processor -- start 2024-04-26T23:19:04.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.369+0000 7f25185e7640 1 -- start start 2024-04-26T23:19:04.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.370+0000 7f25185e7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f251006f790 0x7f25101f4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:04.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.370+0000 7f25185e7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2510072c50 0x7f2510154ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:04.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.370+0000 7f25185e7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2510156460 0x7f25100b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:04.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.370+0000 7f25185e7640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f25100cbbe0 con 0x7f2510156460 2024-04-26T23:19:04.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.370+0000 7f25185e7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2510072700 con 0x7f251006f790 2024-04-26T23:19:04.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.370+0000 7f25185e7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f251006e260 con 0x7f2510072c50 2024-04-26T23:19:04.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.370+0000 7f2516b84640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2510156460 0x7f25100b3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:04.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.370+0000 7f2516b84640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2510156460 0x7f25100b3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54834/0 (socket says 172.21.15.47:54834) 2024-04-26T23:19:04.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.370+0000 7f2516b84640 1 -- 172.21.15.47:0/1822633164 learned_addr learned my addr 172.21.15.47:0/1822633164 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:04.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.371+0000 7f2516383640 1 --2- 172.21.15.47:0/1822633164 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f251006f790 0x7f25101f4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:04.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.371+0000 7f2515b82640 1 --2- 172.21.15.47:0/1822633164 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2510072c50 0x7f2510154ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:04.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.371+0000 7f2516b84640 1 -- 172.21.15.47:0/1822633164 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2510072c50 msgr2=0x7f2510154ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:04.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.371+0000 7f2516b84640 1 --2- 172.21.15.47:0/1822633164 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2510072c50 0x7f2510154ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:04.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.371+0000 7f2516b84640 1 -- 172.21.15.47:0/1822633164 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f251006f790 msgr2=0x7f25101f4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:04.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.371+0000 7f2516b84640 1 --2- 172.21.15.47:0/1822633164 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f251006f790 0x7f25101f4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:04.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.371+0000 7f2516b84640 1 -- 172.21.15.47:0/1822633164 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2510153f50 con 0x7f2510156460 2024-04-26T23:19:04.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.371+0000 7f2515b82640 1 --2- 172.21.15.47:0/1822633164 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2510072c50 0x7f2510154ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:19:04.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.371+0000 7f2516b84640 1 --2- 172.21.15.47:0/1822633164 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2510156460 0x7f25100b3fc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f250802c910 tx=0x7f25080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:04.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.371+0000 7f250effd640 1 -- 172.21.15.47:0/1822633164 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2508033620 con 0x7f2510156460 2024-04-26T23:19:04.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.371+0000 7f250effd640 1 -- 172.21.15.47:0/1822633164 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25080337e0 con 0x7f2510156460 2024-04-26T23:19:04.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.371+0000 7f250effd640 1 -- 172.21.15.47:0/1822633164 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2508033bd0 con 0x7f2510156460 2024-04-26T23:19:04.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.371+0000 7f25185e7640 1 -- 172.21.15.47:0/1822633164 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2510154230 con 0x7f2510156460 2024-04-26T23:19:04.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.371+0000 7f25185e7640 1 -- 172.21.15.47:0/1822633164 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25101547f0 con 0x7f2510156460 2024-04-26T23:19:04.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.372+0000 7f25185e7640 1 -- 172.21.15.47:0/1822633164 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25100694f0 con 0x7f2510156460 2024-04-26T23:19:04.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.373+0000 7f250effd640 1 -- 172.21.15.47:0/1822633164 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f250804d060 con 0x7f2510156460 2024-04-26T23:19:04.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.373+0000 7f250effd640 1 --2- 172.21.15.47:0/1822633164 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f24fc06c8f0 0x7f24fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:04.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.373+0000 7f250effd640 1 -- 172.21.15.47:0/1822633164 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f25080aeb30 con 0x7f2510156460 2024-04-26T23:19:04.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.373+0000 7f2516383640 1 --2- 172.21.15.47:0/1822633164 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f24fc06c8f0 0x7f24fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:04.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.374+0000 7f2516383640 1 --2- 172.21.15.47:0/1822633164 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f24fc06c8f0 0x7f24fc06edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2500009f20 tx=0x7f25000094d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:04.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.377+0000 7f250effd640 1 -- 172.21.15.47:0/1822633164 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2508041070 con 0x7f2510156460 2024-04-26T23:19:04.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.601+0000 7f25185e7640 1 -- 172.21.15.47:0/1822633164 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f251006de80 con 0x7f2510156460 2024-04-26T23:19:04.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.601+0000 7f250effd640 1 -- 172.21.15.47:0/1822633164 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f250807ccb0 con 0x7f2510156460 2024-04-26T23:19:04.603 INFO:teuthology.orchestra.run.smithi047.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-26T23:19:04.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.603+0000 7f25185e7640 1 -- 172.21.15.47:0/1822633164 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f24fc06c8f0 msgr2=0x7f24fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:04.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.603+0000 7f25185e7640 1 --2- 172.21.15.47:0/1822633164 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f24fc06c8f0 0x7f24fc06edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2500009f20 tx=0x7f25000094d0 comp rx=0 tx=0).stop 2024-04-26T23:19:04.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.603+0000 7f25185e7640 1 -- 172.21.15.47:0/1822633164 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2510156460 msgr2=0x7f25100b3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:04.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.603+0000 7f25185e7640 1 --2- 172.21.15.47:0/1822633164 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2510156460 0x7f25100b3fc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f250802c910 tx=0x7f25080046c0 comp rx=0 tx=0).stop 2024-04-26T23:19:04.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.603+0000 7f25185e7640 1 -- 172.21.15.47:0/1822633164 shutdown_connections 2024-04-26T23:19:04.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.603+0000 7f25185e7640 1 --2- 172.21.15.47:0/1822633164 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f24fc06c8f0 0x7f24fc06edb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:04.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.603+0000 7f25185e7640 1 --2- 172.21.15.47:0/1822633164 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2510156460 0x7f25100b3fc0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:04.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.603+0000 7f25185e7640 1 --2- 172.21.15.47:0/1822633164 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2510072c50 0x7f2510154ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:04.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.603+0000 7f25185e7640 1 --2- 172.21.15.47:0/1822633164 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f251006f790 0x7f25101f4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:04.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.603+0000 7f25185e7640 1 -- 172.21.15.47:0/1822633164 >> 172.21.15.47:0/1822633164 conn(0x7f25100d0be0 msgr2=0x7f251014f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:04.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.603+0000 7f25185e7640 1 -- 172.21.15.47:0/1822633164 shutdown_connections 2024-04-26T23:19:04.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04.603+0000 7f25185e7640 1 -- 172.21.15.47:0/1822633164 wait complete. 2024-04-26T23:19:04.616 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T23:19:04.616 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T23:19:04.616 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi047.front.sepia.ceph.com 2024-04-26T23:19:04.616 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T23:19:04.616 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T23:19:04.616 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T23:19:04.616 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T23:19:04.616 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T23:19:04.616 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:19:04.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:04.617 DEBUG:teuthology.orchestra.run.smithi047:> ip addr 2024-04-26T23:19:04.633 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T23:19:04.633 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T23:19:04.634 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T23:19:04.634 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:19:04.634 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2024-04-26T23:19:04.634 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:19:04.634 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:19:04.634 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:19:04.635 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f0 2024-04-26T23:19:04.635 INFO:teuthology.orchestra.run.smithi047.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:19:04.635 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:19:04.635 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f1 2024-04-26T23:19:04.636 INFO:teuthology.orchestra.run.smithi047.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T23:19:04.636 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2024-04-26T23:19:04.636 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f0 2024-04-26T23:19:04.636 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-26T23:19:04.636 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42654sec preferred_lft 42654sec 2024-04-26T23:19:04.636 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link noprefixroute 2024-04-26T23:19:04.637 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:19:04.637 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:19:04.637 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2024-04-26T23:19:04.637 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f1 2024-04-26T23:19:04.637 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T23:19:04.638 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:19:04.638 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-04-26T23:19:04.638 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link add name ceph-brx type bridge 2024-04-26T23:19:04.638 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip addr flush dev ceph-brx 2024-04-26T23:19:04.638 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set ceph-brx up 2024-04-26T23:19:04.638 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T23:19:04.638 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-04-26T23:19:04.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:19:04.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:19:04.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:04.773 DEBUG:teuthology.orchestra.run.smithi047:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T23:19:04.796 INFO:teuthology.orchestra.run.smithi047.stdout:1 2024-04-26T23:19:04.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:04.798 DEBUG:teuthology.orchestra.run.smithi047:> route 2024-04-26T23:19:04.855 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2024-04-26T23:19:04.855 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T23:19:04.855 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T23:19:04.856 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T23:19:04.856 INFO:teuthology.orchestra.run.smithi047.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T23:19:04.856 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:19:04.856 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-04-26T23:19:04.856 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T23:19:04.857 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T23:19:04.857 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T23:19:04.857 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-04-26T23:19:04.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:19:04.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:19:04.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:04.982 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-04-26T23:19:04.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:04.999 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list-id 2024-04-26T23:19:05.054 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:19:05.055 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-04-26T23:19:05.055 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T23:19:05.055 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-26T23:19:05.055 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-04-26T23:19:05.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:19:05.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:19:05.173 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-26T23:19:05.173 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:19:05.173 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-04-26T23:19:05.173 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-26T23:19:05.173 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T23:19:05.173 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-26T23:19:05.173 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-26T23:19:05.173 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-26T23:19:05.173 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-04-26T23:19:05.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:19:05.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:19:05.285 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:19:05.285 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-04-26T23:19:05.285 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set brx.0 up 2024-04-26T23:19:05.285 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T23:19:05.285 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-04-26T23:19:05.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:19:05.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:19:05.362 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:19:05.362 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T23:19:05.377 INFO:teuthology.orchestra.run.smithi047.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:19:05.377 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:19:05.377 DEBUG:teuthology.orchestra.run.smithi047:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T23:19:05.432 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T23:19:05.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:05.433 DEBUG:teuthology.orchestra.run.smithi047:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:19:05.515 INFO:teuthology.orchestra.run.smithi047.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:19:05.515 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:19:05.515 INFO:teuthology.orchestra.run.smithi047.stdout:invalid new device string format 2024-04-26T23:19:05.614 INFO:teuthology.orchestra.run.smithi047.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:19:05.615 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:19:05.615 INFO:teuthology.orchestra.run.smithi047.stdout:invalid new device string format 2024-04-26T23:19:05.615 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: resolved to: "172.21.15.47:3300,172.21.15.203:3300,172.21.15.203:3301" 2024-04-26T23:19:05.615 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:19:05.620 INFO:teuthology.orchestra.run.smithi047.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:19:05.620 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:19:05.620 INFO:teuthology.orchestra.run.smithi047.stdout:invalid new device string format 2024-04-26T23:19:05.620 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: resolved to: "172.21.15.47:3300,172.21.15.203:3300,172.21.15.203:3301" 2024-04-26T23:19:05.620 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:19:05.620 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: resolved to: "172.21.15.47:6789,172.21.15.203:6789,172.21.15.203:6790" 2024-04-26T23:19:05.620 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.47:6789,172.21.15.203:6789,172.21.15.203:6790:/ 2024-04-26T23:19:05.620 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=399006d3-4db1-43e6-ba81-a79a276eabfd" will pass to kernel 2024-04-26T23:19:05.621 INFO:teuthology.orchestra.run.smithi047.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T23:19:05.621 INFO:teuthology.orchestra.run.smithi047.stdout: You just mounted a file system that supports labels which does not 2024-04-26T23:19:05.621 INFO:teuthology.orchestra.run.smithi047.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T23:19:05.621 INFO:teuthology.orchestra.run.smithi047.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T23:19:05.621 INFO:teuthology.orchestra.run.smithi047.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T23:19:05.623 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T23:19:05.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:05.623 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T23:19:05.688 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T23:19:05.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:05 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:19:05.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:19:05.786 INFO:teuthology.orchestra.run.smithi047.stdout:client14477 2024-04-26T23:19:05.786 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:19:05.786 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:19:05.802 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T23:19:05.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:05 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:19:05.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:19:05.899 INFO:teuthology.orchestra.run.smithi047.stdout:client14477 2024-04-26T23:19:05.899 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T23:19:05.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:05 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:19:05.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:19:05.955 INFO:teuthology.orchestra.run.smithi047.stdout:client14477 2024-04-26T23:19:05.955 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/399006d3-4db1-43e6-ba81-a79a276eabfd.client14477/status') 2024-04-26T23:19:06.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:19:06.014 INFO:teuthology.orchestra.run.smithi047.stdout:instance: client.14477 (3)172.21.15.47:0/929233429 2024-04-26T23:19:06.014 INFO:teuthology.orchestra.run.smithi047.stdout:blocklisted: false 2024-04-26T23:19:06.015 INFO:teuthology.orchestra.run.smithi047.stderr:0+1 records in 2024-04-26T23:19:06.015 INFO:teuthology.orchestra.run.smithi047.stderr:0+1 records out 2024-04-26T23:19:06.015 INFO:teuthology.orchestra.run.smithi047.stderr:70 bytes copied, 2.3734e-05 s, 2.9 MB/s 2024-04-26T23:19:06.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:19:06.021 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-04-26T23:19:06.022 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T23:19:06.022 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-26T23:19:06.022 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T23:19:06.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.122+0000 7fbbd77f7640 1 Processor -- start 2024-04-26T23:19:06.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.122+0000 7fbbd77f7640 1 -- start start 2024-04-26T23:19:06.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.123+0000 7fbbd77f7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbbd0162880 0x7fbbd00d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:06.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.123+0000 7fbbd77f7640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbbd00b5f20 0x7fbbd00d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:06.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.123+0000 7fbbd77f7640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbbd00d39f0 0x7fbbd00b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:06.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.123+0000 7fbbd77f7640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fbbd00d0580 con 0x7fbbd00b5f20 2024-04-26T23:19:06.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.123+0000 7fbbd77f7640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fbbd00b7e50 con 0x7fbbd00d39f0 2024-04-26T23:19:06.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.123+0000 7fbbd77f7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fbbd00d1800 con 0x7fbbd0162880 2024-04-26T23:19:06.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.123+0000 7fbbd4d92640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbbd00b5f20 0x7fbbd00d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:06.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.123+0000 7fbbd4d92640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbbd00b5f20 0x7fbbd00d34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54844/0 (socket says 172.21.15.47:54844) 2024-04-26T23:19:06.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.123+0000 7fbbd4d92640 1 -- 172.21.15.47:0/785467458 learned_addr learned my addr 172.21.15.47:0/785467458 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:06.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.123+0000 7fbbd5d94640 1 --2- 172.21.15.47:0/785467458 >> v2:172.21.15.203:3300/0 conn(0x7fbbd00d39f0 0x7fbbd00b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:06.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.123+0000 7fbbd4d92640 1 -- 172.21.15.47:0/785467458 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbbd0162880 msgr2=0x7fbbd00d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:06.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.123+0000 7fbbd5593640 1 --2- 172.21.15.47:0/785467458 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbbd0162880 0x7fbbd00d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:06.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.123+0000 7fbbd4d92640 1 --2- 172.21.15.47:0/785467458 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbbd0162880 0x7fbbd00d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:06.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.123+0000 7fbbd4d92640 1 -- 172.21.15.47:0/785467458 >> v2:172.21.15.203:3300/0 conn(0x7fbbd00d39f0 msgr2=0x7fbbd00b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:06.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.123+0000 7fbbd4d92640 1 --2- 172.21.15.47:0/785467458 >> v2:172.21.15.203:3300/0 conn(0x7fbbd00d39f0 0x7fbbd00b69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:06.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.123+0000 7fbbd4d92640 1 -- 172.21.15.47:0/785467458 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbd00d07d0 con 0x7fbbd00b5f20 2024-04-26T23:19:06.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.123+0000 7fbbd5d94640 1 --2- 172.21.15.47:0/785467458 >> v2:172.21.15.203:3300/0 conn(0x7fbbd00d39f0 0x7fbbd00b69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:19:06.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.124+0000 7fbbd4d92640 1 --2- 172.21.15.47:0/785467458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbd00b5f20 0x7fbbd00d34b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fbbd008f770 tx=0x7fbbc002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=844b1a92ac50f917 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:06.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.124+0000 7fbbc7fff640 1 -- 172.21.15.47:0/785467458 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0037070 con 0x7fbbd00b5f20 2024-04-26T23:19:06.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.124+0000 7fbbc7fff640 1 -- 172.21.15.47:0/785467458 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbc0032440 con 0x7fbbd00b5f20 2024-04-26T23:19:06.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.124+0000 7fbbc7fff640 1 -- 172.21.15.47:0/785467458 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0032810 con 0x7fbbd00b5f20 2024-04-26T23:19:06.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.125+0000 7fbbd77f7640 1 -- 172.21.15.47:0/785467458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbd00b5f20 msgr2=0x7fbbd00d34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:06.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.125+0000 7fbbd77f7640 1 --2- 172.21.15.47:0/785467458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbd00b5f20 0x7fbbd00d34b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fbbd008f770 tx=0x7fbbc002a1e0 comp rx=0 tx=0).stop 2024-04-26T23:19:06.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.125+0000 7fbbd77f7640 1 -- 172.21.15.47:0/785467458 shutdown_connections 2024-04-26T23:19:06.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.125+0000 7fbbd77f7640 1 --2- 172.21.15.47:0/785467458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbd00b5f20 0x7fbbd00d34b0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:06.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.125+0000 7fbbd77f7640 1 --2- 172.21.15.47:0/785467458 >> v2:172.21.15.203:3300/0 conn(0x7fbbd00d39f0 0x7fbbd00b69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:06.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.125+0000 7fbbd77f7640 1 --2- 172.21.15.47:0/785467458 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbbd0162880 0x7fbbd00d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:06.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.125+0000 7fbbd77f7640 1 -- 172.21.15.47:0/785467458 >> 172.21.15.47:0/785467458 conn(0x7fbbd00d4f60 msgr2=0x7fbbd00d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:06.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.125+0000 7fbbd77f7640 1 -- 172.21.15.47:0/785467458 shutdown_connections 2024-04-26T23:19:06.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.125+0000 7fbbd77f7640 1 -- 172.21.15.47:0/785467458 wait complete. 2024-04-26T23:19:06.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.126+0000 7fbbd77f7640 1 Processor -- start 2024-04-26T23:19:06.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.126+0000 7fbbd77f7640 1 -- start start 2024-04-26T23:19:06.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.127+0000 7fbbd77f7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbbd00b5f20 0x7fbbd0153550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:06.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.127+0000 7fbbd77f7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbd00d39f0 0x7fbbd00b2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:06.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.127+0000 7fbbd77f7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbbd0162880 0x7fbbd00b2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:06.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.127+0000 7fbbd77f7640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd0162c60 con 0x7fbbd00d39f0 2024-04-26T23:19:06.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.127+0000 7fbbd77f7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd00d0d90 con 0x7fbbd00b5f20 2024-04-26T23:19:06.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.127+0000 7fbbd77f7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fbbd0162de0 con 0x7fbbd0162880 2024-04-26T23:19:06.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.127+0000 7fbbd4d92640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbd00d39f0 0x7fbbd00b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:06.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.127+0000 7fbbd4d92640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbd00d39f0 0x7fbbd00b2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54848/0 (socket says 172.21.15.47:54848) 2024-04-26T23:19:06.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.127+0000 7fbbd4d92640 1 -- 172.21.15.47:0/2404645910 learned_addr learned my addr 172.21.15.47:0/2404645910 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:06.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.127+0000 7fbbd5593640 1 --2- 172.21.15.47:0/2404645910 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbbd00b5f20 0x7fbbd0153550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:06.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.127+0000 7fbbd5d94640 1 --2- 172.21.15.47:0/2404645910 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbbd0162880 0x7fbbd00b2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:06.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.127+0000 7fbbd4d92640 1 -- 172.21.15.47:0/2404645910 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbbd0162880 msgr2=0x7fbbd00b2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:06.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.127+0000 7fbbd4d92640 1 --2- 172.21.15.47:0/2404645910 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbbd0162880 0x7fbbd00b2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:06.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.127+0000 7fbbd4d92640 1 -- 172.21.15.47:0/2404645910 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbbd00b5f20 msgr2=0x7fbbd0153550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:06.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.127+0000 7fbbd4d92640 1 --2- 172.21.15.47:0/2404645910 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbbd00b5f20 0x7fbbd0153550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:06.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.127+0000 7fbbd4d92640 1 -- 172.21.15.47:0/2404645910 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbd00cc1f0 con 0x7fbbd00d39f0 2024-04-26T23:19:06.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.127+0000 7fbbd5d94640 1 --2- 172.21.15.47:0/2404645910 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbbd0162880 0x7fbbd00b2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:19:06.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.127+0000 7fbbd5593640 1 --2- 172.21.15.47:0/2404645910 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbbd00b5f20 0x7fbbd0153550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:19:06.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.127+0000 7fbbd4d92640 1 --2- 172.21.15.47:0/2404645910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbd00d39f0 0x7fbbd00b2560 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0000c00 tx=0x7fbbc00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:06.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.127+0000 7fbbc5ffb640 1 -- 172.21.15.47:0/2404645910 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0041070 con 0x7fbbd00d39f0 2024-04-26T23:19:06.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.127+0000 7fbbc5ffb640 1 -- 172.21.15.47:0/2404645910 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbc003c870 con 0x7fbbd00d39f0 2024-04-26T23:19:06.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.127+0000 7fbbd77f7640 1 -- 172.21.15.47:0/2404645910 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbd00cc4d0 con 0x7fbbd00d39f0 2024-04-26T23:19:06.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.127+0000 7fbbc5ffb640 1 -- 172.21.15.47:0/2404645910 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0037040 con 0x7fbbd00d39f0 2024-04-26T23:19:06.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.128+0000 7fbbd77f7640 1 -- 172.21.15.47:0/2404645910 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbd00b2aa0 con 0x7fbbd00d39f0 2024-04-26T23:19:06.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.128+0000 7fbbd77f7640 1 -- 172.21.15.47:0/2404645910 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbd00694f0 con 0x7fbbd00d39f0 2024-04-26T23:19:06.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.129+0000 7fbbc5ffb640 1 -- 172.21.15.47:0/2404645910 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbbc0044070 con 0x7fbbd00d39f0 2024-04-26T23:19:06.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.129+0000 7fbbc5ffb640 1 --2- 172.21.15.47:0/2404645910 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fbbb406c8f0 0x7fbbb406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:06.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.130+0000 7fbbc5ffb640 1 -- 172.21.15.47:0/2404645910 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fbbc00acaf0 con 0x7fbbd00d39f0 2024-04-26T23:19:06.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.130+0000 7fbbd5593640 1 --2- 172.21.15.47:0/2404645910 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fbbb406c8f0 0x7fbbb406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:06.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.130+0000 7fbbd5593640 1 --2- 172.21.15.47:0/2404645910 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fbbb406c8f0 0x7fbbb406edb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0068ef0 tx=0x7fbbcc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:06.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.134+0000 7fbbc5ffb640 1 -- 172.21.15.47:0/2404645910 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbbc007ad90 con 0x7fbbd00d39f0 2024-04-26T23:19:06.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.358+0000 7fbbd77f7640 1 -- 172.21.15.47:0/2404645910 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fbbd0069d60 con 0x7fbbd00d39f0 2024-04-26T23:19:06.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.359+0000 7fbbc5ffb640 1 -- 172.21.15.47:0/2404645910 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7fbbc007a690 con 0x7fbbd00d39f0 2024-04-26T23:19:06.360 INFO:teuthology.orchestra.run.smithi047.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-26T23:19:06.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.360+0000 7fbbd77f7640 1 -- 172.21.15.47:0/2404645910 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fbbb406c8f0 msgr2=0x7fbbb406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:06.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.360+0000 7fbbd77f7640 1 --2- 172.21.15.47:0/2404645910 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fbbb406c8f0 0x7fbbb406edb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0068ef0 tx=0x7fbbcc008040 comp rx=0 tx=0).stop 2024-04-26T23:19:06.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.360+0000 7fbbd77f7640 1 -- 172.21.15.47:0/2404645910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbd00d39f0 msgr2=0x7fbbd00b2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:06.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.360+0000 7fbbd77f7640 1 --2- 172.21.15.47:0/2404645910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbd00d39f0 0x7fbbd00b2560 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0000c00 tx=0x7fbbc00046c0 comp rx=0 tx=0).stop 2024-04-26T23:19:06.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.361+0000 7fbbd77f7640 1 -- 172.21.15.47:0/2404645910 shutdown_connections 2024-04-26T23:19:06.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.361+0000 7fbbd77f7640 1 --2- 172.21.15.47:0/2404645910 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fbbb406c8f0 0x7fbbb406edb0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:06.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.361+0000 7fbbd77f7640 1 --2- 172.21.15.47:0/2404645910 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbbd0162880 0x7fbbd00b2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:06.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.361+0000 7fbbd77f7640 1 --2- 172.21.15.47:0/2404645910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbd00d39f0 0x7fbbd00b2560 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:06.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.361+0000 7fbbd77f7640 1 --2- 172.21.15.47:0/2404645910 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbbd00b5f20 0x7fbbd0153550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:06.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.361+0000 7fbbd77f7640 1 -- 172.21.15.47:0/2404645910 >> 172.21.15.47:0/2404645910 conn(0x7fbbd00d4f60 msgr2=0x7fbbd0160740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:06.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.361+0000 7fbbd77f7640 1 -- 172.21.15.47:0/2404645910 shutdown_connections 2024-04-26T23:19:06.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06.361+0000 7fbbd77f7640 1 -- 172.21.15.47:0/2404645910 wait complete. 2024-04-26T23:19:06.373 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T23:19:06.373 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T23:19:06.373 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi047.front.sepia.ceph.com 2024-04-26T23:19:06.373 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-26T23:19:06.373 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-26T23:19:06.373 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T23:19:06.373 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T23:19:06.373 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T23:19:06.373 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-26T23:19:06.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:06.374 DEBUG:teuthology.orchestra.run.smithi047:> ip addr 2024-04-26T23:19:06.389 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T23:19:06.389 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T23:19:06.389 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T23:19:06.389 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:19:06.389 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2024-04-26T23:19:06.389 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:19:06.389 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:19:06.389 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:19:06.389 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f0 2024-04-26T23:19:06.390 INFO:teuthology.orchestra.run.smithi047.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:19:06.390 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:19:06.390 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f1 2024-04-26T23:19:06.390 INFO:teuthology.orchestra.run.smithi047.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T23:19:06.390 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2024-04-26T23:19:06.390 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f0 2024-04-26T23:19:06.390 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-26T23:19:06.390 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42653sec preferred_lft 42653sec 2024-04-26T23:19:06.390 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link noprefixroute 2024-04-26T23:19:06.390 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:19:06.390 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:19:06.390 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2024-04-26T23:19:06.390 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f1 2024-04-26T23:19:06.390 INFO:teuthology.orchestra.run.smithi047.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T23:19:06.391 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 32:8d:47:b1:d0:77 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:19:06.391 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T23:19:06.391 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:19:06.391 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::7c6d:6aff:fe00:4b30/64 scope link tentative 2024-04-26T23:19:06.391 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:19:06.391 INFO:teuthology.orchestra.run.smithi047.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T23:19:06.391 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 32:8d:47:b1:d0:77 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T23:19:06.391 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::308d:47ff:feb1:d077/64 scope link tentative 2024-04-26T23:19:06.391 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:19:06.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:06.391 DEBUG:teuthology.orchestra.run.smithi047:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T23:19:06.452 INFO:teuthology.orchestra.run.smithi047.stdout:1 2024-04-26T23:19:06.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:06.454 DEBUG:teuthology.orchestra.run.smithi047:> route 2024-04-26T23:19:06.510 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2024-04-26T23:19:06.510 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T23:19:06.510 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T23:19:06.510 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T23:19:06.510 INFO:teuthology.orchestra.run.smithi047.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T23:19:06.510 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:19:06.510 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-04-26T23:19:06.511 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T23:19:06.511 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T23:19:06.511 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T23:19:06.511 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-04-26T23:19:06.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:19:06.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:19:06.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:06.637 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-04-26T23:19:06.652 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T23:19:06.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:06.653 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list-id 2024-04-26T23:19:06.707 INFO:teuthology.orchestra.run.smithi047.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-26T23:19:06.707 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:19:06.708 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-04-26T23:19:06.708 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T23:19:06.708 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-26T23:19:06.708 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-04-26T23:19:06.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:19:06.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:19:06.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:06.825 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-26T23:19:06.850 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T23:19:06.851 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T23:19:06.851 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T23:19:06.851 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:19:06.851 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2024-04-26T23:19:06.851 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:19:06.851 INFO:teuthology.orchestra.run.smithi047.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T23:19:06.851 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 26:2f:89:f9:15:70 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-26T23:19:06.851 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-26T23:19:06.851 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:19:06.851 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::242f:89ff:fef9:1570/64 scope link tentative 2024-04-26T23:19:06.852 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:19:06.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:06.852 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-26T23:19:06.914 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T23:19:06.914 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T23:19:06.914 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T23:19:06.915 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:19:06.915 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2024-04-26T23:19:06.915 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:19:06.915 INFO:teuthology.orchestra.run.smithi047.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T23:19:06.915 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 26:2f:89:f9:15:70 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-26T23:19:06.915 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-26T23:19:06.915 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:19:06.915 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::242f:89ff:fef9:1570/64 scope link tentative 2024-04-26T23:19:06.915 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:19:06.916 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-26T23:19:06.916 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:19:06.916 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-04-26T23:19:06.916 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-26T23:19:06.916 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-26T23:19:06.917 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-26T23:19:06.917 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-26T23:19:06.917 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-26T23:19:06.917 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-04-26T23:19:07.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:19:07.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:19:07.067 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:19:07.067 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-04-26T23:19:07.068 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set brx.1 up 2024-04-26T23:19:07.068 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set dev brx.1 master ceph-brx 2024-04-26T23:19:07.068 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-04-26T23:19:07.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:19:07.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:19:07.146 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:19:07.146 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-26T23:19:07.161 INFO:teuthology.orchestra.run.smithi047.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-26T23:19:07.161 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:19:07.161 DEBUG:teuthology.orchestra.run.smithi047:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-26T23:19:07.216 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T23:19:07.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:07.217 DEBUG:teuthology.orchestra.run.smithi047:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-26T23:19:07.299 INFO:teuthology.orchestra.run.smithi047.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-26T23:19:07.299 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-26T23:19:07.299 INFO:teuthology.orchestra.run.smithi047.stdout:invalid new device string format 2024-04-26T23:19:07.308 INFO:teuthology.orchestra.run.smithi047.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-26T23:19:07.308 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-26T23:19:07.308 INFO:teuthology.orchestra.run.smithi047.stdout:invalid new device string format 2024-04-26T23:19:07.308 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: resolved to: "172.21.15.47:3300,172.21.15.203:3300,172.21.15.203:3301" 2024-04-26T23:19:07.308 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:19:07.313 INFO:teuthology.orchestra.run.smithi047.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-26T23:19:07.313 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-26T23:19:07.313 INFO:teuthology.orchestra.run.smithi047.stdout:invalid new device string format 2024-04-26T23:19:07.313 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: resolved to: "172.21.15.47:3300,172.21.15.203:3300,172.21.15.203:3301" 2024-04-26T23:19:07.313 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:19:07.313 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: resolved to: "172.21.15.47:6789,172.21.15.203:6789,172.21.15.203:6790" 2024-04-26T23:19:07.313 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.47:6789,172.21.15.203:6789,172.21.15.203:6790:/ 2024-04-26T23:19:07.313 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=399006d3-4db1-43e6-ba81-a79a276eabfd" will pass to kernel 2024-04-26T23:19:07.314 INFO:teuthology.orchestra.run.smithi047.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-04-26T23:19:07.314 INFO:teuthology.orchestra.run.smithi047.stdout: You just mounted a file system that supports labels which does not 2024-04-26T23:19:07.314 INFO:teuthology.orchestra.run.smithi047.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T23:19:07.314 INFO:teuthology.orchestra.run.smithi047.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T23:19:07.314 INFO:teuthology.orchestra.run.smithi047.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T23:19:07.315 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T23:19:07.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:07.315 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-26T23:19:07.379 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T23:19:07.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:19:07.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:19:07.477 INFO:teuthology.orchestra.run.smithi047.stdout:client14483 2024-04-26T23:19:07.477 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:19:07.477 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:19:07.493 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T23:19:07.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:19:07.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:19:07.590 INFO:teuthology.orchestra.run.smithi047.stdout:client14483 2024-04-26T23:19:07.591 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T23:19:07.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:19:07.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:19:07.647 INFO:teuthology.orchestra.run.smithi047.stdout:client14483 2024-04-26T23:19:07.648 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/399006d3-4db1-43e6-ba81-a79a276eabfd.client14483/status') 2024-04-26T23:19:07.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:19:07.706 INFO:teuthology.orchestra.run.smithi047.stdout:instance: client.14483 (3)172.21.15.47:0/2455300429 2024-04-26T23:19:07.706 INFO:teuthology.orchestra.run.smithi047.stdout:blocklisted: false 2024-04-26T23:19:07.707 INFO:teuthology.orchestra.run.smithi047.stderr:0+1 records in 2024-04-26T23:19:07.707 INFO:teuthology.orchestra.run.smithi047.stderr:0+1 records out 2024-04-26T23:19:07.707 INFO:teuthology.orchestra.run.smithi047.stderr:71 bytes copied, 2.4263e-05 s, 2.9 MB/s 2024-04-26T23:19:07.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:19:07.714 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-04-26T23:19:07.714 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T23:19:07.714 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-26T23:19:07.714 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T23:19:07.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.814+0000 7fcdd6c8c640 1 Processor -- start 2024-04-26T23:19:07.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.814+0000 7fcdd6c8c640 1 -- start start 2024-04-26T23:19:07.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.815+0000 7fcdd6c8c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcdd00d3b80 0x7fcdd00b4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:07.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.815+0000 7fcdd6c8c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcdd00b3ff0 0x7fcdd00b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:07.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.815+0000 7fcdd6c8c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdd00d2d70 con 0x7fcdd015f540 2024-04-26T23:19:07.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.815+0000 7fcdd6c8c640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fcdd015f920 con 0x7fcdd00b3ff0 2024-04-26T23:19:07.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.815+0000 7fcdd6c8c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fcdd00d0f30 con 0x7fcdd00d3b80 2024-04-26T23:19:07.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.815+0000 7fcdd5229640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcdd00b3ff0 0x7fcdd00b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:07.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.815+0000 7fcdc7fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcdd00d3b80 0x7fcdd00b4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:07.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.815+0000 7fcdd4a28640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fcdd015f540 0x7fcdd00d3470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.47:47232/0 (socket says 172.21.15.47:47232) 2024-04-26T23:19:07.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.815+0000 7fcdd5229640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcdd00b3ff0 0x7fcdd00b4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.47:48188/0 (socket says 172.21.15.47:48188) 2024-04-26T23:19:07.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.815+0000 7fcdd4a28640 1 -- 172.21.15.47:0/1077029805 learned_addr learned my addr 172.21.15.47:0/1077029805 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:07.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.816+0000 7fcdc77fe640 1 -- 172.21.15.47:0/1077029805 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1155289376 0 0) 0x7fcdd00d2d70 con 0x7fcdd015f540 2024-04-26T23:19:07.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.816+0000 7fcdc77fe640 1 -- 172.21.15.47:0/1077029805 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcdb8003520 con 0x7fcdd015f540 2024-04-26T23:19:07.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.816+0000 7fcdd5229640 1 -- 172.21.15.47:0/1077029805 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcdd00d3b80 msgr2=0x7fcdd00b4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:07.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.816+0000 7fcdd5229640 1 --2- 172.21.15.47:0/1077029805 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcdd00d3b80 0x7fcdd00b4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:07.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.816+0000 7fcdd5229640 1 -- 172.21.15.47:0/1077029805 >> v1:172.21.15.203:6789/0 conn(0x7fcdd015f540 legacy=0x7fcdd00d3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:07.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.816+0000 7fcdd5229640 1 -- 172.21.15.47:0/1077029805 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdd00d3050 con 0x7fcdd00b3ff0 2024-04-26T23:19:07.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.816+0000 7fcdc7fff640 1 --2- 172.21.15.47:0/1077029805 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcdd00d3b80 0x7fcdd00b4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:19:07.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.816+0000 7fcdd5229640 1 --2- 172.21.15.47:0/1077029805 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcdd00b3ff0 0x7fcdd00b4fe0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8009d90 tx=0x7fcdc802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83009daf2ee345b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:07.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.817+0000 7fcdc77fe640 1 -- 172.21.15.47:0/1077029805 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc8007740 con 0x7fcdd00b3ff0 2024-04-26T23:19:07.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.817+0000 7fcdc77fe640 1 -- 172.21.15.47:0/1077029805 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdc80078e0 con 0x7fcdd00b3ff0 2024-04-26T23:19:07.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.817+0000 7fcdc77fe640 1 -- 172.21.15.47:0/1077029805 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc8007cb0 con 0x7fcdd00b3ff0 2024-04-26T23:19:07.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.817+0000 7fcdd6c8c640 1 -- 172.21.15.47:0/1077029805 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcdd00b3ff0 msgr2=0x7fcdd00b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:07.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.817+0000 7fcdd6c8c640 1 --2- 172.21.15.47:0/1077029805 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcdd00b3ff0 0x7fcdd00b4fe0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8009d90 tx=0x7fcdc802a660 comp rx=0 tx=0).stop 2024-04-26T23:19:07.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.817+0000 7fcdd6c8c640 1 -- 172.21.15.47:0/1077029805 shutdown_connections 2024-04-26T23:19:07.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.817+0000 7fcdd6c8c640 1 --2- 172.21.15.47:0/1077029805 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcdd00b3ff0 0x7fcdd00b4fe0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:07.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.817+0000 7fcdd6c8c640 1 --2- 172.21.15.47:0/1077029805 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcdd00d3b80 0x7fcdd00b4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:07.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.817+0000 7fcdd6c8c640 1 -- 172.21.15.47:0/1077029805 >> 172.21.15.47:0/1077029805 conn(0x7fcdd00d4f90 msgr2=0x7fcdd00d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:07.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.817+0000 7fcdd6c8c640 1 -- 172.21.15.47:0/1077029805 shutdown_connections 2024-04-26T23:19:07.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.817+0000 7fcdd6c8c640 1 -- 172.21.15.47:0/1077029805 wait complete. 2024-04-26T23:19:07.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.818+0000 7fcdd6c8c640 1 Processor -- start 2024-04-26T23:19:07.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.818+0000 7fcdd6c8c640 1 -- start start 2024-04-26T23:19:07.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.819+0000 7fcdd6c8c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdd00b3ff0 0x7fcdd00b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:07.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.819+0000 7fcdd6c8c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcdd00d3b80 0x7fcdd0156490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:07.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.819+0000 7fcdd6c8c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcdd015f540 0x7fcdd00b48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:07.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.819+0000 7fcdd6c8c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd0068e50 con 0x7fcdd00b3ff0 2024-04-26T23:19:07.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.819+0000 7fcdd6c8c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd00d0bc0 con 0x7fcdd00d3b80 2024-04-26T23:19:07.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.819+0000 7fcdd6c8c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fcdd00d2780 con 0x7fcdd015f540 2024-04-26T23:19:07.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.819+0000 7fcdd4a28640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdd00b3ff0 0x7fcdd00b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:07.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.819+0000 7fcdd4a28640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdd00b3ff0 0x7fcdd00b5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54852/0 (socket says 172.21.15.47:54852) 2024-04-26T23:19:07.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.819+0000 7fcdd4a28640 1 -- 172.21.15.47:0/3192447001 learned_addr learned my addr 172.21.15.47:0/3192447001 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:07.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.819+0000 7fcdd5229640 1 --2- 172.21.15.47:0/3192447001 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcdd015f540 0x7fcdd00b48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:07.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.819+0000 7fcdc7fff640 1 --2- 172.21.15.47:0/3192447001 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcdd00d3b80 0x7fcdd0156490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:07.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.820+0000 7fcdd4a28640 1 -- 172.21.15.47:0/3192447001 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcdd015f540 msgr2=0x7fcdd00b48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:07.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.820+0000 7fcdd4a28640 1 --2- 172.21.15.47:0/3192447001 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcdd015f540 0x7fcdd00b48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:07.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.820+0000 7fcdd4a28640 1 -- 172.21.15.47:0/3192447001 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcdd00d3b80 msgr2=0x7fcdd0156490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:07.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.820+0000 7fcdd4a28640 1 --2- 172.21.15.47:0/3192447001 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcdd00d3b80 0x7fcdd0156490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:07.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.820+0000 7fcdd4a28640 1 -- 172.21.15.47:0/3192447001 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdd00b8af0 con 0x7fcdd00b3ff0 2024-04-26T23:19:07.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.820+0000 7fcdd4a28640 1 --2- 172.21.15.47:0/3192447001 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdd00b3ff0 0x7fcdd00b5d80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc00c430 tx=0x7fcdcc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:07.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.820+0000 7fcdc57fa640 1 -- 172.21.15.47:0/3192447001 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc017070 con 0x7fcdd00b3ff0 2024-04-26T23:19:07.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.820+0000 7fcdc57fa640 1 -- 172.21.15.47:0/3192447001 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdcc005ce0 con 0x7fcdd00b3ff0 2024-04-26T23:19:07.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.820+0000 7fcdd6c8c640 1 -- 172.21.15.47:0/3192447001 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdd00b8810 con 0x7fcdd00b3ff0 2024-04-26T23:19:07.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.820+0000 7fcdc57fa640 1 -- 172.21.15.47:0/3192447001 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc0072f0 con 0x7fcdd00b3ff0 2024-04-26T23:19:07.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.820+0000 7fcdd6c8c640 1 -- 172.21.15.47:0/3192447001 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdd00c9f20 con 0x7fcdd00b3ff0 2024-04-26T23:19:07.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.821+0000 7fcdd6c8c640 1 -- 172.21.15.47:0/3192447001 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdd00694f0 con 0x7fcdd00b3ff0 2024-04-26T23:19:07.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.821+0000 7fcdc57fa640 1 -- 172.21.15.47:0/3192447001 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcdcc01a070 con 0x7fcdd00b3ff0 2024-04-26T23:19:07.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.822+0000 7fcdc57fa640 1 --2- 172.21.15.47:0/3192447001 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fcdbc06c8f0 0x7fcdbc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:07.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.822+0000 7fcdc57fa640 1 -- 172.21.15.47:0/3192447001 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fcdcc08b210 con 0x7fcdd00b3ff0 2024-04-26T23:19:07.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.822+0000 7fcdc7fff640 1 --2- 172.21.15.47:0/3192447001 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fcdbc06c8f0 0x7fcdbc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:07.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.823+0000 7fcdc7fff640 1 --2- 172.21.15.47:0/3192447001 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fcdbc06c8f0 0x7fcdbc06edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0009560 tx=0x7fcdc0009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:07.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:07.826+0000 7fcdc57fa640 1 -- 172.21.15.47:0/3192447001 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcdcc058d40 con 0x7fcdd00b3ff0 2024-04-26T23:19:08.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:08.055+0000 7fcdd6c8c640 1 -- 172.21.15.47:0/3192447001 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fcdd0069ba0 con 0x7fcdd00b3ff0 2024-04-26T23:19:08.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:08.056+0000 7fcdc57fa640 1 -- 172.21.15.47:0/3192447001 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7fcdcc05d040 con 0x7fcdd00b3ff0 2024-04-26T23:19:08.057 INFO:teuthology.orchestra.run.smithi047.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-26T23:19:08.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:08.057+0000 7fcdd6c8c640 1 -- 172.21.15.47:0/3192447001 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fcdbc06c8f0 msgr2=0x7fcdbc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:08.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:08.058+0000 7fcdd6c8c640 1 --2- 172.21.15.47:0/3192447001 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fcdbc06c8f0 0x7fcdbc06edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0009560 tx=0x7fcdc0009190 comp rx=0 tx=0).stop 2024-04-26T23:19:08.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:08.058+0000 7fcdd6c8c640 1 -- 172.21.15.47:0/3192447001 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdd00b3ff0 msgr2=0x7fcdd00b5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:08.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:08.058+0000 7fcdd6c8c640 1 --2- 172.21.15.47:0/3192447001 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdd00b3ff0 0x7fcdd00b5d80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc00c430 tx=0x7fcdcc00c8f0 comp rx=0 tx=0).stop 2024-04-26T23:19:08.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:08.058+0000 7fcdd6c8c640 1 -- 172.21.15.47:0/3192447001 shutdown_connections 2024-04-26T23:19:08.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:08.058+0000 7fcdd6c8c640 1 --2- 172.21.15.47:0/3192447001 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fcdbc06c8f0 0x7fcdbc06edb0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:08.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:08.058+0000 7fcdd6c8c640 1 --2- 172.21.15.47:0/3192447001 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcdd015f540 0x7fcdd00b48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:08.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:08.058+0000 7fcdd6c8c640 1 --2- 172.21.15.47:0/3192447001 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcdd00d3b80 0x7fcdd0156490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:08.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:08.058+0000 7fcdd6c8c640 1 --2- 172.21.15.47:0/3192447001 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdd00b3ff0 0x7fcdd00b5d80 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:08.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:08.058+0000 7fcdd6c8c640 1 -- 172.21.15.47:0/3192447001 >> 172.21.15.47:0/3192447001 conn(0x7fcdd00d4f90 msgr2=0x7fcdd00d1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:08.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:08.058+0000 7fcdd6c8c640 1 -- 172.21.15.47:0/3192447001 shutdown_connections 2024-04-26T23:19:08.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:08.058+0000 7fcdd6c8c640 1 -- 172.21.15.47:0/3192447001 wait complete. 2024-04-26T23:19:08.070 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T23:19:08.070 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T23:19:08.070 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi203.front.sepia.ceph.com 2024-04-26T23:19:08.070 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-26T23:19:08.070 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-26T23:19:08.070 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T23:19:08.070 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T23:19:08.071 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T23:19:08.071 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-26T23:19:08.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:08.071 DEBUG:teuthology.orchestra.run.smithi203:> ip addr 2024-04-26T23:19:08.089 INFO:teuthology.orchestra.run.smithi203.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T23:19:08.089 INFO:teuthology.orchestra.run.smithi203.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T23:19:08.089 INFO:teuthology.orchestra.run.smithi203.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T23:19:08.089 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:19:08.089 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 ::1/128 scope host 2024-04-26T23:19:08.089 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:19:08.089 INFO:teuthology.orchestra.run.smithi203.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:19:08.089 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:d9:19:44 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:19:08.089 INFO:teuthology.orchestra.run.smithi203.stdout: altname enp7s0f0 2024-04-26T23:19:08.089 INFO:teuthology.orchestra.run.smithi203.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:19:08.089 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:d9:19:45 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:19:08.089 INFO:teuthology.orchestra.run.smithi203.stdout: altname enp7s0f1 2024-04-26T23:19:08.090 INFO:teuthology.orchestra.run.smithi203.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:19:08.090 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:8f:4a:22 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:19:08.090 INFO:teuthology.orchestra.run.smithi203.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T23:19:08.090 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:8f:4a:23 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:19:08.090 INFO:teuthology.orchestra.run.smithi203.stdout: inet 172.21.15.203/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-26T23:19:08.090 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft 42687sec preferred_lft 42687sec 2024-04-26T23:19:08.090 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::52c8:c0e4:16ad:f707/64 scope link noprefixroute 2024-04-26T23:19:08.090 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:19:08.090 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T23:19:08.090 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:19:08.091 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2024-04-26T23:19:08.091 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link add name ceph-brx type bridge 2024-04-26T23:19:08.091 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip addr flush dev ceph-brx 2024-04-26T23:19:08.091 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link set ceph-brx up 2024-04-26T23:19:08.091 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T23:19:08.091 DEBUG:teuthology.orchestra.run.smithi203:> ') 2024-04-26T23:19:08.182 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:19:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:19:08.267 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:19:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:19:08.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:08.274 DEBUG:teuthology.orchestra.run.smithi203:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T23:19:08.297 INFO:teuthology.orchestra.run.smithi203.stdout:1 2024-04-26T23:19:08.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:08.298 DEBUG:teuthology.orchestra.run.smithi203:> route 2024-04-26T23:19:08.364 INFO:teuthology.orchestra.run.smithi203.stdout:Kernel IP routing table 2024-04-26T23:19:08.364 INFO:teuthology.orchestra.run.smithi203.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T23:19:08.364 INFO:teuthology.orchestra.run.smithi203.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T23:19:08.364 INFO:teuthology.orchestra.run.smithi203.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T23:19:08.364 INFO:teuthology.orchestra.run.smithi203.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T23:19:08.364 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:19:08.364 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2024-04-26T23:19:08.364 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T23:19:08.365 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T23:19:08.365 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T23:19:08.365 DEBUG:teuthology.orchestra.run.smithi203:> ') 2024-04-26T23:19:08.451 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:19:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:19:08.634 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:19:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:19:08.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:08.640 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list 2024-04-26T23:19:08.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:08.656 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list-id 2024-04-26T23:19:08.710 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:19:08.711 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2024-04-26T23:19:08.711 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T23:19:08.711 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-26T23:19:08.711 DEBUG:teuthology.orchestra.run.smithi203:> ') 2024-04-26T23:19:08.798 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:19:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:19:08.819 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:19:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:19:08.826 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-26T23:19:08.826 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:19:08.826 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2024-04-26T23:19:08.826 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-26T23:19:08.826 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T23:19:08.826 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-26T23:19:08.826 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-26T23:19:08.826 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-26T23:19:08.826 DEBUG:teuthology.orchestra.run.smithi203:> ') 2024-04-26T23:19:08.874 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:19:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:19:08.941 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:19:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:19:08.947 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:19:08.947 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2024-04-26T23:19:08.947 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link set brx.0 up 2024-04-26T23:19:08.947 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T23:19:08.947 DEBUG:teuthology.orchestra.run.smithi203:> ') 2024-04-26T23:19:08.995 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:19:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:19:09.016 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:19:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:19:09.022 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:19:09.022 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-26T23:19:09.037 INFO:teuthology.orchestra.run.smithi203.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-26T23:19:09.037 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:19:09.037 DEBUG:teuthology.orchestra.run.smithi203:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-26T23:19:09.088 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T23:19:09.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:09.088 DEBUG:teuthology.orchestra.run.smithi203:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T23:19:09.176 INFO:teuthology.orchestra.run.smithi203.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T23:19:09.176 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-26T23:19:09.176 INFO:teuthology.orchestra.run.smithi203.stdout:invalid new device string format 2024-04-26T23:19:09.281 INFO:teuthology.orchestra.run.smithi203.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T23:19:09.281 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-26T23:19:09.281 INFO:teuthology.orchestra.run.smithi203.stdout:invalid new device string format 2024-04-26T23:19:09.281 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: resolved to: "172.21.15.47:3300,172.21.15.203:3300,172.21.15.203:3301" 2024-04-26T23:19:09.281 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:19:09.286 INFO:teuthology.orchestra.run.smithi203.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T23:19:09.286 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-26T23:19:09.286 INFO:teuthology.orchestra.run.smithi203.stdout:invalid new device string format 2024-04-26T23:19:09.286 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: resolved to: "172.21.15.47:3300,172.21.15.203:3300,172.21.15.203:3301" 2024-04-26T23:19:09.286 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:19:09.286 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: resolved to: "172.21.15.47:6789,172.21.15.203:6789,172.21.15.203:6790" 2024-04-26T23:19:09.286 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.47:6789,172.21.15.203:6789,172.21.15.203:6790:/ 2024-04-26T23:19:09.286 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=399006d3-4db1-43e6-ba81-a79a276eabfd" will pass to kernel 2024-04-26T23:19:09.287 INFO:teuthology.orchestra.run.smithi203.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-26T23:19:09.287 INFO:teuthology.orchestra.run.smithi203.stdout: You just mounted a file system that supports labels which does not 2024-04-26T23:19:09.287 INFO:teuthology.orchestra.run.smithi203.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T23:19:09.287 INFO:teuthology.orchestra.run.smithi203.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T23:19:09.287 INFO:teuthology.orchestra.run.smithi203.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T23:19:09.288 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T23:19:09.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:09.289 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-26T23:19:09.312 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T23:19:09.400 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:19:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:19:09.402 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:19:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:19:09.408 INFO:teuthology.orchestra.run.smithi203.stdout:client14492 2024-04-26T23:19:09.408 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-26T23:19:09.408 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:19:09.424 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T23:19:09.511 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:19:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:19:09.513 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:19:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:19:09.518 INFO:teuthology.orchestra.run.smithi203.stdout:client14492 2024-04-26T23:19:09.518 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T23:19:09.566 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:19:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:19:09.568 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:19:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:19:09.574 INFO:teuthology.orchestra.run.smithi203.stdout:client14492 2024-04-26T23:19:09.574 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/399006d3-4db1-43e6-ba81-a79a276eabfd.client14492/status') 2024-04-26T23:19:09.622 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:19:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:19:09.631 INFO:teuthology.orchestra.run.smithi203.stdout:instance: client.14492 (3)192.168.0.1:0/863369814 2024-04-26T23:19:09.631 INFO:teuthology.orchestra.run.smithi203.stdout:blocklisted: false 2024-04-26T23:19:09.631 INFO:teuthology.orchestra.run.smithi203.stderr:0+1 records in 2024-04-26T23:19:09.632 INFO:teuthology.orchestra.run.smithi203.stderr:0+1 records out 2024-04-26T23:19:09.632 INFO:teuthology.orchestra.run.smithi203.stderr:69 bytes copied, 2.3116e-05 s, 3.0 MB/s 2024-04-26T23:19:09.632 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:19:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:19:09.638 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-04-26T23:19:09.638 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T23:19:09.639 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-26T23:19:09.639 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T23:19:09.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.751+0000 7f5e7a396640 1 Processor -- start 2024-04-26T23:19:09.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.751+0000 7f5e7a396640 1 -- start start 2024-04-26T23:19:09.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.752+0000 7f5e7a396640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5e740b4d40 0x7f5e740b6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:09.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.752+0000 7f5e7a396640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e740b6c50 0x7f5e740aed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:09.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.752+0000 7f5e7a396640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e74072fb0 con 0x7f5e740b0b70 2024-04-26T23:19:09.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.752+0000 7f5e7a396640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f5e74073bc0 con 0x7f5e740b4d40 2024-04-26T23:19:09.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.752+0000 7f5e7a396640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f5e74073290 con 0x7f5e740b6c50 2024-04-26T23:19:09.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.752+0000 7f5e73fff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f5e740b0b70 0x7f5e740b0f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:32954/0 (socket says 172.21.15.47:32954) 2024-04-26T23:19:09.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.752+0000 7f5e73fff640 1 -- 172.21.15.47:0/1564836562 learned_addr learned my addr 172.21.15.47:0/1564836562 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:09.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.752+0000 7f5e737fe640 1 --2- 172.21.15.47:0/1564836562 >> v2:172.21.15.203:3300/0 conn(0x7f5e740b4d40 0x7f5e740b6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:09.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.752+0000 7f5e78933640 1 --2- 172.21.15.47:0/1564836562 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e740b6c50 0x7f5e740aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:09.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.752+0000 7f5e72ffd640 1 -- 172.21.15.47:0/1564836562 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2268143010 0 0) 0x7f5e74072fb0 con 0x7f5e740b0b70 2024-04-26T23:19:09.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.752+0000 7f5e72ffd640 1 -- 172.21.15.47:0/1564836562 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5e60003520 con 0x7f5e740b0b70 2024-04-26T23:19:09.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.752+0000 7f5e72ffd640 1 -- 172.21.15.47:0/1564836562 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 899496889 0 0) 0x7f5e60003520 con 0x7f5e740b0b70 2024-04-26T23:19:09.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.752+0000 7f5e737fe640 1 -- 172.21.15.47:0/1564836562 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e740b6c50 msgr2=0x7f5e740aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:09.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.752+0000 7f5e737fe640 1 --2- 172.21.15.47:0/1564836562 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e740b6c50 0x7f5e740aed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:09.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.752+0000 7f5e737fe640 1 -- 172.21.15.47:0/1564836562 >> v1:172.21.15.47:6789/0 conn(0x7f5e740b0b70 legacy=0x7f5e740b0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:09.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.753+0000 7f5e737fe640 1 -- 172.21.15.47:0/1564836562 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e74073570 con 0x7f5e740b4d40 2024-04-26T23:19:09.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.753+0000 7f5e737fe640 1 --2- 172.21.15.47:0/1564836562 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e740b4d40 0x7f5e740b6710 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5e640097a0 tx=0x7f5e6402a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fbf4a47b04ec371 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:09.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.754+0000 7f5e72ffd640 1 -- 172.21.15.47:0/1564836562 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e64037070 con 0x7f5e740b4d40 2024-04-26T23:19:09.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.754+0000 7f5e72ffd640 1 -- 172.21.15.47:0/1564836562 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e6402aba0 con 0x7f5e740b4d40 2024-04-26T23:19:09.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.754+0000 7f5e72ffd640 1 -- 172.21.15.47:0/1564836562 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c001670 con 0x7f5e740b4d40 2024-04-26T23:19:09.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.754+0000 7f5e7a396640 1 -- 172.21.15.47:0/1564836562 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e740b4d40 msgr2=0x7f5e740b6710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:09.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.754+0000 7f5e7a396640 1 --2- 172.21.15.47:0/1564836562 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e740b4d40 0x7f5e740b6710 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5e640097a0 tx=0x7f5e6402a110 comp rx=0 tx=0).stop 2024-04-26T23:19:09.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.754+0000 7f5e7a396640 1 -- 172.21.15.47:0/1564836562 shutdown_connections 2024-04-26T23:19:09.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.754+0000 7f5e7a396640 1 --2- 172.21.15.47:0/1564836562 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e740b6c50 0x7f5e740aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:09.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.754+0000 7f5e7a396640 1 --2- 172.21.15.47:0/1564836562 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e740b4d40 0x7f5e740b6710 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:09.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.754+0000 7f5e7a396640 1 -- 172.21.15.47:0/1564836562 >> 172.21.15.47:0/1564836562 conn(0x7f5e740b39b0 msgr2=0x7f5e740b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:09.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.754+0000 7f5e7a396640 1 -- 172.21.15.47:0/1564836562 shutdown_connections 2024-04-26T23:19:09.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.754+0000 7f5e7a396640 1 -- 172.21.15.47:0/1564836562 wait complete. 2024-04-26T23:19:09.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.755+0000 7f5e7a396640 1 Processor -- start 2024-04-26T23:19:09.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.756+0000 7f5e7a396640 1 -- start start 2024-04-26T23:19:09.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.756+0000 7f5e7a396640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e740b0b70 0x7f5e74159050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:09.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.756+0000 7f5e7a396640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e740b4d40 0x7f5e741e7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:09.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.756+0000 7f5e7a396640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e740b6c50 0x7f5e7415c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:09.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.756+0000 7f5e7a396640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5e741604e0 con 0x7f5e740b4d40 2024-04-26T23:19:09.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.756+0000 7f5e7a396640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5e74160360 con 0x7f5e740b0b70 2024-04-26T23:19:09.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.756+0000 7f5e7a396640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f5e740af2d0 con 0x7f5e740b6c50 2024-04-26T23:19:09.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.756+0000 7f5e737fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e740b4d40 0x7f5e741e7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:09.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.756+0000 7f5e737fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e740b4d40 0x7f5e741e7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41068/0 (socket says 172.21.15.47:41068) 2024-04-26T23:19:09.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.756+0000 7f5e737fe640 1 -- 172.21.15.47:0/2365168578 learned_addr learned my addr 172.21.15.47:0/2365168578 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:09.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.757+0000 7f5e73fff640 1 --2- 172.21.15.47:0/2365168578 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e740b0b70 0x7f5e74159050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:09.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.757+0000 7f5e78933640 1 --2- 172.21.15.47:0/2365168578 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e740b6c50 0x7f5e7415c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:09.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.757+0000 7f5e737fe640 1 -- 172.21.15.47:0/2365168578 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e740b6c50 msgr2=0x7f5e7415c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:09.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.757+0000 7f5e737fe640 1 --2- 172.21.15.47:0/2365168578 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e740b6c50 0x7f5e7415c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:09.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.757+0000 7f5e737fe640 1 -- 172.21.15.47:0/2365168578 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e740b0b70 msgr2=0x7f5e74159050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:09.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.757+0000 7f5e737fe640 1 --2- 172.21.15.47:0/2365168578 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e740b0b70 0x7f5e74159050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:09.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.757+0000 7f5e737fe640 1 -- 172.21.15.47:0/2365168578 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e74156b60 con 0x7f5e740b4d40 2024-04-26T23:19:09.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.757+0000 7f5e73fff640 1 --2- 172.21.15.47:0/2365168578 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e740b0b70 0x7f5e74159050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:19:09.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.757+0000 7f5e737fe640 1 --2- 172.21.15.47:0/2365168578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e740b4d40 0x7f5e741e7880 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5e640098c0 tx=0x7f5e640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:09.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.757+0000 7f5e70ff9640 1 -- 172.21.15.47:0/2365168578 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e64037050 con 0x7f5e740b4d40 2024-04-26T23:19:09.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.757+0000 7f5e70ff9640 1 -- 172.21.15.47:0/2365168578 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e64043070 con 0x7f5e740b4d40 2024-04-26T23:19:09.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.757+0000 7f5e7a396640 1 -- 172.21.15.47:0/2365168578 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e74156e40 con 0x7f5e740b4d40 2024-04-26T23:19:09.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.757+0000 7f5e70ff9640 1 -- 172.21.15.47:0/2365168578 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e64006e10 con 0x7f5e740b4d40 2024-04-26T23:19:09.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.757+0000 7f5e7a396640 1 -- 172.21.15.47:0/2365168578 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e74157400 con 0x7f5e740b4d40 2024-04-26T23:19:09.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.758+0000 7f5e7a396640 1 -- 172.21.15.47:0/2365168578 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e7406d0d0 con 0x7f5e740b4d40 2024-04-26T23:19:09.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.759+0000 7f5e70ff9640 1 -- 172.21.15.47:0/2365168578 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5e64048460 con 0x7f5e740b4d40 2024-04-26T23:19:09.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.759+0000 7f5e70ff9640 1 --2- 172.21.15.47:0/2365168578 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f5e5406c8f0 0x7f5e5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:09.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.759+0000 7f5e70ff9640 1 -- 172.21.15.47:0/2365168578 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f5e640ac680 con 0x7f5e740b4d40 2024-04-26T23:19:09.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.759+0000 7f5e73fff640 1 --2- 172.21.15.47:0/2365168578 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f5e5406c8f0 0x7f5e5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:09.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.760+0000 7f5e73fff640 1 --2- 172.21.15.47:0/2365168578 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f5e5406c8f0 0x7f5e5406edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c005880 tx=0x7f5e5c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:09.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.763+0000 7f5e70ff9640 1 -- 172.21.15.47:0/2365168578 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5e7406d0d0 con 0x7f5e740b4d40 2024-04-26T23:19:09.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.987+0000 7f5e7a396640 1 -- 172.21.15.47:0/2365168578 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5e740694f0 con 0x7f5e740b4d40 2024-04-26T23:19:09.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.988+0000 7f5e70ff9640 1 -- 172.21.15.47:0/2365168578 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f5e6407a770 con 0x7f5e740b4d40 2024-04-26T23:19:09.990 INFO:teuthology.orchestra.run.smithi047.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-26T23:19:09.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.990+0000 7f5e7a396640 1 -- 172.21.15.47:0/2365168578 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f5e5406c8f0 msgr2=0x7f5e5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:09.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.990+0000 7f5e7a396640 1 --2- 172.21.15.47:0/2365168578 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f5e5406c8f0 0x7f5e5406edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c005880 tx=0x7f5e5c009040 comp rx=0 tx=0).stop 2024-04-26T23:19:09.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.990+0000 7f5e7a396640 1 -- 172.21.15.47:0/2365168578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e740b4d40 msgr2=0x7f5e741e7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:09.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.990+0000 7f5e7a396640 1 --2- 172.21.15.47:0/2365168578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e740b4d40 0x7f5e741e7880 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5e640098c0 tx=0x7f5e640046c0 comp rx=0 tx=0).stop 2024-04-26T23:19:09.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.990+0000 7f5e7a396640 1 -- 172.21.15.47:0/2365168578 shutdown_connections 2024-04-26T23:19:09.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.990+0000 7f5e7a396640 1 --2- 172.21.15.47:0/2365168578 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f5e5406c8f0 0x7f5e5406edb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:09.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.990+0000 7f5e7a396640 1 --2- 172.21.15.47:0/2365168578 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e740b6c50 0x7f5e7415c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:09.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.990+0000 7f5e7a396640 1 --2- 172.21.15.47:0/2365168578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e740b4d40 0x7f5e741e7880 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:09.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.990+0000 7f5e7a396640 1 --2- 172.21.15.47:0/2365168578 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e740b0b70 0x7f5e74159050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:09.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.990+0000 7f5e7a396640 1 -- 172.21.15.47:0/2365168578 >> 172.21.15.47:0/2365168578 conn(0x7f5e740b39b0 msgr2=0x7f5e740b0370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:09.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.990+0000 7f5e7a396640 1 -- 172.21.15.47:0/2365168578 shutdown_connections 2024-04-26T23:19:09.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:09.990+0000 7f5e7a396640 1 -- 172.21.15.47:0/2365168578 wait complete. 2024-04-26T23:19:10.002 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T23:19:10.003 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T23:19:10.003 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi203.front.sepia.ceph.com 2024-04-26T23:19:10.003 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-26T23:19:10.003 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-26T23:19:10.003 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T23:19:10.003 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T23:19:10.003 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T23:19:10.003 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-26T23:19:10.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:10.003 DEBUG:teuthology.orchestra.run.smithi203:> ip addr 2024-04-26T23:19:10.019 INFO:teuthology.orchestra.run.smithi203.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T23:19:10.019 INFO:teuthology.orchestra.run.smithi203.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T23:19:10.019 INFO:teuthology.orchestra.run.smithi203.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T23:19:10.019 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:19:10.020 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 ::1/128 scope host 2024-04-26T23:19:10.020 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:19:10.020 INFO:teuthology.orchestra.run.smithi203.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:19:10.020 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:d9:19:44 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:19:10.020 INFO:teuthology.orchestra.run.smithi203.stdout: altname enp7s0f0 2024-04-26T23:19:10.020 INFO:teuthology.orchestra.run.smithi203.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:19:10.020 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:d9:19:45 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:19:10.020 INFO:teuthology.orchestra.run.smithi203.stdout: altname enp7s0f1 2024-04-26T23:19:10.020 INFO:teuthology.orchestra.run.smithi203.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:19:10.020 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:8f:4a:22 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:19:10.020 INFO:teuthology.orchestra.run.smithi203.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T23:19:10.020 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:8f:4a:23 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:19:10.020 INFO:teuthology.orchestra.run.smithi203.stdout: inet 172.21.15.203/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-26T23:19:10.020 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft 42685sec preferred_lft 42685sec 2024-04-26T23:19:10.021 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::52c8:c0e4:16ad:f707/64 scope link noprefixroute 2024-04-26T23:19:10.021 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:19:10.021 INFO:teuthology.orchestra.run.smithi203.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T23:19:10.021 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 66:38:58:a0:89:03 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:19:10.021 INFO:teuthology.orchestra.run.smithi203.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T23:19:10.021 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:19:10.021 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::9c9a:d4ff:fea4:85ba/64 scope link tentative 2024-04-26T23:19:10.021 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:19:10.021 INFO:teuthology.orchestra.run.smithi203.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T23:19:10.021 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 66:38:58:a0:89:03 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T23:19:10.021 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::6438:58ff:fea0:8903/64 scope link tentative 2024-04-26T23:19:10.021 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:19:10.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:10.022 DEBUG:teuthology.orchestra.run.smithi203:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T23:19:10.083 INFO:teuthology.orchestra.run.smithi203.stdout:1 2024-04-26T23:19:10.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:10.085 DEBUG:teuthology.orchestra.run.smithi203:> route 2024-04-26T23:19:10.142 INFO:teuthology.orchestra.run.smithi203.stdout:Kernel IP routing table 2024-04-26T23:19:10.142 INFO:teuthology.orchestra.run.smithi203.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T23:19:10.142 INFO:teuthology.orchestra.run.smithi203.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T23:19:10.142 INFO:teuthology.orchestra.run.smithi203.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T23:19:10.142 INFO:teuthology.orchestra.run.smithi203.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T23:19:10.142 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:19:10.143 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2024-04-26T23:19:10.143 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T23:19:10.143 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T23:19:10.143 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T23:19:10.143 DEBUG:teuthology.orchestra.run.smithi203:> ') 2024-04-26T23:19:10.229 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:19:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:19:10.259 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:19:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:19:10.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:10.265 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list 2024-04-26T23:19:10.280 INFO:teuthology.orchestra.run.smithi203.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-26T23:19:10.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:10.281 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list-id 2024-04-26T23:19:10.335 INFO:teuthology.orchestra.run.smithi203.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-26T23:19:10.336 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:19:10.336 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2024-04-26T23:19:10.336 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T23:19:10.336 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-26T23:19:10.336 DEBUG:teuthology.orchestra.run.smithi203:> ') 2024-04-26T23:19:10.423 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:19:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:19:10.444 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:19:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:19:10.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:10.450 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-26T23:19:10.473 INFO:teuthology.orchestra.run.smithi203.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T23:19:10.473 INFO:teuthology.orchestra.run.smithi203.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T23:19:10.473 INFO:teuthology.orchestra.run.smithi203.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T23:19:10.474 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:19:10.474 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 ::1/128 scope host 2024-04-26T23:19:10.474 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:19:10.474 INFO:teuthology.orchestra.run.smithi203.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T23:19:10.474 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 42:bc:c2:b7:88:25 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-26T23:19:10.474 INFO:teuthology.orchestra.run.smithi203.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-26T23:19:10.474 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:19:10.474 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::40bc:c2ff:feb7:8825/64 scope link 2024-04-26T23:19:10.474 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:19:10.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:10.475 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-26T23:19:10.539 INFO:teuthology.orchestra.run.smithi203.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T23:19:10.539 INFO:teuthology.orchestra.run.smithi203.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T23:19:10.539 INFO:teuthology.orchestra.run.smithi203.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T23:19:10.539 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:19:10.539 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 ::1/128 scope host 2024-04-26T23:19:10.540 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:19:10.540 INFO:teuthology.orchestra.run.smithi203.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T23:19:10.540 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 42:bc:c2:b7:88:25 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-26T23:19:10.540 INFO:teuthology.orchestra.run.smithi203.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-26T23:19:10.540 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:19:10.540 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::40bc:c2ff:feb7:8825/64 scope link 2024-04-26T23:19:10.540 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:19:10.541 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-26T23:19:10.541 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:19:10.541 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2024-04-26T23:19:10.541 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-26T23:19:10.541 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-26T23:19:10.541 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-26T23:19:10.541 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-26T23:19:10.542 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-26T23:19:10.542 DEBUG:teuthology.orchestra.run.smithi203:> ') 2024-04-26T23:19:10.629 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:19:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:19:10.682 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:19:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:19:10.689 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:19:10.689 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2024-04-26T23:19:10.689 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link set brx.1 up 2024-04-26T23:19:10.689 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link set dev brx.1 master ceph-brx 2024-04-26T23:19:10.689 DEBUG:teuthology.orchestra.run.smithi203:> ') 2024-04-26T23:19:10.736 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:19:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:19:10.757 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:19:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:19:10.763 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:19:10.763 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-26T23:19:10.777 INFO:teuthology.orchestra.run.smithi203.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-26T23:19:10.778 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:19:10.778 DEBUG:teuthology.orchestra.run.smithi203:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-26T23:19:10.832 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T23:19:10.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:10.832 DEBUG:teuthology.orchestra.run.smithi203:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-26T23:19:10.913 INFO:teuthology.orchestra.run.smithi203.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-26T23:19:10.913 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-26T23:19:10.913 INFO:teuthology.orchestra.run.smithi203.stdout:invalid new device string format 2024-04-26T23:19:10.921 INFO:teuthology.orchestra.run.smithi203.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-26T23:19:10.922 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-26T23:19:10.922 INFO:teuthology.orchestra.run.smithi203.stdout:invalid new device string format 2024-04-26T23:19:10.922 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: resolved to: "172.21.15.47:3300,172.21.15.203:3300,172.21.15.203:3301" 2024-04-26T23:19:10.922 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:19:10.926 INFO:teuthology.orchestra.run.smithi203.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-26T23:19:10.926 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-26T23:19:10.927 INFO:teuthology.orchestra.run.smithi203.stdout:invalid new device string format 2024-04-26T23:19:10.927 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: resolved to: "172.21.15.47:3300,172.21.15.203:3300,172.21.15.203:3301" 2024-04-26T23:19:10.927 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:19:10.927 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: resolved to: "172.21.15.47:6789,172.21.15.203:6789,172.21.15.203:6790" 2024-04-26T23:19:10.927 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.47:6789,172.21.15.203:6789,172.21.15.203:6790:/ 2024-04-26T23:19:10.927 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=399006d3-4db1-43e6-ba81-a79a276eabfd" will pass to kernel 2024-04-26T23:19:10.929 INFO:teuthology.orchestra.run.smithi203.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-04-26T23:19:10.929 INFO:teuthology.orchestra.run.smithi203.stdout: You just mounted a file system that supports labels which does not 2024-04-26T23:19:10.929 INFO:teuthology.orchestra.run.smithi203.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T23:19:10.929 INFO:teuthology.orchestra.run.smithi203.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T23:19:10.929 INFO:teuthology.orchestra.run.smithi203.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T23:19:10.931 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T23:19:10.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:10.931 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-26T23:19:10.994 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T23:19:11.084 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:19:11 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:19:11.085 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:19:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:19:11.091 INFO:teuthology.orchestra.run.smithi203.stdout:client4714 2024-04-26T23:19:11.091 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-26T23:19:11.091 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:19:11.106 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T23:19:11.195 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:19:11 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:19:11.197 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:19:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:19:11.202 INFO:teuthology.orchestra.run.smithi203.stdout:client4714 2024-04-26T23:19:11.203 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T23:19:11.250 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:19:11 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:19:11.251 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:19:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:19:11.257 INFO:teuthology.orchestra.run.smithi203.stdout:client4714 2024-04-26T23:19:11.257 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/399006d3-4db1-43e6-ba81-a79a276eabfd.client4714/status') 2024-04-26T23:19:11.305 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:19:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:19:11.315 INFO:teuthology.orchestra.run.smithi203.stdout:instance: client.4714 (3)192.168.0.2:0/2902659737 2024-04-26T23:19:11.315 INFO:teuthology.orchestra.run.smithi203.stdout:blocklisted: false 2024-04-26T23:19:11.315 INFO:teuthology.orchestra.run.smithi203.stderr:0+1 records in 2024-04-26T23:19:11.315 INFO:teuthology.orchestra.run.smithi203.stderr:0+1 records out 2024-04-26T23:19:11.315 INFO:teuthology.orchestra.run.smithi203.stderr:69 bytes copied, 3.8863e-05 s, 1.8 MB/s 2024-04-26T23:19:11.316 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-26T23:19:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:19:11.322 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-26T23:19:11.340 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:19:11.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.453+0000 7fdf90504640 1 Processor -- start 2024-04-26T23:19:11.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.453+0000 7fdf90504640 1 -- start start 2024-04-26T23:19:11.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.454+0000 7fdf90504640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdf8815f3a0 0x7fdf880d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:11.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.454+0000 7fdf90504640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdf880d2330 0x7fdf880b2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:11.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.454+0000 7fdf90504640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf880d0e20 con 0x7fdf880d1770 2024-04-26T23:19:11.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.454+0000 7fdf90504640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fdf880b1d70 con 0x7fdf8815f3a0 2024-04-26T23:19:11.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.454+0000 7fdf90504640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fdf880b2050 con 0x7fdf880d2330 2024-04-26T23:19:11.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.454+0000 7fdf8e2a0640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdf8815f3a0 0x7fdf880d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:11.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.454+0000 7fdf8e2a0640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdf8815f3a0 0x7fdf880d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41082/0 (socket says 172.21.15.47:41082) 2024-04-26T23:19:11.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.454+0000 7fdf8e2a0640 1 -- 172.21.15.47:0/3363128574 learned_addr learned my addr 172.21.15.47:0/3363128574 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:11.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.454+0000 7fdf8da9f640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fdf880d1770 0x7fdf880d1c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:32960/0 (socket says 172.21.15.47:32960) 2024-04-26T23:19:11.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.454+0000 7fdf8e2a0640 1 -- 172.21.15.47:0/3363128574 >> v2:172.21.15.203:3300/0 conn(0x7fdf880d2330 msgr2=0x7fdf880b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:11.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.454+0000 7fdf8eaa1640 1 --2- 172.21.15.47:0/3363128574 >> v2:172.21.15.203:3300/0 conn(0x7fdf880d2330 0x7fdf880b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:11.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.454+0000 7fdf8d29e640 1 -- 172.21.15.47:0/3363128574 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1078047741 0 0) 0x7fdf880d0e20 con 0x7fdf880d1770 2024-04-26T23:19:11.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.454+0000 7fdf8e2a0640 1 --2- 172.21.15.47:0/3363128574 >> v2:172.21.15.203:3300/0 conn(0x7fdf880d2330 0x7fdf880b2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:11.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.454+0000 7fdf8e2a0640 1 -- 172.21.15.47:0/3363128574 >> v1:172.21.15.47:6789/0 conn(0x7fdf880d1770 legacy=0x7fdf880d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:11.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.454+0000 7fdf8e2a0640 1 -- 172.21.15.47:0/3363128574 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf880b2330 con 0x7fdf8815f3a0 2024-04-26T23:19:11.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.454+0000 7fdf8eaa1640 1 --2- 172.21.15.47:0/3363128574 >> v2:172.21.15.203:3300/0 conn(0x7fdf880d2330 0x7fdf880b2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:19:11.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.455+0000 7fdf8e2a0640 1 --2- 172.21.15.47:0/3363128574 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf8815f3a0 0x7fdf880d06f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fdf880c89f0 tx=0x7fdf7c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70770a5d6cb8933 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:11.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.455+0000 7fdf8d29e640 1 -- 172.21.15.47:0/3363128574 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c037070 con 0x7fdf8815f3a0 2024-04-26T23:19:11.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.455+0000 7fdf8d29e640 1 -- 172.21.15.47:0/3363128574 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf7c032440 con 0x7fdf8815f3a0 2024-04-26T23:19:11.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.455+0000 7fdf8d29e640 1 -- 172.21.15.47:0/3363128574 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c032810 con 0x7fdf8815f3a0 2024-04-26T23:19:11.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.456+0000 7fdf90504640 1 -- 172.21.15.47:0/3363128574 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf8815f3a0 msgr2=0x7fdf880d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:11.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.456+0000 7fdf90504640 1 --2- 172.21.15.47:0/3363128574 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf8815f3a0 0x7fdf880d06f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fdf880c89f0 tx=0x7fdf7c02a270 comp rx=0 tx=0).stop 2024-04-26T23:19:11.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.456+0000 7fdf90504640 1 -- 172.21.15.47:0/3363128574 shutdown_connections 2024-04-26T23:19:11.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.456+0000 7fdf90504640 1 --2- 172.21.15.47:0/3363128574 >> v2:172.21.15.203:3300/0 conn(0x7fdf880d2330 0x7fdf880b2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:11.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.456+0000 7fdf90504640 1 --2- 172.21.15.47:0/3363128574 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf8815f3a0 0x7fdf880d06f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:11.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.456+0000 7fdf90504640 1 -- 172.21.15.47:0/3363128574 >> 172.21.15.47:0/3363128574 conn(0x7fdf880d4f90 msgr2=0x7fdf880d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:11.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.456+0000 7fdf90504640 1 -- 172.21.15.47:0/3363128574 shutdown_connections 2024-04-26T23:19:11.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.456+0000 7fdf90504640 1 -- 172.21.15.47:0/3363128574 wait complete. 2024-04-26T23:19:11.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.457+0000 7fdf90504640 1 Processor -- start 2024-04-26T23:19:11.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.457+0000 7fdf90504640 1 -- start start 2024-04-26T23:19:11.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.458+0000 7fdf90504640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf880d1770 0x7fdf88160310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:11.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.458+0000 7fdf90504640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf880d2330 0x7fdf88160850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:11.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.458+0000 7fdf90504640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf8815f3a0 0x7fdf881f8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:11.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.458+0000 7fdf90504640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fdf880b2f10 con 0x7fdf880d1770 2024-04-26T23:19:11.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.458+0000 7fdf90504640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdf880b2d90 con 0x7fdf880d2330 2024-04-26T23:19:11.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.458+0000 7fdf90504640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fdf880b3090 con 0x7fdf8815f3a0 2024-04-26T23:19:11.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.458+0000 7fdf8e2a0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf880d1770 0x7fdf88160310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:11.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.458+0000 7fdf8e2a0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf880d1770 0x7fdf88160310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41086/0 (socket says 172.21.15.47:41086) 2024-04-26T23:19:11.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.458+0000 7fdf8e2a0640 1 -- 172.21.15.47:0/1017427676 learned_addr learned my addr 172.21.15.47:0/1017427676 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:11.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.458+0000 7fdf8da9f640 1 --2- 172.21.15.47:0/1017427676 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf880d2330 0x7fdf88160850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:11.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.458+0000 7fdf8eaa1640 1 --2- 172.21.15.47:0/1017427676 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf8815f3a0 0x7fdf881f8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:11.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.458+0000 7fdf8e2a0640 1 -- 172.21.15.47:0/1017427676 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf8815f3a0 msgr2=0x7fdf881f8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:11.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.458+0000 7fdf8e2a0640 1 --2- 172.21.15.47:0/1017427676 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf8815f3a0 0x7fdf881f8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:11.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.458+0000 7fdf8e2a0640 1 -- 172.21.15.47:0/1017427676 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf880d2330 msgr2=0x7fdf88160850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:11.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.458+0000 7fdf8e2a0640 1 --2- 172.21.15.47:0/1017427676 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf880d2330 0x7fdf88160850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:11.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.458+0000 7fdf8e2a0640 1 -- 172.21.15.47:0/1017427676 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf881f58c0 con 0x7fdf880d1770 2024-04-26T23:19:11.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.458+0000 7fdf8eaa1640 1 --2- 172.21.15.47:0/1017427676 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf8815f3a0 0x7fdf881f8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:19:11.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.459+0000 7fdf8e2a0640 1 --2- 172.21.15.47:0/1017427676 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf880d1770 0x7fdf88160310 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c02a2b0 tx=0x7fdf7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:11.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.459+0000 7fdf7affd640 1 -- 172.21.15.47:0/1017427676 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c037040 con 0x7fdf880d1770 2024-04-26T23:19:11.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.459+0000 7fdf7affd640 1 -- 172.21.15.47:0/1017427676 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf7c042070 con 0x7fdf880d1770 2024-04-26T23:19:11.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.459+0000 7fdf7affd640 1 -- 172.21.15.47:0/1017427676 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c030bb0 con 0x7fdf880d1770 2024-04-26T23:19:11.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.459+0000 7fdf90504640 1 -- 172.21.15.47:0/1017427676 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf88162650 con 0x7fdf880d1770 2024-04-26T23:19:11.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.459+0000 7fdf90504640 1 -- 172.21.15.47:0/1017427676 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf881f54e0 con 0x7fdf880d1770 2024-04-26T23:19:11.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.460+0000 7fdf90504640 1 -- 172.21.15.47:0/1017427676 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf880694f0 con 0x7fdf880d1770 2024-04-26T23:19:11.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.461+0000 7fdf7affd640 1 -- 172.21.15.47:0/1017427676 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdf7c047460 con 0x7fdf880d1770 2024-04-26T23:19:11.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.461+0000 7fdf7affd640 1 --2- 172.21.15.47:0/1017427676 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdf6c06c8f0 0x7fdf6c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:11.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.461+0000 7fdf7affd640 1 -- 172.21.15.47:0/1017427676 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6764+0+0 (secure 0 0 0) 0x7fdf7c0aca70 con 0x7fdf880d1770 2024-04-26T23:19:11.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.461+0000 7fdf8da9f640 1 --2- 172.21.15.47:0/1017427676 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdf6c06c8f0 0x7fdf6c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:11.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.462+0000 7fdf8da9f640 1 --2- 172.21.15.47:0/1017427676 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdf6c06c8f0 0x7fdf6c06edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdf8815e590 tx=0x7fdf84007d50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:11.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.466+0000 7fdf7affd640 1 -- 172.21.15.47:0/1017427676 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdf7c07ab10 con 0x7fdf880d1770 2024-04-26T23:19:11.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.661+0000 7fdf90504640 1 -- 172.21.15.47:0/1017427676 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdf880cb730 con 0x7fdf880d1770 2024-04-26T23:19:11.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.662+0000 7fdf7affd640 1 -- 172.21.15.47:0/1017427676 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18857 (secure 0 0 0) 0x7fdf7c07a410 con 0x7fdf880d1770 2024-04-26T23:19:11.664 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:19:11.664 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":25,"fsid":"399006d3-4db1-43e6-ba81-a79a276eabfd","created":"2024-04-26T23:18:06.731211+0000","modified":"2024-04-26T23:19:11.120161+0000","last_up_change":"2024-04-26T23:18:17.808736+0000","last_in_change":"2024-04-26T23:18:08.945859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:18:16.937492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:18:29.033575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:18:30.318499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:18:34.014536+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ee091567-fea0-4a14-bf4e-0e16c20bb2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6819","nonce":326520779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6821","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6823","nonce":326520779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6829","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6831","nonce":326520779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6825","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6827","nonce":326520779}]},"public_addr":"172.21.15.47:6819/326520779","cluster_addr":"172.21.15.47:6823/326520779","heartbeat_back_addr":"172.21.15.47:6831/326520779","heartbeat_front_addr":"172.21.15.47:6827/326520779","state":["exists","up"]},{"osd":1,"uuid":"c14440a0-bba5-4931-a6db-b736c08e869b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6809","nonce":1912610931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6811","nonce":1912610931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6815","nonce":1912610931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6813","nonce":1912610931}]},"public_addr":"172.21.15.47:6809/1912610931","cluster_addr":"172.21.15.47:6811/1912610931","heartbeat_back_addr":"172.21.15.47:6815/1912610931","heartbeat_front_addr":"172.21.15.47:6813/1912610931","state":["exists","up"]},{"osd":2,"uuid":"3940f8de-c9b2-4cd3-a911-9c301729eb30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6801","nonce":2569869305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6803","nonce":2569869305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6807","nonce":2569869305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6805","nonce":2569869305}]},"public_addr":"172.21.15.47:6801/2569869305","cluster_addr":"172.21.15.47:6803/2569869305","heartbeat_back_addr":"172.21.15.47:6807/2569869305","heartbeat_front_addr":"172.21.15.47:6805/2569869305","state":["exists","up"]},{"osd":3,"uuid":"1286a67e-8958-426a-8c2e-bc85498c619c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6818","nonce":2791411683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6822","nonce":2791411683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6830","nonce":2791411683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6826","nonce":2791411683}]},"public_addr":"172.21.15.47:6818/2791411683","cluster_addr":"172.21.15.47:6822/2791411683","heartbeat_back_addr":"172.21.15.47:6830/2791411683","heartbeat_front_addr":"172.21.15.47:6826/2791411683","state":["exists","up"]},{"osd":4,"uuid":"d8ee2c20-e5b7-46c2-9999-b234070126cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6809","nonce":441313104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6811","nonce":441313104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6815","nonce":441313104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6813","nonce":441313104}]},"public_addr":"172.21.15.203:6809/441313104","cluster_addr":"172.21.15.203:6811/441313104","heartbeat_back_addr":"172.21.15.203:6815/441313104","heartbeat_front_addr":"172.21.15.203:6813/441313104","state":["exists","up"]},{"osd":5,"uuid":"b453a8f0-f324-4840-9ae0-d67bdadb9f8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6801","nonce":56838444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6803","nonce":56838444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6807","nonce":56838444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6805","nonce":56838444}]},"public_addr":"172.21.15.203:6801/56838444","cluster_addr":"172.21.15.203:6803/56838444","heartbeat_back_addr":"172.21.15.203:6807/56838444","heartbeat_front_addr":"172.21.15.203:6805/56838444","state":["exists","up"]},{"osd":6,"uuid":"a6d31951-2d4b-4e8d-a510-6283b01dc158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6825","nonce":2738517080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6827","nonce":2738517080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6831","nonce":2738517080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6829","nonce":2738517080}]},"public_addr":"172.21.15.203:6825/2738517080","cluster_addr":"172.21.15.203:6827/2738517080","heartbeat_back_addr":"172.21.15.203:6831/2738517080","heartbeat_front_addr":"172.21.15.203:6829/2738517080","state":["exists","up"]},{"osd":7,"uuid":"7119b8c0-fe96-402d-9e75-7b509d9d44d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6817","nonce":1315565058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6819","nonce":1315565058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6823","nonce":1315565058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6821","nonce":1315565058}]},"public_addr":"172.21.15.203:6817/1315565058","cluster_addr":"172.21.15.203:6819/1315565058","heartbeat_back_addr":"172.21.15.203:6823/1315565058","heartbeat_front_addr":"172.21.15.203:6821/1315565058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.121921+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.137494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.084814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.111593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.072430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.125066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.097867+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.057061+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":5,"to":1}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:19:11.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.664+0000 7fdf90504640 1 -- 172.21.15.47:0/1017427676 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdf6c06c8f0 msgr2=0x7fdf6c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:11.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.664+0000 7fdf90504640 1 --2- 172.21.15.47:0/1017427676 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdf6c06c8f0 0x7fdf6c06edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdf8815e590 tx=0x7fdf84007d50 comp rx=0 tx=0).stop 2024-04-26T23:19:11.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.664+0000 7fdf90504640 1 -- 172.21.15.47:0/1017427676 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf880d1770 msgr2=0x7fdf88160310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:11.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.664+0000 7fdf90504640 1 --2- 172.21.15.47:0/1017427676 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf880d1770 0x7fdf88160310 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c02a2b0 tx=0x7fdf7c0046c0 comp rx=0 tx=0).stop 2024-04-26T23:19:11.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.665+0000 7fdf90504640 1 -- 172.21.15.47:0/1017427676 shutdown_connections 2024-04-26T23:19:11.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.665+0000 7fdf90504640 1 --2- 172.21.15.47:0/1017427676 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdf6c06c8f0 0x7fdf6c06edb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:11.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.665+0000 7fdf90504640 1 --2- 172.21.15.47:0/1017427676 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf8815f3a0 0x7fdf881f8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:11.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.665+0000 7fdf90504640 1 --2- 172.21.15.47:0/1017427676 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf880d2330 0x7fdf88160850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:11.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.665+0000 7fdf90504640 1 --2- 172.21.15.47:0/1017427676 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf880d1770 0x7fdf88160310 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:11.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.665+0000 7fdf90504640 1 -- 172.21.15.47:0/1017427676 >> 172.21.15.47:0/1017427676 conn(0x7fdf880d4f90 msgr2=0x7fdf880d42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:11.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.665+0000 7fdf90504640 1 -- 172.21.15.47:0/1017427676 shutdown_connections 2024-04-26T23:19:11.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.665+0000 7fdf90504640 1 -- 172.21.15.47:0/1017427676 wait complete. 2024-04-26T23:19:11.678 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:18:16.937492+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:18:29.033575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:18:30.318499+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T23:18:34.014536+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T23:19:11.678 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:19:11.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.780+0000 7f95eca7e640 1 Processor -- start 2024-04-26T23:19:11.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.780+0000 7f95eca7e640 1 -- start start 2024-04-26T23:19:11.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.781+0000 7f95eca7e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95e80d1a00 0x7f95e80d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:11.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.781+0000 7f95eca7e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f95e80b40d0 0x7f95e80b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:11.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.781+0000 7f95eca7e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95e80d0a10 con 0x7f95e80d2320 2024-04-26T23:19:11.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.781+0000 7f95eca7e640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f95e80d0cf0 con 0x7f95e80b40d0 2024-04-26T23:19:11.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.781+0000 7f95eca7e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f95e80d0fd0 con 0x7f95e80d1a00 2024-04-26T23:19:11.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.781+0000 7f95e6d9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f95e80b40d0 0x7f95e80b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:11.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.782+0000 7f95e6d9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f95e80b40d0 0x7f95e80b4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41090/0 (socket says 172.21.15.47:41090) 2024-04-26T23:19:11.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.782+0000 7f95e6d9d640 1 -- 172.21.15.47:0/607719677 learned_addr learned my addr 172.21.15.47:0/607719677 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:11.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.782+0000 7f95e659c640 1 --2- 172.21.15.47:0/607719677 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95e80d1a00 0x7f95e80d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:11.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.782+0000 7f95e5d9b640 1 --1- 172.21.15.47:0/607719677 >> v1:172.21.15.203:6789/0 conn(0x7f95e80d2320 0x7f95e81625f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.47:49756/0 (socket says 172.21.15.47:49756) 2024-04-26T23:19:11.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.782+0000 7f95e6d9d640 1 -- 172.21.15.47:0/607719677 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95e80d1a00 msgr2=0x7f95e80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:11.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.782+0000 7f95e6d9d640 1 --2- 172.21.15.47:0/607719677 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95e80d1a00 0x7f95e80d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:11.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.782+0000 7f95e559a640 1 -- 172.21.15.47:0/607719677 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3163669077 0 0) 0x7f95e80d0a10 con 0x7f95e80d2320 2024-04-26T23:19:11.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.782+0000 7f95e6d9d640 1 -- 172.21.15.47:0/607719677 >> v1:172.21.15.203:6789/0 conn(0x7f95e80d2320 legacy=0x7f95e81625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:11.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.782+0000 7f95e6d9d640 1 -- 172.21.15.47:0/607719677 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95e80d40a0 con 0x7f95e80b40d0 2024-04-26T23:19:11.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.783+0000 7f95e6d9d640 1 --2- 172.21.15.47:0/607719677 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95e80b40d0 0x7f95e80b4b00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f95d8009e50 tx=0x7f95d802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26e8f54974dc2ef5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:11.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.783+0000 7f95e559a640 1 -- 172.21.15.47:0/607719677 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d8036070 con 0x7f95e80b40d0 2024-04-26T23:19:11.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.783+0000 7f95e559a640 1 -- 172.21.15.47:0/607719677 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95d8031600 con 0x7f95e80b40d0 2024-04-26T23:19:11.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.783+0000 7f95e559a640 1 -- 172.21.15.47:0/607719677 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d80319d0 con 0x7f95e80b40d0 2024-04-26T23:19:11.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.783+0000 7f95eca7e640 1 -- 172.21.15.47:0/607719677 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95e80b40d0 msgr2=0x7f95e80b4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:11.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.783+0000 7f95eca7e640 1 --2- 172.21.15.47:0/607719677 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95e80b40d0 0x7f95e80b4b00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f95d8009e50 tx=0x7f95d802a680 comp rx=0 tx=0).stop 2024-04-26T23:19:11.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.783+0000 7f95eca7e640 1 -- 172.21.15.47:0/607719677 shutdown_connections 2024-04-26T23:19:11.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.783+0000 7f95eca7e640 1 --2- 172.21.15.47:0/607719677 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95e80b40d0 0x7f95e80b4b00 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:11.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.783+0000 7f95eca7e640 1 --2- 172.21.15.47:0/607719677 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95e80d1a00 0x7f95e80d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:11.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.783+0000 7f95eca7e640 1 -- 172.21.15.47:0/607719677 >> 172.21.15.47:0/607719677 conn(0x7f95e80d4fd0 msgr2=0x7f95e80d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:11.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.783+0000 7f95eca7e640 1 -- 172.21.15.47:0/607719677 shutdown_connections 2024-04-26T23:19:11.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.783+0000 7f95eca7e640 1 -- 172.21.15.47:0/607719677 wait complete. 2024-04-26T23:19:11.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.785+0000 7f95eca7e640 1 Processor -- start 2024-04-26T23:19:11.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.785+0000 7f95eca7e640 1 -- start start 2024-04-26T23:19:11.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.785+0000 7f95eca7e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95e80b40d0 0x7f95e81f87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:11.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.785+0000 7f95eca7e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95e80d1a00 0x7f95e80c8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:11.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.785+0000 7f95eca7e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95e80d2320 0x7f95e80b1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:11.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.785+0000 7f95eca7e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f95e80b5040 con 0x7f95e80d1a00 2024-04-26T23:19:11.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.785+0000 7f95eca7e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f95e8162f50 con 0x7f95e80b40d0 2024-04-26T23:19:11.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.785+0000 7f95eca7e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f95e80b51c0 con 0x7f95e80d2320 2024-04-26T23:19:11.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.786+0000 7f95e5d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95e80d1a00 0x7f95e80c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:11.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.786+0000 7f95e5d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95e80d1a00 0x7f95e80c8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41098/0 (socket says 172.21.15.47:41098) 2024-04-26T23:19:11.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.786+0000 7f95e5d9b640 1 -- 172.21.15.47:0/1739922586 learned_addr learned my addr 172.21.15.47:0/1739922586 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:11.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.786+0000 7f95e659c640 1 --2- 172.21.15.47:0/1739922586 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95e80b40d0 0x7f95e81f87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:11.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.786+0000 7f95e6d9d640 1 --2- 172.21.15.47:0/1739922586 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95e80d2320 0x7f95e80b1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:11.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.786+0000 7f95e5d9b640 1 -- 172.21.15.47:0/1739922586 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95e80d2320 msgr2=0x7f95e80b1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:11.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.786+0000 7f95e5d9b640 1 --2- 172.21.15.47:0/1739922586 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95e80d2320 0x7f95e80b1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:11.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.786+0000 7f95e5d9b640 1 -- 172.21.15.47:0/1739922586 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95e80b40d0 msgr2=0x7f95e81f87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:11.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.786+0000 7f95e5d9b640 1 --2- 172.21.15.47:0/1739922586 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95e80b40d0 0x7f95e81f87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:11.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.786+0000 7f95e5d9b640 1 -- 172.21.15.47:0/1739922586 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95e81f9630 con 0x7f95e80d1a00 2024-04-26T23:19:11.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.786+0000 7f95e6d9d640 1 --2- 172.21.15.47:0/1739922586 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95e80d2320 0x7f95e80b1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:19:11.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.786+0000 7f95e5d9b640 1 --2- 172.21.15.47:0/1739922586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95e80d1a00 0x7f95e80c8810 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f95d000c430 tx=0x7f95d000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:11.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.786+0000 7f95deffd640 1 -- 172.21.15.47:0/1739922586 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d00074e0 con 0x7f95e80d1a00 2024-04-26T23:19:11.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.786+0000 7f95deffd640 1 -- 172.21.15.47:0/1739922586 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95d00047e0 con 0x7f95e80d1a00 2024-04-26T23:19:11.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.786+0000 7f95eca7e640 1 -- 172.21.15.47:0/1739922586 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95e81f8df0 con 0x7f95e80d1a00 2024-04-26T23:19:11.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.786+0000 7f95deffd640 1 -- 172.21.15.47:0/1739922586 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d0004bd0 con 0x7f95e80d1a00 2024-04-26T23:19:11.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.786+0000 7f95eca7e640 1 -- 172.21.15.47:0/1739922586 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95e80ca450 con 0x7f95e80d1a00 2024-04-26T23:19:11.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.787+0000 7f95eca7e640 1 -- 172.21.15.47:0/1739922586 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95e80694f0 con 0x7f95e80d1a00 2024-04-26T23:19:11.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.788+0000 7f95deffd640 1 -- 172.21.15.47:0/1739922586 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f95d0011070 con 0x7f95e80d1a00 2024-04-26T23:19:11.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.788+0000 7f95deffd640 1 --2- 172.21.15.47:0/1739922586 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f95c806c8f0 0x7f95c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:11.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.789+0000 7f95deffd640 1 -- 172.21.15.47:0/1739922586 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6764+0+0 (secure 0 0 0) 0x7f95d008baf0 con 0x7f95e80d1a00 2024-04-26T23:19:11.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.789+0000 7f95e659c640 1 --2- 172.21.15.47:0/1739922586 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f95c806c8f0 0x7f95c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:11.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.789+0000 7f95e659c640 1 --2- 172.21.15.47:0/1739922586 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f95c806c8f0 0x7f95c806edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f95e0009b40 tx=0x7f95e0009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:11.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.793+0000 7f95deffd640 1 -- 172.21.15.47:0/1739922586 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f95e81f8df0 con 0x7f95e80d1a00 2024-04-26T23:19:11.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.980+0000 7f95eca7e640 1 -- 172.21.15.47:0/1739922586 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f95e80d1f80 con 0x7f95e80d1a00 2024-04-26T23:19:11.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.980+0000 7f95deffd640 1 -- 172.21.15.47:0/1739922586 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f95d0059af0 con 0x7f95e80d1a00 2024-04-26T23:19:11.982 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-26T23:19:11.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.982+0000 7f95eca7e640 1 -- 172.21.15.47:0/1739922586 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f95c806c8f0 msgr2=0x7f95c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:11.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.982+0000 7f95eca7e640 1 --2- 172.21.15.47:0/1739922586 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f95c806c8f0 0x7f95c806edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f95e0009b40 tx=0x7f95e0009190 comp rx=0 tx=0).stop 2024-04-26T23:19:11.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.982+0000 7f95eca7e640 1 -- 172.21.15.47:0/1739922586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95e80d1a00 msgr2=0x7f95e80c8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:11.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.982+0000 7f95eca7e640 1 --2- 172.21.15.47:0/1739922586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95e80d1a00 0x7f95e80c8810 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f95d000c430 tx=0x7f95d000c8f0 comp rx=0 tx=0).stop 2024-04-26T23:19:11.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.982+0000 7f95eca7e640 1 -- 172.21.15.47:0/1739922586 shutdown_connections 2024-04-26T23:19:11.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.982+0000 7f95eca7e640 1 --2- 172.21.15.47:0/1739922586 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f95c806c8f0 0x7f95c806edb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:11.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.982+0000 7f95eca7e640 1 --2- 172.21.15.47:0/1739922586 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95e80d2320 0x7f95e80b1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:11.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.982+0000 7f95eca7e640 1 --2- 172.21.15.47:0/1739922586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95e80d1a00 0x7f95e80c8810 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:11.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.982+0000 7f95eca7e640 1 --2- 172.21.15.47:0/1739922586 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95e80b40d0 0x7f95e81f87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:11.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.982+0000 7f95eca7e640 1 -- 172.21.15.47:0/1739922586 >> 172.21.15.47:0/1739922586 conn(0x7f95e80d4fd0 msgr2=0x7f95e806d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:11.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.982+0000 7f95eca7e640 1 -- 172.21.15.47:0/1739922586 shutdown_connections 2024-04-26T23:19:11.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:11.982+0000 7f95eca7e640 1 -- 172.21.15.47:0/1739922586 wait complete. 2024-04-26T23:19:11.995 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:19:12.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.099+0000 7f491ad24640 1 Processor -- start 2024-04-26T23:19:12.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.099+0000 7f491ad24640 1 -- start start 2024-04-26T23:19:12.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.100+0000 7f491ad24640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f491415f550 0x7f49140d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:12.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.100+0000 7f491ad24640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f49140d3bc0 0x7f49140d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:12.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.100+0000 7f491ad24640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49140d1040 con 0x7f491415f170 2024-04-26T23:19:12.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.100+0000 7f491ad24640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f49140c9430 con 0x7f49140d3bc0 2024-04-26T23:19:12.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.100+0000 7f491ad24640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f49140c9680 con 0x7f491415f550 2024-04-26T23:19:12.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.100+0000 7f49192c1640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f49140d3bc0 0x7f49140d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:12.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.100+0000 7f49192c1640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f49140d3bc0 0x7f49140d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41110/0 (socket says 172.21.15.47:41110) 2024-04-26T23:19:12.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.100+0000 7f49192c1640 1 -- 172.21.15.47:0/2767284540 learned_addr learned my addr 172.21.15.47:0/2767284540 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:12.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.100+0000 7f4913fff640 1 --2- 172.21.15.47:0/2767284540 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f491415f550 0x7f49140d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:12.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.100+0000 7f49192c1640 1 -- 172.21.15.47:0/2767284540 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f491415f550 msgr2=0x7f49140d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:12.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.100+0000 7f49192c1640 1 --2- 172.21.15.47:0/2767284540 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f491415f550 0x7f49140d5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:12.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.100+0000 7f49192c1640 1 -- 172.21.15.47:0/2767284540 >> v1:172.21.15.203:6789/0 conn(0x7f491415f170 legacy=0x7f49140d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:12.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.100+0000 7f49192c1640 1 -- 172.21.15.47:0/2767284540 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49140c9960 con 0x7f49140d3bc0 2024-04-26T23:19:12.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.101+0000 7f49192c1640 1 --2- 172.21.15.47:0/2767284540 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49140d3bc0 0x7f49140d2cb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f490c009e50 tx=0x7f490c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee7e95bf373bccfb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:12.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.101+0000 7f49137fe640 1 -- 172.21.15.47:0/2767284540 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c036070 con 0x7f49140d3bc0 2024-04-26T23:19:12.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.101+0000 7f49137fe640 1 -- 172.21.15.47:0/2767284540 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f490c031600 con 0x7f49140d3bc0 2024-04-26T23:19:12.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.101+0000 7f49137fe640 1 -- 172.21.15.47:0/2767284540 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c0319d0 con 0x7f49140d3bc0 2024-04-26T23:19:12.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.101+0000 7f491ad24640 1 -- 172.21.15.47:0/2767284540 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49140d3bc0 msgr2=0x7f49140d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:12.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.101+0000 7f491ad24640 1 --2- 172.21.15.47:0/2767284540 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49140d3bc0 0x7f49140d2cb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f490c009e50 tx=0x7f490c02a710 comp rx=0 tx=0).stop 2024-04-26T23:19:12.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.102+0000 7f491ad24640 1 -- 172.21.15.47:0/2767284540 shutdown_connections 2024-04-26T23:19:12.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.102+0000 7f491ad24640 1 --2- 172.21.15.47:0/2767284540 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49140d3bc0 0x7f49140d2cb0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:12.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.102+0000 7f491ad24640 1 --2- 172.21.15.47:0/2767284540 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f491415f550 0x7f49140d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:12.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.102+0000 7f491ad24640 1 -- 172.21.15.47:0/2767284540 >> 172.21.15.47:0/2767284540 conn(0x7f49140d4f60 msgr2=0x7f49140d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:12.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.102+0000 7f491ad24640 1 -- 172.21.15.47:0/2767284540 shutdown_connections 2024-04-26T23:19:12.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.102+0000 7f491ad24640 1 -- 172.21.15.47:0/2767284540 wait complete. 2024-04-26T23:19:12.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.103+0000 7f491ad24640 1 Processor -- start 2024-04-26T23:19:12.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.103+0000 7f491ad24640 1 -- start start 2024-04-26T23:19:12.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.104+0000 7f491ad24640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49140d3bc0 0x7f491414f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:12.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.104+0000 7f491ad24640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f491415f170 0x7f49140cd490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:12.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.104+0000 7f491ad24640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49140ce790 0x7f49140cec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:12.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.104+0000 7f491ad24640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f49140ca5c0 con 0x7f49140ce790 2024-04-26T23:19:12.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.104+0000 7f491ad24640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f49140ca440 con 0x7f49140d3bc0 2024-04-26T23:19:12.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.104+0000 7f491ad24640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f49140ca740 con 0x7f491415f170 2024-04-26T23:19:12.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.104+0000 7f49192c1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49140ce790 0x7f49140cec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:12.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.104+0000 7f49192c1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49140ce790 0x7f49140cec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41116/0 (socket says 172.21.15.47:41116) 2024-04-26T23:19:12.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.104+0000 7f49192c1640 1 -- 172.21.15.47:0/2497087675 learned_addr learned my addr 172.21.15.47:0/2497087675 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:12.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.104+0000 7f4913fff640 1 --2- 172.21.15.47:0/2497087675 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f491415f170 0x7f49140cd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:12.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.104+0000 7f4918ac0640 1 --2- 172.21.15.47:0/2497087675 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49140d3bc0 0x7f491414f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:12.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.104+0000 7f49192c1640 1 -- 172.21.15.47:0/2497087675 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f491415f170 msgr2=0x7f49140cd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:12.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.104+0000 7f49192c1640 1 --2- 172.21.15.47:0/2497087675 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f491415f170 0x7f49140cd490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:12.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.104+0000 7f49192c1640 1 -- 172.21.15.47:0/2497087675 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49140d3bc0 msgr2=0x7f491414f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:12.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.104+0000 7f49192c1640 1 --2- 172.21.15.47:0/2497087675 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49140d3bc0 0x7f491414f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:12.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.104+0000 7f49192c1640 1 -- 172.21.15.47:0/2497087675 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49140cc3f0 con 0x7f49140ce790 2024-04-26T23:19:12.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.104+0000 7f49192c1640 1 --2- 172.21.15.47:0/2497087675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49140ce790 0x7f49140cec40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f490c02ad00 tx=0x7f490c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:12.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.104+0000 7f49117fa640 1 -- 172.21.15.47:0/2497087675 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c036030 con 0x7f49140ce790 2024-04-26T23:19:12.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.105+0000 7f49117fa640 1 -- 172.21.15.47:0/2497087675 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f490c040070 con 0x7f49140ce790 2024-04-26T23:19:12.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.105+0000 7f491ad24640 1 -- 172.21.15.47:0/2497087675 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49140cc6d0 con 0x7f49140ce790 2024-04-26T23:19:12.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.105+0000 7f49117fa640 1 -- 172.21.15.47:0/2497087675 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c009510 con 0x7f49140ce790 2024-04-26T23:19:12.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.105+0000 7f491ad24640 1 -- 172.21.15.47:0/2497087675 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49140ccc90 con 0x7f49140ce790 2024-04-26T23:19:12.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.106+0000 7f491ad24640 1 -- 172.21.15.47:0/2497087675 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49140694f0 con 0x7f49140ce790 2024-04-26T23:19:12.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.106+0000 7f49117fa640 1 -- 172.21.15.47:0/2497087675 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f490c007bf0 con 0x7f49140ce790 2024-04-26T23:19:12.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.107+0000 7f49117fa640 1 --2- 172.21.15.47:0/2497087675 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f490006c920 0x7f490006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:12.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.107+0000 7f49117fa640 1 -- 172.21.15.47:0/2497087675 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6764+0+0 (secure 0 0 0) 0x7f490c0ac540 con 0x7f49140ce790 2024-04-26T23:19:12.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.107+0000 7f4918ac0640 1 --2- 172.21.15.47:0/2497087675 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f490006c920 0x7f490006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:12.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.107+0000 7f4918ac0640 1 --2- 172.21.15.47:0/2497087675 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f490006c920 0x7f490006ede0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4904001780 tx=0x7f4904009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:12.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.111+0000 7f49117fa640 1 -- 172.21.15.47:0/2497087675 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f490c079f00 con 0x7f49140ce790 2024-04-26T23:19:12.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.302+0000 7f491ad24640 1 -- 172.21.15.47:0/2497087675 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f491414f850 con 0x7f49140ce790 2024-04-26T23:19:12.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.302+0000 7f49117fa640 1 -- 172.21.15.47:0/2497087675 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f490c07e200 con 0x7f49140ce790 2024-04-26T23:19:12.304 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-26T23:19:12.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.304+0000 7f491ad24640 1 -- 172.21.15.47:0/2497087675 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f490006c920 msgr2=0x7f490006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:12.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.304+0000 7f491ad24640 1 --2- 172.21.15.47:0/2497087675 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f490006c920 0x7f490006ede0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4904001780 tx=0x7f4904009040 comp rx=0 tx=0).stop 2024-04-26T23:19:12.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.304+0000 7f491ad24640 1 -- 172.21.15.47:0/2497087675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49140ce790 msgr2=0x7f49140cec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:12.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.304+0000 7f491ad24640 1 --2- 172.21.15.47:0/2497087675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49140ce790 0x7f49140cec40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f490c02ad00 tx=0x7f490c004a40 comp rx=0 tx=0).stop 2024-04-26T23:19:12.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.304+0000 7f491ad24640 1 -- 172.21.15.47:0/2497087675 shutdown_connections 2024-04-26T23:19:12.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.304+0000 7f491ad24640 1 --2- 172.21.15.47:0/2497087675 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f490006c920 0x7f490006ede0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:12.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.304+0000 7f491ad24640 1 --2- 172.21.15.47:0/2497087675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49140ce790 0x7f49140cec40 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:12.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.304+0000 7f491ad24640 1 --2- 172.21.15.47:0/2497087675 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f491415f170 0x7f49140cd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:12.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.304+0000 7f491ad24640 1 --2- 172.21.15.47:0/2497087675 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49140d3bc0 0x7f491414f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:12.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.304+0000 7f491ad24640 1 -- 172.21.15.47:0/2497087675 >> 172.21.15.47:0/2497087675 conn(0x7f49140d4f60 msgr2=0x7f49140d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:12.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.304+0000 7f491ad24640 1 -- 172.21.15.47:0/2497087675 shutdown_connections 2024-04-26T23:19:12.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.304+0000 7f491ad24640 1 -- 172.21.15.47:0/2497087675 wait complete. 2024-04-26T23:19:12.317 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:19:12.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.421+0000 7fb9da293640 1 Processor -- start 2024-04-26T23:19:12.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.421+0000 7fb9da293640 1 -- start start 2024-04-26T23:19:12.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.422+0000 7fb9da293640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb9d415f170 0x7fb9d40d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:12.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.422+0000 7fb9da293640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9d40d1040 con 0x7fb9d415f550 2024-04-26T23:19:12.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.422+0000 7fb9da293640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9d40c9680 con 0x7fb9d40d3bc0 2024-04-26T23:19:12.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.422+0000 7fb9da293640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fb9d40c9430 con 0x7fb9d415f170 2024-04-26T23:19:12.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.422+0000 7fb9d3fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb9d415f170 0x7fb9d40d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:12.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.422+0000 7fb9d37fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb9d415f550 0x7fb9d40d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:32976/0 (socket says 172.21.15.47:32976) 2024-04-26T23:19:12.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.422+0000 7fb9d37fe640 1 -- 172.21.15.47:0/4054129467 learned_addr learned my addr 172.21.15.47:0/4054129467 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:12.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.422+0000 7fb9d3fff640 1 -- 172.21.15.47:0/4054129467 >> v1:172.21.15.203:6789/0 conn(0x7fb9d40d3bc0 legacy=0x7fb9d40ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:12.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.422+0000 7fb9d3fff640 1 -- 172.21.15.47:0/4054129467 >> v1:172.21.15.47:6789/0 conn(0x7fb9d415f550 legacy=0x7fb9d40d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:12.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.422+0000 7fb9d2ffd640 1 -- 172.21.15.47:0/4054129467 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3423132760 0 0) 0x7fb9c8001110 con 0x7fb9d415f550 2024-04-26T23:19:12.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.422+0000 7fb9d3fff640 1 -- 172.21.15.47:0/4054129467 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9d40c9960 con 0x7fb9d415f170 2024-04-26T23:19:12.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.423+0000 7fb9d3fff640 1 --2- 172.21.15.47:0/4054129467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9d415f170 0x7fb9d40d5a80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb9d40c95b0 tx=0x7fb9c402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=790735d8aca5601e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:12.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.423+0000 7fb9d2ffd640 1 -- 172.21.15.47:0/4054129467 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c4037070 con 0x7fb9d415f170 2024-04-26T23:19:12.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.423+0000 7fb9d2ffd640 1 -- 172.21.15.47:0/4054129467 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9c4032440 con 0x7fb9d415f170 2024-04-26T23:19:12.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.423+0000 7fb9d2ffd640 1 -- 172.21.15.47:0/4054129467 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c4032810 con 0x7fb9d415f170 2024-04-26T23:19:12.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.424+0000 7fb9da293640 1 -- 172.21.15.47:0/4054129467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9d415f170 msgr2=0x7fb9d40d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:12.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.424+0000 7fb9da293640 1 --2- 172.21.15.47:0/4054129467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9d415f170 0x7fb9d40d5a80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb9d40c95b0 tx=0x7fb9c402a200 comp rx=0 tx=0).stop 2024-04-26T23:19:12.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.424+0000 7fb9da293640 1 -- 172.21.15.47:0/4054129467 shutdown_connections 2024-04-26T23:19:12.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.424+0000 7fb9da293640 1 --2- 172.21.15.47:0/4054129467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9d415f170 0x7fb9d40d5a80 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:12.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.424+0000 7fb9da293640 1 -- 172.21.15.47:0/4054129467 >> 172.21.15.47:0/4054129467 conn(0x7fb9d40d4f60 msgr2=0x7fb9d40d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:12.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.424+0000 7fb9da293640 1 -- 172.21.15.47:0/4054129467 shutdown_connections 2024-04-26T23:19:12.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.424+0000 7fb9da293640 1 -- 172.21.15.47:0/4054129467 wait complete. 2024-04-26T23:19:12.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.425+0000 7fb9da293640 1 Processor -- start 2024-04-26T23:19:12.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.425+0000 7fb9da293640 1 -- start start 2024-04-26T23:19:12.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.426+0000 7fb9da293640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb9d40d3bc0 0x7fb9d40cd040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:12.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.426+0000 7fb9da293640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb9d415f170 0x7fb9d415a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:12.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.426+0000 7fb9da293640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9d415bf80 0x7fb9d4158010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:12.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.426+0000 7fb9da293640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d40d2e30 con 0x7fb9d415bf80 2024-04-26T23:19:12.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.426+0000 7fb9da293640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d40d2cb0 con 0x7fb9d415f170 2024-04-26T23:19:12.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.426+0000 7fb9da293640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb9d40d2fb0 con 0x7fb9d40d3bc0 2024-04-26T23:19:12.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.426+0000 7fb9d8830640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9d415bf80 0x7fb9d4158010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:12.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.426+0000 7fb9d8830640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9d415bf80 0x7fb9d4158010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41134/0 (socket says 172.21.15.47:41134) 2024-04-26T23:19:12.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.426+0000 7fb9d8830640 1 -- 172.21.15.47:0/1763631224 learned_addr learned my addr 172.21.15.47:0/1763631224 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:12.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.426+0000 7fb9d3fff640 1 --2- 172.21.15.47:0/1763631224 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb9d40d3bc0 0x7fb9d40cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:12.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.426+0000 7fb9d37fe640 1 --2- 172.21.15.47:0/1763631224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb9d415f170 0x7fb9d415a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:12.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.426+0000 7fb9d8830640 1 -- 172.21.15.47:0/1763631224 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb9d40d3bc0 msgr2=0x7fb9d40cd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:12.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.426+0000 7fb9d8830640 1 --2- 172.21.15.47:0/1763631224 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb9d40d3bc0 0x7fb9d40cd040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:12.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.426+0000 7fb9d8830640 1 -- 172.21.15.47:0/1763631224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb9d415f170 msgr2=0x7fb9d415a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:12.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.426+0000 7fb9d8830640 1 --2- 172.21.15.47:0/1763631224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb9d415f170 0x7fb9d415a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:12.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.426+0000 7fb9d8830640 1 -- 172.21.15.47:0/1763631224 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9d415ac60 con 0x7fb9d415bf80 2024-04-26T23:19:12.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.426+0000 7fb9d8830640 1 --2- 172.21.15.47:0/1763631224 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9d415bf80 0x7fb9d4158010 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc00c780 tx=0x7fb9cc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:12.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.427+0000 7fb9d0ff9640 1 -- 172.21.15.47:0/1763631224 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9cc004b60 con 0x7fb9d415bf80 2024-04-26T23:19:12.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.427+0000 7fb9d0ff9640 1 -- 172.21.15.47:0/1763631224 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9cc004d00 con 0x7fb9d415bf80 2024-04-26T23:19:12.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.427+0000 7fb9da293640 1 -- 172.21.15.47:0/1763631224 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9d415aee0 con 0x7fb9d415bf80 2024-04-26T23:19:12.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.427+0000 7fb9d0ff9640 1 -- 172.21.15.47:0/1763631224 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9cc00aa90 con 0x7fb9d415bf80 2024-04-26T23:19:12.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.427+0000 7fb9da293640 1 -- 172.21.15.47:0/1763631224 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9d415b4a0 con 0x7fb9d415bf80 2024-04-26T23:19:12.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.428+0000 7fb9da293640 1 -- 172.21.15.47:0/1763631224 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9d415bc70 con 0x7fb9d415bf80 2024-04-26T23:19:12.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.428+0000 7fb9d0ff9640 1 -- 172.21.15.47:0/1763631224 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb9cc00ac30 con 0x7fb9d415bf80 2024-04-26T23:19:12.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.428+0000 7fb9d0ff9640 1 --2- 172.21.15.47:0/1763631224 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb9c006c8f0 0x7fb9c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:12.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.429+0000 7fb9d0ff9640 1 -- 172.21.15.47:0/1763631224 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7fb9cc08c190 con 0x7fb9d415bf80 2024-04-26T23:19:12.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.429+0000 7fb9d3fff640 1 --2- 172.21.15.47:0/1763631224 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb9c006c8f0 0x7fb9c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:12.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.429+0000 7fb9d3fff640 1 --2- 172.21.15.47:0/1763631224 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb9c006c8f0 0x7fb9c006edb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4034000 tx=0x7fb9c402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:12.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.433+0000 7fb9d0ff9640 1 -- 172.21.15.47:0/1763631224 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb9cc020070 con 0x7fb9d415bf80 2024-04-26T23:19:12.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.618+0000 7fb9da293640 1 -- 172.21.15.47:0/1763631224 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb9d4074590 con 0x7fb9d415bf80 2024-04-26T23:19:12.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.618+0000 7fb9d0ff9640 1 -- 172.21.15.47:0/1763631224 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7fb9d415ac60 con 0x7fb9d415bf80 2024-04-26T23:19:12.620 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-26T23:19:12.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.620+0000 7fb9da293640 1 -- 172.21.15.47:0/1763631224 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb9c006c8f0 msgr2=0x7fb9c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:12.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.620+0000 7fb9da293640 1 --2- 172.21.15.47:0/1763631224 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb9c006c8f0 0x7fb9c006edb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4034000 tx=0x7fb9c402e000 comp rx=0 tx=0).stop 2024-04-26T23:19:12.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.620+0000 7fb9da293640 1 -- 172.21.15.47:0/1763631224 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9d415bf80 msgr2=0x7fb9d4158010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:12.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.620+0000 7fb9da293640 1 --2- 172.21.15.47:0/1763631224 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9d415bf80 0x7fb9d4158010 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc00c780 tx=0x7fb9cc00cc40 comp rx=0 tx=0).stop 2024-04-26T23:19:12.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.620+0000 7fb9da293640 1 -- 172.21.15.47:0/1763631224 shutdown_connections 2024-04-26T23:19:12.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.620+0000 7fb9da293640 1 --2- 172.21.15.47:0/1763631224 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb9c006c8f0 0x7fb9c006edb0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:12.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.620+0000 7fb9da293640 1 --2- 172.21.15.47:0/1763631224 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9d415bf80 0x7fb9d4158010 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:12.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.620+0000 7fb9da293640 1 --2- 172.21.15.47:0/1763631224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb9d415f170 0x7fb9d415a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:12.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.620+0000 7fb9da293640 1 --2- 172.21.15.47:0/1763631224 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb9d40d3bc0 0x7fb9d40cd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:12.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.620+0000 7fb9da293640 1 -- 172.21.15.47:0/1763631224 >> 172.21.15.47:0/1763631224 conn(0x7fb9d40d4f60 msgr2=0x7fb9d40d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:12.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.621+0000 7fb9da293640 1 -- 172.21.15.47:0/1763631224 shutdown_connections 2024-04-26T23:19:12.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.621+0000 7fb9da293640 1 -- 172.21.15.47:0/1763631224 wait complete. 2024-04-26T23:19:12.633 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T23:19:12.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.735+0000 7fb830946640 1 Processor -- start 2024-04-26T23:19:12.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.735+0000 7fb830946640 1 -- start start 2024-04-26T23:19:12.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.736+0000 7fb830946640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb82c15f170 0x7fb82c0d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:12.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.736+0000 7fb830946640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb82c0d3bc0 0x7fb82c0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:12.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.736+0000 7fb830946640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb82c0d1040 con 0x7fb82c15f550 2024-04-26T23:19:12.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.736+0000 7fb830946640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fb82c0c9430 con 0x7fb82c15f170 2024-04-26T23:19:12.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.736+0000 7fb830946640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb82c0c9680 con 0x7fb82c0d3bc0 2024-04-26T23:19:12.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.736+0000 7fb82a59c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb82c15f170 0x7fb82c0d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:12.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.736+0000 7fb82a59c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb82c15f170 0x7fb82c0d5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41146/0 (socket says 172.21.15.47:41146) 2024-04-26T23:19:12.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.736+0000 7fb82a59c640 1 -- 172.21.15.47:0/3384381510 learned_addr learned my addr 172.21.15.47:0/3384381510 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:12.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.737+0000 7fb82ad9d640 1 --2- 172.21.15.47:0/3384381510 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb82c0d3bc0 0x7fb82c0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:12.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.737+0000 7fb82a59c640 1 -- 172.21.15.47:0/3384381510 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb82c0d3bc0 msgr2=0x7fb82c0d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:12.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.737+0000 7fb82a59c640 1 --2- 172.21.15.47:0/3384381510 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb82c0d3bc0 0x7fb82c0d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:12.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.737+0000 7fb82a59c640 1 -- 172.21.15.47:0/3384381510 >> v1:172.21.15.203:6789/0 conn(0x7fb82c15f550 legacy=0x7fb82c0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:12.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.737+0000 7fb82a59c640 1 -- 172.21.15.47:0/3384381510 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb82c0c9960 con 0x7fb82c15f170 2024-04-26T23:19:12.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.738+0000 7fb82a59c640 1 --2- 172.21.15.47:0/3384381510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb82c15f170 0x7fb82c0d5a80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb8240098a0 tx=0x7fb82402a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e53980aeaf7c3b34 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:12.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.738+0000 7fb82959a640 1 -- 172.21.15.47:0/3384381510 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb824037070 con 0x7fb82c15f170 2024-04-26T23:19:12.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.738+0000 7fb82959a640 1 -- 172.21.15.47:0/3384381510 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb824032440 con 0x7fb82c15f170 2024-04-26T23:19:12.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.738+0000 7fb82959a640 1 -- 172.21.15.47:0/3384381510 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb824032810 con 0x7fb82c15f170 2024-04-26T23:19:12.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.738+0000 7fb830946640 1 -- 172.21.15.47:0/3384381510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb82c15f170 msgr2=0x7fb82c0d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:12.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.738+0000 7fb830946640 1 --2- 172.21.15.47:0/3384381510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb82c15f170 0x7fb82c0d5a80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb8240098a0 tx=0x7fb82402a290 comp rx=0 tx=0).stop 2024-04-26T23:19:12.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.738+0000 7fb830946640 1 -- 172.21.15.47:0/3384381510 shutdown_connections 2024-04-26T23:19:12.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.738+0000 7fb830946640 1 --2- 172.21.15.47:0/3384381510 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb82c0d3bc0 0x7fb82c0d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:12.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.738+0000 7fb830946640 1 --2- 172.21.15.47:0/3384381510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb82c15f170 0x7fb82c0d5a80 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:12.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.738+0000 7fb830946640 1 -- 172.21.15.47:0/3384381510 >> 172.21.15.47:0/3384381510 conn(0x7fb82c0d4f60 msgr2=0x7fb82c0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:12.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.738+0000 7fb830946640 1 -- 172.21.15.47:0/3384381510 shutdown_connections 2024-04-26T23:19:12.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.738+0000 7fb830946640 1 -- 172.21.15.47:0/3384381510 wait complete. 2024-04-26T23:19:12.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.740+0000 7fb830946640 1 Processor -- start 2024-04-26T23:19:12.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.740+0000 7fb830946640 1 -- start start 2024-04-26T23:19:12.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.740+0000 7fb830946640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb82c0d3bc0 0x7fb82c09f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:12.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.740+0000 7fb830946640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb82c15f170 0x7fb82c0ce3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:12.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.740+0000 7fb830946640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb82c0b3830 0x7fb82c0a11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:12.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.740+0000 7fb830946640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb82c0ca5c0 con 0x7fb82c15f170 2024-04-26T23:19:12.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.740+0000 7fb830946640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb82c0ca440 con 0x7fb82c0b3830 2024-04-26T23:19:12.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.740+0000 7fb830946640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb82c0ca740 con 0x7fb82c0d3bc0 2024-04-26T23:19:12.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.741+0000 7fb829d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb82c15f170 0x7fb82c0ce3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:12.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.741+0000 7fb829d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb82c15f170 0x7fb82c0ce3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41158/0 (socket says 172.21.15.47:41158) 2024-04-26T23:19:12.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.741+0000 7fb829d9b640 1 -- 172.21.15.47:0/536055922 learned_addr learned my addr 172.21.15.47:0/536055922 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:12.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.741+0000 7fb82ad9d640 1 --2- 172.21.15.47:0/536055922 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb82c0b3830 0x7fb82c0a11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:12.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.741+0000 7fb82a59c640 1 --2- 172.21.15.47:0/536055922 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb82c0d3bc0 0x7fb82c09f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:12.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.741+0000 7fb829d9b640 1 -- 172.21.15.47:0/536055922 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb82c0d3bc0 msgr2=0x7fb82c09f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:12.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.741+0000 7fb829d9b640 1 --2- 172.21.15.47:0/536055922 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb82c0d3bc0 0x7fb82c09f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:12.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.741+0000 7fb829d9b640 1 -- 172.21.15.47:0/536055922 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb82c0b3830 msgr2=0x7fb82c0a11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:12.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.741+0000 7fb829d9b640 1 --2- 172.21.15.47:0/536055922 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb82c0b3830 0x7fb82c0a11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:12.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.741+0000 7fb829d9b640 1 -- 172.21.15.47:0/536055922 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb82c0a03c0 con 0x7fb82c15f170 2024-04-26T23:19:12.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.741+0000 7fb829d9b640 1 --2- 172.21.15.47:0/536055922 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb82c15f170 0x7fb82c0ce3e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb81400c3c0 tx=0x7fb81400c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:12.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.741+0000 7fb822ffd640 1 -- 172.21.15.47:0/536055922 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8140052d0 con 0x7fb82c15f170 2024-04-26T23:19:12.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.741+0000 7fb822ffd640 1 -- 172.21.15.47:0/536055922 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb814005490 con 0x7fb82c15f170 2024-04-26T23:19:12.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.741+0000 7fb830946640 1 -- 172.21.15.47:0/536055922 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb82c0a2920 con 0x7fb82c15f170 2024-04-26T23:19:12.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.741+0000 7fb822ffd640 1 -- 172.21.15.47:0/536055922 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb814007260 con 0x7fb82c15f170 2024-04-26T23:19:12.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.741+0000 7fb830946640 1 -- 172.21.15.47:0/536055922 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb82c0a0cb0 con 0x7fb82c15f170 2024-04-26T23:19:12.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.742+0000 7fb830946640 1 -- 172.21.15.47:0/536055922 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb82c0694f0 con 0x7fb82c15f170 2024-04-26T23:19:12.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.743+0000 7fb822ffd640 1 -- 172.21.15.47:0/536055922 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb814007400 con 0x7fb82c15f170 2024-04-26T23:19:12.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.743+0000 7fb822ffd640 1 --2- 172.21.15.47:0/536055922 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb80c06c8f0 0x7fb80c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:12.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.743+0000 7fb822ffd640 1 -- 172.21.15.47:0/536055922 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7fb81408b400 con 0x7fb82c15f170 2024-04-26T23:19:12.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.743+0000 7fb82a59c640 1 --2- 172.21.15.47:0/536055922 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb80c06c8f0 0x7fb80c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:12.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.744+0000 7fb82a59c640 1 --2- 172.21.15.47:0/536055922 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb80c06c8f0 0x7fb80c06edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb824034000 tx=0x7fb82402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:12.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.748+0000 7fb822ffd640 1 -- 172.21.15.47:0/536055922 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb82c0d1040 con 0x7fb82c15f170 2024-04-26T23:19:12.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.936+0000 7fb830946640 1 -- 172.21.15.47:0/536055922 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fb82c0a07a0 con 0x7fb82c15f170 2024-04-26T23:19:12.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.937+0000 7fb822ffd640 1 -- 172.21.15.47:0/536055922 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fb8140594a0 con 0x7fb82c15f170 2024-04-26T23:19:12.939 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-26T23:19:12.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.939+0000 7fb830946640 1 -- 172.21.15.47:0/536055922 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb80c06c8f0 msgr2=0x7fb80c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:12.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.939+0000 7fb830946640 1 --2- 172.21.15.47:0/536055922 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb80c06c8f0 0x7fb80c06edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb824034000 tx=0x7fb82402e000 comp rx=0 tx=0).stop 2024-04-26T23:19:12.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.939+0000 7fb830946640 1 -- 172.21.15.47:0/536055922 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb82c15f170 msgr2=0x7fb82c0ce3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:12.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.939+0000 7fb830946640 1 --2- 172.21.15.47:0/536055922 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb82c15f170 0x7fb82c0ce3e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb81400c3c0 tx=0x7fb81400c7c0 comp rx=0 tx=0).stop 2024-04-26T23:19:12.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.939+0000 7fb830946640 1 -- 172.21.15.47:0/536055922 shutdown_connections 2024-04-26T23:19:12.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.939+0000 7fb830946640 1 --2- 172.21.15.47:0/536055922 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb80c06c8f0 0x7fb80c06edb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:12.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.939+0000 7fb830946640 1 --2- 172.21.15.47:0/536055922 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb82c0b3830 0x7fb82c0a11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:12.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.939+0000 7fb830946640 1 --2- 172.21.15.47:0/536055922 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb82c15f170 0x7fb82c0ce3e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:12.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.939+0000 7fb830946640 1 --2- 172.21.15.47:0/536055922 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb82c0d3bc0 0x7fb82c09f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:12.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.939+0000 7fb830946640 1 -- 172.21.15.47:0/536055922 >> 172.21.15.47:0/536055922 conn(0x7fb82c0d4f60 msgr2=0x7fb82c0c89c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:12.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.939+0000 7fb830946640 1 -- 172.21.15.47:0/536055922 shutdown_connections 2024-04-26T23:19:12.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:12.939+0000 7fb830946640 1 -- 172.21.15.47:0/536055922 wait complete. 2024-04-26T23:19:12.952 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:19:13.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.052+0000 7f8b345b7640 1 Processor -- start 2024-04-26T23:19:13.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.052+0000 7f8b345b7640 1 -- start start 2024-04-26T23:19:13.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.053+0000 7f8b345b7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8b2c15b600 0x7f8b2c071f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:13.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.053+0000 7f8b345b7640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8b2c0cf5e0 0x7f8b2c0ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:13.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.053+0000 7f8b345b7640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8b2c0cbbd0 0x7f8b2c0cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:13.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.053+0000 7f8b345b7640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f8b2c0cb9b0 con 0x7f8b2c0cbbd0 2024-04-26T23:19:13.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.053+0000 7f8b345b7640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f8b2c072450 con 0x7f8b2c0cf5e0 2024-04-26T23:19:13.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.053+0000 7f8b345b7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f8b2c15b9e0 con 0x7f8b2c15b600 2024-04-26T23:19:13.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.053+0000 7f8b32b54640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8b2c0cbbd0 0x7f8b2c0cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:13.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.053+0000 7f8b32b54640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8b2c0cbbd0 0x7f8b2c0cef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41164/0 (socket says 172.21.15.47:41164) 2024-04-26T23:19:13.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.053+0000 7f8b32b54640 1 -- 172.21.15.47:0/3048465964 learned_addr learned my addr 172.21.15.47:0/3048465964 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:13.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.053+0000 7f8b31b52640 1 --2- 172.21.15.47:0/3048465964 >> v2:172.21.15.203:3300/0 conn(0x7f8b2c0cf5e0 0x7f8b2c0ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:13.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.053+0000 7f8b32353640 1 --2- 172.21.15.47:0/3048465964 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8b2c15b600 0x7f8b2c071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:13.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.053+0000 7f8b32b54640 1 -- 172.21.15.47:0/3048465964 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8b2c15b600 msgr2=0x7f8b2c071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:13.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.053+0000 7f8b32b54640 1 --2- 172.21.15.47:0/3048465964 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8b2c15b600 0x7f8b2c071f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:13.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.053+0000 7f8b32b54640 1 -- 172.21.15.47:0/3048465964 >> v2:172.21.15.203:3300/0 conn(0x7f8b2c0cf5e0 msgr2=0x7f8b2c0ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:13.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.053+0000 7f8b32b54640 1 --2- 172.21.15.47:0/3048465964 >> v2:172.21.15.203:3300/0 conn(0x7f8b2c0cf5e0 0x7f8b2c0ce9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:13.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.053+0000 7f8b32b54640 1 -- 172.21.15.47:0/3048465964 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b2c0d0740 con 0x7f8b2c0cbbd0 2024-04-26T23:19:13.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.053+0000 7f8b31b52640 1 --2- 172.21.15.47:0/3048465964 >> v2:172.21.15.203:3300/0 conn(0x7f8b2c0cf5e0 0x7f8b2c0ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:19:13.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.054+0000 7f8b32b54640 1 --2- 172.21.15.47:0/3048465964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b2c0cbbd0 0x7f8b2c0cef10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c0b3850 tx=0x7f8b2402c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48e4019e3ebda309 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:13.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.054+0000 7f8b31351640 1 -- 172.21.15.47:0/3048465964 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b24035400 con 0x7f8b2c0cbbd0 2024-04-26T23:19:13.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.054+0000 7f8b31351640 1 -- 172.21.15.47:0/3048465964 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b240355a0 con 0x7f8b2c0cbbd0 2024-04-26T23:19:13.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.054+0000 7f8b31351640 1 -- 172.21.15.47:0/3048465964 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b24035970 con 0x7f8b2c0cbbd0 2024-04-26T23:19:13.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.054+0000 7f8b345b7640 1 -- 172.21.15.47:0/3048465964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b2c0cbbd0 msgr2=0x7f8b2c0cef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:13.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.055+0000 7f8b345b7640 1 --2- 172.21.15.47:0/3048465964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b2c0cbbd0 0x7f8b2c0cef10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c0b3850 tx=0x7f8b2402c8a0 comp rx=0 tx=0).stop 2024-04-26T23:19:13.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.055+0000 7f8b345b7640 1 -- 172.21.15.47:0/3048465964 shutdown_connections 2024-04-26T23:19:13.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.055+0000 7f8b345b7640 1 --2- 172.21.15.47:0/3048465964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b2c0cbbd0 0x7f8b2c0cef10 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:13.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.055+0000 7f8b345b7640 1 --2- 172.21.15.47:0/3048465964 >> v2:172.21.15.203:3300/0 conn(0x7f8b2c0cf5e0 0x7f8b2c0ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:13.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.055+0000 7f8b345b7640 1 --2- 172.21.15.47:0/3048465964 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8b2c15b600 0x7f8b2c071f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:13.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.055+0000 7f8b345b7640 1 -- 172.21.15.47:0/3048465964 >> 172.21.15.47:0/3048465964 conn(0x7f8b2c0d0be0 msgr2=0x7f8b2c0d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:13.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.055+0000 7f8b345b7640 1 -- 172.21.15.47:0/3048465964 shutdown_connections 2024-04-26T23:19:13.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.055+0000 7f8b345b7640 1 -- 172.21.15.47:0/3048465964 wait complete. 2024-04-26T23:19:13.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.056+0000 7f8b345b7640 1 Processor -- start 2024-04-26T23:19:13.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.056+0000 7f8b345b7640 1 -- start start 2024-04-26T23:19:13.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.057+0000 7f8b345b7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b2c0cbbd0 0x7f8b2c15f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:13.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.057+0000 7f8b345b7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b2c0cf5e0 0x7f8b2c1f5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:13.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.057+0000 7f8b345b7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8b2c15b600 0x7f8b2c155dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:13.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.057+0000 7f8b345b7640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2c0d02f0 con 0x7f8b2c0cf5e0 2024-04-26T23:19:13.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.057+0000 7f8b345b7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2c0cb640 con 0x7f8b2c0cbbd0 2024-04-26T23:19:13.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.057+0000 7f8b345b7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f8b2c0d0470 con 0x7f8b2c15b600 2024-04-26T23:19:13.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.057+0000 7f8b31b52640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b2c0cf5e0 0x7f8b2c1f5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:13.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.057+0000 7f8b31b52640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b2c0cf5e0 0x7f8b2c1f5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41174/0 (socket says 172.21.15.47:41174) 2024-04-26T23:19:13.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.057+0000 7f8b31b52640 1 -- 172.21.15.47:0/2996127597 learned_addr learned my addr 172.21.15.47:0/2996127597 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:13.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.057+0000 7f8b32353640 1 --2- 172.21.15.47:0/2996127597 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b2c0cbbd0 0x7f8b2c15f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:13.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.057+0000 7f8b32b54640 1 --2- 172.21.15.47:0/2996127597 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8b2c15b600 0x7f8b2c155dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:13.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.057+0000 7f8b31b52640 1 -- 172.21.15.47:0/2996127597 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8b2c15b600 msgr2=0x7f8b2c155dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:13.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.057+0000 7f8b31b52640 1 --2- 172.21.15.47:0/2996127597 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8b2c15b600 0x7f8b2c155dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:13.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.057+0000 7f8b31b52640 1 -- 172.21.15.47:0/2996127597 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b2c0cbbd0 msgr2=0x7f8b2c15f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:13.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.057+0000 7f8b31b52640 1 --2- 172.21.15.47:0/2996127597 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b2c0cbbd0 0x7f8b2c15f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:13.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.057+0000 7f8b31b52640 1 -- 172.21.15.47:0/2996127597 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b2c1f5b10 con 0x7f8b2c0cf5e0 2024-04-26T23:19:13.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.057+0000 7f8b31b52640 1 --2- 172.21.15.47:0/2996127597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b2c0cf5e0 0x7f8b2c1f5050 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c00c7c0 tx=0x7f8b1c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:13.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.057+0000 7f8b22ffd640 1 -- 172.21.15.47:0/2996127597 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b1c015070 con 0x7f8b2c0cf5e0 2024-04-26T23:19:13.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.057+0000 7f8b22ffd640 1 -- 172.21.15.47:0/2996127597 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b1c007a30 con 0x7f8b2c0cf5e0 2024-04-26T23:19:13.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.057+0000 7f8b22ffd640 1 -- 172.21.15.47:0/2996127597 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b1c007e00 con 0x7f8b2c0cf5e0 2024-04-26T23:19:13.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.057+0000 7f8b345b7640 1 -- 172.21.15.47:0/2996127597 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b2c1f5df0 con 0x7f8b2c0cf5e0 2024-04-26T23:19:13.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.057+0000 7f8b345b7640 1 -- 172.21.15.47:0/2996127597 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b2c0c5c40 con 0x7f8b2c0cf5e0 2024-04-26T23:19:13.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.058+0000 7f8b345b7640 1 -- 172.21.15.47:0/2996127597 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b2c0694f0 con 0x7f8b2c0cf5e0 2024-04-26T23:19:13.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.059+0000 7f8b22ffd640 1 -- 172.21.15.47:0/2996127597 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8b1c018070 con 0x7f8b2c0cf5e0 2024-04-26T23:19:13.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.059+0000 7f8b22ffd640 1 --2- 172.21.15.47:0/2996127597 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f8b1006c8f0 0x7f8b1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:13.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.059+0000 7f8b22ffd640 1 -- 172.21.15.47:0/2996127597 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f8b1c08bd80 con 0x7f8b2c0cf5e0 2024-04-26T23:19:13.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.060+0000 7f8b32353640 1 --2- 172.21.15.47:0/2996127597 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f8b1006c8f0 0x7f8b1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:13.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.060+0000 7f8b32353640 1 --2- 172.21.15.47:0/2996127597 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f8b1006c8f0 0x7f8b1006edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8b2800aa20 tx=0x7f8b28008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:13.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.064+0000 7f8b22ffd640 1 -- 172.21.15.47:0/2996127597 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8b1c0597d0 con 0x7f8b2c0cf5e0 2024-04-26T23:19:13.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.251+0000 7f8b345b7640 1 -- 172.21.15.47:0/2996127597 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8b2c1f5fc0 con 0x7f8b2c0cf5e0 2024-04-26T23:19:13.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.252+0000 7f8b22ffd640 1 -- 172.21.15.47:0/2996127597 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18857 (secure 0 0 0) 0x7f8b1c05dad0 con 0x7f8b2c0cf5e0 2024-04-26T23:19:13.254 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:19:13.254 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":26,"fsid":"399006d3-4db1-43e6-ba81-a79a276eabfd","created":"2024-04-26T23:18:06.731211+0000","modified":"2024-04-26T23:19:12.124051+0000","last_up_change":"2024-04-26T23:18:17.808736+0000","last_in_change":"2024-04-26T23:18:08.945859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:18:16.937492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:18:29.033575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:18:30.318499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:18:34.014536+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ee091567-fea0-4a14-bf4e-0e16c20bb2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6819","nonce":326520779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6821","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6823","nonce":326520779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6829","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6831","nonce":326520779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6825","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6827","nonce":326520779}]},"public_addr":"172.21.15.47:6819/326520779","cluster_addr":"172.21.15.47:6823/326520779","heartbeat_back_addr":"172.21.15.47:6831/326520779","heartbeat_front_addr":"172.21.15.47:6827/326520779","state":["exists","up"]},{"osd":1,"uuid":"c14440a0-bba5-4931-a6db-b736c08e869b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6809","nonce":1912610931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6811","nonce":1912610931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6815","nonce":1912610931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6813","nonce":1912610931}]},"public_addr":"172.21.15.47:6809/1912610931","cluster_addr":"172.21.15.47:6811/1912610931","heartbeat_back_addr":"172.21.15.47:6815/1912610931","heartbeat_front_addr":"172.21.15.47:6813/1912610931","state":["exists","up"]},{"osd":2,"uuid":"3940f8de-c9b2-4cd3-a911-9c301729eb30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6801","nonce":2569869305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6803","nonce":2569869305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6807","nonce":2569869305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6805","nonce":2569869305}]},"public_addr":"172.21.15.47:6801/2569869305","cluster_addr":"172.21.15.47:6803/2569869305","heartbeat_back_addr":"172.21.15.47:6807/2569869305","heartbeat_front_addr":"172.21.15.47:6805/2569869305","state":["exists","up"]},{"osd":3,"uuid":"1286a67e-8958-426a-8c2e-bc85498c619c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6818","nonce":2791411683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6822","nonce":2791411683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6830","nonce":2791411683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6826","nonce":2791411683}]},"public_addr":"172.21.15.47:6818/2791411683","cluster_addr":"172.21.15.47:6822/2791411683","heartbeat_back_addr":"172.21.15.47:6830/2791411683","heartbeat_front_addr":"172.21.15.47:6826/2791411683","state":["exists","up"]},{"osd":4,"uuid":"d8ee2c20-e5b7-46c2-9999-b234070126cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6809","nonce":441313104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6811","nonce":441313104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6815","nonce":441313104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6813","nonce":441313104}]},"public_addr":"172.21.15.203:6809/441313104","cluster_addr":"172.21.15.203:6811/441313104","heartbeat_back_addr":"172.21.15.203:6815/441313104","heartbeat_front_addr":"172.21.15.203:6813/441313104","state":["exists","up"]},{"osd":5,"uuid":"b453a8f0-f324-4840-9ae0-d67bdadb9f8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6801","nonce":56838444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6803","nonce":56838444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6807","nonce":56838444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6805","nonce":56838444}]},"public_addr":"172.21.15.203:6801/56838444","cluster_addr":"172.21.15.203:6803/56838444","heartbeat_back_addr":"172.21.15.203:6807/56838444","heartbeat_front_addr":"172.21.15.203:6805/56838444","state":["exists","up"]},{"osd":6,"uuid":"a6d31951-2d4b-4e8d-a510-6283b01dc158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6825","nonce":2738517080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6827","nonce":2738517080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6831","nonce":2738517080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6829","nonce":2738517080}]},"public_addr":"172.21.15.203:6825/2738517080","cluster_addr":"172.21.15.203:6827/2738517080","heartbeat_back_addr":"172.21.15.203:6831/2738517080","heartbeat_front_addr":"172.21.15.203:6829/2738517080","state":["exists","up"]},{"osd":7,"uuid":"7119b8c0-fe96-402d-9e75-7b509d9d44d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6817","nonce":1315565058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6819","nonce":1315565058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6823","nonce":1315565058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6821","nonce":1315565058}]},"public_addr":"172.21.15.203:6817/1315565058","cluster_addr":"172.21.15.203:6819/1315565058","heartbeat_back_addr":"172.21.15.203:6823/1315565058","heartbeat_front_addr":"172.21.15.203:6821/1315565058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.121921+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.137494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.084814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.111593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.072430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.125066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.097867+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.057061+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":5,"to":1}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:19:13.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.254+0000 7f8b345b7640 1 -- 172.21.15.47:0/2996127597 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f8b1006c8f0 msgr2=0x7f8b1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:13.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.254+0000 7f8b345b7640 1 --2- 172.21.15.47:0/2996127597 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f8b1006c8f0 0x7f8b1006edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8b2800aa20 tx=0x7f8b28008040 comp rx=0 tx=0).stop 2024-04-26T23:19:13.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.254+0000 7f8b345b7640 1 -- 172.21.15.47:0/2996127597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b2c0cf5e0 msgr2=0x7f8b2c1f5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:13.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.254+0000 7f8b345b7640 1 --2- 172.21.15.47:0/2996127597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b2c0cf5e0 0x7f8b2c1f5050 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c00c7c0 tx=0x7f8b1c00cc80 comp rx=0 tx=0).stop 2024-04-26T23:19:13.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.254+0000 7f8b345b7640 1 -- 172.21.15.47:0/2996127597 shutdown_connections 2024-04-26T23:19:13.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.254+0000 7f8b345b7640 1 --2- 172.21.15.47:0/2996127597 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f8b1006c8f0 0x7f8b1006edb0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:13.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.254+0000 7f8b345b7640 1 --2- 172.21.15.47:0/2996127597 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8b2c15b600 0x7f8b2c155dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:13.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.254+0000 7f8b345b7640 1 --2- 172.21.15.47:0/2996127597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b2c0cf5e0 0x7f8b2c1f5050 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:13.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.254+0000 7f8b345b7640 1 --2- 172.21.15.47:0/2996127597 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b2c0cbbd0 0x7f8b2c15f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:13.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.254+0000 7f8b345b7640 1 -- 172.21.15.47:0/2996127597 >> 172.21.15.47:0/2996127597 conn(0x7f8b2c0d0be0 msgr2=0x7f8b2c15a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:13.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.254+0000 7f8b345b7640 1 -- 172.21.15.47:0/2996127597 shutdown_connections 2024-04-26T23:19:13.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.254+0000 7f8b345b7640 1 -- 172.21.15.47:0/2996127597 wait complete. 2024-04-26T23:19:13.267 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:18:16.937492+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:18:29.033575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:18:30.318499+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T23:18:34.014536+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T23:19:13.268 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:19:13.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.369+0000 7fd54e3f2640 1 Processor -- start 2024-04-26T23:19:13.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.369+0000 7fd54e3f2640 1 -- start start 2024-04-26T23:19:13.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.370+0000 7fd54e3f2640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd54815f170 0x7fd5480d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:13.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.370+0000 7fd54e3f2640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5480d1040 con 0x7fd5480d3bc0 2024-04-26T23:19:13.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.370+0000 7fd54e3f2640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5480c9680 con 0x7fd54815f550 2024-04-26T23:19:13.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.370+0000 7fd54e3f2640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fd5480c9430 con 0x7fd54815f170 2024-04-26T23:19:13.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.370+0000 7fd547fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd54815f170 0x7fd5480d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:13.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.370+0000 7fd547fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd54815f170 0x7fd5480d5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41180/0 (socket says 172.21.15.47:41180) 2024-04-26T23:19:13.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.370+0000 7fd547fff640 1 -- 172.21.15.47:0/2633689775 learned_addr learned my addr 172.21.15.47:0/2633689775 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:13.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.370+0000 7fd54c98f640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd5480d3bc0 0x7fd5480ca440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:32978/0 (socket says 172.21.15.47:32978) 2024-04-26T23:19:13.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.370+0000 7fd547fff640 1 -- 172.21.15.47:0/2633689775 >> v1:172.21.15.203:6789/0 conn(0x7fd54815f550 legacy=0x7fd5480d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:13.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.370+0000 7fd547fff640 1 -- 172.21.15.47:0/2633689775 >> v1:172.21.15.47:6789/0 conn(0x7fd5480d3bc0 legacy=0x7fd5480ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:13.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.370+0000 7fd546ffd640 1 -- 172.21.15.47:0/2633689775 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4057539099 0 0) 0x7fd540001490 con 0x7fd5480d3bc0 2024-04-26T23:19:13.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.370+0000 7fd547fff640 1 -- 172.21.15.47:0/2633689775 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5480c9960 con 0x7fd54815f170 2024-04-26T23:19:13.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.371+0000 7fd547fff640 1 --2- 172.21.15.47:0/2633689775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd54815f170 0x7fd5480d5a80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd5480c95b0 tx=0x7fd53002a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a85d77ca7f59dfe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:13.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.371+0000 7fd546ffd640 1 -- 172.21.15.47:0/2633689775 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd530037070 con 0x7fd54815f170 2024-04-26T23:19:13.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.371+0000 7fd546ffd640 1 -- 172.21.15.47:0/2633689775 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd530032440 con 0x7fd54815f170 2024-04-26T23:19:13.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.371+0000 7fd546ffd640 1 -- 172.21.15.47:0/2633689775 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd530032810 con 0x7fd54815f170 2024-04-26T23:19:13.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.372+0000 7fd54e3f2640 1 -- 172.21.15.47:0/2633689775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd54815f170 msgr2=0x7fd5480d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:13.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.372+0000 7fd54e3f2640 1 --2- 172.21.15.47:0/2633689775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd54815f170 0x7fd5480d5a80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd5480c95b0 tx=0x7fd53002a2c0 comp rx=0 tx=0).stop 2024-04-26T23:19:13.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.372+0000 7fd54e3f2640 1 -- 172.21.15.47:0/2633689775 shutdown_connections 2024-04-26T23:19:13.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.372+0000 7fd54e3f2640 1 --2- 172.21.15.47:0/2633689775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd54815f170 0x7fd5480d5a80 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:13.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.372+0000 7fd54e3f2640 1 -- 172.21.15.47:0/2633689775 >> 172.21.15.47:0/2633689775 conn(0x7fd5480d4f60 msgr2=0x7fd5480d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:13.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.372+0000 7fd54e3f2640 1 -- 172.21.15.47:0/2633689775 shutdown_connections 2024-04-26T23:19:13.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.372+0000 7fd54e3f2640 1 -- 172.21.15.47:0/2633689775 wait complete. 2024-04-26T23:19:13.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.373+0000 7fd54e3f2640 1 Processor -- start 2024-04-26T23:19:13.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.373+0000 7fd54e3f2640 1 -- start start 2024-04-26T23:19:13.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.374+0000 7fd54e3f2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5480d3bc0 0x7fd5480cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:13.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.374+0000 7fd54e3f2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd54815f170 0x7fd548152340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:13.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.374+0000 7fd54e3f2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5480a0570 0x7fd5480a0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:13.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.374+0000 7fd54e3f2640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd5480d2e30 con 0x7fd54815f170 2024-04-26T23:19:13.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.374+0000 7fd54e3f2640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd5480d2cb0 con 0x7fd5480d3bc0 2024-04-26T23:19:13.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.374+0000 7fd54e3f2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd5480d2fb0 con 0x7fd5480a0570 2024-04-26T23:19:13.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.374+0000 7fd5477fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd54815f170 0x7fd548152340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:13.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.374+0000 7fd5477fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd54815f170 0x7fd548152340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41186/0 (socket says 172.21.15.47:41186) 2024-04-26T23:19:13.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.374+0000 7fd5477fe640 1 -- 172.21.15.47:0/954741914 learned_addr learned my addr 172.21.15.47:0/954741914 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:13.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.374+0000 7fd547fff640 1 --2- 172.21.15.47:0/954741914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5480d3bc0 0x7fd5480cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:13.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.374+0000 7fd54c98f640 1 --2- 172.21.15.47:0/954741914 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5480a0570 0x7fd5480a0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:13.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.374+0000 7fd5477fe640 1 -- 172.21.15.47:0/954741914 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5480a0570 msgr2=0x7fd5480a0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:13.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.374+0000 7fd5477fe640 1 --2- 172.21.15.47:0/954741914 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5480a0570 0x7fd5480a0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:13.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.374+0000 7fd5477fe640 1 -- 172.21.15.47:0/954741914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5480d3bc0 msgr2=0x7fd5480cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:13.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.374+0000 7fd5477fe640 1 --2- 172.21.15.47:0/954741914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5480d3bc0 0x7fd5480cdde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:13.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.374+0000 7fd5477fe640 1 -- 172.21.15.47:0/954741914 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5480cd090 con 0x7fd54815f170 2024-04-26T23:19:13.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.374+0000 7fd54c98f640 1 --2- 172.21.15.47:0/954741914 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5480a0570 0x7fd5480a0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:19:13.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.375+0000 7fd5477fe640 1 --2- 172.21.15.47:0/954741914 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd54815f170 0x7fd548152340 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd538009fd0 tx=0x7fd53800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:13.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.375+0000 7fd544ff9640 1 -- 172.21.15.47:0/954741914 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd538015020 con 0x7fd54815f170 2024-04-26T23:19:13.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.375+0000 7fd544ff9640 1 -- 172.21.15.47:0/954741914 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd538011830 con 0x7fd54815f170 2024-04-26T23:19:13.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.375+0000 7fd54e3f2640 1 -- 172.21.15.47:0/954741914 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5480cd300 con 0x7fd54815f170 2024-04-26T23:19:13.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.375+0000 7fd544ff9640 1 -- 172.21.15.47:0/954741914 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd538011c00 con 0x7fd54815f170 2024-04-26T23:19:13.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.375+0000 7fd54e3f2640 1 -- 172.21.15.47:0/954741914 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd548152880 con 0x7fd54815f170 2024-04-26T23:19:13.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.376+0000 7fd54e3f2640 1 -- 172.21.15.47:0/954741914 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5480694f0 con 0x7fd54815f170 2024-04-26T23:19:13.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.376+0000 7fd544ff9640 1 -- 172.21.15.47:0/954741914 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd538018020 con 0x7fd54815f170 2024-04-26T23:19:13.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.377+0000 7fd544ff9640 1 --2- 172.21.15.47:0/954741914 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd52806c8f0 0x7fd52806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:13.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.377+0000 7fd544ff9640 1 -- 172.21.15.47:0/954741914 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7fd53808bd20 con 0x7fd54815f170 2024-04-26T23:19:13.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.377+0000 7fd547fff640 1 --2- 172.21.15.47:0/954741914 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd52806c8f0 0x7fd52806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:13.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.377+0000 7fd547fff640 1 --2- 172.21.15.47:0/954741914 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd52806c8f0 0x7fd52806edb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd530034000 tx=0x7fd53002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:13.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.381+0000 7fd544ff9640 1 -- 172.21.15.47:0/954741914 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd53801f020 con 0x7fd54815f170 2024-04-26T23:19:13.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.566+0000 7fd54e3f2640 1 -- 172.21.15.47:0/954741914 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd5480c8c80 con 0x7fd54815f170 2024-04-26T23:19:13.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.566+0000 7fd544ff9640 1 -- 172.21.15.47:0/954741914 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7fd538059dc0 con 0x7fd54815f170 2024-04-26T23:19:13.568 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-26T23:19:13.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.568+0000 7fd54e3f2640 1 -- 172.21.15.47:0/954741914 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd52806c8f0 msgr2=0x7fd52806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:13.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.568+0000 7fd54e3f2640 1 --2- 172.21.15.47:0/954741914 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd52806c8f0 0x7fd52806edb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd530034000 tx=0x7fd53002e000 comp rx=0 tx=0).stop 2024-04-26T23:19:13.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.568+0000 7fd54e3f2640 1 -- 172.21.15.47:0/954741914 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd54815f170 msgr2=0x7fd548152340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:13.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.568+0000 7fd54e3f2640 1 --2- 172.21.15.47:0/954741914 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd54815f170 0x7fd548152340 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd538009fd0 tx=0x7fd53800b870 comp rx=0 tx=0).stop 2024-04-26T23:19:13.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.568+0000 7fd54e3f2640 1 -- 172.21.15.47:0/954741914 shutdown_connections 2024-04-26T23:19:13.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.568+0000 7fd54e3f2640 1 --2- 172.21.15.47:0/954741914 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd52806c8f0 0x7fd52806edb0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:13.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.568+0000 7fd54e3f2640 1 --2- 172.21.15.47:0/954741914 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5480a0570 0x7fd5480a0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:13.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.568+0000 7fd54e3f2640 1 --2- 172.21.15.47:0/954741914 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd54815f170 0x7fd548152340 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:13.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.568+0000 7fd54e3f2640 1 --2- 172.21.15.47:0/954741914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5480d3bc0 0x7fd5480cdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:13.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.568+0000 7fd54e3f2640 1 -- 172.21.15.47:0/954741914 >> 172.21.15.47:0/954741914 conn(0x7fd5480d4f60 msgr2=0x7fd5480d1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:13.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.568+0000 7fd54e3f2640 1 -- 172.21.15.47:0/954741914 shutdown_connections 2024-04-26T23:19:13.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.568+0000 7fd54e3f2640 1 -- 172.21.15.47:0/954741914 wait complete. 2024-04-26T23:19:13.581 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:19:13.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.688+0000 7fdf3ad46640 1 Processor -- start 2024-04-26T23:19:13.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.689+0000 7fdf3ad46640 1 -- start start 2024-04-26T23:19:13.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.689+0000 7fdf3ad46640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf34072fb0 0x7fdf340b56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:13.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.689+0000 7fdf3ad46640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdf34073390 0x7fdf3406f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:13.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.689+0000 7fdf3ad46640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf340b5e30 con 0x7fdf3406fc80 2024-04-26T23:19:13.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.689+0000 7fdf3ad46640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fdf340b5c00 con 0x7fdf34073390 2024-04-26T23:19:13.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.689+0000 7fdf3ad46640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fdf3406f2b0 con 0x7fdf34072fb0 2024-04-26T23:19:13.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.690+0000 7fdf38ae2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf34072fb0 0x7fdf340b56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:13.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.690+0000 7fdf392e3640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fdf3406fc80 0x7fdf34070110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.47:49788/0 (socket says 172.21.15.47:49788) 2024-04-26T23:19:13.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.690+0000 7fdf33fff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdf34073390 0x7fdf3406f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:13.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.690+0000 7fdf392e3640 1 -- 172.21.15.47:0/1322262775 learned_addr learned my addr 172.21.15.47:0/1322262775 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:13.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.690+0000 7fdf337fe640 1 -- 172.21.15.47:0/1322262775 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2606237759 0 0) 0x7fdf340b5e30 con 0x7fdf3406fc80 2024-04-26T23:19:13.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.690+0000 7fdf337fe640 1 -- 172.21.15.47:0/1322262775 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf1c003520 con 0x7fdf3406fc80 2024-04-26T23:19:13.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.690+0000 7fdf38ae2640 1 -- 172.21.15.47:0/1322262775 >> v2:172.21.15.203:3300/0 conn(0x7fdf34073390 msgr2=0x7fdf3406f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:13.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.690+0000 7fdf38ae2640 1 --2- 172.21.15.47:0/1322262775 >> v2:172.21.15.203:3300/0 conn(0x7fdf34073390 0x7fdf3406f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:13.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.690+0000 7fdf38ae2640 1 -- 172.21.15.47:0/1322262775 >> v1:172.21.15.203:6789/0 conn(0x7fdf3406fc80 legacy=0x7fdf34070110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:13.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.690+0000 7fdf38ae2640 1 -- 172.21.15.47:0/1322262775 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf340b6110 con 0x7fdf34072fb0 2024-04-26T23:19:13.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.690+0000 7fdf33fff640 1 --2- 172.21.15.47:0/1322262775 >> v2:172.21.15.203:3300/0 conn(0x7fdf34073390 0x7fdf3406f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:19:13.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.691+0000 7fdf38ae2640 1 --2- 172.21.15.47:0/1322262775 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf34072fb0 0x7fdf340b56c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fdf240097a0 tx=0x7fdf2402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6103423635d4715d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:13.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.691+0000 7fdf337fe640 1 -- 172.21.15.47:0/1322262775 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf24037070 con 0x7fdf34072fb0 2024-04-26T23:19:13.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.691+0000 7fdf337fe640 1 -- 172.21.15.47:0/1322262775 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf24032440 con 0x7fdf34072fb0 2024-04-26T23:19:13.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.691+0000 7fdf337fe640 1 -- 172.21.15.47:0/1322262775 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf240327f0 con 0x7fdf34072fb0 2024-04-26T23:19:13.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.692+0000 7fdf3ad46640 1 -- 172.21.15.47:0/1322262775 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf34072fb0 msgr2=0x7fdf340b56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:13.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.692+0000 7fdf3ad46640 1 --2- 172.21.15.47:0/1322262775 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf34072fb0 0x7fdf340b56c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fdf240097a0 tx=0x7fdf2402a180 comp rx=0 tx=0).stop 2024-04-26T23:19:13.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.692+0000 7fdf3ad46640 1 -- 172.21.15.47:0/1322262775 shutdown_connections 2024-04-26T23:19:13.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.692+0000 7fdf3ad46640 1 --2- 172.21.15.47:0/1322262775 >> v2:172.21.15.203:3300/0 conn(0x7fdf34073390 0x7fdf3406f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:13.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.692+0000 7fdf3ad46640 1 --2- 172.21.15.47:0/1322262775 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf34072fb0 0x7fdf340b56c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:13.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.692+0000 7fdf3ad46640 1 -- 172.21.15.47:0/1322262775 >> 172.21.15.47:0/1322262775 conn(0x7fdf340b39b0 msgr2=0x7fdf340b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:13.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.692+0000 7fdf3ad46640 1 -- 172.21.15.47:0/1322262775 shutdown_connections 2024-04-26T23:19:13.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.692+0000 7fdf3ad46640 1 -- 172.21.15.47:0/1322262775 wait complete. 2024-04-26T23:19:13.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.693+0000 7fdf3ad46640 1 Processor -- start 2024-04-26T23:19:13.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.693+0000 7fdf3ad46640 1 -- start start 2024-04-26T23:19:13.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.694+0000 7fdf3ad46640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf3406fc80 0x7fdf3414c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:13.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.694+0000 7fdf3ad46640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf34072fb0 0x7fdf340cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:13.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.694+0000 7fdf3ad46640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf34073390 0x7fdf3415ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:13.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.694+0000 7fdf3ad46640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fdf3415c160 con 0x7fdf34072fb0 2024-04-26T23:19:13.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.694+0000 7fdf3ad46640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdf340737f0 con 0x7fdf34073390 2024-04-26T23:19:13.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.694+0000 7fdf3ad46640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fdf3415c2e0 con 0x7fdf3406fc80 2024-04-26T23:19:13.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.694+0000 7fdf33fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf34072fb0 0x7fdf340cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:13.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.694+0000 7fdf33fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf34072fb0 0x7fdf340cc5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41198/0 (socket says 172.21.15.47:41198) 2024-04-26T23:19:13.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.694+0000 7fdf33fff640 1 -- 172.21.15.47:0/3428976896 learned_addr learned my addr 172.21.15.47:0/3428976896 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:13.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.694+0000 7fdf38ae2640 1 --2- 172.21.15.47:0/3428976896 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf3406fc80 0x7fdf3414c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:13.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.694+0000 7fdf392e3640 1 --2- 172.21.15.47:0/3428976896 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf34073390 0x7fdf3415ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:13.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.694+0000 7fdf33fff640 1 -- 172.21.15.47:0/3428976896 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf3406fc80 msgr2=0x7fdf3414c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:13.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.694+0000 7fdf33fff640 1 --2- 172.21.15.47:0/3428976896 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf3406fc80 0x7fdf3414c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:13.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.694+0000 7fdf33fff640 1 -- 172.21.15.47:0/3428976896 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf34073390 msgr2=0x7fdf3415ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:13.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.694+0000 7fdf33fff640 1 --2- 172.21.15.47:0/3428976896 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf34073390 0x7fdf3415ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:13.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.694+0000 7fdf33fff640 1 -- 172.21.15.47:0/3428976896 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf340ccb00 con 0x7fdf34072fb0 2024-04-26T23:19:13.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.694+0000 7fdf392e3640 1 --2- 172.21.15.47:0/3428976896 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf34073390 0x7fdf3415ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:19:13.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.695+0000 7fdf33fff640 1 --2- 172.21.15.47:0/3428976896 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf34072fb0 0x7fdf340cc5c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdf2800ebe0 tx=0x7fdf2800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:13.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.695+0000 7fdf317fa640 1 -- 172.21.15.47:0/3428976896 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf28017070 con 0x7fdf34072fb0 2024-04-26T23:19:13.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.695+0000 7fdf317fa640 1 -- 172.21.15.47:0/3428976896 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf2800cc30 con 0x7fdf34072fb0 2024-04-26T23:19:13.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.695+0000 7fdf3ad46640 1 -- 172.21.15.47:0/3428976896 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf340ccdd0 con 0x7fdf34072fb0 2024-04-26T23:19:13.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.695+0000 7fdf317fa640 1 -- 172.21.15.47:0/3428976896 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf28013470 con 0x7fdf34072fb0 2024-04-26T23:19:13.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.695+0000 7fdf3ad46640 1 -- 172.21.15.47:0/3428976896 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf340cd390 con 0x7fdf34072fb0 2024-04-26T23:19:13.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.696+0000 7fdf3ad46640 1 -- 172.21.15.47:0/3428976896 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf340cdab0 con 0x7fdf34072fb0 2024-04-26T23:19:13.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.696+0000 7fdf317fa640 1 -- 172.21.15.47:0/3428976896 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdf2801a070 con 0x7fdf34072fb0 2024-04-26T23:19:13.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.697+0000 7fdf317fa640 1 --2- 172.21.15.47:0/3428976896 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdf2006c8f0 0x7fdf2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:13.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.697+0000 7fdf317fa640 1 -- 172.21.15.47:0/3428976896 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7fdf2808bdb0 con 0x7fdf34072fb0 2024-04-26T23:19:13.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.697+0000 7fdf38ae2640 1 --2- 172.21.15.47:0/3428976896 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdf2006c8f0 0x7fdf2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:13.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.698+0000 7fdf38ae2640 1 --2- 172.21.15.47:0/3428976896 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdf2006c8f0 0x7fdf2006edb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdf24034000 tx=0x7fdf2402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:13.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.701+0000 7fdf317fa640 1 -- 172.21.15.47:0/3428976896 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdf28059800 con 0x7fdf34072fb0 2024-04-26T23:19:13.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.889+0000 7fdf3ad46640 1 -- 172.21.15.47:0/3428976896 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdf3406f770 con 0x7fdf34072fb0 2024-04-26T23:19:13.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.889+0000 7fdf317fa640 1 -- 172.21.15.47:0/3428976896 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fdf2805db00 con 0x7fdf34072fb0 2024-04-26T23:19:13.891 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-26T23:19:13.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.891+0000 7fdf3ad46640 1 -- 172.21.15.47:0/3428976896 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdf2006c8f0 msgr2=0x7fdf2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:13.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.891+0000 7fdf3ad46640 1 --2- 172.21.15.47:0/3428976896 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdf2006c8f0 0x7fdf2006edb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdf24034000 tx=0x7fdf2402e000 comp rx=0 tx=0).stop 2024-04-26T23:19:13.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.891+0000 7fdf3ad46640 1 -- 172.21.15.47:0/3428976896 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf34072fb0 msgr2=0x7fdf340cc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:13.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.891+0000 7fdf3ad46640 1 --2- 172.21.15.47:0/3428976896 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf34072fb0 0x7fdf340cc5c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdf2800ebe0 tx=0x7fdf2800c520 comp rx=0 tx=0).stop 2024-04-26T23:19:13.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.891+0000 7fdf3ad46640 1 -- 172.21.15.47:0/3428976896 shutdown_connections 2024-04-26T23:19:13.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.891+0000 7fdf3ad46640 1 --2- 172.21.15.47:0/3428976896 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdf2006c8f0 0x7fdf2006edb0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:13.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.891+0000 7fdf3ad46640 1 --2- 172.21.15.47:0/3428976896 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf34073390 0x7fdf3415ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:13.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.891+0000 7fdf3ad46640 1 --2- 172.21.15.47:0/3428976896 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf34072fb0 0x7fdf340cc5c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:13.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.891+0000 7fdf3ad46640 1 --2- 172.21.15.47:0/3428976896 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf3406fc80 0x7fdf3414c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:13.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.891+0000 7fdf3ad46640 1 -- 172.21.15.47:0/3428976896 >> 172.21.15.47:0/3428976896 conn(0x7fdf340b39b0 msgr2=0x7fdf340b2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:13.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.891+0000 7fdf3ad46640 1 -- 172.21.15.47:0/3428976896 shutdown_connections 2024-04-26T23:19:13.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:13.891+0000 7fdf3ad46640 1 -- 172.21.15.47:0/3428976896 wait complete. 2024-04-26T23:19:13.905 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:19:14.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.012+0000 7f3c4e8da640 1 Processor -- start 2024-04-26T23:19:14.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.012+0000 7f3c4e8da640 1 -- start start 2024-04-26T23:19:14.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.013+0000 7f3c4e8da640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c4815f170 0x7f3c480d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:14.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.013+0000 7f3c4e8da640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3c4815f550 0x7f3c480d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:14.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.013+0000 7f3c4e8da640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c480d1040 con 0x7f3c480d34b0 2024-04-26T23:19:14.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.013+0000 7f3c4e8da640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f3c480c9430 con 0x7f3c4815f550 2024-04-26T23:19:14.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.013+0000 7f3c4e8da640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3c480c9680 con 0x7f3c4815f170 2024-04-26T23:19:14.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.013+0000 7f3c47fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c4815f170 0x7f3c480d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:14.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.013+0000 7f3c477fe640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3c4815f550 0x7f3c480d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:14.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.013+0000 7f3c4ce77640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f3c480d34b0 0x7f3c480cfc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.47:49794/0 (socket says 172.21.15.47:49794) 2024-04-26T23:19:14.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.013+0000 7f3c4ce77640 1 -- 172.21.15.47:0/3992032619 learned_addr learned my addr 172.21.15.47:0/3992032619 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:14.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.014+0000 7f3c46ffd640 1 -- 172.21.15.47:0/3992032619 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3983030545 0 0) 0x7f3c40001490 con 0x7f3c480d34b0 2024-04-26T23:19:14.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.014+0000 7f3c46ffd640 1 -- 172.21.15.47:0/3992032619 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c30003520 con 0x7f3c480d34b0 2024-04-26T23:19:14.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.014+0000 7f3c47fff640 1 -- 172.21.15.47:0/3992032619 >> v2:172.21.15.203:3300/0 conn(0x7f3c4815f550 msgr2=0x7f3c480d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:14.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.014+0000 7f3c47fff640 1 --2- 172.21.15.47:0/3992032619 >> v2:172.21.15.203:3300/0 conn(0x7f3c4815f550 0x7f3c480d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:14.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.014+0000 7f3c47fff640 1 -- 172.21.15.47:0/3992032619 >> v1:172.21.15.203:6789/0 conn(0x7f3c480d34b0 legacy=0x7f3c480cfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:14.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.014+0000 7f3c47fff640 1 -- 172.21.15.47:0/3992032619 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c480c9960 con 0x7f3c4815f170 2024-04-26T23:19:14.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.014+0000 7f3c477fe640 1 --2- 172.21.15.47:0/3992032619 >> v2:172.21.15.203:3300/0 conn(0x7f3c4815f550 0x7f3c480d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:19:14.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.014+0000 7f3c47fff640 1 --2- 172.21.15.47:0/3992032619 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c4815f170 0x7f3c480d5a80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3c380097a0 tx=0x7f3c3802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=376132c63f14fcfc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:14.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.015+0000 7f3c46ffd640 1 -- 172.21.15.47:0/3992032619 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c38037070 con 0x7f3c4815f170 2024-04-26T23:19:14.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.015+0000 7f3c46ffd640 1 -- 172.21.15.47:0/3992032619 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c38032440 con 0x7f3c4815f170 2024-04-26T23:19:14.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.015+0000 7f3c46ffd640 1 -- 172.21.15.47:0/3992032619 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c380327f0 con 0x7f3c4815f170 2024-04-26T23:19:14.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.015+0000 7f3c4e8da640 1 -- 172.21.15.47:0/3992032619 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c4815f170 msgr2=0x7f3c480d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:14.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.015+0000 7f3c4e8da640 1 --2- 172.21.15.47:0/3992032619 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c4815f170 0x7f3c480d5a80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3c380097a0 tx=0x7f3c3802a180 comp rx=0 tx=0).stop 2024-04-26T23:19:14.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.015+0000 7f3c4e8da640 1 -- 172.21.15.47:0/3992032619 shutdown_connections 2024-04-26T23:19:14.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.015+0000 7f3c4e8da640 1 --2- 172.21.15.47:0/3992032619 >> v2:172.21.15.203:3300/0 conn(0x7f3c4815f550 0x7f3c480d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:14.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.015+0000 7f3c4e8da640 1 --2- 172.21.15.47:0/3992032619 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c4815f170 0x7f3c480d5a80 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:14.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.015+0000 7f3c4e8da640 1 -- 172.21.15.47:0/3992032619 >> 172.21.15.47:0/3992032619 conn(0x7f3c480d4f60 msgr2=0x7f3c480d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:14.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.015+0000 7f3c4e8da640 1 -- 172.21.15.47:0/3992032619 shutdown_connections 2024-04-26T23:19:14.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.015+0000 7f3c4e8da640 1 -- 172.21.15.47:0/3992032619 wait complete. 2024-04-26T23:19:14.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.016+0000 7f3c4e8da640 1 Processor -- start 2024-04-26T23:19:14.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.017+0000 7f3c4e8da640 1 -- start start 2024-04-26T23:19:14.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.017+0000 7f3c4e8da640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c480d34b0 0x7f3c480b9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:14.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.017+0000 7f3c4e8da640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c4815f170 0x7f3c480b7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:14.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.017+0000 7f3c4e8da640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c480ce900 0x7f3c48147620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:14.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.017+0000 7f3c4e8da640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3c480cb5d0 con 0x7f3c480d34b0 2024-04-26T23:19:14.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.017+0000 7f3c4e8da640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3c480cb450 con 0x7f3c480ce900 2024-04-26T23:19:14.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.017+0000 7f3c4e8da640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3c480cb750 con 0x7f3c4815f170 2024-04-26T23:19:14.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.017+0000 7f3c47fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c480d34b0 0x7f3c480b9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:14.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.017+0000 7f3c47fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c480d34b0 0x7f3c480b9510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41214/0 (socket says 172.21.15.47:41214) 2024-04-26T23:19:14.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.017+0000 7f3c47fff640 1 -- 172.21.15.47:0/7172750 learned_addr learned my addr 172.21.15.47:0/7172750 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:14.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.017+0000 7f3c477fe640 1 --2- 172.21.15.47:0/7172750 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c4815f170 0x7f3c480b7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:14.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.018+0000 7f3c4ce77640 1 --2- 172.21.15.47:0/7172750 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c480ce900 0x7f3c48147620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:14.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.018+0000 7f3c47fff640 1 -- 172.21.15.47:0/7172750 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c4815f170 msgr2=0x7f3c480b7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:14.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.018+0000 7f3c47fff640 1 --2- 172.21.15.47:0/7172750 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c4815f170 0x7f3c480b7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:14.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.018+0000 7f3c47fff640 1 -- 172.21.15.47:0/7172750 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c480ce900 msgr2=0x7f3c48147620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:14.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.018+0000 7f3c47fff640 1 --2- 172.21.15.47:0/7172750 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c480ce900 0x7f3c48147620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:14.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.018+0000 7f3c47fff640 1 -- 172.21.15.47:0/7172750 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c480b8470 con 0x7f3c480d34b0 2024-04-26T23:19:14.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.018+0000 7f3c4ce77640 1 --2- 172.21.15.47:0/7172750 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c480ce900 0x7f3c48147620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:19:14.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.018+0000 7f3c47fff640 1 --2- 172.21.15.47:0/7172750 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c480d34b0 0x7f3c480b9510 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3c38035040 tx=0x7f3c380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:14.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.018+0000 7f3c44ff9640 1 -- 172.21.15.47:0/7172750 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c38037040 con 0x7f3c480d34b0 2024-04-26T23:19:14.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.018+0000 7f3c44ff9640 1 -- 172.21.15.47:0/7172750 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c38042070 con 0x7f3c480d34b0 2024-04-26T23:19:14.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.018+0000 7f3c44ff9640 1 -- 172.21.15.47:0/7172750 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3803b930 con 0x7f3c480d34b0 2024-04-26T23:19:14.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.018+0000 7f3c4e8da640 1 -- 172.21.15.47:0/7172750 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c480b8750 con 0x7f3c480d34b0 2024-04-26T23:19:14.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.018+0000 7f3c4e8da640 1 -- 172.21.15.47:0/7172750 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c480b8d10 con 0x7f3c480d34b0 2024-04-26T23:19:14.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.019+0000 7f3c4e8da640 1 -- 172.21.15.47:0/7172750 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c480694f0 con 0x7f3c480d34b0 2024-04-26T23:19:14.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.020+0000 7f3c44ff9640 1 -- 172.21.15.47:0/7172750 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3c3803bad0 con 0x7f3c480d34b0 2024-04-26T23:19:14.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.020+0000 7f3c44ff9640 1 --2- 172.21.15.47:0/7172750 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3c3406c920 0x7f3c3406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:14.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.020+0000 7f3c44ff9640 1 -- 172.21.15.47:0/7172750 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f3c380ac700 con 0x7f3c480d34b0 2024-04-26T23:19:14.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.020+0000 7f3c477fe640 1 --2- 172.21.15.47:0/7172750 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3c3406c920 0x7f3c3406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:14.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.021+0000 7f3c477fe640 1 --2- 172.21.15.47:0/7172750 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3c3406c920 0x7f3c3406ede0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c009560 tx=0x7f3c3c009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:14.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.024+0000 7f3c44ff9640 1 -- 172.21.15.47:0/7172750 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3c3807a710 con 0x7f3c480d34b0 2024-04-26T23:19:14.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.211+0000 7f3c4e8da640 1 -- 172.21.15.47:0/7172750 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3c480d2e50 con 0x7f3c480d34b0 2024-04-26T23:19:14.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.211+0000 7f3c44ff9640 1 -- 172.21.15.47:0/7172750 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f3c3807a010 con 0x7f3c480d34b0 2024-04-26T23:19:14.214 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-26T23:19:14.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.213+0000 7f3c4e8da640 1 -- 172.21.15.47:0/7172750 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3c3406c920 msgr2=0x7f3c3406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:14.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.213+0000 7f3c4e8da640 1 --2- 172.21.15.47:0/7172750 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3c3406c920 0x7f3c3406ede0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c009560 tx=0x7f3c3c009190 comp rx=0 tx=0).stop 2024-04-26T23:19:14.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.213+0000 7f3c4e8da640 1 -- 172.21.15.47:0/7172750 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c480d34b0 msgr2=0x7f3c480b9510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:14.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.213+0000 7f3c4e8da640 1 --2- 172.21.15.47:0/7172750 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c480d34b0 0x7f3c480b9510 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3c38035040 tx=0x7f3c380046c0 comp rx=0 tx=0).stop 2024-04-26T23:19:14.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.213+0000 7f3c4e8da640 1 -- 172.21.15.47:0/7172750 shutdown_connections 2024-04-26T23:19:14.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.213+0000 7f3c4e8da640 1 --2- 172.21.15.47:0/7172750 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3c3406c920 0x7f3c3406ede0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:14.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.213+0000 7f3c4e8da640 1 --2- 172.21.15.47:0/7172750 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c480ce900 0x7f3c48147620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:14.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.213+0000 7f3c4e8da640 1 --2- 172.21.15.47:0/7172750 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c4815f170 0x7f3c480b7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:14.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.213+0000 7f3c4e8da640 1 --2- 172.21.15.47:0/7172750 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c480d34b0 0x7f3c480b9510 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:14.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.213+0000 7f3c4e8da640 1 -- 172.21.15.47:0/7172750 >> 172.21.15.47:0/7172750 conn(0x7f3c480d4f60 msgr2=0x7f3c480c89a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:14.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.213+0000 7f3c4e8da640 1 -- 172.21.15.47:0/7172750 shutdown_connections 2024-04-26T23:19:14.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.213+0000 7f3c4e8da640 1 -- 172.21.15.47:0/7172750 wait complete. 2024-04-26T23:19:14.226 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T23:19:14.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.329+0000 7fafb3ff2640 1 Processor -- start 2024-04-26T23:19:14.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.330+0000 7fafb3ff2640 1 -- start start 2024-04-26T23:19:14.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.330+0000 7fafb3ff2640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fafac15f770 0x7fafac0b1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:14.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.330+0000 7fafb3ff2640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafac06e580 con 0x7fafac0d1920 2024-04-26T23:19:14.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.330+0000 7fafb3ff2640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafac06e860 con 0x7fafac0d0d60 2024-04-26T23:19:14.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.330+0000 7fafb3ff2640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fafac06f3e0 con 0x7fafac15f770 2024-04-26T23:19:14.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.331+0000 7fafb258f640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fafac0d1920 0x7fafac0cdb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:32980/0 (socket says 172.21.15.47:32980) 2024-04-26T23:19:14.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.331+0000 7fafb258f640 1 -- 172.21.15.47:0/226738351 learned_addr learned my addr 172.21.15.47:0/226738351 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:14.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.331+0000 7fafb0d8c640 1 -- 172.21.15.47:0/226738351 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1020307367 0 0) 0x7fafac06e580 con 0x7fafac0d1920 2024-04-26T23:19:14.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.331+0000 7fafb0d8c640 1 -- 172.21.15.47:0/226738351 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faf98003520 con 0x7fafac0d1920 2024-04-26T23:19:14.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.331+0000 7fafb1d8e640 1 --2- 172.21.15.47:0/226738351 >> v2:172.21.15.203:3300/0 conn(0x7fafac15f770 0x7fafac0b1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:14.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.331+0000 7fafb0d8c640 1 -- 172.21.15.47:0/226738351 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2766611562 0 0) 0x7faf98003520 con 0x7fafac0d1920 2024-04-26T23:19:14.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.331+0000 7fafb0d8c640 1 -- 172.21.15.47:0/226738351 >> v2:172.21.15.203:3300/0 conn(0x7fafac15f770 msgr2=0x7fafac0b1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:14.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.331+0000 7fafb0d8c640 1 --2- 172.21.15.47:0/226738351 >> v2:172.21.15.203:3300/0 conn(0x7fafac15f770 0x7fafac0b1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:14.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.331+0000 7fafb0d8c640 1 -- 172.21.15.47:0/226738351 >> v1:172.21.15.203:6789/0 conn(0x7fafac0d0d60 legacy=0x7fafac0d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:14.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.331+0000 7fafb0d8c640 1 -- 172.21.15.47:0/226738351 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafac06eb40 con 0x7fafac0d1920 2024-04-26T23:19:14.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.331+0000 7fafb0d8c640 1 -- 172.21.15.47:0/226738351 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7fafa40019f0 con 0x7fafac0d1920 2024-04-26T23:19:14.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.331+0000 7fafb0d8c640 1 -- 172.21.15.47:0/226738351 >> v1:172.21.15.47:6789/0 conn(0x7fafac0d1920 legacy=0x7fafac0cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:14.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.331+0000 7fafb0d8c640 1 --2- 172.21.15.47:0/226738351 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf980039b0 0x7faf98003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:14.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.331+0000 7fafb0d8c640 1 --2- 172.21.15.47:0/226738351 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf98004800 0x7faf98004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:14.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.331+0000 7fafb0d8c640 1 --2- 172.21.15.47:0/226738351 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf980051c0 0x7faf9800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:14.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.331+0000 7fafb0d8c640 1 -- 172.21.15.47:0/226738351 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fafac06f3e0 con 0x7faf980051c0 2024-04-26T23:19:14.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.331+0000 7fafb0d8c640 1 -- 172.21.15.47:0/226738351 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7faf9800c110 con 0x7faf980039b0 2024-04-26T23:19:14.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.331+0000 7fafb0d8c640 1 -- 172.21.15.47:0/226738351 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7faf9800c310 con 0x7faf98004800 2024-04-26T23:19:14.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.331+0000 7fafb258f640 1 --2- 172.21.15.47:0/226738351 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf980051c0 0x7faf9800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:14.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.331+0000 7fafb1d8e640 1 --2- 172.21.15.47:0/226738351 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf980039b0 0x7faf98003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:14.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.331+0000 7fafb158d640 1 --2- 172.21.15.47:0/226738351 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf98004800 0x7faf98004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:14.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.332+0000 7fafb258f640 1 -- 172.21.15.47:0/226738351 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf98004800 msgr2=0x7faf98004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:14.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.332+0000 7fafb258f640 1 --2- 172.21.15.47:0/226738351 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf98004800 0x7faf98004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:14.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.332+0000 7fafb258f640 1 -- 172.21.15.47:0/226738351 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf980039b0 msgr2=0x7faf98003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:14.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.332+0000 7fafb258f640 1 --2- 172.21.15.47:0/226738351 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf980039b0 0x7faf98003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:14.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.332+0000 7fafb258f640 1 -- 172.21.15.47:0/226738351 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf98003520 con 0x7faf980051c0 2024-04-26T23:19:14.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.332+0000 7fafb258f640 1 -- 172.21.15.47:0/226738351 reap_dead start 2024-04-26T23:19:14.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.332+0000 7fafb258f640 1 --2- 172.21.15.47:0/226738351 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf980051c0 0x7faf9800b9c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fafa400a850 tx=0x7faf98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87c3445daa6ff254 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:14.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.332+0000 7fafb0d8c640 1 -- 172.21.15.47:0/226738351 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa400a4a0 con 0x7faf980051c0 2024-04-26T23:19:14.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.332+0000 7fafb0d8c640 1 -- 172.21.15.47:0/226738351 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafa402f070 con 0x7faf980051c0 2024-04-26T23:19:14.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.332+0000 7fafb0d8c640 1 -- 172.21.15.47:0/226738351 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa402bce0 con 0x7faf980051c0 2024-04-26T23:19:14.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.333+0000 7fafb3ff2640 1 -- 172.21.15.47:0/226738351 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf980051c0 msgr2=0x7faf9800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:14.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.333+0000 7fafb3ff2640 1 --2- 172.21.15.47:0/226738351 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf980051c0 0x7faf9800b9c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fafa400a850 tx=0x7faf98005830 comp rx=0 tx=0).stop 2024-04-26T23:19:14.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.333+0000 7fafb3ff2640 1 -- 172.21.15.47:0/226738351 shutdown_connections 2024-04-26T23:19:14.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.333+0000 7fafb3ff2640 1 --2- 172.21.15.47:0/226738351 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf980051c0 0x7faf9800b9c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:14.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.333+0000 7fafb3ff2640 1 -- 172.21.15.47:0/226738351 >> 172.21.15.47:0/226738351 conn(0x7fafac0d2e60 msgr2=0x7fafac0d3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:14.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.333+0000 7fafb3ff2640 1 -- 172.21.15.47:0/226738351 shutdown_connections 2024-04-26T23:19:14.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.333+0000 7fafb3ff2640 1 -- 172.21.15.47:0/226738351 wait complete. 2024-04-26T23:19:14.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.334+0000 7fafb3ff2640 1 Processor -- start 2024-04-26T23:19:14.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.334+0000 7fafb3ff2640 1 -- start start 2024-04-26T23:19:14.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.335+0000 7fafb3ff2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf980051c0 0x7fafac1523f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:14.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.335+0000 7fafb3ff2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fafac0c8d50 0x7fafac152d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:14.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.335+0000 7fafb3ff2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fafac0c91d0 0x7fafac15dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:14.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.335+0000 7fafb3ff2640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fafac0b2360 con 0x7fafac0c91d0 2024-04-26T23:19:14.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.335+0000 7fafb3ff2640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fafac0b21e0 con 0x7fafac0c8d50 2024-04-26T23:19:14.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.335+0000 7fafb3ff2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fafac0cd6e0 con 0x7faf980051c0 2024-04-26T23:19:14.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.335+0000 7fafb258f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fafac0c91d0 0x7fafac15dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:14.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.335+0000 7fafb258f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fafac0c91d0 0x7fafac15dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41242/0 (socket says 172.21.15.47:41242) 2024-04-26T23:19:14.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.335+0000 7fafb258f640 1 -- 172.21.15.47:0/2684601295 learned_addr learned my addr 172.21.15.47:0/2684601295 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:14.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.335+0000 7fafb1d8e640 1 --2- 172.21.15.47:0/2684601295 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf980051c0 0x7fafac1523f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:14.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.335+0000 7fafb158d640 1 --2- 172.21.15.47:0/2684601295 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fafac0c8d50 0x7fafac152d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:14.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.335+0000 7fafb258f640 1 -- 172.21.15.47:0/2684601295 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf980051c0 msgr2=0x7fafac1523f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:14.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.335+0000 7fafb258f640 1 --2- 172.21.15.47:0/2684601295 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf980051c0 0x7fafac1523f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:14.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.335+0000 7fafb258f640 1 -- 172.21.15.47:0/2684601295 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fafac0c8d50 msgr2=0x7fafac152d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:14.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.335+0000 7fafb258f640 1 --2- 172.21.15.47:0/2684601295 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fafac0c8d50 0x7fafac152d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:14.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.335+0000 7fafb258f640 1 -- 172.21.15.47:0/2684601295 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafac0b5c00 con 0x7fafac0c91d0 2024-04-26T23:19:14.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.335+0000 7fafb158d640 1 --2- 172.21.15.47:0/2684601295 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fafac0c8d50 0x7fafac152d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:19:14.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.336+0000 7fafb258f640 1 --2- 172.21.15.47:0/2684601295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fafac0c91d0 0x7fafac15dbd0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fafa4000f80 tx=0x7fafa402d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:14.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.336+0000 7fafa27fc640 1 -- 172.21.15.47:0/2684601295 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa4045070 con 0x7fafac0c91d0 2024-04-26T23:19:14.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.336+0000 7fafa27fc640 1 -- 172.21.15.47:0/2684601295 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafa40406a0 con 0x7fafac0c91d0 2024-04-26T23:19:14.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.336+0000 7fafb3ff2640 1 -- 172.21.15.47:0/2684601295 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafac0b5ee0 con 0x7fafac0c91d0 2024-04-26T23:19:14.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.336+0000 7fafa27fc640 1 -- 172.21.15.47:0/2684601295 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa4040a90 con 0x7fafac0c91d0 2024-04-26T23:19:14.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.336+0000 7fafb3ff2640 1 -- 172.21.15.47:0/2684601295 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafac0b64a0 con 0x7fafac0c91d0 2024-04-26T23:19:14.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.337+0000 7fafb3ff2640 1 -- 172.21.15.47:0/2684601295 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafac0d0820 con 0x7fafac0c91d0 2024-04-26T23:19:14.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.337+0000 7fafa27fc640 1 -- 172.21.15.47:0/2684601295 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fafa4048070 con 0x7fafac0c91d0 2024-04-26T23:19:14.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.338+0000 7fafa27fc640 1 --2- 172.21.15.47:0/2684601295 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7faf9006c920 0x7faf9006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:14.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.338+0000 7fafa27fc640 1 -- 172.21.15.47:0/2684601295 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7fafa40ac5e0 con 0x7fafac0c91d0 2024-04-26T23:19:14.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.338+0000 7fafb1d8e640 1 --2- 172.21.15.47:0/2684601295 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7faf9006c920 0x7faf9006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:14.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.338+0000 7fafb1d8e640 1 --2- 172.21.15.47:0/2684601295 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7faf9006c920 0x7faf9006ede0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fafa8006fd0 tx=0x7fafa8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:14.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.342+0000 7fafa27fc640 1 -- 172.21.15.47:0/2684601295 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fafa407a550 con 0x7fafac0c91d0 2024-04-26T23:19:14.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.526+0000 7fafb3ff2640 1 -- 172.21.15.47:0/2684601295 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fafac06fe50 con 0x7fafac0c91d0 2024-04-26T23:19:14.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.526+0000 7fafa27fc640 1 -- 172.21.15.47:0/2684601295 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fafa4079e50 con 0x7fafac0c91d0 2024-04-26T23:19:14.528 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-26T23:19:14.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.528+0000 7fafb3ff2640 1 -- 172.21.15.47:0/2684601295 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7faf9006c920 msgr2=0x7faf9006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:14.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.528+0000 7fafb3ff2640 1 --2- 172.21.15.47:0/2684601295 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7faf9006c920 0x7faf9006ede0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fafa8006fd0 tx=0x7fafa8008040 comp rx=0 tx=0).stop 2024-04-26T23:19:14.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.528+0000 7fafb3ff2640 1 -- 172.21.15.47:0/2684601295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fafac0c91d0 msgr2=0x7fafac15dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:14.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.528+0000 7fafb3ff2640 1 --2- 172.21.15.47:0/2684601295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fafac0c91d0 0x7fafac15dbd0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fafa4000f80 tx=0x7fafa402d070 comp rx=0 tx=0).stop 2024-04-26T23:19:14.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.528+0000 7fafb3ff2640 1 -- 172.21.15.47:0/2684601295 shutdown_connections 2024-04-26T23:19:14.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.528+0000 7fafb3ff2640 1 --2- 172.21.15.47:0/2684601295 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7faf9006c920 0x7faf9006ede0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:14.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.528+0000 7fafb3ff2640 1 --2- 172.21.15.47:0/2684601295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fafac0c91d0 0x7fafac15dbd0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:14.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.528+0000 7fafb3ff2640 1 --2- 172.21.15.47:0/2684601295 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fafac0c8d50 0x7fafac152d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:14.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.528+0000 7fafb3ff2640 1 --2- 172.21.15.47:0/2684601295 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf980051c0 0x7fafac1523f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:14.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.528+0000 7fafb3ff2640 1 -- 172.21.15.47:0/2684601295 >> 172.21.15.47:0/2684601295 conn(0x7fafac0d2e60 msgr2=0x7fafac0b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:14.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.528+0000 7fafb3ff2640 1 -- 172.21.15.47:0/2684601295 shutdown_connections 2024-04-26T23:19:14.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.528+0000 7fafb3ff2640 1 -- 172.21.15.47:0/2684601295 wait complete. 2024-04-26T23:19:14.541 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:19:14.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.640+0000 7f022e5f0640 1 Processor -- start 2024-04-26T23:19:14.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.640+0000 7f022e5f0640 1 -- start start 2024-04-26T23:19:14.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.641+0000 7f022e5f0640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f02281608f0 0x7f02280d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:14.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.641+0000 7f022e5f0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02280cffa0 0x7f02280b2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:14.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.641+0000 7f022e5f0640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02280cde20 con 0x7f02280d0a90 2024-04-26T23:19:14.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.641+0000 7f022e5f0640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f02280cea30 con 0x7f02281608f0 2024-04-26T23:19:14.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.641+0000 7f022e5f0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f02280ce100 con 0x7f02280cffa0 2024-04-26T23:19:14.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.641+0000 7f0227fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f02281608f0 0x7f02280d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:14.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.641+0000 7f0227fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f02281608f0 0x7f02280d1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41244/0 (socket says 172.21.15.47:41244) 2024-04-26T23:19:14.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.641+0000 7f0227fff640 1 -- 172.21.15.47:0/4030834046 learned_addr learned my addr 172.21.15.47:0/4030834046 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:14.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.642+0000 7f0227fff640 1 -- 172.21.15.47:0/4030834046 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02280cffa0 msgr2=0x7f02280b2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:14.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.642+0000 7f022cb8d640 1 --2- 172.21.15.47:0/4030834046 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02280cffa0 0x7f02280b2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:14.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.642+0000 7f0227fff640 1 --2- 172.21.15.47:0/4030834046 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02280cffa0 0x7f02280b2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:14.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.642+0000 7f0227fff640 1 -- 172.21.15.47:0/4030834046 >> v1:172.21.15.203:6789/0 conn(0x7f02280d0a90 legacy=0x7f02280cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:14.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.642+0000 7f0227fff640 1 -- 172.21.15.47:0/4030834046 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02280ce3e0 con 0x7f02281608f0 2024-04-26T23:19:14.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.642+0000 7f0227fff640 1 --2- 172.21.15.47:0/4030834046 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02281608f0 0x7f02280d1bf0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0218009870 tx=0x7f021802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f68f9f7b44241b3f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:14.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.643+0000 7f0226ffd640 1 -- 172.21.15.47:0/4030834046 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0218037070 con 0x7f02281608f0 2024-04-26T23:19:14.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.643+0000 7f0226ffd640 1 -- 172.21.15.47:0/4030834046 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0218032440 con 0x7f02281608f0 2024-04-26T23:19:14.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.643+0000 7f0226ffd640 1 -- 172.21.15.47:0/4030834046 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0218032810 con 0x7f02281608f0 2024-04-26T23:19:14.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.643+0000 7f022e5f0640 1 -- 172.21.15.47:0/4030834046 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02281608f0 msgr2=0x7f02280d1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:14.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.643+0000 7f022e5f0640 1 --2- 172.21.15.47:0/4030834046 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02281608f0 0x7f02280d1bf0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0218009870 tx=0x7f021802a210 comp rx=0 tx=0).stop 2024-04-26T23:19:14.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.643+0000 7f022e5f0640 1 -- 172.21.15.47:0/4030834046 shutdown_connections 2024-04-26T23:19:14.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.643+0000 7f022e5f0640 1 --2- 172.21.15.47:0/4030834046 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02280cffa0 0x7f02280b2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:14.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.643+0000 7f022e5f0640 1 --2- 172.21.15.47:0/4030834046 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02281608f0 0x7f02280d1bf0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:14.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.643+0000 7f022e5f0640 1 -- 172.21.15.47:0/4030834046 >> 172.21.15.47:0/4030834046 conn(0x7f02280d2ec0 msgr2=0x7f02280d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:14.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.643+0000 7f022e5f0640 1 -- 172.21.15.47:0/4030834046 shutdown_connections 2024-04-26T23:19:14.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.643+0000 7f022e5f0640 1 -- 172.21.15.47:0/4030834046 wait complete. 2024-04-26T23:19:14.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.644+0000 7f022e5f0640 1 Processor -- start 2024-04-26T23:19:14.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.645+0000 7f022e5f0640 1 -- start start 2024-04-26T23:19:14.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.645+0000 7f022e5f0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02280cffa0 0x7f02280c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:14.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.645+0000 7f022e5f0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02280d0a90 0x7f02281516d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:14.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.645+0000 7f022e5f0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02281608f0 0x7f022806dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:14.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.645+0000 7f022e5f0640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f02280ceec0 con 0x7f02280cffa0 2024-04-26T23:19:14.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.645+0000 7f022e5f0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f02280d0ef0 con 0x7f02280d0a90 2024-04-26T23:19:14.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.645+0000 7f022e5f0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f02280cda40 con 0x7f02281608f0 2024-04-26T23:19:14.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.645+0000 7f0227fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02280cffa0 0x7f02280c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:14.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.645+0000 7f0227fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02280cffa0 0x7f02280c93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41258/0 (socket says 172.21.15.47:41258) 2024-04-26T23:19:14.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.645+0000 7f0227fff640 1 -- 172.21.15.47:0/43203290 learned_addr learned my addr 172.21.15.47:0/43203290 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:14.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.645+0000 7f022cb8d640 1 --2- 172.21.15.47:0/43203290 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02281608f0 0x7f022806dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:14.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.645+0000 7f02277fe640 1 --2- 172.21.15.47:0/43203290 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02280d0a90 0x7f02281516d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:14.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.646+0000 7f0227fff640 1 -- 172.21.15.47:0/43203290 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02281608f0 msgr2=0x7f022806dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:14.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.646+0000 7f0227fff640 1 --2- 172.21.15.47:0/43203290 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02281608f0 0x7f022806dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:14.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.646+0000 7f0227fff640 1 -- 172.21.15.47:0/43203290 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02280d0a90 msgr2=0x7f02281516d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:14.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.646+0000 7f0227fff640 1 --2- 172.21.15.47:0/43203290 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02280d0a90 0x7f02281516d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:14.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.646+0000 7f0227fff640 1 -- 172.21.15.47:0/43203290 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f022806e570 con 0x7f02280cffa0 2024-04-26T23:19:14.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.646+0000 7f02277fe640 1 --2- 172.21.15.47:0/43203290 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02280d0a90 0x7f02281516d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:19:14.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.646+0000 7f0227fff640 1 --2- 172.21.15.47:0/43203290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02280cffa0 0x7f02280c93c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0218009870 tx=0x7f02180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:14.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.646+0000 7f0224ff9640 1 -- 172.21.15.47:0/43203290 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0218041070 con 0x7f02280cffa0 2024-04-26T23:19:14.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.646+0000 7f0224ff9640 1 -- 172.21.15.47:0/43203290 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f021803c870 con 0x7f02280cffa0 2024-04-26T23:19:14.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.646+0000 7f0224ff9640 1 -- 172.21.15.47:0/43203290 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0218037040 con 0x7f02280cffa0 2024-04-26T23:19:14.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.646+0000 7f022e5f0640 1 -- 172.21.15.47:0/43203290 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f022806e850 con 0x7f02280cffa0 2024-04-26T23:19:14.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.646+0000 7f022e5f0640 1 -- 172.21.15.47:0/43203290 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0228151c10 con 0x7f02280cffa0 2024-04-26T23:19:14.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.647+0000 7f022e5f0640 1 -- 172.21.15.47:0/43203290 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02280694f0 con 0x7f02280cffa0 2024-04-26T23:19:14.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.648+0000 7f0224ff9640 1 -- 172.21.15.47:0/43203290 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0218044070 con 0x7f02280cffa0 2024-04-26T23:19:14.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.648+0000 7f0224ff9640 1 --2- 172.21.15.47:0/43203290 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f021406c8f0 0x7f021406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:14.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.648+0000 7f0224ff9640 1 -- 172.21.15.47:0/43203290 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f02180acb70 con 0x7f02280cffa0 2024-04-26T23:19:14.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.648+0000 7f02277fe640 1 --2- 172.21.15.47:0/43203290 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f021406c8f0 0x7f021406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:14.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.649+0000 7f02277fe640 1 --2- 172.21.15.47:0/43203290 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f021406c8f0 0x7f021406edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f021c006fd0 tx=0x7f021c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:14.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.652+0000 7f0224ff9640 1 -- 172.21.15.47:0/43203290 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f021807a6e0 con 0x7f02280cffa0 2024-04-26T23:19:14.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.837+0000 7f022e5f0640 1 -- 172.21.15.47:0/43203290 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f02280c6600 con 0x7f02280cffa0 2024-04-26T23:19:14.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.838+0000 7f0224ff9640 1 -- 172.21.15.47:0/43203290 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18857 (secure 0 0 0) 0x7f021807e9e0 con 0x7f02280cffa0 2024-04-26T23:19:14.840 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:19:14.840 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":26,"fsid":"399006d3-4db1-43e6-ba81-a79a276eabfd","created":"2024-04-26T23:18:06.731211+0000","modified":"2024-04-26T23:19:12.124051+0000","last_up_change":"2024-04-26T23:18:17.808736+0000","last_in_change":"2024-04-26T23:18:08.945859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:18:16.937492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:18:29.033575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:18:30.318499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:18:34.014536+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ee091567-fea0-4a14-bf4e-0e16c20bb2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6819","nonce":326520779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6821","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6823","nonce":326520779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6829","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6831","nonce":326520779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6825","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6827","nonce":326520779}]},"public_addr":"172.21.15.47:6819/326520779","cluster_addr":"172.21.15.47:6823/326520779","heartbeat_back_addr":"172.21.15.47:6831/326520779","heartbeat_front_addr":"172.21.15.47:6827/326520779","state":["exists","up"]},{"osd":1,"uuid":"c14440a0-bba5-4931-a6db-b736c08e869b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6809","nonce":1912610931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6811","nonce":1912610931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6815","nonce":1912610931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6813","nonce":1912610931}]},"public_addr":"172.21.15.47:6809/1912610931","cluster_addr":"172.21.15.47:6811/1912610931","heartbeat_back_addr":"172.21.15.47:6815/1912610931","heartbeat_front_addr":"172.21.15.47:6813/1912610931","state":["exists","up"]},{"osd":2,"uuid":"3940f8de-c9b2-4cd3-a911-9c301729eb30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6801","nonce":2569869305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6803","nonce":2569869305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6807","nonce":2569869305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6805","nonce":2569869305}]},"public_addr":"172.21.15.47:6801/2569869305","cluster_addr":"172.21.15.47:6803/2569869305","heartbeat_back_addr":"172.21.15.47:6807/2569869305","heartbeat_front_addr":"172.21.15.47:6805/2569869305","state":["exists","up"]},{"osd":3,"uuid":"1286a67e-8958-426a-8c2e-bc85498c619c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6818","nonce":2791411683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6822","nonce":2791411683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6830","nonce":2791411683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6826","nonce":2791411683}]},"public_addr":"172.21.15.47:6818/2791411683","cluster_addr":"172.21.15.47:6822/2791411683","heartbeat_back_addr":"172.21.15.47:6830/2791411683","heartbeat_front_addr":"172.21.15.47:6826/2791411683","state":["exists","up"]},{"osd":4,"uuid":"d8ee2c20-e5b7-46c2-9999-b234070126cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6809","nonce":441313104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6811","nonce":441313104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6815","nonce":441313104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6813","nonce":441313104}]},"public_addr":"172.21.15.203:6809/441313104","cluster_addr":"172.21.15.203:6811/441313104","heartbeat_back_addr":"172.21.15.203:6815/441313104","heartbeat_front_addr":"172.21.15.203:6813/441313104","state":["exists","up"]},{"osd":5,"uuid":"b453a8f0-f324-4840-9ae0-d67bdadb9f8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6801","nonce":56838444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6803","nonce":56838444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6807","nonce":56838444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6805","nonce":56838444}]},"public_addr":"172.21.15.203:6801/56838444","cluster_addr":"172.21.15.203:6803/56838444","heartbeat_back_addr":"172.21.15.203:6807/56838444","heartbeat_front_addr":"172.21.15.203:6805/56838444","state":["exists","up"]},{"osd":6,"uuid":"a6d31951-2d4b-4e8d-a510-6283b01dc158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6825","nonce":2738517080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6827","nonce":2738517080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6831","nonce":2738517080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6829","nonce":2738517080}]},"public_addr":"172.21.15.203:6825/2738517080","cluster_addr":"172.21.15.203:6827/2738517080","heartbeat_back_addr":"172.21.15.203:6831/2738517080","heartbeat_front_addr":"172.21.15.203:6829/2738517080","state":["exists","up"]},{"osd":7,"uuid":"7119b8c0-fe96-402d-9e75-7b509d9d44d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6817","nonce":1315565058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6819","nonce":1315565058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6823","nonce":1315565058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6821","nonce":1315565058}]},"public_addr":"172.21.15.203:6817/1315565058","cluster_addr":"172.21.15.203:6819/1315565058","heartbeat_back_addr":"172.21.15.203:6823/1315565058","heartbeat_front_addr":"172.21.15.203:6821/1315565058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.121921+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.137494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.084814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.111593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.072430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.125066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.097867+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.057061+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":5,"to":1}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:19:14.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.840+0000 7f022e5f0640 1 -- 172.21.15.47:0/43203290 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f021406c8f0 msgr2=0x7f021406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:14.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.840+0000 7f022e5f0640 1 --2- 172.21.15.47:0/43203290 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f021406c8f0 0x7f021406edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f021c006fd0 tx=0x7f021c008040 comp rx=0 tx=0).stop 2024-04-26T23:19:14.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.840+0000 7f022e5f0640 1 -- 172.21.15.47:0/43203290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02280cffa0 msgr2=0x7f02280c93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:14.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.840+0000 7f022e5f0640 1 --2- 172.21.15.47:0/43203290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02280cffa0 0x7f02280c93c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0218009870 tx=0x7f02180046c0 comp rx=0 tx=0).stop 2024-04-26T23:19:14.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.840+0000 7f022e5f0640 1 -- 172.21.15.47:0/43203290 shutdown_connections 2024-04-26T23:19:14.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.840+0000 7f022e5f0640 1 --2- 172.21.15.47:0/43203290 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f021406c8f0 0x7f021406edb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:14.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.840+0000 7f022e5f0640 1 --2- 172.21.15.47:0/43203290 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02281608f0 0x7f022806dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:14.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.840+0000 7f022e5f0640 1 --2- 172.21.15.47:0/43203290 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02280d0a90 0x7f02281516d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:14.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.840+0000 7f022e5f0640 1 --2- 172.21.15.47:0/43203290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02280cffa0 0x7f02280c93c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:14.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.840+0000 7f022e5f0640 1 -- 172.21.15.47:0/43203290 >> 172.21.15.47:0/43203290 conn(0x7f02280d2ec0 msgr2=0x7f02280b6980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:14.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.840+0000 7f022e5f0640 1 -- 172.21.15.47:0/43203290 shutdown_connections 2024-04-26T23:19:14.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.840+0000 7f022e5f0640 1 -- 172.21.15.47:0/43203290 wait complete. 2024-04-26T23:19:14.853 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:18:16.937492+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:18:29.033575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:18:30.318499+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T23:18:34.014536+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T23:19:14.854 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:19:14.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.956+0000 7f4134cc8640 1 Processor -- start 2024-04-26T23:19:14.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.956+0000 7f4134cc8640 1 -- start start 2024-04-26T23:19:14.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.957+0000 7f4134cc8640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f413015f170 0x7f41300d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:14.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.957+0000 7f4134cc8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41300d3bc0 0x7f41300d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:14.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.957+0000 7f4134cc8640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41300d1040 con 0x7f413015f550 2024-04-26T23:19:14.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.957+0000 7f4134cc8640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f41300c9430 con 0x7f413015f170 2024-04-26T23:19:14.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.957+0000 7f4134cc8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f41300c9680 con 0x7f41300d3bc0 2024-04-26T23:19:14.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.958+0000 7f412e59c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f413015f170 0x7f41300d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:14.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.958+0000 7f412dd9b640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f413015f550 0x7f41300d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.47:49818/0 (socket says 172.21.15.47:49818) 2024-04-26T23:19:14.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.958+0000 7f412ed9d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41300d3bc0 0x7f41300d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:14.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.958+0000 7f412dd9b640 1 -- 172.21.15.47:0/99568607 learned_addr learned my addr 172.21.15.47:0/99568607 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:14.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.958+0000 7f412d59a640 1 -- 172.21.15.47:0/99568607 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1230695269 0 0) 0x7f4118001110 con 0x7f413015f550 2024-04-26T23:19:14.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.958+0000 7f412d59a640 1 -- 172.21.15.47:0/99568607 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f411c003520 con 0x7f413015f550 2024-04-26T23:19:14.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.958+0000 7f412ed9d640 1 -- 172.21.15.47:0/99568607 >> v2:172.21.15.203:3300/0 conn(0x7f413015f170 msgr2=0x7f41300d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:14.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.958+0000 7f412ed9d640 1 --2- 172.21.15.47:0/99568607 >> v2:172.21.15.203:3300/0 conn(0x7f413015f170 0x7f41300d5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:14.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.958+0000 7f412ed9d640 1 -- 172.21.15.47:0/99568607 >> v1:172.21.15.203:6789/0 conn(0x7f413015f550 legacy=0x7f41300d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:14.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.958+0000 7f412ed9d640 1 -- 172.21.15.47:0/99568607 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41300c9960 con 0x7f41300d3bc0 2024-04-26T23:19:14.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.958+0000 7f412e59c640 1 --2- 172.21.15.47:0/99568607 >> v2:172.21.15.203:3300/0 conn(0x7f413015f170 0x7f41300d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:19:14.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.959+0000 7f412ed9d640 1 --2- 172.21.15.47:0/99568607 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41300d3bc0 0x7f41300d2cb0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4120009dc0 tx=0x7f412002a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30ed62fb9e4604af server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:14.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.959+0000 7f412d59a640 1 -- 172.21.15.47:0/99568607 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412002ae60 con 0x7f41300d3bc0 2024-04-26T23:19:14.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.959+0000 7f412d59a640 1 -- 172.21.15.47:0/99568607 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4120007740 con 0x7f41300d3bc0 2024-04-26T23:19:14.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.959+0000 7f412d59a640 1 -- 172.21.15.47:0/99568607 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4120007af0 con 0x7f41300d3bc0 2024-04-26T23:19:14.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.959+0000 7f4134cc8640 1 -- 172.21.15.47:0/99568607 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41300d3bc0 msgr2=0x7f41300d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:14.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.959+0000 7f4134cc8640 1 --2- 172.21.15.47:0/99568607 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41300d3bc0 0x7f41300d2cb0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4120009dc0 tx=0x7f412002a680 comp rx=0 tx=0).stop 2024-04-26T23:19:14.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.960+0000 7f4134cc8640 1 -- 172.21.15.47:0/99568607 shutdown_connections 2024-04-26T23:19:14.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.960+0000 7f4134cc8640 1 --2- 172.21.15.47:0/99568607 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41300d3bc0 0x7f41300d2cb0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:14.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.960+0000 7f4134cc8640 1 --2- 172.21.15.47:0/99568607 >> v2:172.21.15.203:3300/0 conn(0x7f413015f170 0x7f41300d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:14.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.960+0000 7f4134cc8640 1 -- 172.21.15.47:0/99568607 >> 172.21.15.47:0/99568607 conn(0x7f41300d4f60 msgr2=0x7f41300d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:14.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.960+0000 7f4134cc8640 1 -- 172.21.15.47:0/99568607 shutdown_connections 2024-04-26T23:19:14.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.960+0000 7f4134cc8640 1 -- 172.21.15.47:0/99568607 wait complete. 2024-04-26T23:19:14.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.961+0000 7f4134cc8640 1 Processor -- start 2024-04-26T23:19:14.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.961+0000 7f4134cc8640 1 -- start start 2024-04-26T23:19:14.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.962+0000 7f4134cc8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41300d3bc0 0x7f4130158510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:14.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.962+0000 7f4134cc8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f413015f170 0x7f41300d1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:14.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.962+0000 7f4134cc8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41300d2420 0x7f41300ce190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:14.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.962+0000 7f4134cc8640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f41300ca5c0 con 0x7f41300d2420 2024-04-26T23:19:14.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.962+0000 7f4134cc8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f41300ca440 con 0x7f41300d3bc0 2024-04-26T23:19:14.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.962+0000 7f4134cc8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f41300ca740 con 0x7f413015f170 2024-04-26T23:19:14.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.962+0000 7f412ed9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41300d2420 0x7f41300ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:14.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.962+0000 7f412ed9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41300d2420 0x7f41300ce190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41260/0 (socket says 172.21.15.47:41260) 2024-04-26T23:19:14.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.962+0000 7f412ed9d640 1 -- 172.21.15.47:0/1801982980 learned_addr learned my addr 172.21.15.47:0/1801982980 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:14.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.962+0000 7f412dd9b640 1 --2- 172.21.15.47:0/1801982980 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f413015f170 0x7f41300d1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:14.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.962+0000 7f412e59c640 1 --2- 172.21.15.47:0/1801982980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41300d3bc0 0x7f4130158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:14.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.962+0000 7f412ed9d640 1 -- 172.21.15.47:0/1801982980 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f413015f170 msgr2=0x7f41300d1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:14.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.962+0000 7f412ed9d640 1 --2- 172.21.15.47:0/1801982980 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f413015f170 0x7f41300d1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:14.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.962+0000 7f412ed9d640 1 -- 172.21.15.47:0/1801982980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41300d3bc0 msgr2=0x7f4130158510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:14.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.962+0000 7f412ed9d640 1 --2- 172.21.15.47:0/1801982980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41300d3bc0 0x7f4130158510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:14.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.962+0000 7f412ed9d640 1 -- 172.21.15.47:0/1801982980 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41300cef00 con 0x7f41300d2420 2024-04-26T23:19:14.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.962+0000 7f412e59c640 1 --2- 172.21.15.47:0/1801982980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41300d3bc0 0x7f4130158510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:19:14.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.962+0000 7f412ed9d640 1 --2- 172.21.15.47:0/1801982980 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41300d2420 0x7f41300ce190 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4120009dc0 tx=0x7f4120004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:14.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.963+0000 7f4126ffd640 1 -- 172.21.15.47:0/1801982980 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412002ae60 con 0x7f41300d2420 2024-04-26T23:19:14.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.963+0000 7f4126ffd640 1 -- 172.21.15.47:0/1801982980 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4120003e10 con 0x7f41300d2420 2024-04-26T23:19:14.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.963+0000 7f4134cc8640 1 -- 172.21.15.47:0/1801982980 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41300cf1e0 con 0x7f41300d2420 2024-04-26T23:19:14.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.963+0000 7f4126ffd640 1 -- 172.21.15.47:0/1801982980 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4120004490 con 0x7f41300d2420 2024-04-26T23:19:14.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.963+0000 7f4134cc8640 1 -- 172.21.15.47:0/1801982980 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4130158f60 con 0x7f41300d2420 2024-04-26T23:19:14.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.964+0000 7f4134cc8640 1 -- 172.21.15.47:0/1801982980 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41300694f0 con 0x7f41300d2420 2024-04-26T23:19:14.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.964+0000 7f4126ffd640 1 -- 172.21.15.47:0/1801982980 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4120039040 con 0x7f41300d2420 2024-04-26T23:19:14.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.964+0000 7f4126ffd640 1 --2- 172.21.15.47:0/1801982980 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f411006c8f0 0x7f411006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:14.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.965+0000 7f4126ffd640 1 -- 172.21.15.47:0/1801982980 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f41200acda0 con 0x7f41300d2420 2024-04-26T23:19:14.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.965+0000 7f412e59c640 1 --2- 172.21.15.47:0/1801982980 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f411006c8f0 0x7f411006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:14.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.965+0000 7f412e59c640 1 --2- 172.21.15.47:0/1801982980 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f411006c8f0 0x7f411006edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4128009ba0 tx=0x7f4128008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:14.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:14.969+0000 7f4126ffd640 1 -- 172.21.15.47:0/1801982980 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4120040030 con 0x7f41300d2420 2024-04-26T23:19:15.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.154+0000 7f4134cc8640 1 -- 172.21.15.47:0/1801982980 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f41301593a0 con 0x7f41300d2420 2024-04-26T23:19:15.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.154+0000 7f4126ffd640 1 -- 172.21.15.47:0/1801982980 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f412007a7a0 con 0x7f41300d2420 2024-04-26T23:19:15.156 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-26T23:19:15.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.156+0000 7f4134cc8640 1 -- 172.21.15.47:0/1801982980 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f411006c8f0 msgr2=0x7f411006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:15.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.156+0000 7f4134cc8640 1 --2- 172.21.15.47:0/1801982980 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f411006c8f0 0x7f411006edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4128009ba0 tx=0x7f4128008040 comp rx=0 tx=0).stop 2024-04-26T23:19:15.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.156+0000 7f4134cc8640 1 -- 172.21.15.47:0/1801982980 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41300d2420 msgr2=0x7f41300ce190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:15.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.156+0000 7f4134cc8640 1 --2- 172.21.15.47:0/1801982980 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41300d2420 0x7f41300ce190 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4120009dc0 tx=0x7f4120004a40 comp rx=0 tx=0).stop 2024-04-26T23:19:15.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.156+0000 7f4134cc8640 1 -- 172.21.15.47:0/1801982980 shutdown_connections 2024-04-26T23:19:15.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.156+0000 7f4134cc8640 1 --2- 172.21.15.47:0/1801982980 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f411006c8f0 0x7f411006edb0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:15.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.156+0000 7f4134cc8640 1 --2- 172.21.15.47:0/1801982980 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41300d2420 0x7f41300ce190 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:15.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.156+0000 7f4134cc8640 1 --2- 172.21.15.47:0/1801982980 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f413015f170 0x7f41300d1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:15.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.156+0000 7f4134cc8640 1 --2- 172.21.15.47:0/1801982980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41300d3bc0 0x7f4130158510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:15.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.156+0000 7f4134cc8640 1 -- 172.21.15.47:0/1801982980 >> 172.21.15.47:0/1801982980 conn(0x7f41300d4f60 msgr2=0x7f41300d1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:15.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.156+0000 7f4134cc8640 1 -- 172.21.15.47:0/1801982980 shutdown_connections 2024-04-26T23:19:15.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.156+0000 7f4134cc8640 1 -- 172.21.15.47:0/1801982980 wait complete. 2024-04-26T23:19:15.168 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:19:15.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.271+0000 7f4f59ee1640 1 Processor -- start 2024-04-26T23:19:15.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.271+0000 7f4f59ee1640 1 -- start start 2024-04-26T23:19:15.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.272+0000 7f4f59ee1640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4f54156460 0x7f4f540cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:15.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.272+0000 7f4f59ee1640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4f5406fea0 0x7f4f54072c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:15.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.272+0000 7f4f59ee1640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f54156840 con 0x7f4f540cba10 2024-04-26T23:19:15.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.272+0000 7f4f59ee1640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f4f540cbf20 con 0x7f4f5406fea0 2024-04-26T23:19:15.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.272+0000 7f4f59ee1640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f4f540cc200 con 0x7f4f54156460 2024-04-26T23:19:15.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.272+0000 7f4f53fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4f5406fea0 0x7f4f54072c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:15.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.272+0000 7f4f53fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4f5406fea0 0x7f4f54072c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41268/0 (socket says 172.21.15.47:41268) 2024-04-26T23:19:15.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.272+0000 7f4f53fff640 1 -- 172.21.15.47:0/1819694070 learned_addr learned my addr 172.21.15.47:0/1819694070 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:15.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.272+0000 7f4f53fff640 1 -- 172.21.15.47:0/1819694070 >> v2:172.21.15.203:3300/0 conn(0x7f4f54156460 msgr2=0x7f4f540cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:15.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.272+0000 7f4f537fe640 1 --2- 172.21.15.47:0/1819694070 >> v2:172.21.15.203:3300/0 conn(0x7f4f54156460 0x7f4f540cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:15.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.272+0000 7f4f53fff640 1 --2- 172.21.15.47:0/1819694070 >> v2:172.21.15.203:3300/0 conn(0x7f4f54156460 0x7f4f540cb4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:15.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.272+0000 7f4f53fff640 1 -- 172.21.15.47:0/1819694070 >> v1:172.21.15.203:6789/0 conn(0x7f4f540cba10 legacy=0x7f4f5406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:15.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.272+0000 7f4f53fff640 1 -- 172.21.15.47:0/1819694070 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f540cc4e0 con 0x7f4f5406fea0 2024-04-26T23:19:15.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.273+0000 7f4f53fff640 1 --2- 172.21.15.47:0/1819694070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f5406fea0 0x7f4f54072c50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4f540b05d0 tx=0x7f4f4c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cc3701b12de42ac server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:15.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.273+0000 7f4f52ffd640 1 -- 172.21.15.47:0/1819694070 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c036070 con 0x7f4f5406fea0 2024-04-26T23:19:15.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.273+0000 7f4f52ffd640 1 -- 172.21.15.47:0/1819694070 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f4c031600 con 0x7f4f5406fea0 2024-04-26T23:19:15.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.273+0000 7f4f52ffd640 1 -- 172.21.15.47:0/1819694070 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c0319d0 con 0x7f4f5406fea0 2024-04-26T23:19:15.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.274+0000 7f4f59ee1640 1 -- 172.21.15.47:0/1819694070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f5406fea0 msgr2=0x7f4f54072c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:15.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.274+0000 7f4f59ee1640 1 --2- 172.21.15.47:0/1819694070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f5406fea0 0x7f4f54072c50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4f540b05d0 tx=0x7f4f4c02a6f0 comp rx=0 tx=0).stop 2024-04-26T23:19:15.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.274+0000 7f4f59ee1640 1 -- 172.21.15.47:0/1819694070 shutdown_connections 2024-04-26T23:19:15.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.274+0000 7f4f59ee1640 1 --2- 172.21.15.47:0/1819694070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f5406fea0 0x7f4f54072c50 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:15.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.274+0000 7f4f59ee1640 1 --2- 172.21.15.47:0/1819694070 >> v2:172.21.15.203:3300/0 conn(0x7f4f54156460 0x7f4f540cb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:15.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.274+0000 7f4f59ee1640 1 -- 172.21.15.47:0/1819694070 >> 172.21.15.47:0/1819694070 conn(0x7f4f540d0be0 msgr2=0x7f4f540d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:15.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.274+0000 7f4f59ee1640 1 -- 172.21.15.47:0/1819694070 shutdown_connections 2024-04-26T23:19:15.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.274+0000 7f4f59ee1640 1 -- 172.21.15.47:0/1819694070 wait complete. 2024-04-26T23:19:15.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.275+0000 7f4f59ee1640 1 Processor -- start 2024-04-26T23:19:15.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.275+0000 7f4f59ee1640 1 -- start start 2024-04-26T23:19:15.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.276+0000 7f4f59ee1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f5406fea0 0x7f4f541f3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:15.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.276+0000 7f4f59ee1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f540cba10 0x7f4f541f2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:15.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.276+0000 7f4f59ee1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f54156460 0x7f4f540c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:15.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.276+0000 7f4f59ee1640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5406e260 con 0x7f4f54156460 2024-04-26T23:19:15.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.276+0000 7f4f59ee1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4f54072700 con 0x7f4f5406fea0 2024-04-26T23:19:15.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.276+0000 7f4f59ee1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f4f5406e3e0 con 0x7f4f540cba10 2024-04-26T23:19:15.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.276+0000 7f4f53fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f54156460 0x7f4f540c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:15.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.276+0000 7f4f53fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f54156460 0x7f4f540c7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41282/0 (socket says 172.21.15.47:41282) 2024-04-26T23:19:15.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.276+0000 7f4f53fff640 1 -- 172.21.15.47:0/3751831753 learned_addr learned my addr 172.21.15.47:0/3751831753 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:15.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.276+0000 7f4f537fe640 1 --2- 172.21.15.47:0/3751831753 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f5406fea0 0x7f4f541f3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:15.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.276+0000 7f4f53fff640 1 -- 172.21.15.47:0/3751831753 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f540cba10 msgr2=0x7f4f541f2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:15.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.276+0000 7f4f53fff640 1 --2- 172.21.15.47:0/3751831753 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f540cba10 0x7f4f541f2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:15.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.276+0000 7f4f53fff640 1 -- 172.21.15.47:0/3751831753 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f5406fea0 msgr2=0x7f4f541f3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:15.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.276+0000 7f4f53fff640 1 --2- 172.21.15.47:0/3751831753 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f5406fea0 0x7f4f541f3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:15.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.276+0000 7f4f53fff640 1 -- 172.21.15.47:0/3751831753 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f540c9a70 con 0x7f4f54156460 2024-04-26T23:19:15.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.276+0000 7f4f53fff640 1 --2- 172.21.15.47:0/3751831753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f54156460 0x7f4f540c7cb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c009f40 tx=0x7f4f4c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:15.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.276+0000 7f4f50ff9640 1 -- 172.21.15.47:0/3751831753 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c036050 con 0x7f4f54156460 2024-04-26T23:19:15.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.276+0000 7f4f50ff9640 1 -- 172.21.15.47:0/3751831753 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f4c042070 con 0x7f4f54156460 2024-04-26T23:19:15.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.276+0000 7f4f50ff9640 1 -- 172.21.15.47:0/3751831753 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c009510 con 0x7f4f54156460 2024-04-26T23:19:15.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.276+0000 7f4f59ee1640 1 -- 172.21.15.47:0/3751831753 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f540c9d50 con 0x7f4f54156460 2024-04-26T23:19:15.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.277+0000 7f4f59ee1640 1 -- 172.21.15.47:0/3751831753 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f540ca310 con 0x7f4f54156460 2024-04-26T23:19:15.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.278+0000 7f4f59ee1640 1 -- 172.21.15.47:0/3751831753 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f5406e8b0 con 0x7f4f54156460 2024-04-26T23:19:15.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.278+0000 7f4f50ff9640 1 -- 172.21.15.47:0/3751831753 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4f4c007bb0 con 0x7f4f54156460 2024-04-26T23:19:15.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.278+0000 7f4f50ff9640 1 --2- 172.21.15.47:0/3751831753 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4f3406c920 0x7f4f3406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:15.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.279+0000 7f4f50ff9640 1 -- 172.21.15.47:0/3751831753 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f4f4c0acf70 con 0x7f4f54156460 2024-04-26T23:19:15.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.279+0000 7f4f537fe640 1 --2- 172.21.15.47:0/3751831753 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4f3406c920 0x7f4f3406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:15.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.279+0000 7f4f537fe640 1 --2- 172.21.15.47:0/3751831753 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4f3406c920 0x7f4f3406ede0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4f4400b7c0 tx=0x7f4f44008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:15.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.283+0000 7f4f50ff9640 1 -- 172.21.15.47:0/3751831753 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4f5406e8b0 con 0x7f4f54156460 2024-04-26T23:19:15.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.466+0000 7f4f59ee1640 1 -- 172.21.15.47:0/3751831753 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4f540694f0 con 0x7f4f54156460 2024-04-26T23:19:15.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.467+0000 7f4f50ff9640 1 -- 172.21.15.47:0/3751831753 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f4f4c07af80 con 0x7f4f54156460 2024-04-26T23:19:15.469 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-26T23:19:15.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.468+0000 7f4f59ee1640 1 -- 172.21.15.47:0/3751831753 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4f3406c920 msgr2=0x7f4f3406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:15.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.468+0000 7f4f59ee1640 1 --2- 172.21.15.47:0/3751831753 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4f3406c920 0x7f4f3406ede0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4f4400b7c0 tx=0x7f4f44008040 comp rx=0 tx=0).stop 2024-04-26T23:19:15.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.468+0000 7f4f59ee1640 1 -- 172.21.15.47:0/3751831753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f54156460 msgr2=0x7f4f540c7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:15.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.468+0000 7f4f59ee1640 1 --2- 172.21.15.47:0/3751831753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f54156460 0x7f4f540c7cb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c009f40 tx=0x7f4f4c004a40 comp rx=0 tx=0).stop 2024-04-26T23:19:15.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.469+0000 7f4f59ee1640 1 -- 172.21.15.47:0/3751831753 shutdown_connections 2024-04-26T23:19:15.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.469+0000 7f4f59ee1640 1 --2- 172.21.15.47:0/3751831753 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4f3406c920 0x7f4f3406ede0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:15.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.469+0000 7f4f59ee1640 1 --2- 172.21.15.47:0/3751831753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f54156460 0x7f4f540c7cb0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:15.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.469+0000 7f4f59ee1640 1 --2- 172.21.15.47:0/3751831753 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f540cba10 0x7f4f541f2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:15.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.469+0000 7f4f59ee1640 1 --2- 172.21.15.47:0/3751831753 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f5406fea0 0x7f4f541f3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:15.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.469+0000 7f4f59ee1640 1 -- 172.21.15.47:0/3751831753 >> 172.21.15.47:0/3751831753 conn(0x7f4f540d0be0 msgr2=0x7f4f540b0780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:15.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.469+0000 7f4f59ee1640 1 -- 172.21.15.47:0/3751831753 shutdown_connections 2024-04-26T23:19:15.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.469+0000 7f4f59ee1640 1 -- 172.21.15.47:0/3751831753 wait complete. 2024-04-26T23:19:15.481 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:19:15.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.584+0000 7fd1eacf1640 1 Processor -- start 2024-04-26T23:19:15.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.584+0000 7fd1eacf1640 1 -- start start 2024-04-26T23:19:15.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.585+0000 7fd1eacf1640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd1e4070350 0x7fd1e4070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:15.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.585+0000 7fd1eacf1640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1e40d2390 con 0x7fd1e4071090 2024-04-26T23:19:15.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.585+0000 7fd1eacf1640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1e406d3f0 con 0x7fd1e40cfde0 2024-04-26T23:19:15.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.585+0000 7fd1eacf1640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fd1e40afab0 con 0x7fd1e4070350 2024-04-26T23:19:15.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.585+0000 7fd1e3fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd1e4070350 0x7fd1e4070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:15.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.585+0000 7fd1e928e640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd1e4071090 0x7fd1e415e9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:32988/0 (socket says 172.21.15.47:32988) 2024-04-26T23:19:15.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.585+0000 7fd1e3fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd1e4070350 0x7fd1e4070b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41298/0 (socket says 172.21.15.47:41298) 2024-04-26T23:19:15.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.585+0000 7fd1e928e640 1 -- 172.21.15.47:0/2579698060 learned_addr learned my addr 172.21.15.47:0/2579698060 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:15.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.586+0000 7fd1e3fff640 1 -- 172.21.15.47:0/2579698060 >> v1:172.21.15.203:6789/0 conn(0x7fd1e40cfde0 legacy=0x7fd1e40d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:15.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.586+0000 7fd1e3fff640 1 -- 172.21.15.47:0/2579698060 >> v1:172.21.15.47:6789/0 conn(0x7fd1e4071090 legacy=0x7fd1e415e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:15.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.586+0000 7fd1e37fe640 1 -- 172.21.15.47:0/2579698060 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4261783282 0 0) 0x7fd1e40d2390 con 0x7fd1e4071090 2024-04-26T23:19:15.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.586+0000 7fd1e3fff640 1 -- 172.21.15.47:0/2579698060 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1e406d6d0 con 0x7fd1e4070350 2024-04-26T23:19:15.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.586+0000 7fd1e3fff640 1 --2- 172.21.15.47:0/2579698060 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1e4070350 0x7fd1e4070b50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd1d80097a0 tx=0x7fd1d802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1c1d26d70b85436 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:15.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.586+0000 7fd1e37fe640 1 -- 172.21.15.47:0/2579698060 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d8037070 con 0x7fd1e4070350 2024-04-26T23:19:15.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.587+0000 7fd1e37fe640 1 -- 172.21.15.47:0/2579698060 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1d8032440 con 0x7fd1e4070350 2024-04-26T23:19:15.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.587+0000 7fd1e37fe640 1 -- 172.21.15.47:0/2579698060 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d8032810 con 0x7fd1e4070350 2024-04-26T23:19:15.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.587+0000 7fd1eacf1640 1 -- 172.21.15.47:0/2579698060 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1e4070350 msgr2=0x7fd1e4070b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:15.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.587+0000 7fd1eacf1640 1 --2- 172.21.15.47:0/2579698060 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1e4070350 0x7fd1e4070b50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd1d80097a0 tx=0x7fd1d802a1a0 comp rx=0 tx=0).stop 2024-04-26T23:19:15.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.587+0000 7fd1eacf1640 1 -- 172.21.15.47:0/2579698060 shutdown_connections 2024-04-26T23:19:15.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.587+0000 7fd1eacf1640 1 --2- 172.21.15.47:0/2579698060 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1e4070350 0x7fd1e4070b50 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:15.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.587+0000 7fd1eacf1640 1 -- 172.21.15.47:0/2579698060 >> 172.21.15.47:0/2579698060 conn(0x7fd1e40d2d80 msgr2=0x7fd1e40d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:15.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.587+0000 7fd1eacf1640 1 -- 172.21.15.47:0/2579698060 shutdown_connections 2024-04-26T23:19:15.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.587+0000 7fd1eacf1640 1 -- 172.21.15.47:0/2579698060 wait complete. 2024-04-26T23:19:15.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.588+0000 7fd1eacf1640 1 Processor -- start 2024-04-26T23:19:15.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.588+0000 7fd1eacf1640 1 -- start start 2024-04-26T23:19:15.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.589+0000 7fd1eacf1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1e4070350 0x7fd1e40b4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:15.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.589+0000 7fd1eacf1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1e4071090 0x7fd1e409fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:15.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.589+0000 7fd1eacf1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1e40cfde0 0x7fd1e41616b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:15.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.589+0000 7fd1eacf1640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e40714c0 con 0x7fd1e40cfde0 2024-04-26T23:19:15.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.589+0000 7fd1eacf1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e406dd20 con 0x7fd1e4071090 2024-04-26T23:19:15.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.589+0000 7fd1eacf1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd1e406f1c0 con 0x7fd1e4070350 2024-04-26T23:19:15.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.589+0000 7fd1e928e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1e40cfde0 0x7fd1e41616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:15.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.589+0000 7fd1e928e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1e40cfde0 0x7fd1e41616b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41312/0 (socket says 172.21.15.47:41312) 2024-04-26T23:19:15.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.589+0000 7fd1e928e640 1 -- 172.21.15.47:0/1120115578 learned_addr learned my addr 172.21.15.47:0/1120115578 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:15.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.589+0000 7fd1e3fff640 1 --2- 172.21.15.47:0/1120115578 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1e4071090 0x7fd1e409fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:15.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.589+0000 7fd1e8a8d640 1 --2- 172.21.15.47:0/1120115578 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1e4070350 0x7fd1e40b4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:15.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.590+0000 7fd1e928e640 1 -- 172.21.15.47:0/1120115578 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1e4070350 msgr2=0x7fd1e40b4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:15.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.590+0000 7fd1e928e640 1 --2- 172.21.15.47:0/1120115578 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1e4070350 0x7fd1e40b4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:15.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.590+0000 7fd1e928e640 1 -- 172.21.15.47:0/1120115578 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1e4071090 msgr2=0x7fd1e409fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:15.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.590+0000 7fd1e928e640 1 --2- 172.21.15.47:0/1120115578 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1e4071090 0x7fd1e409fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:15.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.590+0000 7fd1e928e640 1 -- 172.21.15.47:0/1120115578 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1e41f4fb0 con 0x7fd1e40cfde0 2024-04-26T23:19:15.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.590+0000 7fd1e3fff640 1 --2- 172.21.15.47:0/1120115578 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1e4071090 0x7fd1e409fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:19:15.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.590+0000 7fd1e8a8d640 1 --2- 172.21.15.47:0/1120115578 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1e4070350 0x7fd1e40b4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:19:15.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.590+0000 7fd1e928e640 1 --2- 172.21.15.47:0/1120115578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1e40cfde0 0x7fd1e41616b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc00c7a0 tx=0x7fd1dc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:15.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.590+0000 7fd1e17fa640 1 -- 172.21.15.47:0/1120115578 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc018070 con 0x7fd1e40cfde0 2024-04-26T23:19:15.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.590+0000 7fd1e17fa640 1 -- 172.21.15.47:0/1120115578 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1dc00a640 con 0x7fd1e40cfde0 2024-04-26T23:19:15.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.590+0000 7fd1e17fa640 1 -- 172.21.15.47:0/1120115578 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc00aab0 con 0x7fd1e40cfde0 2024-04-26T23:19:15.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.590+0000 7fd1eacf1640 1 -- 172.21.15.47:0/1120115578 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1e41f5290 con 0x7fd1e40cfde0 2024-04-26T23:19:15.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.590+0000 7fd1eacf1640 1 -- 172.21.15.47:0/1120115578 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1e41f5850 con 0x7fd1e40cfde0 2024-04-26T23:19:15.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.591+0000 7fd1eacf1640 1 -- 172.21.15.47:0/1120115578 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1e40694f0 con 0x7fd1e40cfde0 2024-04-26T23:19:15.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.592+0000 7fd1e17fa640 1 -- 172.21.15.47:0/1120115578 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd1dc01b070 con 0x7fd1e40cfde0 2024-04-26T23:19:15.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.592+0000 7fd1e17fa640 1 --2- 172.21.15.47:0/1120115578 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd1d006c8f0 0x7fd1d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:15.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.592+0000 7fd1e17fa640 1 -- 172.21.15.47:0/1120115578 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7fd1dc08b4f0 con 0x7fd1e40cfde0 2024-04-26T23:19:15.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.592+0000 7fd1e8a8d640 1 --2- 172.21.15.47:0/1120115578 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd1d006c8f0 0x7fd1d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:15.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.593+0000 7fd1e8a8d640 1 --2- 172.21.15.47:0/1120115578 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd1d006c8f0 0x7fd1d006edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4007fd0 tx=0x7fd1d4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:15.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.596+0000 7fd1e17fa640 1 -- 172.21.15.47:0/1120115578 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd1dc059590 con 0x7fd1e40cfde0 2024-04-26T23:19:15.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.783+0000 7fd1eacf1640 1 -- 172.21.15.47:0/1120115578 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd1e40d2520 con 0x7fd1e40cfde0 2024-04-26T23:19:15.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.783+0000 7fd1e17fa640 1 -- 172.21.15.47:0/1120115578 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7fd1dc058e90 con 0x7fd1e40cfde0 2024-04-26T23:19:15.785 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-26T23:19:15.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.785+0000 7fd1eacf1640 1 -- 172.21.15.47:0/1120115578 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd1d006c8f0 msgr2=0x7fd1d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:15.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.785+0000 7fd1eacf1640 1 --2- 172.21.15.47:0/1120115578 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd1d006c8f0 0x7fd1d006edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4007fd0 tx=0x7fd1d4009040 comp rx=0 tx=0).stop 2024-04-26T23:19:15.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.785+0000 7fd1eacf1640 1 -- 172.21.15.47:0/1120115578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1e40cfde0 msgr2=0x7fd1e41616b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:15.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.785+0000 7fd1eacf1640 1 --2- 172.21.15.47:0/1120115578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1e40cfde0 0x7fd1e41616b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc00c7a0 tx=0x7fd1dc00cc60 comp rx=0 tx=0).stop 2024-04-26T23:19:15.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.785+0000 7fd1eacf1640 1 -- 172.21.15.47:0/1120115578 shutdown_connections 2024-04-26T23:19:15.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.785+0000 7fd1eacf1640 1 --2- 172.21.15.47:0/1120115578 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd1d006c8f0 0x7fd1d006edb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:15.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.785+0000 7fd1eacf1640 1 --2- 172.21.15.47:0/1120115578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1e40cfde0 0x7fd1e41616b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:15.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.785+0000 7fd1eacf1640 1 --2- 172.21.15.47:0/1120115578 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1e4071090 0x7fd1e409fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:15.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.785+0000 7fd1eacf1640 1 --2- 172.21.15.47:0/1120115578 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1e4070350 0x7fd1e40b4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:15.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.785+0000 7fd1eacf1640 1 -- 172.21.15.47:0/1120115578 >> 172.21.15.47:0/1120115578 conn(0x7fd1e40d2d80 msgr2=0x7fd1e4160d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:15.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.785+0000 7fd1eacf1640 1 -- 172.21.15.47:0/1120115578 shutdown_connections 2024-04-26T23:19:15.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.785+0000 7fd1eacf1640 1 -- 172.21.15.47:0/1120115578 wait complete. 2024-04-26T23:19:15.798 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T23:19:15.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.900+0000 7f4ac9daf640 1 Processor -- start 2024-04-26T23:19:15.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.900+0000 7f4ac9daf640 1 -- start start 2024-04-26T23:19:15.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.901+0000 7f4ac9daf640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4ac40cfde0 0x7f4ac40d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:15.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.901+0000 7f4ac9daf640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ac40d2390 con 0x7f4ac4070350 2024-04-26T23:19:15.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.901+0000 7f4ac9daf640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ac406d3f0 con 0x7f4ac4071260 2024-04-26T23:19:15.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.901+0000 7f4ac9daf640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f4ac40afab0 con 0x7f4ac40cfde0 2024-04-26T23:19:15.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.901+0000 7f4ac37fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4ac40cfde0 0x7f4ac40d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:15.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.901+0000 7f4ac2ffd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4ac4070350 0x7f4ac4070b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33002/0 (socket says 172.21.15.47:33002) 2024-04-26T23:19:15.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.901+0000 7f4ac37fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4ac40cfde0 0x7f4ac40d1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41324/0 (socket says 172.21.15.47:41324) 2024-04-26T23:19:15.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.901+0000 7f4ac2ffd640 1 -- 172.21.15.47:0/1801643428 learned_addr learned my addr 172.21.15.47:0/1801643428 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:15.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.901+0000 7f4ac37fe640 1 -- 172.21.15.47:0/1801643428 >> v1:172.21.15.203:6789/0 conn(0x7f4ac4071260 legacy=0x7f4ac415e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:15.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.901+0000 7f4ac37fe640 1 -- 172.21.15.47:0/1801643428 >> v1:172.21.15.47:6789/0 conn(0x7f4ac4070350 legacy=0x7f4ac4070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:15.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.901+0000 7f4ac37fe640 1 -- 172.21.15.47:0/1801643428 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ac406d6d0 con 0x7f4ac40cfde0 2024-04-26T23:19:15.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.901+0000 7f4ac27fc640 1 -- 172.21.15.47:0/1801643428 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3236799621 0 0) 0x7f4ac40d2390 con 0x7f4ac4070350 2024-04-26T23:19:15.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.902+0000 7f4ac37fe640 1 --2- 172.21.15.47:0/1801643428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ac40cfde0 0x7f4ac40d1b60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4ab40097a0 tx=0x7f4ab402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a2d6c150f71c2a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:15.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.902+0000 7f4ac27fc640 1 -- 172.21.15.47:0/1801643428 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab4037070 con 0x7f4ac40cfde0 2024-04-26T23:19:15.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.902+0000 7f4ac27fc640 1 -- 172.21.15.47:0/1801643428 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ab402ae50 con 0x7f4ac40cfde0 2024-04-26T23:19:15.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.902+0000 7f4ac27fc640 1 -- 172.21.15.47:0/1801643428 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab4032660 con 0x7f4ac40cfde0 2024-04-26T23:19:15.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.903+0000 7f4ac9daf640 1 -- 172.21.15.47:0/1801643428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ac40cfde0 msgr2=0x7f4ac40d1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:15.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.903+0000 7f4ac9daf640 1 --2- 172.21.15.47:0/1801643428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ac40cfde0 0x7f4ac40d1b60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4ab40097a0 tx=0x7f4ab402a140 comp rx=0 tx=0).stop 2024-04-26T23:19:15.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.903+0000 7f4ac9daf640 1 -- 172.21.15.47:0/1801643428 shutdown_connections 2024-04-26T23:19:15.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.903+0000 7f4ac9daf640 1 --2- 172.21.15.47:0/1801643428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ac40cfde0 0x7f4ac40d1b60 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:15.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.903+0000 7f4ac9daf640 1 -- 172.21.15.47:0/1801643428 >> 172.21.15.47:0/1801643428 conn(0x7f4ac40d2d80 msgr2=0x7f4ac40d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:15.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.903+0000 7f4ac9daf640 1 -- 172.21.15.47:0/1801643428 shutdown_connections 2024-04-26T23:19:15.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.903+0000 7f4ac9daf640 1 -- 172.21.15.47:0/1801643428 wait complete. 2024-04-26T23:19:15.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.904+0000 7f4ac9daf640 1 Processor -- start 2024-04-26T23:19:15.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.904+0000 7f4ac9daf640 1 -- start start 2024-04-26T23:19:15.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.905+0000 7f4ac9daf640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ac4070350 0x7f4ac4161160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:15.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.905+0000 7f4ac9daf640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ac4071260 0x7f4ac409dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:15.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.905+0000 7f4ac9daf640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ac40cfde0 0x7f4ac415cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:15.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.905+0000 7f4ac9daf640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac40d20a0 con 0x7f4ac4071260 2024-04-26T23:19:15.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.905+0000 7f4ac9daf640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac406dd20 con 0x7f4ac40cfde0 2024-04-26T23:19:15.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.905+0000 7f4ac9daf640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f4ac406f1c0 con 0x7f4ac4070350 2024-04-26T23:19:15.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.905+0000 7f4ac2ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ac4071260 0x7f4ac409dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:15.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.905+0000 7f4ac2ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ac4071260 0x7f4ac409dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41328/0 (socket says 172.21.15.47:41328) 2024-04-26T23:19:15.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.905+0000 7f4ac2ffd640 1 -- 172.21.15.47:0/3399135060 learned_addr learned my addr 172.21.15.47:0/3399135060 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:15.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.905+0000 7f4ac3fff640 1 --2- 172.21.15.47:0/3399135060 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ac40cfde0 0x7f4ac415cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:15.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.905+0000 7f4ac37fe640 1 --2- 172.21.15.47:0/3399135060 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ac4070350 0x7f4ac4161160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:15.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.905+0000 7f4ac2ffd640 1 -- 172.21.15.47:0/3399135060 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ac4070350 msgr2=0x7f4ac4161160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:15.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.905+0000 7f4ac2ffd640 1 --2- 172.21.15.47:0/3399135060 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ac4070350 0x7f4ac4161160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:15.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.905+0000 7f4ac2ffd640 1 -- 172.21.15.47:0/3399135060 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ac40cfde0 msgr2=0x7f4ac415cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:15.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.905+0000 7f4ac2ffd640 1 --2- 172.21.15.47:0/3399135060 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ac40cfde0 0x7f4ac415cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:15.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.905+0000 7f4ac2ffd640 1 -- 172.21.15.47:0/3399135060 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ac415ac90 con 0x7f4ac4071260 2024-04-26T23:19:15.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.905+0000 7f4ac37fe640 1 --2- 172.21.15.47:0/3399135060 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ac4070350 0x7f4ac4161160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:19:15.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.905+0000 7f4ac2ffd640 1 --2- 172.21.15.47:0/3399135060 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ac4071260 0x7f4ac409dfe0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4ab800c4d0 tx=0x7f4ab800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:15.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.905+0000 7f4aabfff640 1 -- 172.21.15.47:0/3399135060 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab8018070 con 0x7f4ac4071260 2024-04-26T23:19:15.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.905+0000 7f4aabfff640 1 -- 172.21.15.47:0/3399135060 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ab8007500 con 0x7f4ac4071260 2024-04-26T23:19:15.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.905+0000 7f4ac9daf640 1 -- 172.21.15.47:0/3399135060 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ac415af70 con 0x7f4ac4071260 2024-04-26T23:19:15.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.905+0000 7f4aabfff640 1 -- 172.21.15.47:0/3399135060 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab8004a20 con 0x7f4ac4071260 2024-04-26T23:19:15.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.906+0000 7f4ac9daf640 1 -- 172.21.15.47:0/3399135060 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ac415b530 con 0x7f4ac4071260 2024-04-26T23:19:15.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.906+0000 7f4ac9daf640 1 -- 172.21.15.47:0/3399135060 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ac40694f0 con 0x7f4ac4071260 2024-04-26T23:19:15.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.907+0000 7f4aabfff640 1 -- 172.21.15.47:0/3399135060 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4ab801b070 con 0x7f4ac4071260 2024-04-26T23:19:15.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.907+0000 7f4aabfff640 1 --2- 172.21.15.47:0/3399135060 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4ab006c8f0 0x7f4ab006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:15.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.908+0000 7f4aabfff640 1 -- 172.21.15.47:0/3399135060 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f4ab808b2b0 con 0x7f4ac4071260 2024-04-26T23:19:15.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.908+0000 7f4ac37fe640 1 --2- 172.21.15.47:0/3399135060 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4ab006c8f0 0x7f4ab006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:15.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.908+0000 7f4ac37fe640 1 --2- 172.21.15.47:0/3399135060 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4ab006c8f0 0x7f4ab006edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4034000 tx=0x7f4ab402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:15.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:15.912+0000 7f4aabfff640 1 -- 172.21.15.47:0/3399135060 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4ab80592c0 con 0x7f4ac4071260 2024-04-26T23:19:16.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.100+0000 7f4ac9daf640 1 -- 172.21.15.47:0/3399135060 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f4ac4070d60 con 0x7f4ac4071260 2024-04-26T23:19:16.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.101+0000 7f4aabfff640 1 -- 172.21.15.47:0/3399135060 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f4ab8058bc0 con 0x7f4ac4071260 2024-04-26T23:19:16.103 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-26T23:19:16.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.103+0000 7f4ac9daf640 1 -- 172.21.15.47:0/3399135060 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4ab006c8f0 msgr2=0x7f4ab006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:16.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.103+0000 7f4ac9daf640 1 --2- 172.21.15.47:0/3399135060 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4ab006c8f0 0x7f4ab006edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4034000 tx=0x7f4ab402e000 comp rx=0 tx=0).stop 2024-04-26T23:19:16.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.103+0000 7f4ac9daf640 1 -- 172.21.15.47:0/3399135060 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ac4071260 msgr2=0x7f4ac409dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:16.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.103+0000 7f4ac9daf640 1 --2- 172.21.15.47:0/3399135060 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ac4071260 0x7f4ac409dfe0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4ab800c4d0 tx=0x7f4ab800c990 comp rx=0 tx=0).stop 2024-04-26T23:19:16.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.103+0000 7f4ac9daf640 1 -- 172.21.15.47:0/3399135060 shutdown_connections 2024-04-26T23:19:16.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.103+0000 7f4ac9daf640 1 --2- 172.21.15.47:0/3399135060 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4ab006c8f0 0x7f4ab006edb0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:16.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.103+0000 7f4ac9daf640 1 --2- 172.21.15.47:0/3399135060 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ac40cfde0 0x7f4ac415cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:16.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.103+0000 7f4ac9daf640 1 --2- 172.21.15.47:0/3399135060 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ac4071260 0x7f4ac409dfe0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:16.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.103+0000 7f4ac9daf640 1 --2- 172.21.15.47:0/3399135060 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ac4070350 0x7f4ac4161160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:16.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.103+0000 7f4ac9daf640 1 -- 172.21.15.47:0/3399135060 >> 172.21.15.47:0/3399135060 conn(0x7f4ac40d2d80 msgr2=0x7f4ac40b4040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:16.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.103+0000 7f4ac9daf640 1 -- 172.21.15.47:0/3399135060 shutdown_connections 2024-04-26T23:19:16.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.103+0000 7f4ac9daf640 1 -- 172.21.15.47:0/3399135060 wait complete. 2024-04-26T23:19:16.116 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:19:16.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.215+0000 7f22372d1640 1 Processor -- start 2024-04-26T23:19:16.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.215+0000 7f22372d1640 1 -- start start 2024-04-26T23:19:16.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.216+0000 7f22372d1640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2230162810 0x7f2230070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:16.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.216+0000 7f22372d1640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f22300b2340 0x7f22300cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:16.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.216+0000 7f22372d1640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22300d10d0 con 0x7f22300cd930 2024-04-26T23:19:16.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.216+0000 7f22372d1640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f22300d0750 con 0x7f2230162810 2024-04-26T23:19:16.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.216+0000 7f22372d1640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f22300d16a0 con 0x7f22300b2340 2024-04-26T23:19:16.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.216+0000 7f223506d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2230162810 0x7f2230070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:16.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.216+0000 7f223506d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2230162810 0x7f2230070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41334/0 (socket says 172.21.15.47:41334) 2024-04-26T23:19:16.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.216+0000 7f223586e640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f22300cd930 0x7f22300d1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33018/0 (socket says 172.21.15.47:33018) 2024-04-26T23:19:16.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.216+0000 7f223506d640 1 -- 172.21.15.47:0/3591687381 learned_addr learned my addr 172.21.15.47:0/3591687381 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:16.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.216+0000 7f223486c640 1 --2- 172.21.15.47:0/3591687381 >> v2:172.21.15.203:3300/0 conn(0x7f22300b2340 0x7f22300cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:16.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.216+0000 7f223506d640 1 -- 172.21.15.47:0/3591687381 >> v2:172.21.15.203:3300/0 conn(0x7f22300b2340 msgr2=0x7f22300cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:16.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.216+0000 7f223506d640 1 --2- 172.21.15.47:0/3591687381 >> v2:172.21.15.203:3300/0 conn(0x7f22300b2340 0x7f22300cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:16.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.216+0000 7f223506d640 1 -- 172.21.15.47:0/3591687381 >> v1:172.21.15.47:6789/0 conn(0x7f22300cd930 legacy=0x7f22300d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:16.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.216+0000 7f223506d640 1 -- 172.21.15.47:0/3591687381 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22300d1370 con 0x7f2230162810 2024-04-26T23:19:16.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.217+0000 7f223506d640 1 --2- 172.21.15.47:0/3591687381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2230162810 0x7f2230070350 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f22300cd0a0 tx=0x7f222002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25c83ecf6406bee3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:16.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.217+0000 7f222ffff640 1 -- 172.21.15.47:0/3591687381 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2220036070 con 0x7f2230162810 2024-04-26T23:19:16.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.217+0000 7f222ffff640 1 -- 172.21.15.47:0/3591687381 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2220031600 con 0x7f2230162810 2024-04-26T23:19:16.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.217+0000 7f222ffff640 1 -- 172.21.15.47:0/3591687381 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22200319d0 con 0x7f2230162810 2024-04-26T23:19:16.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.218+0000 7f22372d1640 1 -- 172.21.15.47:0/3591687381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2230162810 msgr2=0x7f2230070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:16.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.218+0000 7f22372d1640 1 --2- 172.21.15.47:0/3591687381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2230162810 0x7f2230070350 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f22300cd0a0 tx=0x7f222002a730 comp rx=0 tx=0).stop 2024-04-26T23:19:16.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.218+0000 7f22372d1640 1 -- 172.21.15.47:0/3591687381 shutdown_connections 2024-04-26T23:19:16.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.218+0000 7f22372d1640 1 --2- 172.21.15.47:0/3591687381 >> v2:172.21.15.203:3300/0 conn(0x7f22300b2340 0x7f22300cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:16.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.218+0000 7f22372d1640 1 --2- 172.21.15.47:0/3591687381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2230162810 0x7f2230070350 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:16.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.218+0000 7f22372d1640 1 -- 172.21.15.47:0/3591687381 >> 172.21.15.47:0/3591687381 conn(0x7f22300d2da0 msgr2=0x7f22300d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:16.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.218+0000 7f22372d1640 1 -- 172.21.15.47:0/3591687381 shutdown_connections 2024-04-26T23:19:16.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.218+0000 7f22372d1640 1 -- 172.21.15.47:0/3591687381 wait complete. 2024-04-26T23:19:16.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.219+0000 7f22372d1640 1 Processor -- start 2024-04-26T23:19:16.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.219+0000 7f22372d1640 1 -- start start 2024-04-26T23:19:16.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.220+0000 7f22372d1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22300b2340 0x7f22301f8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:16.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.220+0000 7f22372d1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22300cd930 0x7f22301f2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:16.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.220+0000 7f22372d1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2230162810 0x7f22301f4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:16.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.220+0000 7f22372d1640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2230070890 con 0x7f22300b2340 2024-04-26T23:19:16.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.220+0000 7f22372d1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f22300d0470 con 0x7f22300cd930 2024-04-26T23:19:16.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.220+0000 7f22372d1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f22300d2240 con 0x7f2230162810 2024-04-26T23:19:16.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.220+0000 7f223506d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22300b2340 0x7f22301f8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:16.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.220+0000 7f223506d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22300b2340 0x7f22301f8d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41340/0 (socket says 172.21.15.47:41340) 2024-04-26T23:19:16.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.220+0000 7f223506d640 1 -- 172.21.15.47:0/3601493710 learned_addr learned my addr 172.21.15.47:0/3601493710 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:16.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.220+0000 7f223486c640 1 --2- 172.21.15.47:0/3601493710 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22300cd930 0x7f22301f2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:16.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.220+0000 7f223586e640 1 --2- 172.21.15.47:0/3601493710 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2230162810 0x7f22301f4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:16.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.220+0000 7f223506d640 1 -- 172.21.15.47:0/3601493710 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2230162810 msgr2=0x7f22301f4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:16.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.220+0000 7f223506d640 1 --2- 172.21.15.47:0/3601493710 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2230162810 0x7f22301f4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:16.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.220+0000 7f223506d640 1 -- 172.21.15.47:0/3601493710 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22300cd930 msgr2=0x7f22301f2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:16.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.220+0000 7f223506d640 1 --2- 172.21.15.47:0/3601493710 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22300cd930 0x7f22301f2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:16.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.220+0000 7f223506d640 1 -- 172.21.15.47:0/3601493710 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22301f54e0 con 0x7f22300b2340 2024-04-26T23:19:16.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.220+0000 7f223586e640 1 --2- 172.21.15.47:0/3601493710 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2230162810 0x7f22301f4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:19:16.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.220+0000 7f223506d640 1 --2- 172.21.15.47:0/3601493710 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22300b2340 0x7f22301f8d70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2220034040 tx=0x7f2220004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:16.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.220+0000 7f222dffb640 1 -- 172.21.15.47:0/3601493710 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f222003f020 con 0x7f22300b2340 2024-04-26T23:19:16.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.220+0000 7f222dffb640 1 -- 172.21.15.47:0/3601493710 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2220037040 con 0x7f22300b2340 2024-04-26T23:19:16.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.220+0000 7f22372d1640 1 -- 172.21.15.47:0/3601493710 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22301f57c0 con 0x7f22300b2340 2024-04-26T23:19:16.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.220+0000 7f222dffb640 1 -- 172.21.15.47:0/3601493710 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2220036070 con 0x7f22300b2340 2024-04-26T23:19:16.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.221+0000 7f22372d1640 1 -- 172.21.15.47:0/3601493710 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f223014c4d0 con 0x7f22300b2340 2024-04-26T23:19:16.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.222+0000 7f22372d1640 1 -- 172.21.15.47:0/3601493710 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22300694f0 con 0x7f22300b2340 2024-04-26T23:19:16.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.222+0000 7f222dffb640 1 -- 172.21.15.47:0/3601493710 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2220042020 con 0x7f22300b2340 2024-04-26T23:19:16.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.223+0000 7f222dffb640 1 --2- 172.21.15.47:0/3601493710 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f221c06c8f0 0x7f221c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:16.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.223+0000 7f222dffb640 1 -- 172.21.15.47:0/3601493710 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f22200ac480 con 0x7f22300b2340 2024-04-26T23:19:16.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.223+0000 7f223486c640 1 --2- 172.21.15.47:0/3601493710 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f221c06c8f0 0x7f221c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:16.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.223+0000 7f223486c640 1 --2- 172.21.15.47:0/3601493710 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f221c06c8f0 0x7f221c06edb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f22300d26b0 tx=0x7f2224008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:16.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.227+0000 7f222dffb640 1 -- 172.21.15.47:0/3601493710 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2220049030 con 0x7f22300b2340 2024-04-26T23:19:16.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.413+0000 7f22372d1640 1 -- 172.21.15.47:0/3601493710 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f22300d1eb0 con 0x7f22300b2340 2024-04-26T23:19:16.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.414+0000 7f222dffb640 1 -- 172.21.15.47:0/3601493710 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18857 (secure 0 0 0) 0x7f2220007c70 con 0x7f22300b2340 2024-04-26T23:19:16.416 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:19:16.416 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":26,"fsid":"399006d3-4db1-43e6-ba81-a79a276eabfd","created":"2024-04-26T23:18:06.731211+0000","modified":"2024-04-26T23:19:12.124051+0000","last_up_change":"2024-04-26T23:18:17.808736+0000","last_in_change":"2024-04-26T23:18:08.945859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:18:16.937492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:18:29.033575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:18:30.318499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:18:34.014536+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ee091567-fea0-4a14-bf4e-0e16c20bb2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6819","nonce":326520779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6821","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6823","nonce":326520779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6829","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6831","nonce":326520779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6825","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6827","nonce":326520779}]},"public_addr":"172.21.15.47:6819/326520779","cluster_addr":"172.21.15.47:6823/326520779","heartbeat_back_addr":"172.21.15.47:6831/326520779","heartbeat_front_addr":"172.21.15.47:6827/326520779","state":["exists","up"]},{"osd":1,"uuid":"c14440a0-bba5-4931-a6db-b736c08e869b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6809","nonce":1912610931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6811","nonce":1912610931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6815","nonce":1912610931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6813","nonce":1912610931}]},"public_addr":"172.21.15.47:6809/1912610931","cluster_addr":"172.21.15.47:6811/1912610931","heartbeat_back_addr":"172.21.15.47:6815/1912610931","heartbeat_front_addr":"172.21.15.47:6813/1912610931","state":["exists","up"]},{"osd":2,"uuid":"3940f8de-c9b2-4cd3-a911-9c301729eb30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6801","nonce":2569869305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6803","nonce":2569869305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6807","nonce":2569869305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6805","nonce":2569869305}]},"public_addr":"172.21.15.47:6801/2569869305","cluster_addr":"172.21.15.47:6803/2569869305","heartbeat_back_addr":"172.21.15.47:6807/2569869305","heartbeat_front_addr":"172.21.15.47:6805/2569869305","state":["exists","up"]},{"osd":3,"uuid":"1286a67e-8958-426a-8c2e-bc85498c619c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6818","nonce":2791411683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6822","nonce":2791411683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6830","nonce":2791411683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6826","nonce":2791411683}]},"public_addr":"172.21.15.47:6818/2791411683","cluster_addr":"172.21.15.47:6822/2791411683","heartbeat_back_addr":"172.21.15.47:6830/2791411683","heartbeat_front_addr":"172.21.15.47:6826/2791411683","state":["exists","up"]},{"osd":4,"uuid":"d8ee2c20-e5b7-46c2-9999-b234070126cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6809","nonce":441313104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6811","nonce":441313104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6815","nonce":441313104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6813","nonce":441313104}]},"public_addr":"172.21.15.203:6809/441313104","cluster_addr":"172.21.15.203:6811/441313104","heartbeat_back_addr":"172.21.15.203:6815/441313104","heartbeat_front_addr":"172.21.15.203:6813/441313104","state":["exists","up"]},{"osd":5,"uuid":"b453a8f0-f324-4840-9ae0-d67bdadb9f8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6801","nonce":56838444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6803","nonce":56838444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6807","nonce":56838444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6805","nonce":56838444}]},"public_addr":"172.21.15.203:6801/56838444","cluster_addr":"172.21.15.203:6803/56838444","heartbeat_back_addr":"172.21.15.203:6807/56838444","heartbeat_front_addr":"172.21.15.203:6805/56838444","state":["exists","up"]},{"osd":6,"uuid":"a6d31951-2d4b-4e8d-a510-6283b01dc158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6825","nonce":2738517080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6827","nonce":2738517080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6831","nonce":2738517080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6829","nonce":2738517080}]},"public_addr":"172.21.15.203:6825/2738517080","cluster_addr":"172.21.15.203:6827/2738517080","heartbeat_back_addr":"172.21.15.203:6831/2738517080","heartbeat_front_addr":"172.21.15.203:6829/2738517080","state":["exists","up"]},{"osd":7,"uuid":"7119b8c0-fe96-402d-9e75-7b509d9d44d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6817","nonce":1315565058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6819","nonce":1315565058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6823","nonce":1315565058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6821","nonce":1315565058}]},"public_addr":"172.21.15.203:6817/1315565058","cluster_addr":"172.21.15.203:6819/1315565058","heartbeat_back_addr":"172.21.15.203:6823/1315565058","heartbeat_front_addr":"172.21.15.203:6821/1315565058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.121921+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.137494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.084814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.111593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.072430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.125066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.097867+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.057061+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":5,"to":1}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:19:16.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.416+0000 7f22372d1640 1 -- 172.21.15.47:0/3601493710 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f221c06c8f0 msgr2=0x7f221c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:16.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.416+0000 7f22372d1640 1 --2- 172.21.15.47:0/3601493710 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f221c06c8f0 0x7f221c06edb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f22300d26b0 tx=0x7f2224008040 comp rx=0 tx=0).stop 2024-04-26T23:19:16.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.416+0000 7f22372d1640 1 -- 172.21.15.47:0/3601493710 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22300b2340 msgr2=0x7f22301f8d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:16.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.416+0000 7f22372d1640 1 --2- 172.21.15.47:0/3601493710 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22300b2340 0x7f22301f8d70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2220034040 tx=0x7f2220004a40 comp rx=0 tx=0).stop 2024-04-26T23:19:16.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.416+0000 7f22372d1640 1 -- 172.21.15.47:0/3601493710 shutdown_connections 2024-04-26T23:19:16.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.416+0000 7f22372d1640 1 --2- 172.21.15.47:0/3601493710 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f221c06c8f0 0x7f221c06edb0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:16.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.416+0000 7f22372d1640 1 --2- 172.21.15.47:0/3601493710 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2230162810 0x7f22301f4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:16.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.416+0000 7f22372d1640 1 --2- 172.21.15.47:0/3601493710 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22300cd930 0x7f22301f2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:16.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.416+0000 7f22372d1640 1 --2- 172.21.15.47:0/3601493710 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22300b2340 0x7f22301f8d70 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:16.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.416+0000 7f22372d1640 1 -- 172.21.15.47:0/3601493710 >> 172.21.15.47:0/3601493710 conn(0x7f22300d2da0 msgr2=0x7f2230070b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:16.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.416+0000 7f22372d1640 1 -- 172.21.15.47:0/3601493710 shutdown_connections 2024-04-26T23:19:16.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.416+0000 7f22372d1640 1 -- 172.21.15.47:0/3601493710 wait complete. 2024-04-26T23:19:16.429 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:18:16.937492+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:18:29.033575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:18:30.318499+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T23:18:34.014536+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T23:19:16.430 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:19:16.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.532+0000 7f9732de4640 1 Processor -- start 2024-04-26T23:19:16.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.532+0000 7f9732de4640 1 -- start start 2024-04-26T23:19:16.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.532+0000 7f9732de4640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f972c1628f0 0x7f972c0d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:16.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.532+0000 7f9732de4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f972c0d1550 0x7f972c0d1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:16.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.532+0000 7f9732de4640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f972c0d0890 con 0x7f972c162cd0 2024-04-26T23:19:16.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.532+0000 7f9732de4640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f972c0d1130 con 0x7f972c1628f0 2024-04-26T23:19:16.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.532+0000 7f9732de4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f972c0d0b70 con 0x7f972c0d1550 2024-04-26T23:19:16.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.532+0000 7f9730b80640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f972c1628f0 0x7f972c0d1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:16.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.532+0000 7f9730b80640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f972c1628f0 0x7f972c0d1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41354/0 (socket says 172.21.15.47:41354) 2024-04-26T23:19:16.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.532+0000 7f9730b80640 1 -- 172.21.15.47:0/495038446 learned_addr learned my addr 172.21.15.47:0/495038446 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:16.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.533+0000 7f9731381640 1 --2- 172.21.15.47:0/495038446 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f972c0d1550 0x7f972c0d1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:16.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.533+0000 7f9730b80640 1 -- 172.21.15.47:0/495038446 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f972c0d1550 msgr2=0x7f972c0d1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:16.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.533+0000 7f9730b80640 1 --2- 172.21.15.47:0/495038446 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f972c0d1550 0x7f972c0d1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:16.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.533+0000 7f9730b80640 1 -- 172.21.15.47:0/495038446 >> v1:172.21.15.203:6789/0 conn(0x7f972c162cd0 legacy=0x7f972c0d2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:16.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.533+0000 7f9730b80640 1 -- 172.21.15.47:0/495038446 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f972c0d0e50 con 0x7f972c1628f0 2024-04-26T23:19:16.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.533+0000 7f9730b80640 1 --2- 172.21.15.47:0/495038446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f972c1628f0 0x7f972c0d1a00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9728009870 tx=0x7f972802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95ea51a3bd84eaba server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:16.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.534+0000 7f97237fe640 1 -- 172.21.15.47:0/495038446 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9728037070 con 0x7f972c1628f0 2024-04-26T23:19:16.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.534+0000 7f97237fe640 1 -- 172.21.15.47:0/495038446 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9728032440 con 0x7f972c1628f0 2024-04-26T23:19:16.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.534+0000 7f97237fe640 1 -- 172.21.15.47:0/495038446 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9728032810 con 0x7f972c1628f0 2024-04-26T23:19:16.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.534+0000 7f9732de4640 1 -- 172.21.15.47:0/495038446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f972c1628f0 msgr2=0x7f972c0d1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:16.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.534+0000 7f9732de4640 1 --2- 172.21.15.47:0/495038446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f972c1628f0 0x7f972c0d1a00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9728009870 tx=0x7f972802a180 comp rx=0 tx=0).stop 2024-04-26T23:19:16.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.534+0000 7f9732de4640 1 -- 172.21.15.47:0/495038446 shutdown_connections 2024-04-26T23:19:16.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.534+0000 7f9732de4640 1 --2- 172.21.15.47:0/495038446 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f972c0d1550 0x7f972c0d1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:16.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.534+0000 7f9732de4640 1 --2- 172.21.15.47:0/495038446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f972c1628f0 0x7f972c0d1a00 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:16.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.534+0000 7f9732de4640 1 -- 172.21.15.47:0/495038446 >> 172.21.15.47:0/495038446 conn(0x7f972c0d4ff0 msgr2=0x7f972c0d5400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:16.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.534+0000 7f9732de4640 1 -- 172.21.15.47:0/495038446 shutdown_connections 2024-04-26T23:19:16.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.534+0000 7f9732de4640 1 -- 172.21.15.47:0/495038446 wait complete. 2024-04-26T23:19:16.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.535+0000 7f9732de4640 1 Processor -- start 2024-04-26T23:19:16.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.535+0000 7f9732de4640 1 -- start start 2024-04-26T23:19:16.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.536+0000 7f9732de4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f972c0d1550 0x7f972c14efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:16.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.536+0000 7f9732de4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f972c1628f0 0x7f972c1f8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:16.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.536+0000 7f9732de4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f972c162cd0 0x7f972c1f8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:16.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.536+0000 7f9732de4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f972c0b5df0 con 0x7f972c0d1550 2024-04-26T23:19:16.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.536+0000 7f9732de4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f972c0b5c70 con 0x7f972c1628f0 2024-04-26T23:19:16.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.536+0000 7f9732de4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f972c0b5f70 con 0x7f972c162cd0 2024-04-26T23:19:16.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.536+0000 7f9730b80640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f972c0d1550 0x7f972c14efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:16.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.536+0000 7f9730b80640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f972c0d1550 0x7f972c14efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41364/0 (socket says 172.21.15.47:41364) 2024-04-26T23:19:16.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.536+0000 7f9730b80640 1 -- 172.21.15.47:0/4198602460 learned_addr learned my addr 172.21.15.47:0/4198602460 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:16.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.537+0000 7f9731381640 1 --2- 172.21.15.47:0/4198602460 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f972c162cd0 0x7f972c1f8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:16.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.537+0000 7f9723fff640 1 --2- 172.21.15.47:0/4198602460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f972c1628f0 0x7f972c1f8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:16.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.537+0000 7f9730b80640 1 -- 172.21.15.47:0/4198602460 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f972c162cd0 msgr2=0x7f972c1f8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:16.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.537+0000 7f9730b80640 1 --2- 172.21.15.47:0/4198602460 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f972c162cd0 0x7f972c1f8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:16.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.537+0000 7f9730b80640 1 -- 172.21.15.47:0/4198602460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f972c1628f0 msgr2=0x7f972c1f8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:16.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.537+0000 7f9730b80640 1 --2- 172.21.15.47:0/4198602460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f972c1628f0 0x7f972c1f8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:16.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.537+0000 7f9730b80640 1 -- 172.21.15.47:0/4198602460 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f972c0baec0 con 0x7f972c0d1550 2024-04-26T23:19:16.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.537+0000 7f9730b80640 1 --2- 172.21.15.47:0/4198602460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f972c0d1550 0x7f972c14efe0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9728009870 tx=0x7f97280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:16.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.537+0000 7f97217fa640 1 -- 172.21.15.47:0/4198602460 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9728041020 con 0x7f972c0d1550 2024-04-26T23:19:16.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.537+0000 7f97217fa640 1 -- 172.21.15.47:0/4198602460 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9728030970 con 0x7f972c0d1550 2024-04-26T23:19:16.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.537+0000 7f9732de4640 1 -- 172.21.15.47:0/4198602460 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f972c14f790 con 0x7f972c0d1550 2024-04-26T23:19:16.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.537+0000 7f97217fa640 1 -- 172.21.15.47:0/4198602460 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9728037040 con 0x7f972c0d1550 2024-04-26T23:19:16.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.537+0000 7f9732de4640 1 -- 172.21.15.47:0/4198602460 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f972c0babe0 con 0x7f972c0d1550 2024-04-26T23:19:16.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.538+0000 7f9732de4640 1 -- 172.21.15.47:0/4198602460 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f972c0694f0 con 0x7f972c0d1550 2024-04-26T23:19:16.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.539+0000 7f97217fa640 1 -- 172.21.15.47:0/4198602460 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9728044020 con 0x7f972c0d1550 2024-04-26T23:19:16.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.539+0000 7f97217fa640 1 --2- 172.21.15.47:0/4198602460 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f971806c8f0 0x7f971806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:16.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.539+0000 7f97217fa640 1 -- 172.21.15.47:0/4198602460 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f97280ac8d0 con 0x7f972c0d1550 2024-04-26T23:19:16.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.539+0000 7f9723fff640 1 --2- 172.21.15.47:0/4198602460 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f971806c8f0 0x7f971806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:16.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.540+0000 7f9723fff640 1 --2- 172.21.15.47:0/4198602460 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f971806c8f0 0x7f971806edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f972c0c8dc0 tx=0x7f971c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:16.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.544+0000 7f97217fa640 1 -- 172.21.15.47:0/4198602460 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f972804b030 con 0x7f972c0d1550 2024-04-26T23:19:16.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.730+0000 7f9732de4640 1 -- 172.21.15.47:0/4198602460 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f972c0d0250 con 0x7f972c0d1550 2024-04-26T23:19:16.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.730+0000 7f97217fa640 1 -- 172.21.15.47:0/4198602460 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f972807a970 con 0x7f972c0d1550 2024-04-26T23:19:16.732 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-26T23:19:16.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.732+0000 7f9732de4640 1 -- 172.21.15.47:0/4198602460 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f971806c8f0 msgr2=0x7f971806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:16.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.732+0000 7f9732de4640 1 --2- 172.21.15.47:0/4198602460 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f971806c8f0 0x7f971806edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f972c0c8dc0 tx=0x7f971c009040 comp rx=0 tx=0).stop 2024-04-26T23:19:16.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.732+0000 7f9732de4640 1 -- 172.21.15.47:0/4198602460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f972c0d1550 msgr2=0x7f972c14efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:16.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.732+0000 7f9732de4640 1 --2- 172.21.15.47:0/4198602460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f972c0d1550 0x7f972c14efe0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9728009870 tx=0x7f97280046c0 comp rx=0 tx=0).stop 2024-04-26T23:19:16.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.732+0000 7f9732de4640 1 -- 172.21.15.47:0/4198602460 shutdown_connections 2024-04-26T23:19:16.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.732+0000 7f9732de4640 1 --2- 172.21.15.47:0/4198602460 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f971806c8f0 0x7f971806edb0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:16.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.732+0000 7f9732de4640 1 --2- 172.21.15.47:0/4198602460 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f972c162cd0 0x7f972c1f8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:16.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.732+0000 7f9732de4640 1 --2- 172.21.15.47:0/4198602460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f972c1628f0 0x7f972c1f8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:16.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.732+0000 7f9732de4640 1 --2- 172.21.15.47:0/4198602460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f972c0d1550 0x7f972c14efe0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:16.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.732+0000 7f9732de4640 1 -- 172.21.15.47:0/4198602460 >> 172.21.15.47:0/4198602460 conn(0x7f972c0d4ff0 msgr2=0x7f972c0d53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:16.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.732+0000 7f9732de4640 1 -- 172.21.15.47:0/4198602460 shutdown_connections 2024-04-26T23:19:16.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.732+0000 7f9732de4640 1 -- 172.21.15.47:0/4198602460 wait complete. 2024-04-26T23:19:16.745 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:19:16.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.847+0000 7f692a719640 1 Processor -- start 2024-04-26T23:19:16.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.847+0000 7f692a719640 1 -- start start 2024-04-26T23:19:16.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.848+0000 7f692a719640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f692415f170 0x7f69240d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:16.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.848+0000 7f692a719640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f692415f550 0x7f69240d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:16.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.848+0000 7f692a719640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f69240d34b0 0x7f69240d3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:16.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.848+0000 7f692a719640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f69240d3e60 con 0x7f692415f550 2024-04-26T23:19:16.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.848+0000 7f692a719640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f69240c9430 con 0x7f692415f170 2024-04-26T23:19:16.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.848+0000 7f692a719640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f69240d1040 con 0x7f69240d34b0 2024-04-26T23:19:16.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.848+0000 7f69237fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f692415f550 0x7f69240d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:16.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.848+0000 7f69237fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f692415f550 0x7f69240d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41372/0 (socket says 172.21.15.47:41372) 2024-04-26T23:19:16.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.848+0000 7f69237fe640 1 -- 172.21.15.47:0/1837840870 learned_addr learned my addr 172.21.15.47:0/1837840870 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:16.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.848+0000 7f6923fff640 1 --2- 172.21.15.47:0/1837840870 >> v2:172.21.15.203:3300/0 conn(0x7f692415f170 0x7f69240d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:16.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.848+0000 7f6928cb6640 1 --2- 172.21.15.47:0/1837840870 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f69240d34b0 0x7f69240d3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:16.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.849+0000 7f69237fe640 1 -- 172.21.15.47:0/1837840870 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f69240d34b0 msgr2=0x7f69240d3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:16.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.849+0000 7f69237fe640 1 --2- 172.21.15.47:0/1837840870 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f69240d34b0 0x7f69240d3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:16.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.849+0000 7f69237fe640 1 -- 172.21.15.47:0/1837840870 >> v2:172.21.15.203:3300/0 conn(0x7f692415f170 msgr2=0x7f69240d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:16.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.849+0000 7f69237fe640 1 --2- 172.21.15.47:0/1837840870 >> v2:172.21.15.203:3300/0 conn(0x7f692415f170 0x7f69240d5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:16.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.849+0000 7f69237fe640 1 -- 172.21.15.47:0/1837840870 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69240c9680 con 0x7f692415f550 2024-04-26T23:19:16.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.849+0000 7f69237fe640 1 --2- 172.21.15.47:0/1837840870 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f692415f550 0x7f69240d2cb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6914009870 tx=0x7f691402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=656bf1bb8de882fa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:16.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.849+0000 7f6922ffd640 1 -- 172.21.15.47:0/1837840870 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6914037070 con 0x7f692415f550 2024-04-26T23:19:16.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.849+0000 7f6922ffd640 1 -- 172.21.15.47:0/1837840870 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6914032440 con 0x7f692415f550 2024-04-26T23:19:16.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.849+0000 7f6922ffd640 1 -- 172.21.15.47:0/1837840870 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6914032810 con 0x7f692415f550 2024-04-26T23:19:16.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.850+0000 7f692a719640 1 -- 172.21.15.47:0/1837840870 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f692415f550 msgr2=0x7f69240d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:16.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.850+0000 7f692a719640 1 --2- 172.21.15.47:0/1837840870 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f692415f550 0x7f69240d2cb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6914009870 tx=0x7f691402a210 comp rx=0 tx=0).stop 2024-04-26T23:19:16.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.850+0000 7f692a719640 1 -- 172.21.15.47:0/1837840870 shutdown_connections 2024-04-26T23:19:16.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.850+0000 7f692a719640 1 --2- 172.21.15.47:0/1837840870 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f692415f550 0x7f69240d2cb0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:16.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.850+0000 7f692a719640 1 --2- 172.21.15.47:0/1837840870 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f69240d34b0 0x7f69240d3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:16.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.850+0000 7f692a719640 1 --2- 172.21.15.47:0/1837840870 >> v2:172.21.15.203:3300/0 conn(0x7f692415f170 0x7f69240d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:16.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.850+0000 7f692a719640 1 -- 172.21.15.47:0/1837840870 >> 172.21.15.47:0/1837840870 conn(0x7f69240d4f60 msgr2=0x7f69240d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:16.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.850+0000 7f692a719640 1 -- 172.21.15.47:0/1837840870 shutdown_connections 2024-04-26T23:19:16.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.850+0000 7f692a719640 1 -- 172.21.15.47:0/1837840870 wait complete. 2024-04-26T23:19:16.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.851+0000 7f692a719640 1 Processor -- start 2024-04-26T23:19:16.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.851+0000 7f692a719640 1 -- start start 2024-04-26T23:19:16.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.852+0000 7f692a719640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69240d34b0 0x7f69240ce320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:16.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.852+0000 7f692a719640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f692415f170 0x7f69240cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:16.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.852+0000 7f692a719640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f692415bf10 0x7f69240ba0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:16.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.852+0000 7f692a719640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f69240cfc50 con 0x7f692415bf10 2024-04-26T23:19:16.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.852+0000 7f692a719640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f69240c9c40 con 0x7f69240d34b0 2024-04-26T23:19:16.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.852+0000 7f692a719640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f69240cfdd0 con 0x7f692415f170 2024-04-26T23:19:16.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.852+0000 7f6928cb6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f692415bf10 0x7f69240ba0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:16.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.852+0000 7f6928cb6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f692415bf10 0x7f69240ba0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41388/0 (socket says 172.21.15.47:41388) 2024-04-26T23:19:16.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.852+0000 7f6928cb6640 1 -- 172.21.15.47:0/3119722961 learned_addr learned my addr 172.21.15.47:0/3119722961 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:16.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.853+0000 7f6923fff640 1 --2- 172.21.15.47:0/3119722961 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69240d34b0 0x7f69240ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:16.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.853+0000 7f69237fe640 1 --2- 172.21.15.47:0/3119722961 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f692415f170 0x7f69240cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:16.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.853+0000 7f6928cb6640 1 -- 172.21.15.47:0/3119722961 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f692415f170 msgr2=0x7f69240cbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:16.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.853+0000 7f6928cb6640 1 --2- 172.21.15.47:0/3119722961 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f692415f170 0x7f69240cbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:16.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.853+0000 7f6928cb6640 1 -- 172.21.15.47:0/3119722961 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69240d34b0 msgr2=0x7f69240ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:16.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.853+0000 7f6928cb6640 1 --2- 172.21.15.47:0/3119722961 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69240d34b0 0x7f69240ce320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:16.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.853+0000 7f6928cb6640 1 -- 172.21.15.47:0/3119722961 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69240ce860 con 0x7f692415bf10 2024-04-26T23:19:16.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.853+0000 7f6923fff640 1 --2- 172.21.15.47:0/3119722961 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69240d34b0 0x7f69240ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:19:16.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.853+0000 7f69237fe640 1 --2- 172.21.15.47:0/3119722961 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f692415f170 0x7f69240cbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:19:16.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.853+0000 7f6928cb6640 1 --2- 172.21.15.47:0/3119722961 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f692415bf10 0x7f69240ba0a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f691c00eed0 tx=0x7f691c00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:16.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.853+0000 7f6920ff9640 1 -- 172.21.15.47:0/3119722961 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691c018020 con 0x7f692415bf10 2024-04-26T23:19:16.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.853+0000 7f6920ff9640 1 -- 172.21.15.47:0/3119722961 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f691c014400 con 0x7f692415bf10 2024-04-26T23:19:16.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.853+0000 7f692a719640 1 -- 172.21.15.47:0/3119722961 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69240cc7d0 con 0x7f692415bf10 2024-04-26T23:19:16.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.853+0000 7f6920ff9640 1 -- 172.21.15.47:0/3119722961 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691c0147f0 con 0x7f692415bf10 2024-04-26T23:19:16.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.853+0000 7f692a719640 1 -- 172.21.15.47:0/3119722961 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69240ccd90 con 0x7f692415bf10 2024-04-26T23:19:16.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.854+0000 7f692a719640 1 -- 172.21.15.47:0/3119722961 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69240694f0 con 0x7f692415bf10 2024-04-26T23:19:16.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.855+0000 7f6920ff9640 1 -- 172.21.15.47:0/3119722961 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f691c005d10 con 0x7f692415bf10 2024-04-26T23:19:16.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.855+0000 7f6920ff9640 1 --2- 172.21.15.47:0/3119722961 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f690406c8f0 0x7f690406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:16.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.855+0000 7f6920ff9640 1 -- 172.21.15.47:0/3119722961 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f691c08c5e0 con 0x7f692415bf10 2024-04-26T23:19:16.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.855+0000 7f6923fff640 1 --2- 172.21.15.47:0/3119722961 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f690406c8f0 0x7f690406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:16.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.856+0000 7f6923fff640 1 --2- 172.21.15.47:0/3119722961 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f690406c8f0 0x7f690406edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f690c009ac0 tx=0x7f690c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:16.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:16.860+0000 7f6920ff9640 1 -- 172.21.15.47:0/3119722961 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f691c022070 con 0x7f692415bf10 2024-04-26T23:19:17.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.048+0000 7f692a719640 1 -- 172.21.15.47:0/3119722961 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f69240cb950 con 0x7f692415bf10 2024-04-26T23:19:17.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.048+0000 7f6920ff9640 1 -- 172.21.15.47:0/3119722961 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f691c05a680 con 0x7f692415bf10 2024-04-26T23:19:17.050 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-26T23:19:17.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.050+0000 7f692a719640 1 -- 172.21.15.47:0/3119722961 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f690406c8f0 msgr2=0x7f690406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:17.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.050+0000 7f692a719640 1 --2- 172.21.15.47:0/3119722961 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f690406c8f0 0x7f690406edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f690c009ac0 tx=0x7f690c008040 comp rx=0 tx=0).stop 2024-04-26T23:19:17.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.050+0000 7f692a719640 1 -- 172.21.15.47:0/3119722961 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f692415bf10 msgr2=0x7f69240ba0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:17.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.050+0000 7f692a719640 1 --2- 172.21.15.47:0/3119722961 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f692415bf10 0x7f69240ba0a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f691c00eed0 tx=0x7f691c00b480 comp rx=0 tx=0).stop 2024-04-26T23:19:17.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.050+0000 7f692a719640 1 -- 172.21.15.47:0/3119722961 shutdown_connections 2024-04-26T23:19:17.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.050+0000 7f692a719640 1 --2- 172.21.15.47:0/3119722961 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f690406c8f0 0x7f690406edb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:17.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.050+0000 7f692a719640 1 --2- 172.21.15.47:0/3119722961 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f692415bf10 0x7f69240ba0a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:17.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.050+0000 7f692a719640 1 --2- 172.21.15.47:0/3119722961 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f692415f170 0x7f69240cbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:17.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.050+0000 7f692a719640 1 --2- 172.21.15.47:0/3119722961 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69240d34b0 0x7f69240ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:17.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.050+0000 7f692a719640 1 -- 172.21.15.47:0/3119722961 >> 172.21.15.47:0/3119722961 conn(0x7f69240d4f60 msgr2=0x7f69240c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:17.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.050+0000 7f692a719640 1 -- 172.21.15.47:0/3119722961 shutdown_connections 2024-04-26T23:19:17.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.050+0000 7f692a719640 1 -- 172.21.15.47:0/3119722961 wait complete. 2024-04-26T23:19:17.063 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:19:17.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.167+0000 7f3e7b86b640 1 Processor -- start 2024-04-26T23:19:17.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.167+0000 7f3e7b86b640 1 -- start start 2024-04-26T23:19:17.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.168+0000 7f3e7b86b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e740b5b10 0x7f3e7416a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:17.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.168+0000 7f3e7b86b640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e740d0a10 con 0x7f3e740b40d0 2024-04-26T23:19:17.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.168+0000 7f3e7b86b640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e740d0cf0 con 0x7f3e740d1a00 2024-04-26T23:19:17.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.168+0000 7f3e7b86b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3e740d0fd0 con 0x7f3e740b5b10 2024-04-26T23:19:17.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.168+0000 7f3e78e06640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f3e740b40d0 0x7f3e741625f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33028/0 (socket says 172.21.15.47:33028) 2024-04-26T23:19:17.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.168+0000 7f3e78e06640 1 -- 172.21.15.47:0/563879426 learned_addr learned my addr 172.21.15.47:0/563879426 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:17.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.168+0000 7f3e79e08640 1 --2- 172.21.15.47:0/563879426 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e740b5b10 0x7f3e7416a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:17.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.168+0000 7f3e6bfff640 1 -- 172.21.15.47:0/563879426 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2323596111 0 0) 0x7f3e740d0a10 con 0x7f3e740b40d0 2024-04-26T23:19:17.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.168+0000 7f3e6bfff640 1 -- 172.21.15.47:0/563879426 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3e60003520 con 0x7f3e740b40d0 2024-04-26T23:19:17.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.169+0000 7f3e6bfff640 1 -- 172.21.15.47:0/563879426 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 758354484 0 0) 0x7f3e60003520 con 0x7f3e740b40d0 2024-04-26T23:19:17.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.169+0000 7f3e6bfff640 1 -- 172.21.15.47:0/563879426 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e740b5b10 msgr2=0x7f3e7416a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:17.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.169+0000 7f3e6bfff640 1 --2- 172.21.15.47:0/563879426 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e740b5b10 0x7f3e7416a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:17.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.169+0000 7f3e6bfff640 1 -- 172.21.15.47:0/563879426 >> v1:172.21.15.203:6789/0 conn(0x7f3e740d1a00 legacy=0x7f3e740d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:17.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.169+0000 7f3e6bfff640 1 -- 172.21.15.47:0/563879426 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e740d40a0 con 0x7f3e740b40d0 2024-04-26T23:19:17.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.169+0000 7f3e6bfff640 1 -- 172.21.15.47:0/563879426 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7f3e64001670 con 0x7f3e740b40d0 2024-04-26T23:19:17.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.169+0000 7f3e6bfff640 1 -- 172.21.15.47:0/563879426 >> v1:172.21.15.47:6789/0 conn(0x7f3e740b40d0 legacy=0x7f3e741625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:17.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.169+0000 7f3e6bfff640 1 --2- 172.21.15.47:0/563879426 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e740b5b10 0x7f3e600039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:17.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.169+0000 7f3e6bfff640 1 --2- 172.21.15.47:0/563879426 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e60003f60 0x7f3e60004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:17.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.169+0000 7f3e6bfff640 1 --2- 172.21.15.47:0/563879426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e60004d40 0x7f3e600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:17.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.169+0000 7f3e6bfff640 1 -- 172.21.15.47:0/563879426 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3e740d0fd0 con 0x7f3e60003f60 2024-04-26T23:19:17.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.169+0000 7f3e6bfff640 1 -- 172.21.15.47:0/563879426 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6000bb50 con 0x7f3e60004d40 2024-04-26T23:19:17.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.169+0000 7f3e6bfff640 1 -- 172.21.15.47:0/563879426 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3e6000bd50 con 0x7f3e740b5b10 2024-04-26T23:19:17.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.169+0000 7f3e78e06640 1 --2- 172.21.15.47:0/563879426 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e60003f60 0x7f3e60004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:17.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.169+0000 7f3e79607640 1 --2- 172.21.15.47:0/563879426 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e740b5b10 0x7f3e600039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:17.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.169+0000 7f3e79e08640 1 --2- 172.21.15.47:0/563879426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e60004d40 0x7f3e600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:17.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.169+0000 7f3e78e06640 1 -- 172.21.15.47:0/563879426 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e740b5b10 msgr2=0x7f3e600039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:17.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.169+0000 7f3e78e06640 1 --2- 172.21.15.47:0/563879426 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e740b5b10 0x7f3e600039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:17.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.169+0000 7f3e78e06640 1 -- 172.21.15.47:0/563879426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e60004d40 msgr2=0x7f3e600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:17.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.169+0000 7f3e78e06640 1 --2- 172.21.15.47:0/563879426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e60004d40 0x7f3e600051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:17.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.169+0000 7f3e78e06640 1 -- 172.21.15.47:0/563879426 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e60003520 con 0x7f3e60003f60 2024-04-26T23:19:17.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.169+0000 7f3e79607640 1 --2- 172.21.15.47:0/563879426 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e740b5b10 0x7f3e600039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:19:17.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.170+0000 7f3e78e06640 1 --2- 172.21.15.47:0/563879426 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e60003f60 0x7f3e60004800 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3e6400c210 tx=0x7f3e60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aca70a78e2a7b044 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:17.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.170+0000 7f3e6bfff640 1 -- 172.21.15.47:0/563879426 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e64007d30 con 0x7f3e60003f60 2024-04-26T23:19:17.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.170+0000 7f3e6bfff640 1 -- 172.21.15.47:0/563879426 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e64005ad0 con 0x7f3e60003f60 2024-04-26T23:19:17.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.170+0000 7f3e6bfff640 1 -- 172.21.15.47:0/563879426 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e64007070 con 0x7f3e60003f60 2024-04-26T23:19:17.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.170+0000 7f3e7b86b640 1 -- 172.21.15.47:0/563879426 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e60003f60 msgr2=0x7f3e60004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:17.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.170+0000 7f3e7b86b640 1 --2- 172.21.15.47:0/563879426 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e60003f60 0x7f3e60004800 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3e6400c210 tx=0x7f3e60005830 comp rx=0 tx=0).stop 2024-04-26T23:19:17.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.170+0000 7f3e79e08640 1 -- 172.21.15.47:0/563879426 reap_dead start 2024-04-26T23:19:17.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.170+0000 7f3e7b86b640 1 -- 172.21.15.47:0/563879426 shutdown_connections 2024-04-26T23:19:17.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.170+0000 7f3e7b86b640 1 -- 172.21.15.47:0/563879426 >> 172.21.15.47:0/563879426 conn(0x7f3e740d4fd0 msgr2=0x7f3e740d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:17.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.171+0000 7f3e7b86b640 1 -- 172.21.15.47:0/563879426 shutdown_connections 2024-04-26T23:19:17.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.171+0000 7f3e7b86b640 1 -- 172.21.15.47:0/563879426 wait complete. 2024-04-26T23:19:17.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.172+0000 7f3e7b86b640 1 Processor -- start 2024-04-26T23:19:17.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.172+0000 7f3e7b86b640 1 -- start start 2024-04-26T23:19:17.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.172+0000 7f3e7b86b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e60003f60 0x7f3e740cec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:17.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.173+0000 7f3e7b86b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e740cc0a0 0x7f3e7415e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:17.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.173+0000 7f3e7b86b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e740cc520 0x7f3e7415e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:17.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.173+0000 7f3e7b86b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3e74162e80 con 0x7f3e60003f60 2024-04-26T23:19:17.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.173+0000 7f3e7b86b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3e74162d00 con 0x7f3e740cc0a0 2024-04-26T23:19:17.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.173+0000 7f3e7b86b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3e74163000 con 0x7f3e740cc520 2024-04-26T23:19:17.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.173+0000 7f3e79607640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e60003f60 0x7f3e740cec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:17.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.173+0000 7f3e79607640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e60003f60 0x7f3e740cec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41414/0 (socket says 172.21.15.47:41414) 2024-04-26T23:19:17.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.173+0000 7f3e79607640 1 -- 172.21.15.47:0/1826799396 learned_addr learned my addr 172.21.15.47:0/1826799396 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:17.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.173+0000 7f3e79e08640 1 --2- 172.21.15.47:0/1826799396 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e740cc520 0x7f3e7415e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:17.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.173+0000 7f3e78e06640 1 --2- 172.21.15.47:0/1826799396 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e740cc0a0 0x7f3e7415e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:17.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.173+0000 7f3e79607640 1 -- 172.21.15.47:0/1826799396 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e740cc520 msgr2=0x7f3e7415e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:17.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.173+0000 7f3e79607640 1 --2- 172.21.15.47:0/1826799396 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e740cc520 0x7f3e7415e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:17.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.173+0000 7f3e79607640 1 -- 172.21.15.47:0/1826799396 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e740cc0a0 msgr2=0x7f3e7415e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:17.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.173+0000 7f3e79607640 1 --2- 172.21.15.47:0/1826799396 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e740cc0a0 0x7f3e7415e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:17.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.173+0000 7f3e79607640 1 -- 172.21.15.47:0/1826799396 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e740ccad0 con 0x7f3e60003f60 2024-04-26T23:19:17.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.173+0000 7f3e79e08640 1 --2- 172.21.15.47:0/1826799396 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e740cc520 0x7f3e7415e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:19:17.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.173+0000 7f3e79607640 1 --2- 172.21.15.47:0/1826799396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e60003f60 0x7f3e740cec80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3e7000e550 tx=0x7f3e7000ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:17.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.173+0000 7f3e69ffb640 1 -- 172.21.15.47:0/1826799396 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e700052e0 con 0x7f3e60003f60 2024-04-26T23:19:17.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.173+0000 7f3e69ffb640 1 -- 172.21.15.47:0/1826799396 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e70018070 con 0x7f3e60003f60 2024-04-26T23:19:17.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.173+0000 7f3e7b86b640 1 -- 172.21.15.47:0/1826799396 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e740ccdb0 con 0x7f3e60003f60 2024-04-26T23:19:17.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.173+0000 7f3e69ffb640 1 -- 172.21.15.47:0/1826799396 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e70003830 con 0x7f3e60003f60 2024-04-26T23:19:17.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.174+0000 7f3e7b86b640 1 -- 172.21.15.47:0/1826799396 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e740cd370 con 0x7f3e60003f60 2024-04-26T23:19:17.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.175+0000 7f3e7b86b640 1 -- 172.21.15.47:0/1826799396 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e740694f0 con 0x7f3e60003f60 2024-04-26T23:19:17.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.175+0000 7f3e69ffb640 1 -- 172.21.15.47:0/1826799396 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3e700039d0 con 0x7f3e60003f60 2024-04-26T23:19:17.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.175+0000 7f3e69ffb640 1 --2- 172.21.15.47:0/1826799396 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3e5806c920 0x7f3e5806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:17.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.176+0000 7f3e69ffb640 1 -- 172.21.15.47:0/1826799396 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f3e7008bb60 con 0x7f3e60003f60 2024-04-26T23:19:17.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.176+0000 7f3e78e06640 1 --2- 172.21.15.47:0/1826799396 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3e5806c920 0x7f3e5806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:17.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.176+0000 7f3e78e06640 1 --2- 172.21.15.47:0/1826799396 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3e5806c920 0x7f3e5806ede0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3e6400c210 tx=0x7f3e6402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:17.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.180+0000 7f3e69ffb640 1 -- 172.21.15.47:0/1826799396 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3e70059c00 con 0x7f3e60003f60 2024-04-26T23:19:17.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.365+0000 7f3e7b86b640 1 -- 172.21.15.47:0/1826799396 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3e740d1f80 con 0x7f3e60003f60 2024-04-26T23:19:17.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.365+0000 7f3e69ffb640 1 -- 172.21.15.47:0/1826799396 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f3e70059500 con 0x7f3e60003f60 2024-04-26T23:19:17.367 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-26T23:19:17.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.367+0000 7f3e7b86b640 1 -- 172.21.15.47:0/1826799396 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3e5806c920 msgr2=0x7f3e5806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:17.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.367+0000 7f3e7b86b640 1 --2- 172.21.15.47:0/1826799396 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3e5806c920 0x7f3e5806ede0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3e6400c210 tx=0x7f3e6402e000 comp rx=0 tx=0).stop 2024-04-26T23:19:17.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.367+0000 7f3e7b86b640 1 -- 172.21.15.47:0/1826799396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e60003f60 msgr2=0x7f3e740cec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:17.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.367+0000 7f3e7b86b640 1 --2- 172.21.15.47:0/1826799396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e60003f60 0x7f3e740cec80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3e7000e550 tx=0x7f3e7000ea10 comp rx=0 tx=0).stop 2024-04-26T23:19:17.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.367+0000 7f3e7b86b640 1 -- 172.21.15.47:0/1826799396 shutdown_connections 2024-04-26T23:19:17.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.367+0000 7f3e7b86b640 1 --2- 172.21.15.47:0/1826799396 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3e5806c920 0x7f3e5806ede0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:17.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.367+0000 7f3e7b86b640 1 --2- 172.21.15.47:0/1826799396 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e740cc520 0x7f3e7415e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:17.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.367+0000 7f3e7b86b640 1 --2- 172.21.15.47:0/1826799396 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e740cc0a0 0x7f3e7415e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:17.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.367+0000 7f3e7b86b640 1 --2- 172.21.15.47:0/1826799396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e60003f60 0x7f3e740cec80 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:17.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.367+0000 7f3e7b86b640 1 -- 172.21.15.47:0/1826799396 >> 172.21.15.47:0/1826799396 conn(0x7f3e740d4fd0 msgr2=0x7f3e740cfe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:17.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.367+0000 7f3e7b86b640 1 -- 172.21.15.47:0/1826799396 shutdown_connections 2024-04-26T23:19:17.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.367+0000 7f3e7b86b640 1 -- 172.21.15.47:0/1826799396 wait complete. 2024-04-26T23:19:17.380 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T23:19:17.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.483+0000 7f586e462640 1 Processor -- start 2024-04-26T23:19:17.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.483+0000 7f586e462640 1 -- start start 2024-04-26T23:19:17.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.483+0000 7f586e462640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f586806f790 0x7f586806fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:17.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.483+0000 7f586e462640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5868072c50 0x7f5868073060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:17.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.483+0000 7f586e462640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5868156840 con 0x7f5868156460 2024-04-26T23:19:17.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.483+0000 7f586e462640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f58680cbf20 con 0x7f5868072c50 2024-04-26T23:19:17.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.483+0000 7f586e462640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f58680cc200 con 0x7f586806f790 2024-04-26T23:19:17.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.484+0000 7f586c9ff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5868072c50 0x7f5868073060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:17.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.484+0000 7f586c9ff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5868072c50 0x7f5868073060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41420/0 (socket says 172.21.15.47:41420) 2024-04-26T23:19:17.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.484+0000 7f586c9ff640 1 -- 172.21.15.47:0/4254570130 learned_addr learned my addr 172.21.15.47:0/4254570130 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:17.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.484+0000 7f58677fe640 1 --2- 172.21.15.47:0/4254570130 >> v2:172.21.15.203:3300/0 conn(0x7f586806f790 0x7f586806fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:17.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.484+0000 7f586c9ff640 1 -- 172.21.15.47:0/4254570130 >> v2:172.21.15.203:3300/0 conn(0x7f586806f790 msgr2=0x7f586806fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:17.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.484+0000 7f586c9ff640 1 --2- 172.21.15.47:0/4254570130 >> v2:172.21.15.203:3300/0 conn(0x7f586806f790 0x7f586806fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:17.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.484+0000 7f586c9ff640 1 -- 172.21.15.47:0/4254570130 >> v1:172.21.15.203:6789/0 conn(0x7f5868156460 legacy=0x7f58680cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:17.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.484+0000 7f586c9ff640 1 -- 172.21.15.47:0/4254570130 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58680cc4e0 con 0x7f5868072c50 2024-04-26T23:19:17.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.485+0000 7f586c9ff640 1 --2- 172.21.15.47:0/4254570130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5868072c50 0x7f5868073060 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f58680d0b90 tx=0x7f586002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b64dd01c841af65 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:17.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.485+0000 7f5866ffd640 1 -- 172.21.15.47:0/4254570130 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5860036070 con 0x7f5868072c50 2024-04-26T23:19:17.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.485+0000 7f5866ffd640 1 -- 172.21.15.47:0/4254570130 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5860031600 con 0x7f5868072c50 2024-04-26T23:19:17.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.485+0000 7f5866ffd640 1 -- 172.21.15.47:0/4254570130 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58600319d0 con 0x7f5868072c50 2024-04-26T23:19:17.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.486+0000 7f586e462640 1 -- 172.21.15.47:0/4254570130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5868072c50 msgr2=0x7f5868073060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:17.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.486+0000 7f586e462640 1 --2- 172.21.15.47:0/4254570130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5868072c50 0x7f5868073060 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f58680d0b90 tx=0x7f586002a730 comp rx=0 tx=0).stop 2024-04-26T23:19:17.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.486+0000 7f586e462640 1 -- 172.21.15.47:0/4254570130 shutdown_connections 2024-04-26T23:19:17.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.486+0000 7f586e462640 1 --2- 172.21.15.47:0/4254570130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5868072c50 0x7f5868073060 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:17.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.486+0000 7f586e462640 1 --2- 172.21.15.47:0/4254570130 >> v2:172.21.15.203:3300/0 conn(0x7f586806f790 0x7f586806fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:17.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.486+0000 7f586e462640 1 -- 172.21.15.47:0/4254570130 >> 172.21.15.47:0/4254570130 conn(0x7f58680d0be0 msgr2=0x7f58680d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:17.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.486+0000 7f586e462640 1 -- 172.21.15.47:0/4254570130 shutdown_connections 2024-04-26T23:19:17.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.486+0000 7f586e462640 1 -- 172.21.15.47:0/4254570130 wait complete. 2024-04-26T23:19:17.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.487+0000 7f586e462640 1 Processor -- start 2024-04-26T23:19:17.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.487+0000 7f586e462640 1 -- start start 2024-04-26T23:19:17.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.488+0000 7f586e462640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f586806f790 0x7f5868151cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:17.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.488+0000 7f586e462640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5868072c50 0x7f58681e4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:17.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.488+0000 7f586e462640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5868156460 0x7f58681e49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:17.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.488+0000 7f586e462640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f58680cbbe0 con 0x7f586806f790 2024-04-26T23:19:17.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.488+0000 7f586e462640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5868072700 con 0x7f5868072c50 2024-04-26T23:19:17.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.488+0000 7f586e462640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f586806e260 con 0x7f5868156460 2024-04-26T23:19:17.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.488+0000 7f5867fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f586806f790 0x7f5868151cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:17.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.488+0000 7f5867fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f586806f790 0x7f5868151cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41432/0 (socket says 172.21.15.47:41432) 2024-04-26T23:19:17.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.488+0000 7f5867fff640 1 -- 172.21.15.47:0/1793517844 learned_addr learned my addr 172.21.15.47:0/1793517844 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:17.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.488+0000 7f58677fe640 1 --2- 172.21.15.47:0/1793517844 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5868072c50 0x7f58681e4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:17.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.488+0000 7f586c9ff640 1 --2- 172.21.15.47:0/1793517844 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5868156460 0x7f58681e49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:17.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.488+0000 7f5867fff640 1 -- 172.21.15.47:0/1793517844 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5868156460 msgr2=0x7f58681e49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:17.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.488+0000 7f5867fff640 1 --2- 172.21.15.47:0/1793517844 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5868156460 0x7f58681e49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:17.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.488+0000 7f5867fff640 1 -- 172.21.15.47:0/1793517844 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5868072c50 msgr2=0x7f58681e4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:17.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.488+0000 7f5867fff640 1 --2- 172.21.15.47:0/1793517844 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5868072c50 0x7f58681e4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:17.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.488+0000 7f5867fff640 1 -- 172.21.15.47:0/1793517844 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5868143e00 con 0x7f586806f790 2024-04-26T23:19:17.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.488+0000 7f58677fe640 1 --2- 172.21.15.47:0/1793517844 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5868072c50 0x7f58681e4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:19:17.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.488+0000 7f5867fff640 1 --2- 172.21.15.47:0/1793517844 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f586806f790 0x7f5868151cc0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f585800c3c0 tx=0x7f585800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:17.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.488+0000 7f5864ff9640 1 -- 172.21.15.47:0/1793517844 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58580052d0 con 0x7f586806f790 2024-04-26T23:19:17.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.488+0000 7f5864ff9640 1 -- 172.21.15.47:0/1793517844 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5858005490 con 0x7f586806f790 2024-04-26T23:19:17.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.488+0000 7f586e462640 1 -- 172.21.15.47:0/1793517844 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58680c64d0 con 0x7f586806f790 2024-04-26T23:19:17.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.488+0000 7f5864ff9640 1 -- 172.21.15.47:0/1793517844 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5858007260 con 0x7f586806f790 2024-04-26T23:19:17.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.489+0000 7f586e462640 1 -- 172.21.15.47:0/1793517844 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58680c6d10 con 0x7f586806f790 2024-04-26T23:19:17.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.490+0000 7f586e462640 1 -- 172.21.15.47:0/1793517844 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58680694f0 con 0x7f586806f790 2024-04-26T23:19:17.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.490+0000 7f5864ff9640 1 -- 172.21.15.47:0/1793517844 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5858007400 con 0x7f586806f790 2024-04-26T23:19:17.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.491+0000 7f5864ff9640 1 --2- 172.21.15.47:0/1793517844 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f585406c8f0 0x7f585406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:17.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.491+0000 7f5864ff9640 1 -- 172.21.15.47:0/1793517844 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f585808b510 con 0x7f586806f790 2024-04-26T23:19:17.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.491+0000 7f58677fe640 1 --2- 172.21.15.47:0/1793517844 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f585406c8f0 0x7f585406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:17.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.491+0000 7f58677fe640 1 --2- 172.21.15.47:0/1793517844 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f585406c8f0 0x7f585406edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f58680c7d60 tx=0x7f585c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:17.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.495+0000 7f5864ff9640 1 -- 172.21.15.47:0/1793517844 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f58680c64d0 con 0x7f586806f790 2024-04-26T23:19:17.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.679+0000 7f586e462640 1 -- 172.21.15.47:0/1793517844 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f586806de80 con 0x7f586806f790 2024-04-26T23:19:17.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.679+0000 7f5864ff9640 1 -- 172.21.15.47:0/1793517844 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f5858059480 con 0x7f586806f790 2024-04-26T23:19:17.681 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-26T23:19:17.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.681+0000 7f586e462640 1 -- 172.21.15.47:0/1793517844 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f585406c8f0 msgr2=0x7f585406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:17.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.681+0000 7f586e462640 1 --2- 172.21.15.47:0/1793517844 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f585406c8f0 0x7f585406edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f58680c7d60 tx=0x7f585c008040 comp rx=0 tx=0).stop 2024-04-26T23:19:17.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.681+0000 7f586e462640 1 -- 172.21.15.47:0/1793517844 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f586806f790 msgr2=0x7f5868151cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:17.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.681+0000 7f586e462640 1 --2- 172.21.15.47:0/1793517844 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f586806f790 0x7f5868151cc0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f585800c3c0 tx=0x7f585800c880 comp rx=0 tx=0).stop 2024-04-26T23:19:17.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.681+0000 7f586e462640 1 -- 172.21.15.47:0/1793517844 shutdown_connections 2024-04-26T23:19:17.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.681+0000 7f586e462640 1 --2- 172.21.15.47:0/1793517844 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f585406c8f0 0x7f585406edb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:17.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.681+0000 7f586e462640 1 --2- 172.21.15.47:0/1793517844 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5868156460 0x7f58681e49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:17.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.681+0000 7f586e462640 1 --2- 172.21.15.47:0/1793517844 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5868072c50 0x7f58681e4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:17.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.681+0000 7f586e462640 1 --2- 172.21.15.47:0/1793517844 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f586806f790 0x7f5868151cc0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:17.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.681+0000 7f586e462640 1 -- 172.21.15.47:0/1793517844 >> 172.21.15.47:0/1793517844 conn(0x7f58680d0be0 msgr2=0x7f5868154cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:17.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.682+0000 7f586e462640 1 -- 172.21.15.47:0/1793517844 shutdown_connections 2024-04-26T23:19:17.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.682+0000 7f586e462640 1 -- 172.21.15.47:0/1793517844 wait complete. 2024-04-26T23:19:17.725 INFO:tasks.cephfs_test_runner:Starting test: test_forbidden_modification (tasks.cephfs.test_pool_perm.TestPoolPerm) That a client who does not have the capability for setting 2024-04-26T23:19:17.725 INFO:tasks.cephfs_test_runner:test_forbidden_modification (tasks.cephfs.test_pool_perm.TestPoolPerm) 2024-04-26T23:19:17.725 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:19:17.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.826+0000 7feb48b5f640 1 Processor -- start 2024-04-26T23:19:17.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.826+0000 7feb48b5f640 1 -- start start 2024-04-26T23:19:17.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.827+0000 7feb48b5f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7feb440d3bc0 0x7feb440d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:17.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.827+0000 7feb48b5f640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb440b7e50 con 0x7feb440b5f20 2024-04-26T23:19:17.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.827+0000 7feb48b5f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb440d1800 con 0x7feb44162880 2024-04-26T23:19:17.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.827+0000 7feb48b5f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7feb440d0580 con 0x7feb440d3bc0 2024-04-26T23:19:17.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.827+0000 7feb42d9d640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7feb440b5f20 0x7feb440b8300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33034/0 (socket says 172.21.15.47:33034) 2024-04-26T23:19:17.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.827+0000 7feb42d9d640 1 -- 172.21.15.47:0/2481855579 learned_addr learned my addr 172.21.15.47:0/2481855579 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:17.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.827+0000 7feb3bfff640 1 --2- 172.21.15.47:0/2481855579 >> v2:172.21.15.203:3300/0 conn(0x7feb440d3bc0 0x7feb440d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:17.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.827+0000 7feb41d9b640 1 -- 172.21.15.47:0/2481855579 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2248545969 0 0) 0x7feb440b7e50 con 0x7feb440b5f20 2024-04-26T23:19:17.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.827+0000 7feb41d9b640 1 -- 172.21.15.47:0/2481855579 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb30003520 con 0x7feb440b5f20 2024-04-26T23:19:17.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.827+0000 7feb41d9b640 1 -- 172.21.15.47:0/2481855579 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 63752340 0 0) 0x7feb30003520 con 0x7feb440b5f20 2024-04-26T23:19:17.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.827+0000 7feb41d9b640 1 -- 172.21.15.47:0/2481855579 >> v2:172.21.15.203:3300/0 conn(0x7feb440d3bc0 msgr2=0x7feb440d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:17.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.827+0000 7feb41d9b640 1 --2- 172.21.15.47:0/2481855579 >> v2:172.21.15.203:3300/0 conn(0x7feb440d3bc0 0x7feb440d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:17.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.827+0000 7feb41d9b640 1 -- 172.21.15.47:0/2481855579 >> v1:172.21.15.203:6789/0 conn(0x7feb44162880 legacy=0x7feb440d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:17.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.827+0000 7feb41d9b640 1 -- 172.21.15.47:0/2481855579 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb440d07d0 con 0x7feb440b5f20 2024-04-26T23:19:17.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.827+0000 7feb41d9b640 1 -- 172.21.15.47:0/2481855579 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7feb3c0019f0 con 0x7feb440b5f20 2024-04-26T23:19:17.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.827+0000 7feb41d9b640 1 -- 172.21.15.47:0/2481855579 >> v1:172.21.15.47:6789/0 conn(0x7feb440b5f20 legacy=0x7feb440b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:17.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.827+0000 7feb41d9b640 1 --2- 172.21.15.47:0/2481855579 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb300039b0 0x7feb30003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:17.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.827+0000 7feb41d9b640 1 --2- 172.21.15.47:0/2481855579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb30004800 0x7feb30004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:17.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.827+0000 7feb41d9b640 1 --2- 172.21.15.47:0/2481855579 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb300051c0 0x7feb3000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:17.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.827+0000 7feb41d9b640 1 -- 172.21.15.47:0/2481855579 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7feb440d0580 con 0x7feb300039b0 2024-04-26T23:19:17.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.827+0000 7feb41d9b640 1 -- 172.21.15.47:0/2481855579 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7feb3000c060 con 0x7feb30004800 2024-04-26T23:19:17.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.827+0000 7feb41d9b640 1 -- 172.21.15.47:0/2481855579 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7feb3000c260 con 0x7feb300051c0 2024-04-26T23:19:17.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.827+0000 7feb3bfff640 1 --2- 172.21.15.47:0/2481855579 >> v2:172.21.15.203:3300/0 conn(0x7feb440d3bc0 0x7feb440d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:19:17.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.827+0000 7feb4259c640 1 --2- 172.21.15.47:0/2481855579 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb300039b0 0x7feb30003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:17.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.828+0000 7feb42d9d640 1 --2- 172.21.15.47:0/2481855579 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb300051c0 0x7feb3000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:17.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.828+0000 7feb3bfff640 1 --2- 172.21.15.47:0/2481855579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb30004800 0x7feb30004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:17.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.828+0000 7feb4259c640 1 -- 172.21.15.47:0/2481855579 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb300051c0 msgr2=0x7feb3000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:17.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.828+0000 7feb4259c640 1 --2- 172.21.15.47:0/2481855579 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb300051c0 0x7feb3000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:17.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.828+0000 7feb4259c640 1 -- 172.21.15.47:0/2481855579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb30004800 msgr2=0x7feb30004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:17.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.828+0000 7feb4259c640 1 --2- 172.21.15.47:0/2481855579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb30004800 0x7feb30004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:17.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.828+0000 7feb4259c640 1 -- 172.21.15.47:0/2481855579 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb30003520 con 0x7feb300039b0 2024-04-26T23:19:17.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.828+0000 7feb42d9d640 1 --2- 172.21.15.47:0/2481855579 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb300051c0 0x7feb3000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:19:17.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.828+0000 7feb42d9d640 1 -- 172.21.15.47:0/2481855579 reap_dead start 2024-04-26T23:19:17.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.828+0000 7feb4259c640 1 --2- 172.21.15.47:0/2481855579 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb300039b0 0x7feb30003dd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7feb34001750 tx=0x7feb30005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9902a763166e741c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:17.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.829+0000 7feb41d9b640 1 -- 172.21.15.47:0/2481855579 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb34001370 con 0x7feb300039b0 2024-04-26T23:19:17.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.829+0000 7feb41d9b640 1 -- 172.21.15.47:0/2481855579 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb340052d0 con 0x7feb300039b0 2024-04-26T23:19:17.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.829+0000 7feb41d9b640 1 -- 172.21.15.47:0/2481855579 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb3402abc0 con 0x7feb300039b0 2024-04-26T23:19:17.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.829+0000 7feb48b5f640 1 -- 172.21.15.47:0/2481855579 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb300039b0 msgr2=0x7feb30003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:17.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.829+0000 7feb48b5f640 1 --2- 172.21.15.47:0/2481855579 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb300039b0 0x7feb30003dd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7feb34001750 tx=0x7feb30005830 comp rx=0 tx=0).stop 2024-04-26T23:19:17.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.829+0000 7feb48b5f640 1 -- 172.21.15.47:0/2481855579 shutdown_connections 2024-04-26T23:19:17.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.829+0000 7feb48b5f640 1 --2- 172.21.15.47:0/2481855579 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb300039b0 0x7feb30003dd0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:17.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.829+0000 7feb48b5f640 1 -- 172.21.15.47:0/2481855579 >> 172.21.15.47:0/2481855579 conn(0x7feb440d4f60 msgr2=0x7feb440d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:17.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.829+0000 7feb48b5f640 1 -- 172.21.15.47:0/2481855579 shutdown_connections 2024-04-26T23:19:17.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.829+0000 7feb48b5f640 1 -- 172.21.15.47:0/2481855579 wait complete. 2024-04-26T23:19:17.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.831+0000 7feb48b5f640 1 Processor -- start 2024-04-26T23:19:17.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.831+0000 7feb48b5f640 1 -- start start 2024-04-26T23:19:17.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.831+0000 7feb48b5f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb300039b0 0x7feb4415d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:17.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.831+0000 7feb48b5f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb440b4220 0x7feb441f80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:17.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.831+0000 7feb48b5f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb440b1b50 0x7feb440b1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:17.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.831+0000 7feb48b5f640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7feb44162c60 con 0x7feb300039b0 2024-04-26T23:19:17.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.831+0000 7feb48b5f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7feb440d0d90 con 0x7feb440b1b50 2024-04-26T23:19:17.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.831+0000 7feb48b5f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7feb44162de0 con 0x7feb440b4220 2024-04-26T23:19:17.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.831+0000 7feb4259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb300039b0 0x7feb4415d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:17.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.831+0000 7feb4259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb300039b0 0x7feb4415d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41458/0 (socket says 172.21.15.47:41458) 2024-04-26T23:19:17.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.831+0000 7feb4259c640 1 -- 172.21.15.47:0/1330087145 learned_addr learned my addr 172.21.15.47:0/1330087145 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:17.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.832+0000 7feb3bfff640 1 --2- 172.21.15.47:0/1330087145 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb440b4220 0x7feb441f80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:17.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.832+0000 7feb4259c640 1 -- 172.21.15.47:0/1330087145 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb440b4220 msgr2=0x7feb441f80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:17.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.832+0000 7feb42d9d640 1 --2- 172.21.15.47:0/1330087145 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb440b1b50 0x7feb440b1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:17.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.832+0000 7feb4259c640 1 --2- 172.21.15.47:0/1330087145 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb440b4220 0x7feb441f80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:17.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.832+0000 7feb4259c640 1 -- 172.21.15.47:0/1330087145 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb440b1b50 msgr2=0x7feb440b1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:17.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.832+0000 7feb4259c640 1 --2- 172.21.15.47:0/1330087145 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb440b1b50 0x7feb440b1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:17.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.832+0000 7feb4259c640 1 -- 172.21.15.47:0/1330087145 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb440b46a0 con 0x7feb300039b0 2024-04-26T23:19:17.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.832+0000 7feb4259c640 1 --2- 172.21.15.47:0/1330087145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb300039b0 0x7feb4415d9d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7feb34000c00 tx=0x7feb34001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:17.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.832+0000 7feb3affd640 1 -- 172.21.15.47:0/1330087145 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb3403f020 con 0x7feb300039b0 2024-04-26T23:19:17.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.832+0000 7feb3affd640 1 -- 172.21.15.47:0/1330087145 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb34005480 con 0x7feb300039b0 2024-04-26T23:19:17.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.832+0000 7feb48b5f640 1 -- 172.21.15.47:0/1330087145 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb441f8840 con 0x7feb300039b0 2024-04-26T23:19:17.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.832+0000 7feb3affd640 1 -- 172.21.15.47:0/1330087145 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb34007a30 con 0x7feb300039b0 2024-04-26T23:19:17.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.832+0000 7feb48b5f640 1 -- 172.21.15.47:0/1330087145 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb441f8e00 con 0x7feb300039b0 2024-04-26T23:19:17.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.833+0000 7feb48b5f640 1 -- 172.21.15.47:0/1330087145 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb440694f0 con 0x7feb300039b0 2024-04-26T23:19:17.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.834+0000 7feb3affd640 1 -- 172.21.15.47:0/1330087145 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feb34042020 con 0x7feb300039b0 2024-04-26T23:19:17.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.834+0000 7feb3affd640 1 --2- 172.21.15.47:0/1330087145 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7feb2406c8f0 0x7feb2406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:17.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.834+0000 7feb3affd640 1 -- 172.21.15.47:0/1330087145 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7feb340abbc0 con 0x7feb300039b0 2024-04-26T23:19:17.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.834+0000 7feb3bfff640 1 --2- 172.21.15.47:0/1330087145 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7feb2406c8f0 0x7feb2406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:17.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.835+0000 7feb3bfff640 1 --2- 172.21.15.47:0/1330087145 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7feb2406c8f0 0x7feb2406edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7feb440b5910 tx=0x7feb2c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:17.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:17.838+0000 7feb3affd640 1 -- 172.21.15.47:0/1330087145 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7feb34049030 con 0x7feb300039b0 2024-04-26T23:19:18.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.024+0000 7feb48b5f640 1 -- 172.21.15.47:0/1330087145 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feb440d2e90 con 0x7feb300039b0 2024-04-26T23:19:18.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.025+0000 7feb3affd640 1 -- 172.21.15.47:0/1330087145 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18857 (secure 0 0 0) 0x7feb440d1800 con 0x7feb300039b0 2024-04-26T23:19:18.027 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:19:18.028 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":26,"fsid":"399006d3-4db1-43e6-ba81-a79a276eabfd","created":"2024-04-26T23:18:06.731211+0000","modified":"2024-04-26T23:19:12.124051+0000","last_up_change":"2024-04-26T23:18:17.808736+0000","last_in_change":"2024-04-26T23:18:08.945859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:18:16.937492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:18:29.033575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:18:30.318499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:18:34.014536+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ee091567-fea0-4a14-bf4e-0e16c20bb2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6819","nonce":326520779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6821","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6823","nonce":326520779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6829","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6831","nonce":326520779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6825","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6827","nonce":326520779}]},"public_addr":"172.21.15.47:6819/326520779","cluster_addr":"172.21.15.47:6823/326520779","heartbeat_back_addr":"172.21.15.47:6831/326520779","heartbeat_front_addr":"172.21.15.47:6827/326520779","state":["exists","up"]},{"osd":1,"uuid":"c14440a0-bba5-4931-a6db-b736c08e869b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6809","nonce":1912610931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6811","nonce":1912610931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6815","nonce":1912610931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6813","nonce":1912610931}]},"public_addr":"172.21.15.47:6809/1912610931","cluster_addr":"172.21.15.47:6811/1912610931","heartbeat_back_addr":"172.21.15.47:6815/1912610931","heartbeat_front_addr":"172.21.15.47:6813/1912610931","state":["exists","up"]},{"osd":2,"uuid":"3940f8de-c9b2-4cd3-a911-9c301729eb30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6801","nonce":2569869305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6803","nonce":2569869305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6807","nonce":2569869305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6805","nonce":2569869305}]},"public_addr":"172.21.15.47:6801/2569869305","cluster_addr":"172.21.15.47:6803/2569869305","heartbeat_back_addr":"172.21.15.47:6807/2569869305","heartbeat_front_addr":"172.21.15.47:6805/2569869305","state":["exists","up"]},{"osd":3,"uuid":"1286a67e-8958-426a-8c2e-bc85498c619c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6818","nonce":2791411683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6822","nonce":2791411683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6830","nonce":2791411683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6826","nonce":2791411683}]},"public_addr":"172.21.15.47:6818/2791411683","cluster_addr":"172.21.15.47:6822/2791411683","heartbeat_back_addr":"172.21.15.47:6830/2791411683","heartbeat_front_addr":"172.21.15.47:6826/2791411683","state":["exists","up"]},{"osd":4,"uuid":"d8ee2c20-e5b7-46c2-9999-b234070126cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6809","nonce":441313104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6811","nonce":441313104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6815","nonce":441313104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6813","nonce":441313104}]},"public_addr":"172.21.15.203:6809/441313104","cluster_addr":"172.21.15.203:6811/441313104","heartbeat_back_addr":"172.21.15.203:6815/441313104","heartbeat_front_addr":"172.21.15.203:6813/441313104","state":["exists","up"]},{"osd":5,"uuid":"b453a8f0-f324-4840-9ae0-d67bdadb9f8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6801","nonce":56838444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6803","nonce":56838444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6807","nonce":56838444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6805","nonce":56838444}]},"public_addr":"172.21.15.203:6801/56838444","cluster_addr":"172.21.15.203:6803/56838444","heartbeat_back_addr":"172.21.15.203:6807/56838444","heartbeat_front_addr":"172.21.15.203:6805/56838444","state":["exists","up"]},{"osd":6,"uuid":"a6d31951-2d4b-4e8d-a510-6283b01dc158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6825","nonce":2738517080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6827","nonce":2738517080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6831","nonce":2738517080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6829","nonce":2738517080}]},"public_addr":"172.21.15.203:6825/2738517080","cluster_addr":"172.21.15.203:6827/2738517080","heartbeat_back_addr":"172.21.15.203:6831/2738517080","heartbeat_front_addr":"172.21.15.203:6829/2738517080","state":["exists","up"]},{"osd":7,"uuid":"7119b8c0-fe96-402d-9e75-7b509d9d44d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6817","nonce":1315565058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6819","nonce":1315565058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6823","nonce":1315565058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6821","nonce":1315565058}]},"public_addr":"172.21.15.203:6817/1315565058","cluster_addr":"172.21.15.203:6819/1315565058","heartbeat_back_addr":"172.21.15.203:6823/1315565058","heartbeat_front_addr":"172.21.15.203:6821/1315565058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.121921+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.137494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.084814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.111593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.072430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.125066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.097867+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.057061+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":5,"to":1}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:19:18.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.027+0000 7feb48b5f640 1 -- 172.21.15.47:0/1330087145 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7feb2406c8f0 msgr2=0x7feb2406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:18.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.027+0000 7feb48b5f640 1 --2- 172.21.15.47:0/1330087145 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7feb2406c8f0 0x7feb2406edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7feb440b5910 tx=0x7feb2c008040 comp rx=0 tx=0).stop 2024-04-26T23:19:18.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.027+0000 7feb48b5f640 1 -- 172.21.15.47:0/1330087145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb300039b0 msgr2=0x7feb4415d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:18.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.027+0000 7feb48b5f640 1 --2- 172.21.15.47:0/1330087145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb300039b0 0x7feb4415d9d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7feb34000c00 tx=0x7feb34001120 comp rx=0 tx=0).stop 2024-04-26T23:19:18.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.027+0000 7feb48b5f640 1 -- 172.21.15.47:0/1330087145 shutdown_connections 2024-04-26T23:19:18.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.027+0000 7feb48b5f640 1 --2- 172.21.15.47:0/1330087145 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7feb2406c8f0 0x7feb2406edb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:18.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.027+0000 7feb48b5f640 1 --2- 172.21.15.47:0/1330087145 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb440b1b50 0x7feb440b1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:18.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.027+0000 7feb48b5f640 1 --2- 172.21.15.47:0/1330087145 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb440b4220 0x7feb441f80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:18.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.027+0000 7feb48b5f640 1 --2- 172.21.15.47:0/1330087145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb300039b0 0x7feb4415d9d0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:18.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.027+0000 7feb48b5f640 1 -- 172.21.15.47:0/1330087145 >> 172.21.15.47:0/1330087145 conn(0x7feb440d4f60 msgr2=0x7feb440d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:18.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.028+0000 7feb48b5f640 1 -- 172.21.15.47:0/1330087145 shutdown_connections 2024-04-26T23:19:18.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.028+0000 7feb48b5f640 1 -- 172.21.15.47:0/1330087145 wait complete. 2024-04-26T23:19:18.040 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:18:16.937492+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:18:29.033575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:18:30.318499+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T23:18:34.014536+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T23:19:18.041 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:19:18.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.142+0000 7f9b0e1af640 1 Processor -- start 2024-04-26T23:19:18.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.143+0000 7f9b0e1af640 1 -- start start 2024-04-26T23:19:18.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.143+0000 7f9b0e1af640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9b080d1a00 0x7f9b080d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:18.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.143+0000 7f9b0e1af640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b080d0a10 con 0x7f9b080d2320 2024-04-26T23:19:18.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.143+0000 7f9b0e1af640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b080d0cf0 con 0x7f9b080b40d0 2024-04-26T23:19:18.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.143+0000 7f9b0e1af640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f9b080d0fd0 con 0x7f9b080d1a00 2024-04-26T23:19:18.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.143+0000 7f9b06ffd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9b080d2320 0x7f9b081625f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33038/0 (socket says 172.21.15.47:33038) 2024-04-26T23:19:18.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.144+0000 7f9b06ffd640 1 -- 172.21.15.47:0/3671924673 learned_addr learned my addr 172.21.15.47:0/3671924673 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:18.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.144+0000 7f9b077fe640 1 --2- 172.21.15.47:0/3671924673 >> v2:172.21.15.203:3300/0 conn(0x7f9b080d1a00 0x7f9b080d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:18.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.144+0000 7f9b067fc640 1 -- 172.21.15.47:0/3671924673 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3110331979 0 0) 0x7f9b080d0a10 con 0x7f9b080d2320 2024-04-26T23:19:18.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.144+0000 7f9b067fc640 1 -- 172.21.15.47:0/3671924673 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9af0003520 con 0x7f9b080d2320 2024-04-26T23:19:18.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.144+0000 7f9b077fe640 1 -- 172.21.15.47:0/3671924673 >> v1:172.21.15.203:6789/0 conn(0x7f9b080b40d0 legacy=0x7f9b080b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:18.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.144+0000 7f9b077fe640 1 -- 172.21.15.47:0/3671924673 >> v1:172.21.15.47:6789/0 conn(0x7f9b080d2320 legacy=0x7f9b081625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:18.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.144+0000 7f9b067fc640 1 -- 172.21.15.47:0/3671924673 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1051629531 0 0) 0x7f9af0003520 con 0x7f9b080d2320 2024-04-26T23:19:18.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.144+0000 7f9b077fe640 1 -- 172.21.15.47:0/3671924673 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b080d40a0 con 0x7f9b080d1a00 2024-04-26T23:19:18.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.145+0000 7f9b077fe640 1 --2- 172.21.15.47:0/3671924673 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b080d1a00 0x7f9b080d1de0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9af80097a0 tx=0x7f9af802a0a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84c6c83419eacec5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:18.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.145+0000 7f9b067fc640 1 -- 172.21.15.47:0/3671924673 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af8037070 con 0x7f9b080d1a00 2024-04-26T23:19:18.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.145+0000 7f9b067fc640 1 -- 172.21.15.47:0/3671924673 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9af802ab30 con 0x7f9b080d1a00 2024-04-26T23:19:18.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.145+0000 7f9b067fc640 1 -- 172.21.15.47:0/3671924673 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9afc001670 con 0x7f9b080d1a00 2024-04-26T23:19:18.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.146+0000 7f9b0e1af640 1 -- 172.21.15.47:0/3671924673 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b080d1a00 msgr2=0x7f9b080d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:18.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.146+0000 7f9b0e1af640 1 --2- 172.21.15.47:0/3671924673 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b080d1a00 0x7f9b080d1de0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9af80097a0 tx=0x7f9af802a0a0 comp rx=0 tx=0).stop 2024-04-26T23:19:18.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.146+0000 7f9b0e1af640 1 -- 172.21.15.47:0/3671924673 shutdown_connections 2024-04-26T23:19:18.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.146+0000 7f9b0e1af640 1 --2- 172.21.15.47:0/3671924673 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b080d1a00 0x7f9b080d1de0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:18.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.146+0000 7f9b0e1af640 1 -- 172.21.15.47:0/3671924673 >> 172.21.15.47:0/3671924673 conn(0x7f9b080d4fd0 msgr2=0x7f9b080d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:18.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.146+0000 7f9b0e1af640 1 -- 172.21.15.47:0/3671924673 shutdown_connections 2024-04-26T23:19:18.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.146+0000 7f9b0e1af640 1 -- 172.21.15.47:0/3671924673 wait complete. 2024-04-26T23:19:18.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.147+0000 7f9b0e1af640 1 Processor -- start 2024-04-26T23:19:18.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.147+0000 7f9b0e1af640 1 -- start start 2024-04-26T23:19:18.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.148+0000 7f9b0e1af640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b080b40d0 0x7f9b080ce880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:18.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.148+0000 7f9b0e1af640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b080d1a00 0x7f9b080ca890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:18.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.148+0000 7f9b0e1af640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b080d2320 0x7f9b08154fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:18.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.148+0000 7f9b0e1af640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9b080b5210 con 0x7f9b080d1a00 2024-04-26T23:19:18.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.148+0000 7f9b0e1af640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9b08162f50 con 0x7f9b080d2320 2024-04-26T23:19:18.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.148+0000 7f9b0e1af640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f9b080b5390 con 0x7f9b080b40d0 2024-04-26T23:19:18.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.148+0000 7f9b06ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b080d1a00 0x7f9b080ca890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:18.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.148+0000 7f9b06ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b080d1a00 0x7f9b080ca890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41460/0 (socket says 172.21.15.47:41460) 2024-04-26T23:19:18.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.148+0000 7f9b06ffd640 1 -- 172.21.15.47:0/2232486460 learned_addr learned my addr 172.21.15.47:0/2232486460 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:18.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.148+0000 7f9b077fe640 1 --2- 172.21.15.47:0/2232486460 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b080b40d0 0x7f9b080ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:18.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.148+0000 7f9b07fff640 1 --2- 172.21.15.47:0/2232486460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b080d2320 0x7f9b08154fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:18.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.148+0000 7f9b06ffd640 1 -- 172.21.15.47:0/2232486460 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b080b40d0 msgr2=0x7f9b080ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:18.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.148+0000 7f9b06ffd640 1 --2- 172.21.15.47:0/2232486460 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b080b40d0 0x7f9b080ce880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:18.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.148+0000 7f9b06ffd640 1 -- 172.21.15.47:0/2232486460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b080d2320 msgr2=0x7f9b08154fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:18.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.148+0000 7f9b06ffd640 1 --2- 172.21.15.47:0/2232486460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b080d2320 0x7f9b08154fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:18.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.148+0000 7f9b06ffd640 1 -- 172.21.15.47:0/2232486460 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b080cc140 con 0x7f9b080d1a00 2024-04-26T23:19:18.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.148+0000 7f9b07fff640 1 --2- 172.21.15.47:0/2232486460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b080d2320 0x7f9b08154fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:19:18.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.149+0000 7f9b06ffd640 1 --2- 172.21.15.47:0/2232486460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b080d1a00 0x7f9b080ca890 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9afc00c700 tx=0x7f9afc00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:18.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.149+0000 7f9aeffff640 1 -- 172.21.15.47:0/2232486460 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9afc017070 con 0x7f9b080d1a00 2024-04-26T23:19:18.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.149+0000 7f9aeffff640 1 -- 172.21.15.47:0/2232486460 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9afc005ad0 con 0x7f9b080d1a00 2024-04-26T23:19:18.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.149+0000 7f9b0e1af640 1 -- 172.21.15.47:0/2232486460 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b080cb200 con 0x7f9b080d1a00 2024-04-26T23:19:18.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.149+0000 7f9aeffff640 1 -- 172.21.15.47:0/2232486460 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9afc007070 con 0x7f9b080d1a00 2024-04-26T23:19:18.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.149+0000 7f9b0e1af640 1 -- 172.21.15.47:0/2232486460 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b080cb7c0 con 0x7f9b080d1a00 2024-04-26T23:19:18.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.150+0000 7f9b0e1af640 1 -- 172.21.15.47:0/2232486460 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b080694f0 con 0x7f9b080d1a00 2024-04-26T23:19:18.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.152+0000 7f9aeffff640 1 -- 172.21.15.47:0/2232486460 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9afc01a070 con 0x7f9b080d1a00 2024-04-26T23:19:18.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.153+0000 7f9aeffff640 1 --2- 172.21.15.47:0/2232486460 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9af406c8f0 0x7f9af406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:18.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.153+0000 7f9aeffff640 1 -- 172.21.15.47:0/2232486460 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f9afc08ba10 con 0x7f9b080d1a00 2024-04-26T23:19:18.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.153+0000 7f9b077fe640 1 --2- 172.21.15.47:0/2232486460 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9af406c8f0 0x7f9af406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:18.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.154+0000 7f9b077fe640 1 --2- 172.21.15.47:0/2232486460 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9af406c8f0 0x7f9af406edb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9af8034000 tx=0x7f9af802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:18.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.156+0000 7f9aeffff640 1 -- 172.21.15.47:0/2232486460 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9afc059540 con 0x7f9b080d1a00 2024-04-26T23:19:18.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.342+0000 7f9b0e1af640 1 -- 172.21.15.47:0/2232486460 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9b080bae50 con 0x7f9b080d1a00 2024-04-26T23:19:18.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.342+0000 7f9aeffff640 1 -- 172.21.15.47:0/2232486460 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f9afc05d840 con 0x7f9b080d1a00 2024-04-26T23:19:18.344 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-26T23:19:18.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.344+0000 7f9b0e1af640 1 -- 172.21.15.47:0/2232486460 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9af406c8f0 msgr2=0x7f9af406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:18.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.344+0000 7f9b0e1af640 1 --2- 172.21.15.47:0/2232486460 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9af406c8f0 0x7f9af406edb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9af8034000 tx=0x7f9af802e000 comp rx=0 tx=0).stop 2024-04-26T23:19:18.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.344+0000 7f9b0e1af640 1 -- 172.21.15.47:0/2232486460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b080d1a00 msgr2=0x7f9b080ca890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:18.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.344+0000 7f9b0e1af640 1 --2- 172.21.15.47:0/2232486460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b080d1a00 0x7f9b080ca890 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9afc00c700 tx=0x7f9afc00cbc0 comp rx=0 tx=0).stop 2024-04-26T23:19:18.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.344+0000 7f9b0e1af640 1 -- 172.21.15.47:0/2232486460 shutdown_connections 2024-04-26T23:19:18.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.344+0000 7f9b0e1af640 1 --2- 172.21.15.47:0/2232486460 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9af406c8f0 0x7f9af406edb0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:18.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.344+0000 7f9b0e1af640 1 --2- 172.21.15.47:0/2232486460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b080d2320 0x7f9b08154fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:18.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.344+0000 7f9b0e1af640 1 --2- 172.21.15.47:0/2232486460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b080d1a00 0x7f9b080ca890 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:18.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.344+0000 7f9b0e1af640 1 --2- 172.21.15.47:0/2232486460 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b080b40d0 0x7f9b080ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:18.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.345+0000 7f9b0e1af640 1 -- 172.21.15.47:0/2232486460 >> 172.21.15.47:0/2232486460 conn(0x7f9b080d4fd0 msgr2=0x7f9b0806cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:18.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.345+0000 7f9b0e1af640 1 -- 172.21.15.47:0/2232486460 shutdown_connections 2024-04-26T23:19:18.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.345+0000 7f9b0e1af640 1 -- 172.21.15.47:0/2232486460 wait complete. 2024-04-26T23:19:18.357 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:19:18.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.459+0000 7f0aa692b640 1 Processor -- start 2024-04-26T23:19:18.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.459+0000 7f0aa692b640 1 -- start start 2024-04-26T23:19:18.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.460+0000 7f0aa692b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0aa015f550 0x7f0aa00d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:18.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.460+0000 7f0aa692b640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0aa00d3bc0 0x7f0aa00d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:18.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.460+0000 7f0aa692b640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0aa00d1040 con 0x7f0aa015f170 2024-04-26T23:19:18.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.460+0000 7f0aa692b640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f0aa00c9430 con 0x7f0aa015f550 2024-04-26T23:19:18.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.460+0000 7f0aa692b640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f0aa00c9680 con 0x7f0aa00d3bc0 2024-04-26T23:19:18.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.460+0000 7f0a9f7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0aa015f550 0x7f0aa00d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:18.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.460+0000 7f0a9f7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0aa015f550 0x7f0aa00d5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41478/0 (socket says 172.21.15.47:41478) 2024-04-26T23:19:18.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.460+0000 7f0a9f7fe640 1 -- 172.21.15.47:0/188528737 learned_addr learned my addr 172.21.15.47:0/188528737 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:18.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.460+0000 7f0aa4ec8640 1 --2- 172.21.15.47:0/188528737 >> v2:172.21.15.203:3300/0 conn(0x7f0aa00d3bc0 0x7f0aa00d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:18.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.460+0000 7f0a9f7fe640 1 -- 172.21.15.47:0/188528737 >> v2:172.21.15.203:3300/0 conn(0x7f0aa00d3bc0 msgr2=0x7f0aa00d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:18.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.460+0000 7f0a9f7fe640 1 --2- 172.21.15.47:0/188528737 >> v2:172.21.15.203:3300/0 conn(0x7f0aa00d3bc0 0x7f0aa00d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:18.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.460+0000 7f0a9f7fe640 1 -- 172.21.15.47:0/188528737 >> v1:172.21.15.203:6789/0 conn(0x7f0aa015f170 legacy=0x7f0aa00d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:18.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.460+0000 7f0a9f7fe640 1 -- 172.21.15.47:0/188528737 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aa00c9960 con 0x7f0aa015f550 2024-04-26T23:19:18.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.461+0000 7f0a9f7fe640 1 --2- 172.21.15.47:0/188528737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0aa015f550 0x7f0aa00d5a80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0a90009870 tx=0x7f0a9002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77326e18c2ec60bd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:18.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.461+0000 7f0a9effd640 1 -- 172.21.15.47:0/188528737 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a90037070 con 0x7f0aa015f550 2024-04-26T23:19:18.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.461+0000 7f0a9effd640 1 -- 172.21.15.47:0/188528737 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a90032440 con 0x7f0aa015f550 2024-04-26T23:19:18.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.461+0000 7f0a9effd640 1 -- 172.21.15.47:0/188528737 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a90032810 con 0x7f0aa015f550 2024-04-26T23:19:18.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.462+0000 7f0aa692b640 1 -- 172.21.15.47:0/188528737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0aa015f550 msgr2=0x7f0aa00d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:18.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.462+0000 7f0aa692b640 1 --2- 172.21.15.47:0/188528737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0aa015f550 0x7f0aa00d5a80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0a90009870 tx=0x7f0a9002a2a0 comp rx=0 tx=0).stop 2024-04-26T23:19:18.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.462+0000 7f0aa692b640 1 -- 172.21.15.47:0/188528737 shutdown_connections 2024-04-26T23:19:18.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.462+0000 7f0aa692b640 1 --2- 172.21.15.47:0/188528737 >> v2:172.21.15.203:3300/0 conn(0x7f0aa00d3bc0 0x7f0aa00d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:18.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.462+0000 7f0aa692b640 1 --2- 172.21.15.47:0/188528737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0aa015f550 0x7f0aa00d5a80 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:18.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.462+0000 7f0aa692b640 1 -- 172.21.15.47:0/188528737 >> 172.21.15.47:0/188528737 conn(0x7f0aa00d4f60 msgr2=0x7f0aa00d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:18.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.462+0000 7f0aa692b640 1 -- 172.21.15.47:0/188528737 shutdown_connections 2024-04-26T23:19:18.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.462+0000 7f0aa692b640 1 -- 172.21.15.47:0/188528737 wait complete. 2024-04-26T23:19:18.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.463+0000 7f0aa692b640 1 Processor -- start 2024-04-26T23:19:18.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.463+0000 7f0aa692b640 1 -- start start 2024-04-26T23:19:18.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.464+0000 7f0aa692b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0aa00d3bc0 0x7f0aa014b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:18.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.464+0000 7f0aa692b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0aa015f170 0x7f0aa00cc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:18.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.464+0000 7f0aa692b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0aa00ccef0 0x7f0aa01ea000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:18.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.464+0000 7f0aa692b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa00ca5c0 con 0x7f0aa015f170 2024-04-26T23:19:18.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.464+0000 7f0aa692b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa00ca440 con 0x7f0aa00ccef0 2024-04-26T23:19:18.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.464+0000 7f0aa692b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f0aa00ca740 con 0x7f0aa00d3bc0 2024-04-26T23:19:18.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.464+0000 7f0a9f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0aa015f170 0x7f0aa00cc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:18.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.464+0000 7f0a9f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0aa015f170 0x7f0aa00cc9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41480/0 (socket says 172.21.15.47:41480) 2024-04-26T23:19:18.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.464+0000 7f0a9f7fe640 1 -- 172.21.15.47:0/3238092799 learned_addr learned my addr 172.21.15.47:0/3238092799 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:18.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.464+0000 7f0aa4ec8640 1 --2- 172.21.15.47:0/3238092799 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0aa00ccef0 0x7f0aa01ea000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:18.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.464+0000 7f0a9ffff640 1 --2- 172.21.15.47:0/3238092799 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0aa00d3bc0 0x7f0aa014b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:18.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.464+0000 7f0a9f7fe640 1 -- 172.21.15.47:0/3238092799 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0aa00d3bc0 msgr2=0x7f0aa014b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:18.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.464+0000 7f0a9f7fe640 1 --2- 172.21.15.47:0/3238092799 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0aa00d3bc0 0x7f0aa014b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:18.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.464+0000 7f0a9f7fe640 1 -- 172.21.15.47:0/3238092799 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0aa00ccef0 msgr2=0x7f0aa01ea000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:18.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.464+0000 7f0a9f7fe640 1 --2- 172.21.15.47:0/3238092799 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0aa00ccef0 0x7f0aa01ea000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:18.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.464+0000 7f0a9f7fe640 1 -- 172.21.15.47:0/3238092799 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aa014b8a0 con 0x7f0aa015f170 2024-04-26T23:19:18.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.464+0000 7f0aa4ec8640 1 --2- 172.21.15.47:0/3238092799 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0aa00ccef0 0x7f0aa01ea000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:19:18.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.464+0000 7f0a9f7fe640 1 --2- 172.21.15.47:0/3238092799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0aa015f170 0x7f0aa00cc9b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0a90000c00 tx=0x7f0a900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:18.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.465+0000 7f0a9cff9640 1 -- 172.21.15.47:0/3238092799 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a90041020 con 0x7f0aa015f170 2024-04-26T23:19:18.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.465+0000 7f0a9cff9640 1 -- 172.21.15.47:0/3238092799 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a900306c0 con 0x7f0aa015f170 2024-04-26T23:19:18.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.465+0000 7f0aa692b640 1 -- 172.21.15.47:0/3238092799 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0aa00cd520 con 0x7f0aa015f170 2024-04-26T23:19:18.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.465+0000 7f0a9cff9640 1 -- 172.21.15.47:0/3238092799 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a90037040 con 0x7f0aa015f170 2024-04-26T23:19:18.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.465+0000 7f0aa692b640 1 -- 172.21.15.47:0/3238092799 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0aa00cdae0 con 0x7f0aa015f170 2024-04-26T23:19:18.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.466+0000 7f0aa692b640 1 -- 172.21.15.47:0/3238092799 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0aa00694f0 con 0x7f0aa015f170 2024-04-26T23:19:18.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.466+0000 7f0a9cff9640 1 -- 172.21.15.47:0/3238092799 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0a90044020 con 0x7f0aa015f170 2024-04-26T23:19:18.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.467+0000 7f0a9cff9640 1 --2- 172.21.15.47:0/3238092799 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f0a8c06c8f0 0x7f0a8c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:18.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.467+0000 7f0a9cff9640 1 -- 172.21.15.47:0/3238092799 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f0a900ac890 con 0x7f0aa015f170 2024-04-26T23:19:18.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.467+0000 7f0a9ffff640 1 --2- 172.21.15.47:0/3238092799 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f0a8c06c8f0 0x7f0a8c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:18.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.467+0000 7f0a9ffff640 1 --2- 172.21.15.47:0/3238092799 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f0a8c06c8f0 0x7f0a8c06edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0a94001640 tx=0x7f0a94009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:18.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.471+0000 7f0a9cff9640 1 -- 172.21.15.47:0/3238092799 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0a9004b030 con 0x7f0aa015f170 2024-04-26T23:19:18.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.654+0000 7f0aa692b640 1 -- 172.21.15.47:0/3238092799 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0aa0074590 con 0x7f0aa015f170 2024-04-26T23:19:18.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.655+0000 7f0a9cff9640 1 -- 172.21.15.47:0/3238092799 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f0aa014b8a0 con 0x7f0aa015f170 2024-04-26T23:19:18.656 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-26T23:19:18.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.656+0000 7f0aa692b640 1 -- 172.21.15.47:0/3238092799 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f0a8c06c8f0 msgr2=0x7f0a8c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:18.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.656+0000 7f0aa692b640 1 --2- 172.21.15.47:0/3238092799 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f0a8c06c8f0 0x7f0a8c06edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0a94001640 tx=0x7f0a94009040 comp rx=0 tx=0).stop 2024-04-26T23:19:18.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.657+0000 7f0aa692b640 1 -- 172.21.15.47:0/3238092799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0aa015f170 msgr2=0x7f0aa00cc9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:18.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.657+0000 7f0aa692b640 1 --2- 172.21.15.47:0/3238092799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0aa015f170 0x7f0aa00cc9b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0a90000c00 tx=0x7f0a900046c0 comp rx=0 tx=0).stop 2024-04-26T23:19:18.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.657+0000 7f0aa692b640 1 -- 172.21.15.47:0/3238092799 shutdown_connections 2024-04-26T23:19:18.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.657+0000 7f0aa692b640 1 --2- 172.21.15.47:0/3238092799 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f0a8c06c8f0 0x7f0a8c06edb0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:18.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.657+0000 7f0aa692b640 1 --2- 172.21.15.47:0/3238092799 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0aa00ccef0 0x7f0aa01ea000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:18.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.657+0000 7f0aa692b640 1 --2- 172.21.15.47:0/3238092799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0aa015f170 0x7f0aa00cc9b0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:18.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.657+0000 7f0aa692b640 1 --2- 172.21.15.47:0/3238092799 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0aa00d3bc0 0x7f0aa014b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:18.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.657+0000 7f0aa692b640 1 -- 172.21.15.47:0/3238092799 >> 172.21.15.47:0/3238092799 conn(0x7f0aa00d4f60 msgr2=0x7f0aa00d2300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:18.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.657+0000 7f0aa692b640 1 -- 172.21.15.47:0/3238092799 shutdown_connections 2024-04-26T23:19:18.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.657+0000 7f0aa692b640 1 -- 172.21.15.47:0/3238092799 wait complete. 2024-04-26T23:19:18.669 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:19:18.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.771+0000 7fe0b4cda640 1 Processor -- start 2024-04-26T23:19:18.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.771+0000 7fe0b4cda640 1 -- start start 2024-04-26T23:19:18.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.772+0000 7fe0b4cda640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0b015f170 0x7fe0b00d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:18.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.772+0000 7fe0b4cda640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0b00d1040 con 0x7fe0b00d3bc0 2024-04-26T23:19:18.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.772+0000 7fe0b4cda640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0b00c9680 con 0x7fe0b015f550 2024-04-26T23:19:18.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.772+0000 7fe0b4cda640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fe0b00c9430 con 0x7fe0b015f170 2024-04-26T23:19:18.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.772+0000 7fe0aed9d640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe0b00d3bc0 0x7fe0b00ca440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33054/0 (socket says 172.21.15.47:33054) 2024-04-26T23:19:18.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.772+0000 7fe0aed9d640 1 -- 172.21.15.47:0/3185048669 learned_addr learned my addr 172.21.15.47:0/3185048669 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:18.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.772+0000 7fe0ae59c640 1 --2- 172.21.15.47:0/3185048669 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0b015f170 0x7fe0b00d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:18.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.772+0000 7fe0ad59a640 1 -- 172.21.15.47:0/3185048669 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 889789750 0 0) 0x7fe0a8001110 con 0x7fe0b00d3bc0 2024-04-26T23:19:18.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.772+0000 7fe0ad59a640 1 -- 172.21.15.47:0/3185048669 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe09c003520 con 0x7fe0b00d3bc0 2024-04-26T23:19:18.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.773+0000 7fe0ad59a640 1 -- 172.21.15.47:0/3185048669 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2162957854 0 0) 0x7fe0b00c9680 con 0x7fe0b015f550 2024-04-26T23:19:18.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.773+0000 7fe0ad59a640 1 -- 172.21.15.47:0/3185048669 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe0a8001110 con 0x7fe0b015f550 2024-04-26T23:19:18.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.773+0000 7fe0ae59c640 1 -- 172.21.15.47:0/3185048669 >> v1:172.21.15.203:6789/0 conn(0x7fe0b015f550 legacy=0x7fe0b00d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:18.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.773+0000 7fe0ae59c640 1 -- 172.21.15.47:0/3185048669 >> v1:172.21.15.47:6789/0 conn(0x7fe0b00d3bc0 legacy=0x7fe0b00ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:18.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.773+0000 7fe0ad59a640 1 -- 172.21.15.47:0/3185048669 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4214195447 0 0) 0x7fe09c003520 con 0x7fe0b00d3bc0 2024-04-26T23:19:18.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.773+0000 7fe0ae59c640 1 -- 172.21.15.47:0/3185048669 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0b00c9960 con 0x7fe0b015f170 2024-04-26T23:19:18.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.773+0000 7fe0ae59c640 1 --2- 172.21.15.47:0/3185048669 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0b015f170 0x7fe0b00d5a80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0009dc0 tx=0x7fe0a002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40ff5e0df6f361ab server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:18.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.774+0000 7fe0ad59a640 1 -- 172.21.15.47:0/3185048669 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a8001840 con 0x7fe0b015f170 2024-04-26T23:19:18.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.774+0000 7fe0ad59a640 1 -- 172.21.15.47:0/3185048669 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0a0007540 con 0x7fe0b015f170 2024-04-26T23:19:18.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.774+0000 7fe0ad59a640 1 -- 172.21.15.47:0/3185048669 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a0007910 con 0x7fe0b015f170 2024-04-26T23:19:18.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.774+0000 7fe0b4cda640 1 -- 172.21.15.47:0/3185048669 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0b015f170 msgr2=0x7fe0b00d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:18.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.774+0000 7fe0b4cda640 1 --2- 172.21.15.47:0/3185048669 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0b015f170 0x7fe0b00d5a80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0009dc0 tx=0x7fe0a002a660 comp rx=0 tx=0).stop 2024-04-26T23:19:18.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.774+0000 7fe0b4cda640 1 -- 172.21.15.47:0/3185048669 shutdown_connections 2024-04-26T23:19:18.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.774+0000 7fe0b4cda640 1 --2- 172.21.15.47:0/3185048669 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0b015f170 0x7fe0b00d5a80 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:18.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.774+0000 7fe0b4cda640 1 -- 172.21.15.47:0/3185048669 >> 172.21.15.47:0/3185048669 conn(0x7fe0b00d4f60 msgr2=0x7fe0b00d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:18.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.774+0000 7fe0b4cda640 1 -- 172.21.15.47:0/3185048669 shutdown_connections 2024-04-26T23:19:18.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.774+0000 7fe0b4cda640 1 -- 172.21.15.47:0/3185048669 wait complete. 2024-04-26T23:19:18.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.776+0000 7fe0b4cda640 1 Processor -- start 2024-04-26T23:19:18.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.776+0000 7fe0b4cda640 1 -- start start 2024-04-26T23:19:18.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.776+0000 7fe0b4cda640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0b00d3bc0 0x7fe0b0155400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:18.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.776+0000 7fe0b4cda640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0b015f170 0x7fe0b00a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:18.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.776+0000 7fe0b4cda640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0b00cddf0 0x7fe0b00ce260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:18.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.776+0000 7fe0b4cda640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b00d2e30 con 0x7fe0b00d3bc0 2024-04-26T23:19:18.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.776+0000 7fe0b4cda640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b00d2cb0 con 0x7fe0b00cddf0 2024-04-26T23:19:18.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.776+0000 7fe0b4cda640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fe0b00d2fb0 con 0x7fe0b015f170 2024-04-26T23:19:18.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.776+0000 7fe0ae59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0b00d3bc0 0x7fe0b0155400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:18.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.776+0000 7fe0ae59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0b00d3bc0 0x7fe0b0155400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41486/0 (socket says 172.21.15.47:41486) 2024-04-26T23:19:18.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.777+0000 7fe0ae59c640 1 -- 172.21.15.47:0/885894798 learned_addr learned my addr 172.21.15.47:0/885894798 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:18.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.777+0000 7fe0add9b640 1 --2- 172.21.15.47:0/885894798 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0b015f170 0x7fe0b00a08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:18.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.777+0000 7fe0aed9d640 1 --2- 172.21.15.47:0/885894798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0b00cddf0 0x7fe0b00ce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:18.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.777+0000 7fe0ae59c640 1 -- 172.21.15.47:0/885894798 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0b015f170 msgr2=0x7fe0b00a08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:18.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.777+0000 7fe0ae59c640 1 --2- 172.21.15.47:0/885894798 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0b015f170 0x7fe0b00a08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:18.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.777+0000 7fe0ae59c640 1 -- 172.21.15.47:0/885894798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0b00cddf0 msgr2=0x7fe0b00ce260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:18.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.777+0000 7fe0ae59c640 1 --2- 172.21.15.47:0/885894798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0b00cddf0 0x7fe0b00ce260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:18.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.777+0000 7fe0ae59c640 1 -- 172.21.15.47:0/885894798 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0b00a2850 con 0x7fe0b00d3bc0 2024-04-26T23:19:18.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.777+0000 7fe0add9b640 1 --2- 172.21.15.47:0/885894798 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0b015f170 0x7fe0b00a08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:19:18.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.777+0000 7fe0ae59c640 1 --2- 172.21.15.47:0/885894798 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0b00d3bc0 0x7fe0b0155400 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0000f80 tx=0x7fe0a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:18.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.777+0000 7fe0a6ffd640 1 -- 172.21.15.47:0/885894798 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a0040070 con 0x7fe0b00d3bc0 2024-04-26T23:19:18.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.777+0000 7fe0a6ffd640 1 -- 172.21.15.47:0/885894798 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0a0031c40 con 0x7fe0b00d3bc0 2024-04-26T23:19:18.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.777+0000 7fe0b4cda640 1 -- 172.21.15.47:0/885894798 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0b009f840 con 0x7fe0b00d3bc0 2024-04-26T23:19:18.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.777+0000 7fe0a6ffd640 1 -- 172.21.15.47:0/885894798 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a0009450 con 0x7fe0b00d3bc0 2024-04-26T23:19:18.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.777+0000 7fe0b4cda640 1 -- 172.21.15.47:0/885894798 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0b009fe00 con 0x7fe0b00d3bc0 2024-04-26T23:19:18.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.778+0000 7fe0b4cda640 1 -- 172.21.15.47:0/885894798 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0b00694f0 con 0x7fe0b00d3bc0 2024-04-26T23:19:18.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.779+0000 7fe0a6ffd640 1 -- 172.21.15.47:0/885894798 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe0a0039040 con 0x7fe0b00d3bc0 2024-04-26T23:19:18.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.779+0000 7fe0a6ffd640 1 --2- 172.21.15.47:0/885894798 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fe09006c8f0 0x7fe09006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:18.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.779+0000 7fe0a6ffd640 1 -- 172.21.15.47:0/885894798 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7fe0a00ad730 con 0x7fe0b00d3bc0 2024-04-26T23:19:18.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.779+0000 7fe0add9b640 1 --2- 172.21.15.47:0/885894798 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fe09006c8f0 0x7fe09006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:18.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.780+0000 7fe0add9b640 1 --2- 172.21.15.47:0/885894798 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fe09006c8f0 0x7fe09006edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe0b00c8380 tx=0x7fe098009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:18.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.783+0000 7fe0a6ffd640 1 -- 172.21.15.47:0/885894798 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe0b00a2850 con 0x7fe0b00d3bc0 2024-04-26T23:19:18.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.968+0000 7fe0b4cda640 1 -- 172.21.15.47:0/885894798 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe0b0069b20 con 0x7fe0b00d3bc0 2024-04-26T23:19:18.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.969+0000 7fe0a6ffd640 1 -- 172.21.15.47:0/885894798 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7fe0a0043360 con 0x7fe0b00d3bc0 2024-04-26T23:19:18.971 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-26T23:19:18.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.970+0000 7fe0b4cda640 1 -- 172.21.15.47:0/885894798 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fe09006c8f0 msgr2=0x7fe09006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:18.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.970+0000 7fe0b4cda640 1 --2- 172.21.15.47:0/885894798 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fe09006c8f0 0x7fe09006edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe0b00c8380 tx=0x7fe098009040 comp rx=0 tx=0).stop 2024-04-26T23:19:18.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.970+0000 7fe0b4cda640 1 -- 172.21.15.47:0/885894798 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0b00d3bc0 msgr2=0x7fe0b0155400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:18.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.970+0000 7fe0b4cda640 1 --2- 172.21.15.47:0/885894798 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0b00d3bc0 0x7fe0b0155400 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0000f80 tx=0x7fe0a0004a40 comp rx=0 tx=0).stop 2024-04-26T23:19:18.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.970+0000 7fe0b4cda640 1 -- 172.21.15.47:0/885894798 shutdown_connections 2024-04-26T23:19:18.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.970+0000 7fe0b4cda640 1 --2- 172.21.15.47:0/885894798 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fe09006c8f0 0x7fe09006edb0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:18.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.970+0000 7fe0b4cda640 1 --2- 172.21.15.47:0/885894798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0b00cddf0 0x7fe0b00ce260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:18.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.970+0000 7fe0b4cda640 1 --2- 172.21.15.47:0/885894798 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0b015f170 0x7fe0b00a08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:18.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.970+0000 7fe0b4cda640 1 --2- 172.21.15.47:0/885894798 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0b00d3bc0 0x7fe0b0155400 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:18.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.970+0000 7fe0b4cda640 1 -- 172.21.15.47:0/885894798 >> 172.21.15.47:0/885894798 conn(0x7fe0b00d4f60 msgr2=0x7fe0b00d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:18.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.971+0000 7fe0b4cda640 1 -- 172.21.15.47:0/885894798 shutdown_connections 2024-04-26T23:19:18.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:18.971+0000 7fe0b4cda640 1 -- 172.21.15.47:0/885894798 wait complete. 2024-04-26T23:19:18.983 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T23:19:19.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.087+0000 7fd5f75af640 1 Processor -- start 2024-04-26T23:19:19.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.087+0000 7fd5f75af640 1 -- start start 2024-04-26T23:19:19.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.088+0000 7fd5f75af640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd5f015f550 0x7fd5f00d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:19.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.088+0000 7fd5f75af640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5f00d1040 con 0x7fd5f00d3bc0 2024-04-26T23:19:19.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.088+0000 7fd5f75af640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5f00c9680 con 0x7fd5f015f170 2024-04-26T23:19:19.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.088+0000 7fd5f75af640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fd5f00c9430 con 0x7fd5f015f550 2024-04-26T23:19:19.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.088+0000 7fd5f5b4c640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd5f00d3bc0 0x7fd5f00ca440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33060/0 (socket says 172.21.15.47:33060) 2024-04-26T23:19:19.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.088+0000 7fd5f5b4c640 1 -- 172.21.15.47:0/1452783216 learned_addr learned my addr 172.21.15.47:0/1452783216 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:19.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.088+0000 7fd5f4b4a640 1 --2- 172.21.15.47:0/1452783216 >> v2:172.21.15.203:3300/0 conn(0x7fd5f015f550 0x7fd5f00d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:19.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.088+0000 7fd5effff640 1 -- 172.21.15.47:0/1452783216 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1665978078 0 0) 0x7fd5e0001490 con 0x7fd5f00d3bc0 2024-04-26T23:19:19.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.088+0000 7fd5effff640 1 -- 172.21.15.47:0/1452783216 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5d8003520 con 0x7fd5f00d3bc0 2024-04-26T23:19:19.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.088+0000 7fd5effff640 1 -- 172.21.15.47:0/1452783216 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1332079112 0 0) 0x7fd5f00c9680 con 0x7fd5f015f170 2024-04-26T23:19:19.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.088+0000 7fd5effff640 1 -- 172.21.15.47:0/1452783216 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5e0001490 con 0x7fd5f015f170 2024-04-26T23:19:19.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.088+0000 7fd5effff640 1 -- 172.21.15.47:0/1452783216 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2154295079 0 0) 0x7fd5d8003520 con 0x7fd5f00d3bc0 2024-04-26T23:19:19.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.088+0000 7fd5effff640 1 -- 172.21.15.47:0/1452783216 >> v2:172.21.15.203:3300/0 conn(0x7fd5f015f550 msgr2=0x7fd5f00d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:19.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.088+0000 7fd5effff640 1 --2- 172.21.15.47:0/1452783216 >> v2:172.21.15.203:3300/0 conn(0x7fd5f015f550 0x7fd5f00d5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:19.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.088+0000 7fd5effff640 1 -- 172.21.15.47:0/1452783216 >> v1:172.21.15.203:6789/0 conn(0x7fd5f015f170 legacy=0x7fd5f00d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:19.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.088+0000 7fd5effff640 1 -- 172.21.15.47:0/1452783216 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5f00c9960 con 0x7fd5f00d3bc0 2024-04-26T23:19:19.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.088+0000 7fd5effff640 1 -- 172.21.15.47:0/1452783216 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7fd5e0001b40 con 0x7fd5f00d3bc0 2024-04-26T23:19:19.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.088+0000 7fd5effff640 1 -- 172.21.15.47:0/1452783216 >> v1:172.21.15.47:6789/0 conn(0x7fd5f00d3bc0 legacy=0x7fd5f00ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:19.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.088+0000 7fd5effff640 1 --2- 172.21.15.47:0/1452783216 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5d8003bc0 0x7fd5d8003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:19.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.088+0000 7fd5effff640 1 --2- 172.21.15.47:0/1452783216 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5d8004800 0x7fd5d8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:19.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.088+0000 7fd5effff640 1 --2- 172.21.15.47:0/1452783216 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5d80051c0 0x7fd5d800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:19.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.088+0000 7fd5effff640 1 -- 172.21.15.47:0/1452783216 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f00c9430 con 0x7fd5d80051c0 2024-04-26T23:19:19.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.088+0000 7fd5effff640 1 -- 172.21.15.47:0/1452783216 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d800c0f0 con 0x7fd5d8004800 2024-04-26T23:19:19.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.088+0000 7fd5effff640 1 -- 172.21.15.47:0/1452783216 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd5d800c2f0 con 0x7fd5d8003bc0 2024-04-26T23:19:19.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.088+0000 7fd5f5b4c640 1 --2- 172.21.15.47:0/1452783216 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5d80051c0 0x7fd5d800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:19.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.088+0000 7fd5f4b4a640 1 --2- 172.21.15.47:0/1452783216 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5d8004800 0x7fd5d8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:19.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.088+0000 7fd5f534b640 1 --2- 172.21.15.47:0/1452783216 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5d8003bc0 0x7fd5d8003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:19.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.088+0000 7fd5f5b4c640 1 -- 172.21.15.47:0/1452783216 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5d8003bc0 msgr2=0x7fd5d8003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:19.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.088+0000 7fd5f5b4c640 1 --2- 172.21.15.47:0/1452783216 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5d8003bc0 0x7fd5d8003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:19.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.088+0000 7fd5f5b4c640 1 -- 172.21.15.47:0/1452783216 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5d8004800 msgr2=0x7fd5d8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:19.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.088+0000 7fd5f5b4c640 1 --2- 172.21.15.47:0/1452783216 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5d8004800 0x7fd5d8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:19.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.088+0000 7fd5f5b4c640 1 -- 172.21.15.47:0/1452783216 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5d8003520 con 0x7fd5d80051c0 2024-04-26T23:19:19.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.089+0000 7fd5f5b4c640 1 -- 172.21.15.47:0/1452783216 reap_dead start 2024-04-26T23:19:19.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.089+0000 7fd5f5b4c640 1 --2- 172.21.15.47:0/1452783216 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5d80051c0 0x7fd5d800b9c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd5e000a880 tx=0x7fd5d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad6b332f7c30d94e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:19.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.089+0000 7fd5effff640 1 -- 172.21.15.47:0/1452783216 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e000a4a0 con 0x7fd5d80051c0 2024-04-26T23:19:19.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.089+0000 7fd5effff640 1 -- 172.21.15.47:0/1452783216 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5e002f070 con 0x7fd5d80051c0 2024-04-26T23:19:19.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.089+0000 7fd5effff640 1 -- 172.21.15.47:0/1452783216 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e002bd00 con 0x7fd5d80051c0 2024-04-26T23:19:19.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.091+0000 7fd5f75af640 1 -- 172.21.15.47:0/1452783216 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5d80051c0 msgr2=0x7fd5d800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:19.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.091+0000 7fd5f75af640 1 --2- 172.21.15.47:0/1452783216 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5d80051c0 0x7fd5d800b9c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd5e000a880 tx=0x7fd5d8005830 comp rx=0 tx=0).stop 2024-04-26T23:19:19.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.091+0000 7fd5f75af640 1 -- 172.21.15.47:0/1452783216 shutdown_connections 2024-04-26T23:19:19.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.091+0000 7fd5f75af640 1 --2- 172.21.15.47:0/1452783216 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5d80051c0 0x7fd5d800b9c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:19.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.091+0000 7fd5f75af640 1 -- 172.21.15.47:0/1452783216 >> 172.21.15.47:0/1452783216 conn(0x7fd5f00d4f60 msgr2=0x7fd5f00d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:19.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.091+0000 7fd5f75af640 1 -- 172.21.15.47:0/1452783216 shutdown_connections 2024-04-26T23:19:19.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.091+0000 7fd5f75af640 1 -- 172.21.15.47:0/1452783216 wait complete. 2024-04-26T23:19:19.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.092+0000 7fd5f75af640 1 Processor -- start 2024-04-26T23:19:19.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.092+0000 7fd5f75af640 1 -- start start 2024-04-26T23:19:19.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.093+0000 7fd5f75af640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5d80051c0 0x7fd5f01e91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:19.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.093+0000 7fd5f75af640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5f0158d80 0x7fd5f015cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:19.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.093+0000 7fd5f75af640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5f0159200 0x7fd5f015d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:19.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.093+0000 7fd5f75af640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f00d5c00 con 0x7fd5f0158d80 2024-04-26T23:19:19.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.093+0000 7fd5f75af640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f00d5a80 con 0x7fd5d80051c0 2024-04-26T23:19:19.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.093+0000 7fd5f75af640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd5f00d5d80 con 0x7fd5f0159200 2024-04-26T23:19:19.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.093+0000 7fd5f4b4a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5f0158d80 0x7fd5f015cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:19.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.093+0000 7fd5f4b4a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5f0158d80 0x7fd5f015cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41516/0 (socket says 172.21.15.47:41516) 2024-04-26T23:19:19.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.093+0000 7fd5f4b4a640 1 -- 172.21.15.47:0/3588738789 learned_addr learned my addr 172.21.15.47:0/3588738789 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:19.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.093+0000 7fd5f534b640 1 --2- 172.21.15.47:0/3588738789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5d80051c0 0x7fd5f01e91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:19.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.093+0000 7fd5f5b4c640 1 --2- 172.21.15.47:0/3588738789 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5f0159200 0x7fd5f015d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:19.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.093+0000 7fd5f4b4a640 1 -- 172.21.15.47:0/3588738789 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5f0159200 msgr2=0x7fd5f015d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:19.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.093+0000 7fd5f4b4a640 1 --2- 172.21.15.47:0/3588738789 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5f0159200 0x7fd5f015d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:19.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.093+0000 7fd5f4b4a640 1 -- 172.21.15.47:0/3588738789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5d80051c0 msgr2=0x7fd5f01e91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:19.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.093+0000 7fd5f4b4a640 1 --2- 172.21.15.47:0/3588738789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5d80051c0 0x7fd5f01e91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:19.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.093+0000 7fd5f4b4a640 1 -- 172.21.15.47:0/3588738789 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5f01f9150 con 0x7fd5f0158d80 2024-04-26T23:19:19.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.093+0000 7fd5f4b4a640 1 --2- 172.21.15.47:0/3588738789 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5f0158d80 0x7fd5f015cf40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4011930 tx=0x7fd5e4011df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:19.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.093+0000 7fd5edffb640 1 -- 172.21.15.47:0/3588738789 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e400f9d0 con 0x7fd5f0158d80 2024-04-26T23:19:19.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.093+0000 7fd5edffb640 1 -- 172.21.15.47:0/3588738789 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5e400fb70 con 0x7fd5f0158d80 2024-04-26T23:19:19.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.093+0000 7fd5edffb640 1 -- 172.21.15.47:0/3588738789 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e4014440 con 0x7fd5f0158d80 2024-04-26T23:19:19.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.093+0000 7fd5f75af640 1 -- 172.21.15.47:0/3588738789 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5f01e86e0 con 0x7fd5f0158d80 2024-04-26T23:19:19.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.093+0000 7fd5f75af640 1 -- 172.21.15.47:0/3588738789 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5f01e8ca0 con 0x7fd5f0158d80 2024-04-26T23:19:19.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.094+0000 7fd5f75af640 1 -- 172.21.15.47:0/3588738789 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5f00694f0 con 0x7fd5f0158d80 2024-04-26T23:19:19.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.094+0000 7fd5edffb640 1 -- 172.21.15.47:0/3588738789 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd5e4012040 con 0x7fd5f0158d80 2024-04-26T23:19:19.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.095+0000 7fd5edffb640 1 --2- 172.21.15.47:0/3588738789 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd5dc06c920 0x7fd5dc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:19.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.095+0000 7fd5edffb640 1 -- 172.21.15.47:0/3588738789 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7fd5e408c360 con 0x7fd5f0158d80 2024-04-26T23:19:19.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.095+0000 7fd5f534b640 1 --2- 172.21.15.47:0/3588738789 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd5dc06c920 0x7fd5dc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:19.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.095+0000 7fd5f534b640 1 --2- 172.21.15.47:0/3588738789 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd5dc06c920 0x7fd5dc06ede0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8001920 tx=0x7fd5e8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:19.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.099+0000 7fd5edffb640 1 -- 172.21.15.47:0/3588738789 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd5e4019030 con 0x7fd5f0158d80 2024-04-26T23:19:19.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.283+0000 7fd5f75af640 1 -- 172.21.15.47:0/3588738789 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fd5f00caaa0 con 0x7fd5f0158d80 2024-04-26T23:19:19.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.283+0000 7fd5edffb640 1 -- 172.21.15.47:0/3588738789 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fd5e4060030 con 0x7fd5f0158d80 2024-04-26T23:19:19.285 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-26T23:19:19.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.285+0000 7fd5f75af640 1 -- 172.21.15.47:0/3588738789 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd5dc06c920 msgr2=0x7fd5dc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:19.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.285+0000 7fd5f75af640 1 --2- 172.21.15.47:0/3588738789 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd5dc06c920 0x7fd5dc06ede0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8001920 tx=0x7fd5e8009040 comp rx=0 tx=0).stop 2024-04-26T23:19:19.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.285+0000 7fd5f75af640 1 -- 172.21.15.47:0/3588738789 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5f0158d80 msgr2=0x7fd5f015cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:19.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.285+0000 7fd5f75af640 1 --2- 172.21.15.47:0/3588738789 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5f0158d80 0x7fd5f015cf40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4011930 tx=0x7fd5e4011df0 comp rx=0 tx=0).stop 2024-04-26T23:19:19.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.285+0000 7fd5f75af640 1 -- 172.21.15.47:0/3588738789 shutdown_connections 2024-04-26T23:19:19.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.285+0000 7fd5f75af640 1 --2- 172.21.15.47:0/3588738789 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd5dc06c920 0x7fd5dc06ede0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:19.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.285+0000 7fd5f75af640 1 --2- 172.21.15.47:0/3588738789 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5f0159200 0x7fd5f015d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:19.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.285+0000 7fd5f75af640 1 --2- 172.21.15.47:0/3588738789 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5f0158d80 0x7fd5f015cf40 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:19.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.285+0000 7fd5f75af640 1 --2- 172.21.15.47:0/3588738789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5d80051c0 0x7fd5f01e91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:19.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.285+0000 7fd5f75af640 1 -- 172.21.15.47:0/3588738789 >> 172.21.15.47:0/3588738789 conn(0x7fd5f00d4f60 msgr2=0x7fd5f006d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:19.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.285+0000 7fd5f75af640 1 -- 172.21.15.47:0/3588738789 shutdown_connections 2024-04-26T23:19:19.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.285+0000 7fd5f75af640 1 -- 172.21.15.47:0/3588738789 wait complete. 2024-04-26T23:19:19.298 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification' 2024-04-26T23:19:19.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.397+0000 7fba206e1640 1 Processor -- start 2024-04-26T23:19:19.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.397+0000 7fba206e1640 1 -- start start 2024-04-26T23:19:19.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.399+0000 7fba206e1640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fba18162880 0x7fba180d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:19.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.399+0000 7fba206e1640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fba180d3bc0 0x7fba180b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:19.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.399+0000 7fba206e1640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba180b7e50 con 0x7fba180b5f20 2024-04-26T23:19:19.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.399+0000 7fba206e1640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fba180d0580 con 0x7fba18162880 2024-04-26T23:19:19.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.399+0000 7fba206e1640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fba180d1800 con 0x7fba180d3bc0 2024-04-26T23:19:19.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.399+0000 7fba1e47d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fba18162880 0x7fba180d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:19.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.399+0000 7fba1dc7c640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fba180b5f20 0x7fba180d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33074/0 (socket says 172.21.15.47:33074) 2024-04-26T23:19:19.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.399+0000 7fba1dc7c640 1 -- 172.21.15.47:0/3014632794 learned_addr learned my addr 172.21.15.47:0/3014632794 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:19.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.399+0000 7fba1e47d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fba18162880 0x7fba180d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41526/0 (socket says 172.21.15.47:41526) 2024-04-26T23:19:19.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.399+0000 7fba1ec7e640 1 --2- 172.21.15.47:0/3014632794 >> v2:172.21.15.203:3300/0 conn(0x7fba180d3bc0 0x7fba180b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:19.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.399+0000 7fba1e47d640 1 -- 172.21.15.47:0/3014632794 >> v2:172.21.15.203:3300/0 conn(0x7fba180d3bc0 msgr2=0x7fba180b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:19.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.399+0000 7fba1e47d640 1 --2- 172.21.15.47:0/3014632794 >> v2:172.21.15.203:3300/0 conn(0x7fba180d3bc0 0x7fba180b8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:19.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.399+0000 7fba1e47d640 1 -- 172.21.15.47:0/3014632794 >> v1:172.21.15.47:6789/0 conn(0x7fba180b5f20 legacy=0x7fba180d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:19.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.399+0000 7fba1d47b640 1 -- 172.21.15.47:0/3014632794 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2356450303 0 0) 0x7fba180b7e50 con 0x7fba180b5f20 2024-04-26T23:19:19.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.399+0000 7fba1e47d640 1 -- 172.21.15.47:0/3014632794 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba180d07d0 con 0x7fba18162880 2024-04-26T23:19:19.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.399+0000 7fba1e47d640 1 --2- 172.21.15.47:0/3014632794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba18162880 0x7fba180d2cb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fba180d11d0 tx=0x7fba1002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=287b945370a04776 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:19.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.399+0000 7fba1d47b640 1 -- 172.21.15.47:0/3014632794 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba10037070 con 0x7fba18162880 2024-04-26T23:19:19.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.399+0000 7fba1d47b640 1 -- 172.21.15.47:0/3014632794 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba10032440 con 0x7fba18162880 2024-04-26T23:19:19.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.399+0000 7fba1d47b640 1 -- 172.21.15.47:0/3014632794 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba10032810 con 0x7fba18162880 2024-04-26T23:19:19.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.400+0000 7fba206e1640 1 -- 172.21.15.47:0/3014632794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba18162880 msgr2=0x7fba180d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:19.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.400+0000 7fba206e1640 1 --2- 172.21.15.47:0/3014632794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba18162880 0x7fba180d2cb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fba180d11d0 tx=0x7fba1002a1a0 comp rx=0 tx=0).stop 2024-04-26T23:19:19.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.400+0000 7fba206e1640 1 -- 172.21.15.47:0/3014632794 shutdown_connections 2024-04-26T23:19:19.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.400+0000 7fba206e1640 1 --2- 172.21.15.47:0/3014632794 >> v2:172.21.15.203:3300/0 conn(0x7fba180d3bc0 0x7fba180b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:19.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.400+0000 7fba206e1640 1 --2- 172.21.15.47:0/3014632794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba18162880 0x7fba180d2cb0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:19.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.400+0000 7fba206e1640 1 -- 172.21.15.47:0/3014632794 >> 172.21.15.47:0/3014632794 conn(0x7fba180d4f60 msgr2=0x7fba180d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:19.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.400+0000 7fba206e1640 1 -- 172.21.15.47:0/3014632794 shutdown_connections 2024-04-26T23:19:19.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.400+0000 7fba206e1640 1 -- 172.21.15.47:0/3014632794 wait complete. 2024-04-26T23:19:19.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.401+0000 7fba206e1640 1 Processor -- start 2024-04-26T23:19:19.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.401+0000 7fba206e1640 1 -- start start 2024-04-26T23:19:19.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.402+0000 7fba206e1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba180b5f20 0x7fba181f4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:19.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.402+0000 7fba206e1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba180d3bc0 0x7fba180b4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:19.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.402+0000 7fba206e1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba18162880 0x7fba180b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:19.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.402+0000 7fba206e1640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fba18162c60 con 0x7fba18162880 2024-04-26T23:19:19.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.402+0000 7fba206e1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fba180d0d90 con 0x7fba180d3bc0 2024-04-26T23:19:19.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.402+0000 7fba206e1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fba18162de0 con 0x7fba180b5f20 2024-04-26T23:19:19.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.402+0000 7fba1ec7e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba18162880 0x7fba180b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:19.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.402+0000 7fba1ec7e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba18162880 0x7fba180b4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41528/0 (socket says 172.21.15.47:41528) 2024-04-26T23:19:19.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.402+0000 7fba1ec7e640 1 -- 172.21.15.47:0/4022682079 learned_addr learned my addr 172.21.15.47:0/4022682079 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:19.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.402+0000 7fba1dc7c640 1 --2- 172.21.15.47:0/4022682079 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba180d3bc0 0x7fba180b4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:19.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.402+0000 7fba1e47d640 1 --2- 172.21.15.47:0/4022682079 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba180b5f20 0x7fba181f4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:19.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.402+0000 7fba1ec7e640 1 -- 172.21.15.47:0/4022682079 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba180b5f20 msgr2=0x7fba181f4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:19.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.402+0000 7fba1ec7e640 1 --2- 172.21.15.47:0/4022682079 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba180b5f20 0x7fba181f4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:19.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.402+0000 7fba1ec7e640 1 -- 172.21.15.47:0/4022682079 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba180d3bc0 msgr2=0x7fba180b4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:19.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.402+0000 7fba1ec7e640 1 --2- 172.21.15.47:0/4022682079 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba180d3bc0 0x7fba180b4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:19.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.402+0000 7fba1ec7e640 1 -- 172.21.15.47:0/4022682079 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba181549c0 con 0x7fba18162880 2024-04-26T23:19:19.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.402+0000 7fba1dc7c640 1 --2- 172.21.15.47:0/4022682079 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba180d3bc0 0x7fba180b4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:19:19.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.403+0000 7fba1ec7e640 1 --2- 172.21.15.47:0/4022682079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba18162880 0x7fba180b4cc0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fba14007d80 tx=0x7fba1400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:19.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.403+0000 7fba0effd640 1 -- 172.21.15.47:0/4022682079 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba14014420 con 0x7fba18162880 2024-04-26T23:19:19.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.403+0000 7fba0effd640 1 -- 172.21.15.47:0/4022682079 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba14019070 con 0x7fba18162880 2024-04-26T23:19:19.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.403+0000 7fba0effd640 1 -- 172.21.15.47:0/4022682079 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba14014800 con 0x7fba18162880 2024-04-26T23:19:19.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.403+0000 7fba206e1640 1 -- 172.21.15.47:0/4022682079 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba180ce5e0 con 0x7fba18162880 2024-04-26T23:19:19.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.403+0000 7fba206e1640 1 -- 172.21.15.47:0/4022682079 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba180b1ec0 con 0x7fba18162880 2024-04-26T23:19:19.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.405+0000 7fba206e1640 1 -- 172.21.15.47:0/4022682079 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba180694f0 con 0x7fba18162880 2024-04-26T23:19:19.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.405+0000 7fba0effd640 1 -- 172.21.15.47:0/4022682079 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fba140149a0 con 0x7fba18162880 2024-04-26T23:19:19.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.405+0000 7fba0effd640 1 --2- 172.21.15.47:0/4022682079 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb9fc06c8f0 0x7fb9fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:19.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.405+0000 7fba0effd640 1 -- 172.21.15.47:0/4022682079 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7fba1408c710 con 0x7fba18162880 2024-04-26T23:19:19.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.405+0000 7fba1e47d640 1 --2- 172.21.15.47:0/4022682079 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb9fc06c8f0 0x7fb9fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:19.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.406+0000 7fba1e47d640 1 --2- 172.21.15.47:0/4022682079 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb9fc06c8f0 0x7fb9fc06edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fba10034000 tx=0x7fba1002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:19.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.410+0000 7fba0effd640 1 -- 172.21.15.47:0/4022682079 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fba181549c0 con 0x7fba18162880 2024-04-26T23:19:19.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:19.666+0000 7fba206e1640 1 -- 172.21.15.47:0/4022682079 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]} v 0) -- 0x7fba180ba2e0 con 0x7fba18162880 2024-04-26T23:19:20.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.156+0000 7fba0effd640 1 -- 172.21.15.47:0/4022682079 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]}]=0 v69) ==== 150+0+0 (secure 0 0 0) 0x7fba1405a7b0 con 0x7fba18162880 2024-04-26T23:19:20.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.159+0000 7fba206e1640 1 -- 172.21.15.47:0/4022682079 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb9fc06c8f0 msgr2=0x7fb9fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:20.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.159+0000 7fba206e1640 1 --2- 172.21.15.47:0/4022682079 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb9fc06c8f0 0x7fb9fc06edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fba10034000 tx=0x7fba1002e000 comp rx=0 tx=0).stop 2024-04-26T23:19:20.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.159+0000 7fba206e1640 1 -- 172.21.15.47:0/4022682079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba18162880 msgr2=0x7fba180b4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:20.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.159+0000 7fba206e1640 1 --2- 172.21.15.47:0/4022682079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba18162880 0x7fba180b4cc0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fba14007d80 tx=0x7fba1400e4b0 comp rx=0 tx=0).stop 2024-04-26T23:19:20.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.159+0000 7fba206e1640 1 -- 172.21.15.47:0/4022682079 shutdown_connections 2024-04-26T23:19:20.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.159+0000 7fba206e1640 1 --2- 172.21.15.47:0/4022682079 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb9fc06c8f0 0x7fb9fc06edb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:20.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.159+0000 7fba206e1640 1 --2- 172.21.15.47:0/4022682079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba18162880 0x7fba180b4cc0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:20.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.159+0000 7fba206e1640 1 --2- 172.21.15.47:0/4022682079 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba180d3bc0 0x7fba180b4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:20.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.159+0000 7fba206e1640 1 --2- 172.21.15.47:0/4022682079 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba180b5f20 0x7fba181f4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:20.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.159+0000 7fba206e1640 1 -- 172.21.15.47:0/4022682079 >> 172.21.15.47:0/4022682079 conn(0x7fba180d4f60 msgr2=0x7fba18160540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:20.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.159+0000 7fba206e1640 1 -- 172.21.15.47:0/4022682079 shutdown_connections 2024-04-26T23:19:20.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.159+0000 7fba206e1640 1 -- 172.21.15.47:0/4022682079 wait complete. 2024-04-26T23:19:20.173 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T23:19:20.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.281+0000 7f51ba445640 1 Processor -- start 2024-04-26T23:19:20.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.281+0000 7f51ba445640 1 -- start start 2024-04-26T23:19:20.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.281+0000 7f51ba445640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f51b40d1a00 0x7f51b40d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:20.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.281+0000 7f51ba445640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f51b40d2320 0x7f51b41625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:20.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.281+0000 7f51ba445640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51b40b40d0 0x7f51b4162b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:20.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.281+0000 7f51ba445640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f51b4163070 con 0x7f51b40d1a00 2024-04-26T23:19:20.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.281+0000 7f51ba445640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f51b40d0a10 con 0x7f51b40d2320 2024-04-26T23:19:20.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.281+0000 7f51ba445640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f51b40d0cf0 con 0x7f51b40b40d0 2024-04-26T23:19:20.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.282+0000 7f51b3fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f51b40d1a00 0x7f51b40d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:20.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.282+0000 7f51b3fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f51b40d1a00 0x7f51b40d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54536/0 (socket says 172.21.15.47:54536) 2024-04-26T23:19:20.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.282+0000 7f51b3fff640 1 -- 172.21.15.47:0/324656885 learned_addr learned my addr 172.21.15.47:0/324656885 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:20.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.282+0000 7f51b37fe640 1 --2- 172.21.15.47:0/324656885 >> v2:172.21.15.203:3300/0 conn(0x7f51b40d2320 0x7f51b41625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:20.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.282+0000 7f51b89e2640 1 --2- 172.21.15.47:0/324656885 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51b40b40d0 0x7f51b4162b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:20.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.282+0000 7f51b3fff640 1 -- 172.21.15.47:0/324656885 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51b40b40d0 msgr2=0x7f51b4162b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:20.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.282+0000 7f51b3fff640 1 --2- 172.21.15.47:0/324656885 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51b40b40d0 0x7f51b4162b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:20.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.282+0000 7f51b3fff640 1 -- 172.21.15.47:0/324656885 >> v2:172.21.15.203:3300/0 conn(0x7f51b40d2320 msgr2=0x7f51b41625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:20.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.282+0000 7f51b3fff640 1 --2- 172.21.15.47:0/324656885 >> v2:172.21.15.203:3300/0 conn(0x7f51b40d2320 0x7f51b41625f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:20.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.282+0000 7f51b3fff640 1 -- 172.21.15.47:0/324656885 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51b40d0fd0 con 0x7f51b40d1a00 2024-04-26T23:19:20.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.283+0000 7f51b3fff640 1 --2- 172.21.15.47:0/324656885 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51b40d1a00 0x7f51b40d1de0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f51b408d380 tx=0x7f51a402a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c727a35fda1274e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:20.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.283+0000 7f51b2ffd640 1 -- 172.21.15.47:0/324656885 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a4037070 con 0x7f51b40d1a00 2024-04-26T23:19:20.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.283+0000 7f51b2ffd640 1 -- 172.21.15.47:0/324656885 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51a402ae00 con 0x7f51b40d1a00 2024-04-26T23:19:20.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.283+0000 7f51b2ffd640 1 -- 172.21.15.47:0/324656885 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a4032660 con 0x7f51b40d1a00 2024-04-26T23:19:20.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.283+0000 7f51ba445640 1 -- 172.21.15.47:0/324656885 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51b40d1a00 msgr2=0x7f51b40d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:20.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.283+0000 7f51ba445640 1 --2- 172.21.15.47:0/324656885 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51b40d1a00 0x7f51b40d1de0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f51b408d380 tx=0x7f51a402a0b0 comp rx=0 tx=0).stop 2024-04-26T23:19:20.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.283+0000 7f51ba445640 1 -- 172.21.15.47:0/324656885 shutdown_connections 2024-04-26T23:19:20.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.283+0000 7f51ba445640 1 --2- 172.21.15.47:0/324656885 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51b40b40d0 0x7f51b4162b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:20.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.283+0000 7f51ba445640 1 --2- 172.21.15.47:0/324656885 >> v2:172.21.15.203:3300/0 conn(0x7f51b40d2320 0x7f51b41625f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:20.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.283+0000 7f51ba445640 1 --2- 172.21.15.47:0/324656885 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51b40d1a00 0x7f51b40d1de0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:20.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.283+0000 7f51ba445640 1 -- 172.21.15.47:0/324656885 >> 172.21.15.47:0/324656885 conn(0x7f51b40d4fd0 msgr2=0x7f51b40d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:20.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.283+0000 7f51ba445640 1 -- 172.21.15.47:0/324656885 shutdown_connections 2024-04-26T23:19:20.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.283+0000 7f51ba445640 1 -- 172.21.15.47:0/324656885 wait complete. 2024-04-26T23:19:20.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.284+0000 7f51ba445640 1 Processor -- start 2024-04-26T23:19:20.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.285+0000 7f51ba445640 1 -- start start 2024-04-26T23:19:20.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.285+0000 7f51ba445640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51b40b40d0 0x7f51b40ce9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:20.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.285+0000 7f51ba445640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51b40d1a00 0x7f51b41605c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:20.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.285+0000 7f51ba445640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51b40d2320 0x7f51b40cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:20.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.285+0000 7f51ba445640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f51b40d3050 con 0x7f51b40d2320 2024-04-26T23:19:20.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.285+0000 7f51ba445640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f51b40d2ed0 con 0x7f51b40d1a00 2024-04-26T23:19:20.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.285+0000 7f51ba445640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f51b40d31d0 con 0x7f51b40b40d0 2024-04-26T23:19:20.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.285+0000 7f51b89e2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51b40d2320 0x7f51b40cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:20.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.286+0000 7f51b89e2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51b40d2320 0x7f51b40cacf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54548/0 (socket says 172.21.15.47:54548) 2024-04-26T23:19:20.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.286+0000 7f51b89e2640 1 -- 172.21.15.47:0/3515876619 learned_addr learned my addr 172.21.15.47:0/3515876619 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:20.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.286+0000 7f51b37fe640 1 --2- 172.21.15.47:0/3515876619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51b40d1a00 0x7f51b41605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:20.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.286+0000 7f51b3fff640 1 --2- 172.21.15.47:0/3515876619 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51b40b40d0 0x7f51b40ce9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:20.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.286+0000 7f51b89e2640 1 -- 172.21.15.47:0/3515876619 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51b40b40d0 msgr2=0x7f51b40ce9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:20.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.286+0000 7f51b89e2640 1 --2- 172.21.15.47:0/3515876619 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51b40b40d0 0x7f51b40ce9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:20.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.286+0000 7f51b89e2640 1 -- 172.21.15.47:0/3515876619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51b40d1a00 msgr2=0x7f51b41605c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:20.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.286+0000 7f51b89e2640 1 --2- 172.21.15.47:0/3515876619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51b40d1a00 0x7f51b41605c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:20.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.286+0000 7f51b89e2640 1 -- 172.21.15.47:0/3515876619 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51b40c9c50 con 0x7f51b40d2320 2024-04-26T23:19:20.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.286+0000 7f51b89e2640 1 --2- 172.21.15.47:0/3515876619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51b40d2320 0x7f51b40cacf0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f51ac00ccf0 tx=0x7f51ac00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:20.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.286+0000 7f51b0ff9640 1 -- 172.21.15.47:0/3515876619 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ac003b90 con 0x7f51b40d2320 2024-04-26T23:19:20.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.286+0000 7f51b0ff9640 1 -- 172.21.15.47:0/3515876619 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51ac003d30 con 0x7f51b40d2320 2024-04-26T23:19:20.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.286+0000 7f51ba445640 1 -- 172.21.15.47:0/3515876619 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51b40c9f30 con 0x7f51b40d2320 2024-04-26T23:19:20.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.286+0000 7f51b0ff9640 1 -- 172.21.15.47:0/3515876619 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ac004240 con 0x7f51b40d2320 2024-04-26T23:19:20.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.286+0000 7f51ba445640 1 -- 172.21.15.47:0/3515876619 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51b40ca4f0 con 0x7f51b40d2320 2024-04-26T23:19:20.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.287+0000 7f51ba445640 1 -- 172.21.15.47:0/3515876619 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51b40694f0 con 0x7f51b40d2320 2024-04-26T23:19:20.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.288+0000 7f51b0ff9640 1 -- 172.21.15.47:0/3515876619 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f51ac0043e0 con 0x7f51b40d2320 2024-04-26T23:19:20.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.288+0000 7f51b0ff9640 1 --2- 172.21.15.47:0/3515876619 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f51a006c8f0 0x7f51a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:20.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.288+0000 7f51b0ff9640 1 -- 172.21.15.47:0/3515876619 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f51ac08c910 con 0x7f51b40d2320 2024-04-26T23:19:20.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.288+0000 7f51b3fff640 1 --2- 172.21.15.47:0/3515876619 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f51a006c8f0 0x7f51a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:20.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.289+0000 7f51b3fff640 1 --2- 172.21.15.47:0/3515876619 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f51a006c8f0 0x7f51a006edb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f51a4034000 tx=0x7f51a402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:20.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.292+0000 7f51b0ff9640 1 -- 172.21.15.47:0/3515876619 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f51ac05a9b0 con 0x7f51b40d2320 2024-04-26T23:19:20.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.478+0000 7f51ba445640 1 -- 172.21.15.47:0/3515876619 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f51b40b5b10 con 0x7f51b40d2320 2024-04-26T23:19:20.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.481+0000 7f51b0ff9640 1 -- 172.21.15.47:0/3515876619 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f51ac05a2b0 con 0x7f51b40d2320 2024-04-26T23:19:20.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.483+0000 7f51ba445640 1 -- 172.21.15.47:0/3515876619 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f51a006c8f0 msgr2=0x7f51a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:20.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.483+0000 7f51ba445640 1 --2- 172.21.15.47:0/3515876619 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f51a006c8f0 0x7f51a006edb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f51a4034000 tx=0x7f51a402e000 comp rx=0 tx=0).stop 2024-04-26T23:19:20.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.483+0000 7f51ba445640 1 -- 172.21.15.47:0/3515876619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51b40d2320 msgr2=0x7f51b40cacf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:20.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.483+0000 7f51ba445640 1 --2- 172.21.15.47:0/3515876619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51b40d2320 0x7f51b40cacf0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f51ac00ccf0 tx=0x7f51ac00a660 comp rx=0 tx=0).stop 2024-04-26T23:19:20.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.483+0000 7f51ba445640 1 -- 172.21.15.47:0/3515876619 shutdown_connections 2024-04-26T23:19:20.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.483+0000 7f51ba445640 1 --2- 172.21.15.47:0/3515876619 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f51a006c8f0 0x7f51a006edb0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:20.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.483+0000 7f51ba445640 1 --2- 172.21.15.47:0/3515876619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51b40d2320 0x7f51b40cacf0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:20.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.483+0000 7f51ba445640 1 --2- 172.21.15.47:0/3515876619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51b40d1a00 0x7f51b41605c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:20.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.483+0000 7f51ba445640 1 --2- 172.21.15.47:0/3515876619 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51b40b40d0 0x7f51b40ce9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:20.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.483+0000 7f51ba445640 1 -- 172.21.15.47:0/3515876619 >> 172.21.15.47:0/3515876619 conn(0x7f51b40d4fd0 msgr2=0x7f51b406cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:20.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.483+0000 7f51ba445640 1 -- 172.21.15.47:0/3515876619 shutdown_connections 2024-04-26T23:19:20.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.483+0000 7f51ba445640 1 -- 172.21.15.47:0/3515876619 wait complete. 2024-04-26T23:19:20.495 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T23:19:20.495 DEBUG:teuthology.orchestra.run.smithi047:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:19:20.532 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:19:20.574 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T23:19:20.574 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:19:20.574 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:19:20.590 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:19:20.590 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:19:20.647 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:19:20.647 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:19:20.703 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:19:20.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.843+0000 7f49da2f1640 1 Processor -- start 2024-04-26T23:19:20.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.843+0000 7f49da2f1640 1 -- start start 2024-04-26T23:19:20.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.844+0000 7f49da2f1640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f49d40b5f20 0x7f49d40d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:20.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.844+0000 7f49da2f1640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49d40b7e50 con 0x7f49d40d3bc0 2024-04-26T23:19:20.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.844+0000 7f49da2f1640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49d40d1800 con 0x7f49d4162880 2024-04-26T23:19:20.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.844+0000 7f49da2f1640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f49d40d0580 con 0x7f49d40b5f20 2024-04-26T23:19:20.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.844+0000 7f49d888e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f49d40b5f20 0x7f49d40d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:20.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.844+0000 7f49d888e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f49d40b5f20 0x7f49d40d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54564/0 (socket says 172.21.15.47:54564) 2024-04-26T23:19:20.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.844+0000 7f49d888e640 1 -- 172.21.15.47:0/460555241 learned_addr learned my addr 172.21.15.47:0/460555241 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:20.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.844+0000 7f49d37fe640 1 --1- 172.21.15.47:0/460555241 >> v1:172.21.15.47:6789/0 conn(0x7f49d40d3bc0 0x7f49d40b8300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:55522/0 (socket says 172.21.15.47:55522) 2024-04-26T23:19:20.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.844+0000 7f49d888e640 1 -- 172.21.15.47:0/460555241 >> v1:172.21.15.203:6789/0 conn(0x7f49d4162880 legacy=0x7f49d40d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:20.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.844+0000 7f49d888e640 1 -- 172.21.15.47:0/460555241 >> v1:172.21.15.47:6789/0 conn(0x7f49d40d3bc0 legacy=0x7f49d40b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:20.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.844+0000 7f49d2ffd640 1 -- 172.21.15.47:0/460555241 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 384716309 0 0) 0x7f49d40b7e50 con 0x7f49d40d3bc0 2024-04-26T23:19:20.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.844+0000 7f49d888e640 1 -- 172.21.15.47:0/460555241 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d40d07d0 con 0x7f49d40b5f20 2024-04-26T23:19:20.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.845+0000 7f49d888e640 1 --2- 172.21.15.47:0/460555241 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49d40b5f20 0x7f49d40d2cb0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f49cc009e50 tx=0x7f49cc02a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a58bdc586c44edc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:20.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.845+0000 7f49d2ffd640 1 -- 172.21.15.47:0/460555241 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49cc036070 con 0x7f49d40b5f20 2024-04-26T23:19:20.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.845+0000 7f49d2ffd640 1 -- 172.21.15.47:0/460555241 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49cc031600 con 0x7f49d40b5f20 2024-04-26T23:19:20.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.845+0000 7f49d2ffd640 1 -- 172.21.15.47:0/460555241 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49cc0319f0 con 0x7f49d40b5f20 2024-04-26T23:19:20.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.846+0000 7f49da2f1640 1 -- 172.21.15.47:0/460555241 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49d40b5f20 msgr2=0x7f49d40d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:20.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.846+0000 7f49da2f1640 1 --2- 172.21.15.47:0/460555241 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49d40b5f20 0x7f49d40d2cb0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f49cc009e50 tx=0x7f49cc02a7c0 comp rx=0 tx=0).stop 2024-04-26T23:19:20.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.846+0000 7f49da2f1640 1 -- 172.21.15.47:0/460555241 shutdown_connections 2024-04-26T23:19:20.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.846+0000 7f49da2f1640 1 --2- 172.21.15.47:0/460555241 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49d40b5f20 0x7f49d40d2cb0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:20.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.846+0000 7f49da2f1640 1 -- 172.21.15.47:0/460555241 >> 172.21.15.47:0/460555241 conn(0x7f49d40d4f60 msgr2=0x7f49d40d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:20.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.846+0000 7f49da2f1640 1 -- 172.21.15.47:0/460555241 shutdown_connections 2024-04-26T23:19:20.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.846+0000 7f49da2f1640 1 -- 172.21.15.47:0/460555241 wait complete. 2024-04-26T23:19:20.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.847+0000 7f49da2f1640 1 Processor -- start 2024-04-26T23:19:20.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.847+0000 7f49da2f1640 1 -- start start 2024-04-26T23:19:20.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.848+0000 7f49da2f1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49d40b5f20 0x7f49d40b2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:20.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.848+0000 7f49da2f1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49d40d3bc0 0x7f49d41f55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:20.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.848+0000 7f49da2f1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49d4162880 0x7f49d41f81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:20.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.848+0000 7f49da2f1640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f49d4162c60 con 0x7f49d4162880 2024-04-26T23:19:20.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.848+0000 7f49da2f1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f49d40d0d90 con 0x7f49d40d3bc0 2024-04-26T23:19:20.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.848+0000 7f49da2f1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f49d4162de0 con 0x7f49d40b5f20 2024-04-26T23:19:20.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.848+0000 7f49d888e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49d4162880 0x7f49d41f81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:20.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.848+0000 7f49d888e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49d4162880 0x7f49d41f81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54574/0 (socket says 172.21.15.47:54574) 2024-04-26T23:19:20.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.848+0000 7f49d888e640 1 -- 172.21.15.47:0/3043588453 learned_addr learned my addr 172.21.15.47:0/3043588453 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:20.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.848+0000 7f49d37fe640 1 --2- 172.21.15.47:0/3043588453 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49d40d3bc0 0x7f49d41f55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:20.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.848+0000 7f49d3fff640 1 --2- 172.21.15.47:0/3043588453 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49d40b5f20 0x7f49d40b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:20.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.848+0000 7f49d888e640 1 -- 172.21.15.47:0/3043588453 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49d40b5f20 msgr2=0x7f49d40b2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:20.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.848+0000 7f49d888e640 1 --2- 172.21.15.47:0/3043588453 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49d40b5f20 0x7f49d40b2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:20.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.848+0000 7f49d888e640 1 -- 172.21.15.47:0/3043588453 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49d40d3bc0 msgr2=0x7f49d41f55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:20.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.848+0000 7f49d888e640 1 --2- 172.21.15.47:0/3043588453 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49d40d3bc0 0x7f49d41f55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:20.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.848+0000 7f49d888e640 1 -- 172.21.15.47:0/3043588453 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d41f4b80 con 0x7f49d4162880 2024-04-26T23:19:20.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.848+0000 7f49d37fe640 1 --2- 172.21.15.47:0/3043588453 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49d40d3bc0 0x7f49d41f55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:19:20.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.848+0000 7f49d888e640 1 --2- 172.21.15.47:0/3043588453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49d4162880 0x7f49d41f81f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f49cc009f70 tx=0x7f49cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:20.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.849+0000 7f49d0ff9640 1 -- 172.21.15.47:0/3043588453 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49cc036050 con 0x7f49d4162880 2024-04-26T23:19:20.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.849+0000 7f49d0ff9640 1 -- 172.21.15.47:0/3043588453 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49cc042070 con 0x7f49d4162880 2024-04-26T23:19:20.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.849+0000 7f49da2f1640 1 -- 172.21.15.47:0/3043588453 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49d41f8c40 con 0x7f49d4162880 2024-04-26T23:19:20.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.849+0000 7f49d0ff9640 1 -- 172.21.15.47:0/3043588453 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49cc009510 con 0x7f49d4162880 2024-04-26T23:19:20.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.849+0000 7f49da2f1640 1 -- 172.21.15.47:0/3043588453 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49d41f4df0 con 0x7f49d4162880 2024-04-26T23:19:20.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.850+0000 7f49da2f1640 1 -- 172.21.15.47:0/3043588453 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49d40694f0 con 0x7f49d4162880 2024-04-26T23:19:20.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.850+0000 7f49d0ff9640 1 -- 172.21.15.47:0/3043588453 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f49cc004490 con 0x7f49d4162880 2024-04-26T23:19:20.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.850+0000 7f49d0ff9640 1 --2- 172.21.15.47:0/3043588453 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f49b406c8f0 0x7f49b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:20.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.851+0000 7f49d0ff9640 1 -- 172.21.15.47:0/3043588453 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f49cc0acea0 con 0x7f49d4162880 2024-04-26T23:19:20.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.851+0000 7f49d3fff640 1 --2- 172.21.15.47:0/3043588453 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f49b406c8f0 0x7f49b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:20.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.851+0000 7f49d3fff640 1 --2- 172.21.15.47:0/3043588453 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f49b406c8f0 0x7f49b406edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f49d40cd0d0 tx=0x7f49bc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:20.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:20.855+0000 7f49d0ff9640 1 -- 172.21.15.47:0/3043588453 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f49cc07af40 con 0x7f49d4162880 2024-04-26T23:19:21.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.042+0000 7f49da2f1640 1 -- 172.21.15.47:0/3043588453 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f49d40cf390 con 0x7f49d4162880 2024-04-26T23:19:21.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.043+0000 7f49d0ff9640 1 -- 172.21.15.47:0/3043588453 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18857 (secure 0 0 0) 0x7f49cc07a840 con 0x7f49d4162880 2024-04-26T23:19:21.044 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:19:21.044 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":26,"fsid":"399006d3-4db1-43e6-ba81-a79a276eabfd","created":"2024-04-26T23:18:06.731211+0000","modified":"2024-04-26T23:19:12.124051+0000","last_up_change":"2024-04-26T23:18:17.808736+0000","last_in_change":"2024-04-26T23:18:08.945859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:18:16.937492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:18:29.033575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:18:30.318499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:18:34.014536+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ee091567-fea0-4a14-bf4e-0e16c20bb2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6819","nonce":326520779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6821","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6823","nonce":326520779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6829","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6831","nonce":326520779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6825","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6827","nonce":326520779}]},"public_addr":"172.21.15.47:6819/326520779","cluster_addr":"172.21.15.47:6823/326520779","heartbeat_back_addr":"172.21.15.47:6831/326520779","heartbeat_front_addr":"172.21.15.47:6827/326520779","state":["exists","up"]},{"osd":1,"uuid":"c14440a0-bba5-4931-a6db-b736c08e869b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6809","nonce":1912610931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6811","nonce":1912610931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6815","nonce":1912610931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6813","nonce":1912610931}]},"public_addr":"172.21.15.47:6809/1912610931","cluster_addr":"172.21.15.47:6811/1912610931","heartbeat_back_addr":"172.21.15.47:6815/1912610931","heartbeat_front_addr":"172.21.15.47:6813/1912610931","state":["exists","up"]},{"osd":2,"uuid":"3940f8de-c9b2-4cd3-a911-9c301729eb30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6801","nonce":2569869305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6803","nonce":2569869305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6807","nonce":2569869305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6805","nonce":2569869305}]},"public_addr":"172.21.15.47:6801/2569869305","cluster_addr":"172.21.15.47:6803/2569869305","heartbeat_back_addr":"172.21.15.47:6807/2569869305","heartbeat_front_addr":"172.21.15.47:6805/2569869305","state":["exists","up"]},{"osd":3,"uuid":"1286a67e-8958-426a-8c2e-bc85498c619c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6818","nonce":2791411683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6822","nonce":2791411683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6830","nonce":2791411683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6826","nonce":2791411683}]},"public_addr":"172.21.15.47:6818/2791411683","cluster_addr":"172.21.15.47:6822/2791411683","heartbeat_back_addr":"172.21.15.47:6830/2791411683","heartbeat_front_addr":"172.21.15.47:6826/2791411683","state":["exists","up"]},{"osd":4,"uuid":"d8ee2c20-e5b7-46c2-9999-b234070126cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6809","nonce":441313104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6811","nonce":441313104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6815","nonce":441313104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6813","nonce":441313104}]},"public_addr":"172.21.15.203:6809/441313104","cluster_addr":"172.21.15.203:6811/441313104","heartbeat_back_addr":"172.21.15.203:6815/441313104","heartbeat_front_addr":"172.21.15.203:6813/441313104","state":["exists","up"]},{"osd":5,"uuid":"b453a8f0-f324-4840-9ae0-d67bdadb9f8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6801","nonce":56838444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6803","nonce":56838444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6807","nonce":56838444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6805","nonce":56838444}]},"public_addr":"172.21.15.203:6801/56838444","cluster_addr":"172.21.15.203:6803/56838444","heartbeat_back_addr":"172.21.15.203:6807/56838444","heartbeat_front_addr":"172.21.15.203:6805/56838444","state":["exists","up"]},{"osd":6,"uuid":"a6d31951-2d4b-4e8d-a510-6283b01dc158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6825","nonce":2738517080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6827","nonce":2738517080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6831","nonce":2738517080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6829","nonce":2738517080}]},"public_addr":"172.21.15.203:6825/2738517080","cluster_addr":"172.21.15.203:6827/2738517080","heartbeat_back_addr":"172.21.15.203:6831/2738517080","heartbeat_front_addr":"172.21.15.203:6829/2738517080","state":["exists","up"]},{"osd":7,"uuid":"7119b8c0-fe96-402d-9e75-7b509d9d44d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6817","nonce":1315565058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6819","nonce":1315565058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6823","nonce":1315565058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6821","nonce":1315565058}]},"public_addr":"172.21.15.203:6817/1315565058","cluster_addr":"172.21.15.203:6819/1315565058","heartbeat_back_addr":"172.21.15.203:6823/1315565058","heartbeat_front_addr":"172.21.15.203:6821/1315565058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.121921+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.137494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.084814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.111593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.072430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.125066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.097867+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.057061+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":5,"to":1}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:19:21.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.045+0000 7f49da2f1640 1 -- 172.21.15.47:0/3043588453 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f49b406c8f0 msgr2=0x7f49b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:21.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.045+0000 7f49da2f1640 1 --2- 172.21.15.47:0/3043588453 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f49b406c8f0 0x7f49b406edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f49d40cd0d0 tx=0x7f49bc009040 comp rx=0 tx=0).stop 2024-04-26T23:19:21.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.045+0000 7f49da2f1640 1 -- 172.21.15.47:0/3043588453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49d4162880 msgr2=0x7f49d41f81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:21.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.045+0000 7f49da2f1640 1 --2- 172.21.15.47:0/3043588453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49d4162880 0x7f49d41f81f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f49cc009f70 tx=0x7f49cc004a40 comp rx=0 tx=0).stop 2024-04-26T23:19:21.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.045+0000 7f49da2f1640 1 -- 172.21.15.47:0/3043588453 shutdown_connections 2024-04-26T23:19:21.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.045+0000 7f49da2f1640 1 --2- 172.21.15.47:0/3043588453 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f49b406c8f0 0x7f49b406edb0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:21.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.045+0000 7f49da2f1640 1 --2- 172.21.15.47:0/3043588453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49d4162880 0x7f49d41f81f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:21.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.045+0000 7f49da2f1640 1 --2- 172.21.15.47:0/3043588453 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49d40d3bc0 0x7f49d41f55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:21.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.045+0000 7f49da2f1640 1 --2- 172.21.15.47:0/3043588453 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49d40b5f20 0x7f49d40b2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:21.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.045+0000 7f49da2f1640 1 -- 172.21.15.47:0/3043588453 >> 172.21.15.47:0/3043588453 conn(0x7f49d40d4f60 msgr2=0x7f49d40d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:21.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.045+0000 7f49da2f1640 1 -- 172.21.15.47:0/3043588453 shutdown_connections 2024-04-26T23:19:21.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.045+0000 7f49da2f1640 1 -- 172.21.15.47:0/3043588453 wait complete. 2024-04-26T23:19:21.057 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:18:16.937492+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:18:29.033575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:18:30.318499+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T23:18:34.014536+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T23:19:21.057 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:19:21.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.160+0000 7f95da96a640 1 Processor -- start 2024-04-26T23:19:21.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.160+0000 7f95da96a640 1 -- start start 2024-04-26T23:19:21.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.161+0000 7f95da96a640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f95d4160720 0x7f95d40ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:21.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.161+0000 7f95da96a640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95d40d0710 con 0x7f95d40ceb60 2024-04-26T23:19:21.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.161+0000 7f95da96a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95d40d0430 con 0x7f95d40d18f0 2024-04-26T23:19:21.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.161+0000 7f95da96a640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f95d40d01e0 con 0x7f95d4160720 2024-04-26T23:19:21.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.161+0000 7f95d3fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f95d4160720 0x7f95d40ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:21.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.161+0000 7f95d37fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f95d40ceb60 0x7f95d40d11e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:55524/0 (socket says 172.21.15.47:55524) 2024-04-26T23:19:21.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.161+0000 7f95d37fe640 1 -- 172.21.15.47:0/3619227461 learned_addr learned my addr 172.21.15.47:0/3619227461 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:21.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.161+0000 7f95d2ffd640 1 -- 172.21.15.47:0/3619227461 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2125935517 0 0) 0x7f95d40d0710 con 0x7f95d40ceb60 2024-04-26T23:19:21.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.161+0000 7f95d2ffd640 1 -- 172.21.15.47:0/3619227461 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f95bc003520 con 0x7f95d40ceb60 2024-04-26T23:19:21.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.162+0000 7f95d3fff640 1 -- 172.21.15.47:0/3619227461 >> v1:172.21.15.203:6789/0 conn(0x7f95d40d18f0 legacy=0x7f95d40b2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:21.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.162+0000 7f95d3fff640 1 -- 172.21.15.47:0/3619227461 >> v1:172.21.15.47:6789/0 conn(0x7f95d40ceb60 legacy=0x7f95d40d11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:21.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.162+0000 7f95d3fff640 1 -- 172.21.15.47:0/3619227461 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95d40cdb40 con 0x7f95d4160720 2024-04-26T23:19:21.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.162+0000 7f95d3fff640 1 --2- 172.21.15.47:0/3619227461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95d4160720 0x7f95d40ce620 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f95c40097a0 tx=0x7f95c402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26edf67305e6c179 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:21.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.162+0000 7f95d2ffd640 1 -- 172.21.15.47:0/3619227461 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c4037070 con 0x7f95d4160720 2024-04-26T23:19:21.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.162+0000 7f95d2ffd640 1 -- 172.21.15.47:0/3619227461 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95c4032440 con 0x7f95d4160720 2024-04-26T23:19:21.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.162+0000 7f95d2ffd640 1 -- 172.21.15.47:0/3619227461 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c4032810 con 0x7f95d4160720 2024-04-26T23:19:21.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.163+0000 7f95da96a640 1 -- 172.21.15.47:0/3619227461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95d4160720 msgr2=0x7f95d40ce620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:21.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.163+0000 7f95da96a640 1 --2- 172.21.15.47:0/3619227461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95d4160720 0x7f95d40ce620 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f95c40097a0 tx=0x7f95c402a230 comp rx=0 tx=0).stop 2024-04-26T23:19:21.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.163+0000 7f95da96a640 1 -- 172.21.15.47:0/3619227461 shutdown_connections 2024-04-26T23:19:21.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.163+0000 7f95da96a640 1 --2- 172.21.15.47:0/3619227461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95d4160720 0x7f95d40ce620 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:21.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.163+0000 7f95da96a640 1 -- 172.21.15.47:0/3619227461 >> 172.21.15.47:0/3619227461 conn(0x7f95d40d2e00 msgr2=0x7f95d40d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:21.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.163+0000 7f95da96a640 1 -- 172.21.15.47:0/3619227461 shutdown_connections 2024-04-26T23:19:21.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.163+0000 7f95da96a640 1 -- 172.21.15.47:0/3619227461 wait complete. 2024-04-26T23:19:21.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.164+0000 7f95da96a640 1 Processor -- start 2024-04-26T23:19:21.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.164+0000 7f95da96a640 1 -- start start 2024-04-26T23:19:21.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.165+0000 7f95da96a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95d40ceb60 0x7f95d4151530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:21.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.165+0000 7f95da96a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95d40d18f0 0x7f95d40a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:21.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.165+0000 7f95da96a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95d4160720 0x7f95d40b34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:21.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.165+0000 7f95da96a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f95d4160b00 con 0x7f95d40ceb60 2024-04-26T23:19:21.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.165+0000 7f95da96a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f95d40ce100 con 0x7f95d40d18f0 2024-04-26T23:19:21.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.165+0000 7f95da96a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f95d409f620 con 0x7f95d4160720 2024-04-26T23:19:21.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.165+0000 7f95d3fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95d40ceb60 0x7f95d4151530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:21.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.165+0000 7f95d3fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95d40ceb60 0x7f95d4151530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54592/0 (socket says 172.21.15.47:54592) 2024-04-26T23:19:21.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.165+0000 7f95d3fff640 1 -- 172.21.15.47:0/547192610 learned_addr learned my addr 172.21.15.47:0/547192610 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:21.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.165+0000 7f95d37fe640 1 --2- 172.21.15.47:0/547192610 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95d40d18f0 0x7f95d40a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:21.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.165+0000 7f95d8f07640 1 --2- 172.21.15.47:0/547192610 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95d4160720 0x7f95d40b34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:21.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.166+0000 7f95d3fff640 1 -- 172.21.15.47:0/547192610 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95d4160720 msgr2=0x7f95d40b34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:21.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.166+0000 7f95d3fff640 1 --2- 172.21.15.47:0/547192610 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95d4160720 0x7f95d40b34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:21.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.166+0000 7f95d3fff640 1 -- 172.21.15.47:0/547192610 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95d40d18f0 msgr2=0x7f95d40a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:21.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.166+0000 7f95d3fff640 1 --2- 172.21.15.47:0/547192610 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95d40d18f0 0x7f95d40a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:21.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.166+0000 7f95d3fff640 1 -- 172.21.15.47:0/547192610 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95d409f900 con 0x7f95d40ceb60 2024-04-26T23:19:21.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.166+0000 7f95d3fff640 1 --2- 172.21.15.47:0/547192610 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95d40ceb60 0x7f95d4151530 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f95c4000c00 tx=0x7f95c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:21.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.166+0000 7f95d0ff9640 1 -- 172.21.15.47:0/547192610 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c4037040 con 0x7f95d40ceb60 2024-04-26T23:19:21.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.166+0000 7f95d0ff9640 1 -- 172.21.15.47:0/547192610 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95c4042070 con 0x7f95d40ceb60 2024-04-26T23:19:21.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.166+0000 7f95d0ff9640 1 -- 172.21.15.47:0/547192610 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c4030940 con 0x7f95d40ceb60 2024-04-26T23:19:21.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.166+0000 7f95da96a640 1 -- 172.21.15.47:0/547192610 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95d409fbe0 con 0x7f95d40ceb60 2024-04-26T23:19:21.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.166+0000 7f95da96a640 1 -- 172.21.15.47:0/547192610 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95d40a01a0 con 0x7f95d40ceb60 2024-04-26T23:19:21.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.167+0000 7f95da96a640 1 -- 172.21.15.47:0/547192610 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95d40a03a0 con 0x7f95d40ceb60 2024-04-26T23:19:21.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.168+0000 7f95d0ff9640 1 -- 172.21.15.47:0/547192610 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f95c4030ae0 con 0x7f95d40ceb60 2024-04-26T23:19:21.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.168+0000 7f95d0ff9640 1 --2- 172.21.15.47:0/547192610 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f95c006c8f0 0x7f95c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:21.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.168+0000 7f95d0ff9640 1 -- 172.21.15.47:0/547192610 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f95c40ac830 con 0x7f95d40ceb60 2024-04-26T23:19:21.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.168+0000 7f95d37fe640 1 --2- 172.21.15.47:0/547192610 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f95c006c8f0 0x7f95c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:21.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.169+0000 7f95d37fe640 1 --2- 172.21.15.47:0/547192610 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f95c006c8f0 0x7f95c006edb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f95d4161e70 tx=0x7f95c8007d30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:21.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.172+0000 7f95d0ff9640 1 -- 172.21.15.47:0/547192610 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f95d40a03a0 con 0x7f95d40ceb60 2024-04-26T23:19:21.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.357+0000 7f95da96a640 1 -- 172.21.15.47:0/547192610 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f95d40694f0 con 0x7f95d40ceb60 2024-04-26T23:19:21.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.357+0000 7f95d0ff9640 1 -- 172.21.15.47:0/547192610 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f95c407a840 con 0x7f95d40ceb60 2024-04-26T23:19:21.358 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-26T23:19:21.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.359+0000 7f95da96a640 1 -- 172.21.15.47:0/547192610 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f95c006c8f0 msgr2=0x7f95c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:21.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.359+0000 7f95da96a640 1 --2- 172.21.15.47:0/547192610 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f95c006c8f0 0x7f95c006edb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f95d4161e70 tx=0x7f95c8007d30 comp rx=0 tx=0).stop 2024-04-26T23:19:21.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.359+0000 7f95da96a640 1 -- 172.21.15.47:0/547192610 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95d40ceb60 msgr2=0x7f95d4151530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:21.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.359+0000 7f95da96a640 1 --2- 172.21.15.47:0/547192610 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95d40ceb60 0x7f95d4151530 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f95c4000c00 tx=0x7f95c40046c0 comp rx=0 tx=0).stop 2024-04-26T23:19:21.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.359+0000 7f95da96a640 1 -- 172.21.15.47:0/547192610 shutdown_connections 2024-04-26T23:19:21.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.359+0000 7f95da96a640 1 --2- 172.21.15.47:0/547192610 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f95c006c8f0 0x7f95c006edb0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:21.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.359+0000 7f95da96a640 1 --2- 172.21.15.47:0/547192610 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95d4160720 0x7f95d40b34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:21.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.359+0000 7f95da96a640 1 --2- 172.21.15.47:0/547192610 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95d40d18f0 0x7f95d40a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:21.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.359+0000 7f95da96a640 1 --2- 172.21.15.47:0/547192610 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95d40ceb60 0x7f95d4151530 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:21.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.359+0000 7f95da96a640 1 -- 172.21.15.47:0/547192610 >> 172.21.15.47:0/547192610 conn(0x7f95d40d2e00 msgr2=0x7f95d41614b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:21.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.359+0000 7f95da96a640 1 -- 172.21.15.47:0/547192610 shutdown_connections 2024-04-26T23:19:21.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.359+0000 7f95da96a640 1 -- 172.21.15.47:0/547192610 wait complete. 2024-04-26T23:19:21.371 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:19:21.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.474+0000 7fbe2b846640 1 Processor -- start 2024-04-26T23:19:21.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.474+0000 7fbe2b846640 1 -- start start 2024-04-26T23:19:21.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.475+0000 7fbe2b846640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbe24162a20 0x7fbe240d1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:21.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.475+0000 7fbe2b846640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe240d1c40 0x7fbe240d20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:21.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.475+0000 7fbe2b846640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbe240d2ed0 0x7fbe240d3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:21.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.475+0000 7fbe2b846640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fbe240d07a0 con 0x7fbe24162a20 2024-04-26T23:19:21.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.475+0000 7fbe2b846640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fbe240c8bc0 con 0x7fbe240d2ed0 2024-04-26T23:19:21.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.475+0000 7fbe2b846640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fbe240d09f0 con 0x7fbe240d1c40 2024-04-26T23:19:21.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.475+0000 7fbe295e2640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbe24162a20 0x7fbe240d1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:21.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.475+0000 7fbe295e2640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbe24162a20 0x7fbe240d1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54598/0 (socket says 172.21.15.47:54598) 2024-04-26T23:19:21.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.475+0000 7fbe295e2640 1 -- 172.21.15.47:0/2537376500 learned_addr learned my addr 172.21.15.47:0/2537376500 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:21.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.475+0000 7fbe28de1640 1 --2- 172.21.15.47:0/2537376500 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe240d1c40 0x7fbe240d20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:21.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.475+0000 7fbe29de3640 1 --2- 172.21.15.47:0/2537376500 >> v2:172.21.15.203:3300/0 conn(0x7fbe240d2ed0 0x7fbe240d3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:21.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.475+0000 7fbe295e2640 1 -- 172.21.15.47:0/2537376500 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe240d1c40 msgr2=0x7fbe240d20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:21.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.476+0000 7fbe295e2640 1 --2- 172.21.15.47:0/2537376500 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe240d1c40 0x7fbe240d20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:21.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.476+0000 7fbe295e2640 1 -- 172.21.15.47:0/2537376500 >> v2:172.21.15.203:3300/0 conn(0x7fbe240d2ed0 msgr2=0x7fbe240d3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:21.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.476+0000 7fbe295e2640 1 --2- 172.21.15.47:0/2537376500 >> v2:172.21.15.203:3300/0 conn(0x7fbe240d2ed0 0x7fbe240d3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:21.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.476+0000 7fbe295e2640 1 -- 172.21.15.47:0/2537376500 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe240d0cd0 con 0x7fbe24162a20 2024-04-26T23:19:21.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.476+0000 7fbe295e2640 1 --2- 172.21.15.47:0/2537376500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe24162a20 0x7fbe240d1700 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fbe240d14a0 tx=0x7fbe1c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f25eb2aa84fccc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:21.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.476+0000 7fbe23fff640 1 -- 172.21.15.47:0/2537376500 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1c037070 con 0x7fbe24162a20 2024-04-26T23:19:21.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.476+0000 7fbe23fff640 1 -- 172.21.15.47:0/2537376500 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe1c032440 con 0x7fbe24162a20 2024-04-26T23:19:21.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.476+0000 7fbe23fff640 1 -- 172.21.15.47:0/2537376500 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1c032810 con 0x7fbe24162a20 2024-04-26T23:19:21.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.477+0000 7fbe2b846640 1 -- 172.21.15.47:0/2537376500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe24162a20 msgr2=0x7fbe240d1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:21.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.477+0000 7fbe2b846640 1 --2- 172.21.15.47:0/2537376500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe24162a20 0x7fbe240d1700 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fbe240d14a0 tx=0x7fbe1c02a140 comp rx=0 tx=0).stop 2024-04-26T23:19:21.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.477+0000 7fbe2b846640 1 -- 172.21.15.47:0/2537376500 shutdown_connections 2024-04-26T23:19:21.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.477+0000 7fbe2b846640 1 --2- 172.21.15.47:0/2537376500 >> v2:172.21.15.203:3300/0 conn(0x7fbe240d2ed0 0x7fbe240d3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:21.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.477+0000 7fbe2b846640 1 --2- 172.21.15.47:0/2537376500 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe240d1c40 0x7fbe240d20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:21.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.477+0000 7fbe2b846640 1 --2- 172.21.15.47:0/2537376500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe24162a20 0x7fbe240d1700 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:21.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.477+0000 7fbe2b846640 1 -- 172.21.15.47:0/2537376500 >> 172.21.15.47:0/2537376500 conn(0x7fbe240d4fd0 msgr2=0x7fbe240d53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:21.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.477+0000 7fbe2b846640 1 -- 172.21.15.47:0/2537376500 shutdown_connections 2024-04-26T23:19:21.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.477+0000 7fbe2b846640 1 -- 172.21.15.47:0/2537376500 wait complete. 2024-04-26T23:19:21.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.478+0000 7fbe2b846640 1 Processor -- start 2024-04-26T23:19:21.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.478+0000 7fbe2b846640 1 -- start start 2024-04-26T23:19:21.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.479+0000 7fbe2b846640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe240d1c40 0x7fbe240ca4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:21.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.479+0000 7fbe2b846640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe240d2ed0 0x7fbe240ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:21.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.479+0000 7fbe2b846640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe24162a20 0x7fbe240b4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:21.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.479+0000 7fbe2b846640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fbe240cbd90 con 0x7fbe240d2ed0 2024-04-26T23:19:21.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.479+0000 7fbe2b846640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbe240d0fb0 con 0x7fbe240d1c40 2024-04-26T23:19:21.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.479+0000 7fbe2b846640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fbe24162e00 con 0x7fbe24162a20 2024-04-26T23:19:21.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.479+0000 7fbe28de1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe240d2ed0 0x7fbe240ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:21.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.479+0000 7fbe28de1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe240d2ed0 0x7fbe240ca9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54612/0 (socket says 172.21.15.47:54612) 2024-04-26T23:19:21.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.479+0000 7fbe28de1640 1 -- 172.21.15.47:0/4141393897 learned_addr learned my addr 172.21.15.47:0/4141393897 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:21.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.479+0000 7fbe295e2640 1 --2- 172.21.15.47:0/4141393897 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe240d1c40 0x7fbe240ca4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:21.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.479+0000 7fbe29de3640 1 --2- 172.21.15.47:0/4141393897 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe24162a20 0x7fbe240b4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:21.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.480+0000 7fbe28de1640 1 -- 172.21.15.47:0/4141393897 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe24162a20 msgr2=0x7fbe240b4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:21.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.480+0000 7fbe28de1640 1 --2- 172.21.15.47:0/4141393897 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe24162a20 0x7fbe240b4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:21.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.480+0000 7fbe28de1640 1 -- 172.21.15.47:0/4141393897 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe240d1c40 msgr2=0x7fbe240ca4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:21.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.480+0000 7fbe28de1640 1 --2- 172.21.15.47:0/4141393897 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe240d1c40 0x7fbe240ca4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:21.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.480+0000 7fbe28de1640 1 -- 172.21.15.47:0/4141393897 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe240b4f50 con 0x7fbe240d2ed0 2024-04-26T23:19:21.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.480+0000 7fbe28de1640 1 --2- 172.21.15.47:0/4141393897 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe240d2ed0 0x7fbe240ca9e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fbe1800d800 tx=0x7fbe1800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:21.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.480+0000 7fbe21ffb640 1 -- 172.21.15.47:0/4141393897 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe18018030 con 0x7fbe240d2ed0 2024-04-26T23:19:21.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.480+0000 7fbe21ffb640 1 -- 172.21.15.47:0/4141393897 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe180096a0 con 0x7fbe240d2ed0 2024-04-26T23:19:21.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.480+0000 7fbe2b846640 1 -- 172.21.15.47:0/4141393897 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe240b5230 con 0x7fbe240d2ed0 2024-04-26T23:19:21.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.480+0000 7fbe21ffb640 1 -- 172.21.15.47:0/4141393897 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe18009870 con 0x7fbe240d2ed0 2024-04-26T23:19:21.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.480+0000 7fbe2b846640 1 -- 172.21.15.47:0/4141393897 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe240b57f0 con 0x7fbe240d2ed0 2024-04-26T23:19:21.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.481+0000 7fbe2b846640 1 -- 172.21.15.47:0/4141393897 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe240694f0 con 0x7fbe240d2ed0 2024-04-26T23:19:21.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.482+0000 7fbe21ffb640 1 -- 172.21.15.47:0/4141393897 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbe1801b070 con 0x7fbe240d2ed0 2024-04-26T23:19:21.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.482+0000 7fbe21ffb640 1 --2- 172.21.15.47:0/4141393897 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fbe1006c8f0 0x7fbe1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:21.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.482+0000 7fbe21ffb640 1 -- 172.21.15.47:0/4141393897 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7fbe1808bfe0 con 0x7fbe240d2ed0 2024-04-26T23:19:21.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.482+0000 7fbe295e2640 1 --2- 172.21.15.47:0/4141393897 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fbe1006c8f0 0x7fbe1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:21.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.483+0000 7fbe295e2640 1 --2- 172.21.15.47:0/4141393897 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fbe1006c8f0 0x7fbe1006edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c034000 tx=0x7fbe1c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:21.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.487+0000 7fbe21ffb640 1 -- 172.21.15.47:0/4141393897 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbe18059a30 con 0x7fbe240d2ed0 2024-04-26T23:19:21.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.671+0000 7fbe2b846640 1 -- 172.21.15.47:0/4141393897 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbe240cc0a0 con 0x7fbe240d2ed0 2024-04-26T23:19:21.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.671+0000 7fbe21ffb640 1 -- 172.21.15.47:0/4141393897 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fbe1805dd30 con 0x7fbe240d2ed0 2024-04-26T23:19:21.672 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-26T23:19:21.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.674+0000 7fbe2b846640 1 -- 172.21.15.47:0/4141393897 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fbe1006c8f0 msgr2=0x7fbe1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:21.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.674+0000 7fbe2b846640 1 --2- 172.21.15.47:0/4141393897 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fbe1006c8f0 0x7fbe1006edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c034000 tx=0x7fbe1c02e000 comp rx=0 tx=0).stop 2024-04-26T23:19:21.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.674+0000 7fbe2b846640 1 -- 172.21.15.47:0/4141393897 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe240d2ed0 msgr2=0x7fbe240ca9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:21.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.674+0000 7fbe2b846640 1 --2- 172.21.15.47:0/4141393897 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe240d2ed0 0x7fbe240ca9e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fbe1800d800 tx=0x7fbe1800dcc0 comp rx=0 tx=0).stop 2024-04-26T23:19:21.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.674+0000 7fbe2b846640 1 -- 172.21.15.47:0/4141393897 shutdown_connections 2024-04-26T23:19:21.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.674+0000 7fbe2b846640 1 --2- 172.21.15.47:0/4141393897 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fbe1006c8f0 0x7fbe1006edb0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:21.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.674+0000 7fbe2b846640 1 --2- 172.21.15.47:0/4141393897 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe24162a20 0x7fbe240b4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:21.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.674+0000 7fbe2b846640 1 --2- 172.21.15.47:0/4141393897 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe240d2ed0 0x7fbe240ca9e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:21.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.674+0000 7fbe2b846640 1 --2- 172.21.15.47:0/4141393897 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe240d1c40 0x7fbe240ca4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:21.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.674+0000 7fbe2b846640 1 -- 172.21.15.47:0/4141393897 >> 172.21.15.47:0/4141393897 conn(0x7fbe240d4fd0 msgr2=0x7fbe2415f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:21.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.674+0000 7fbe2b846640 1 -- 172.21.15.47:0/4141393897 shutdown_connections 2024-04-26T23:19:21.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.674+0000 7fbe2b846640 1 -- 172.21.15.47:0/4141393897 wait complete. 2024-04-26T23:19:21.685 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:19:21.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.788+0000 7f64b3719640 1 Processor -- start 2024-04-26T23:19:21.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.788+0000 7f64b3719640 1 -- start start 2024-04-26T23:19:21.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.789+0000 7f64b3719640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f64ac0d1a00 0x7f64ac0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:21.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.789+0000 7f64b3719640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f64ac0d2320 0x7f64ac1625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:21.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.789+0000 7f64b3719640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64ac0d0a10 con 0x7f64ac0b40d0 2024-04-26T23:19:21.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.789+0000 7f64b3719640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f64ac0d0cf0 con 0x7f64ac0d1a00 2024-04-26T23:19:21.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.789+0000 7f64b3719640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f64ac0d0fd0 con 0x7f64ac0d2320 2024-04-26T23:19:21.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.789+0000 7f64b14b5640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f64ac0d1a00 0x7f64ac0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:21.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.789+0000 7f64b1cb6640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f64ac0b40d0 0x7f64ac0d2ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:55534/0 (socket says 172.21.15.47:55534) 2024-04-26T23:19:21.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.789+0000 7f64b1cb6640 1 -- 172.21.15.47:0/3065067586 learned_addr learned my addr 172.21.15.47:0/3065067586 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:21.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.790+0000 7f64b0cb4640 1 --2- 172.21.15.47:0/3065067586 >> v2:172.21.15.203:3300/0 conn(0x7f64ac0d2320 0x7f64ac1625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:21.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.790+0000 7f64b14b5640 1 -- 172.21.15.47:0/3065067586 >> v2:172.21.15.203:3300/0 conn(0x7f64ac0d2320 msgr2=0x7f64ac1625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:21.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.790+0000 7f64b14b5640 1 --2- 172.21.15.47:0/3065067586 >> v2:172.21.15.203:3300/0 conn(0x7f64ac0d2320 0x7f64ac1625f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:21.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.790+0000 7f64b14b5640 1 -- 172.21.15.47:0/3065067586 >> v1:172.21.15.47:6789/0 conn(0x7f64ac0b40d0 legacy=0x7f64ac0d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:21.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.790+0000 7f64b14b5640 1 -- 172.21.15.47:0/3065067586 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64ac0d40a0 con 0x7f64ac0d1a00 2024-04-26T23:19:21.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.790+0000 7f64abfff640 1 -- 172.21.15.47:0/3065067586 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 902770962 0 0) 0x7f64ac0d0a10 con 0x7f64ac0b40d0 2024-04-26T23:19:21.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.790+0000 7f64b0cb4640 1 --2- 172.21.15.47:0/3065067586 >> v2:172.21.15.203:3300/0 conn(0x7f64ac0d2320 0x7f64ac1625f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:19:21.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.790+0000 7f64b14b5640 1 --2- 172.21.15.47:0/3065067586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64ac0d1a00 0x7f64ac0d1de0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f64ac08d380 tx=0x7f64a402a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bfdb4624aa6a577 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:21.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.791+0000 7f64abfff640 1 -- 172.21.15.47:0/3065067586 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a4037070 con 0x7f64ac0d1a00 2024-04-26T23:19:21.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.791+0000 7f64abfff640 1 -- 172.21.15.47:0/3065067586 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64a402add0 con 0x7f64ac0d1a00 2024-04-26T23:19:21.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.791+0000 7f64abfff640 1 -- 172.21.15.47:0/3065067586 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a4032660 con 0x7f64ac0d1a00 2024-04-26T23:19:21.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.791+0000 7f64b3719640 1 -- 172.21.15.47:0/3065067586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64ac0d1a00 msgr2=0x7f64ac0d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:21.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.791+0000 7f64b3719640 1 --2- 172.21.15.47:0/3065067586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64ac0d1a00 0x7f64ac0d1de0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f64ac08d380 tx=0x7f64a402a080 comp rx=0 tx=0).stop 2024-04-26T23:19:21.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.791+0000 7f64b3719640 1 -- 172.21.15.47:0/3065067586 shutdown_connections 2024-04-26T23:19:21.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.791+0000 7f64b3719640 1 --2- 172.21.15.47:0/3065067586 >> v2:172.21.15.203:3300/0 conn(0x7f64ac0d2320 0x7f64ac1625f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:21.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.791+0000 7f64b3719640 1 --2- 172.21.15.47:0/3065067586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64ac0d1a00 0x7f64ac0d1de0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:21.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.791+0000 7f64b3719640 1 -- 172.21.15.47:0/3065067586 >> 172.21.15.47:0/3065067586 conn(0x7f64ac0d4fd0 msgr2=0x7f64ac0d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:21.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.791+0000 7f64b3719640 1 -- 172.21.15.47:0/3065067586 shutdown_connections 2024-04-26T23:19:21.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.791+0000 7f64b3719640 1 -- 172.21.15.47:0/3065067586 wait complete. 2024-04-26T23:19:21.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.792+0000 7f64b3719640 1 Processor -- start 2024-04-26T23:19:21.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.792+0000 7f64b3719640 1 -- start start 2024-04-26T23:19:21.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.793+0000 7f64b3719640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64ac0b40d0 0x7f64ac0ba870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:21.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.793+0000 7f64b3719640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f64ac0d1a00 0x7f64ac0c86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:21.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.793+0000 7f64b3719640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64ac0d2320 0x7f64ac15e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:21.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.793+0000 7f64b3719640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f64ac162b30 con 0x7f64ac0b40d0 2024-04-26T23:19:21.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.793+0000 7f64b3719640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f64ac0b5d60 con 0x7f64ac0d2320 2024-04-26T23:19:21.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.793+0000 7f64b3719640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f64ac162cb0 con 0x7f64ac0d1a00 2024-04-26T23:19:21.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.793+0000 7f64b14b5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64ac0b40d0 0x7f64ac0ba870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:21.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.793+0000 7f64b14b5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64ac0b40d0 0x7f64ac0ba870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54624/0 (socket says 172.21.15.47:54624) 2024-04-26T23:19:21.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.793+0000 7f64b14b5640 1 -- 172.21.15.47:0/1245208643 learned_addr learned my addr 172.21.15.47:0/1245208643 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:21.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.793+0000 7f64b1cb6640 1 --2- 172.21.15.47:0/1245208643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64ac0d2320 0x7f64ac15e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:21.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.793+0000 7f64b0cb4640 1 --2- 172.21.15.47:0/1245208643 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f64ac0d1a00 0x7f64ac0c86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:21.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.794+0000 7f64b14b5640 1 -- 172.21.15.47:0/1245208643 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f64ac0d1a00 msgr2=0x7f64ac0c86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:21.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.794+0000 7f64b14b5640 1 --2- 172.21.15.47:0/1245208643 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f64ac0d1a00 0x7f64ac0c86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:21.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.794+0000 7f64b14b5640 1 -- 172.21.15.47:0/1245208643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64ac0d2320 msgr2=0x7f64ac15e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:21.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.794+0000 7f64b14b5640 1 --2- 172.21.15.47:0/1245208643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64ac0d2320 0x7f64ac15e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:21.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.794+0000 7f64b14b5640 1 -- 172.21.15.47:0/1245208643 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64ac0cc7a0 con 0x7f64ac0b40d0 2024-04-26T23:19:21.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.794+0000 7f64b14b5640 1 --2- 172.21.15.47:0/1245208643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64ac0b40d0 0x7f64ac0ba870 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f64a4000c00 tx=0x7f64a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:21.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.794+0000 7f64a9ffb640 1 -- 172.21.15.47:0/1245208643 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a4037050 con 0x7f64ac0b40d0 2024-04-26T23:19:21.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.794+0000 7f64a9ffb640 1 -- 172.21.15.47:0/1245208643 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64a4038040 con 0x7f64ac0b40d0 2024-04-26T23:19:21.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.794+0000 7f64b3719640 1 -- 172.21.15.47:0/1245208643 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64ac0cca80 con 0x7f64ac0b40d0 2024-04-26T23:19:21.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.794+0000 7f64a9ffb640 1 -- 172.21.15.47:0/1245208643 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a4004470 con 0x7f64ac0b40d0 2024-04-26T23:19:21.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.794+0000 7f64b3719640 1 -- 172.21.15.47:0/1245208643 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64ac0cd040 con 0x7f64ac0b40d0 2024-04-26T23:19:21.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.795+0000 7f64b3719640 1 -- 172.21.15.47:0/1245208643 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64ac0694f0 con 0x7f64ac0b40d0 2024-04-26T23:19:21.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.796+0000 7f64a9ffb640 1 -- 172.21.15.47:0/1245208643 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f64a402ac90 con 0x7f64ac0b40d0 2024-04-26T23:19:21.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.796+0000 7f64a9ffb640 1 --2- 172.21.15.47:0/1245208643 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f649806c8f0 0x7f649806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:21.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.796+0000 7f64a9ffb640 1 -- 172.21.15.47:0/1245208643 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f64a40ac770 con 0x7f64ac0b40d0 2024-04-26T23:19:21.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.796+0000 7f64b0cb4640 1 --2- 172.21.15.47:0/1245208643 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f649806c8f0 0x7f649806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:21.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.797+0000 7f64b0cb4640 1 --2- 172.21.15.47:0/1245208643 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f649806c8f0 0x7f649806edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f64ac0c84c0 tx=0x7f64a0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:21.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.800+0000 7f64a9ffb640 1 -- 172.21.15.47:0/1245208643 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f64a407a810 con 0x7f64ac0b40d0 2024-04-26T23:19:21.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.983+0000 7f64b3719640 1 -- 172.21.15.47:0/1245208643 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f64ac0d3250 con 0x7f64ac0b40d0 2024-04-26T23:19:21.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.983+0000 7f64a9ffb640 1 -- 172.21.15.47:0/1245208643 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f64a407a110 con 0x7f64ac0b40d0 2024-04-26T23:19:21.984 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-26T23:19:21.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.985+0000 7f64b3719640 1 -- 172.21.15.47:0/1245208643 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f649806c8f0 msgr2=0x7f649806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:21.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.985+0000 7f64b3719640 1 --2- 172.21.15.47:0/1245208643 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f649806c8f0 0x7f649806edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f64ac0c84c0 tx=0x7f64a0008040 comp rx=0 tx=0).stop 2024-04-26T23:19:21.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.985+0000 7f64b3719640 1 -- 172.21.15.47:0/1245208643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64ac0b40d0 msgr2=0x7f64ac0ba870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:21.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.985+0000 7f64b3719640 1 --2- 172.21.15.47:0/1245208643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64ac0b40d0 0x7f64ac0ba870 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f64a4000c00 tx=0x7f64a40046c0 comp rx=0 tx=0).stop 2024-04-26T23:19:21.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.985+0000 7f64b3719640 1 -- 172.21.15.47:0/1245208643 shutdown_connections 2024-04-26T23:19:21.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.985+0000 7f64b3719640 1 --2- 172.21.15.47:0/1245208643 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f649806c8f0 0x7f649806edb0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:21.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.985+0000 7f64b3719640 1 --2- 172.21.15.47:0/1245208643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64ac0d2320 0x7f64ac15e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:21.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.985+0000 7f64b3719640 1 --2- 172.21.15.47:0/1245208643 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f64ac0d1a00 0x7f64ac0c86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:21.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.985+0000 7f64b3719640 1 --2- 172.21.15.47:0/1245208643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64ac0b40d0 0x7f64ac0ba870 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:21.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.985+0000 7f64b3719640 1 -- 172.21.15.47:0/1245208643 >> 172.21.15.47:0/1245208643 conn(0x7f64ac0d4fd0 msgr2=0x7f64ac1613a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:21.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.985+0000 7f64b3719640 1 -- 172.21.15.47:0/1245208643 shutdown_connections 2024-04-26T23:19:21.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:21.985+0000 7f64b3719640 1 -- 172.21.15.47:0/1245208643 wait complete. 2024-04-26T23:19:21.997 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T23:19:22.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.100+0000 7f761b0b0640 1 Processor -- start 2024-04-26T23:19:22.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.101+0000 7f761b0b0640 1 -- start start 2024-04-26T23:19:22.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.101+0000 7f761b0b0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7614156460 0x7f76140cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:22.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.101+0000 7f761b0b0640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7614156840 con 0x7f76140cba10 2024-04-26T23:19:22.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.101+0000 7f761b0b0640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76140cbf20 con 0x7f761406fea0 2024-04-26T23:19:22.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.101+0000 7f761b0b0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f76140cc200 con 0x7f7614156460 2024-04-26T23:19:22.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.101+0000 7f7613fff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f76140cba10 0x7f761406f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:55542/0 (socket says 172.21.15.47:55542) 2024-04-26T23:19:22.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.101+0000 7f7613fff640 1 -- 172.21.15.47:0/3025050483 learned_addr learned my addr 172.21.15.47:0/3025050483 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:22.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.102+0000 7f7618e4c640 1 --2- 172.21.15.47:0/3025050483 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7614156460 0x7f76140cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:22.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.102+0000 7f76137fe640 1 -- 172.21.15.47:0/3025050483 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2329619160 0 0) 0x7f7614156840 con 0x7f76140cba10 2024-04-26T23:19:22.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.102+0000 7f76137fe640 1 -- 172.21.15.47:0/3025050483 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7604003520 con 0x7f76140cba10 2024-04-26T23:19:22.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.102+0000 7f76137fe640 1 -- 172.21.15.47:0/3025050483 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2571205496 0 0) 0x7f7604003520 con 0x7f76140cba10 2024-04-26T23:19:22.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.102+0000 7f76137fe640 1 -- 172.21.15.47:0/3025050483 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7614156460 msgr2=0x7f76140cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:22.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.102+0000 7f76137fe640 1 --2- 172.21.15.47:0/3025050483 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7614156460 0x7f76140cb4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:22.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.102+0000 7f76137fe640 1 -- 172.21.15.47:0/3025050483 >> v1:172.21.15.203:6789/0 conn(0x7f761406fea0 legacy=0x7f7614072c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:22.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.102+0000 7f76137fe640 1 -- 172.21.15.47:0/3025050483 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76140cc4e0 con 0x7f76140cba10 2024-04-26T23:19:22.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.102+0000 7f76137fe640 1 -- 172.21.15.47:0/3025050483 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7f7600001670 con 0x7f76140cba10 2024-04-26T23:19:22.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.102+0000 7f76137fe640 1 -- 172.21.15.47:0/3025050483 >> v1:172.21.15.47:6789/0 conn(0x7f76140cba10 legacy=0x7f761406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:22.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.102+0000 7f76137fe640 1 --2- 172.21.15.47:0/3025050483 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7614156460 0x7f76040039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:22.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.102+0000 7f76137fe640 1 --2- 172.21.15.47:0/3025050483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7604003f60 0x7f7604004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:22.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.102+0000 7f76137fe640 1 --2- 172.21.15.47:0/3025050483 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7604004d40 0x7f76040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:22.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.102+0000 7f76137fe640 1 -- 172.21.15.47:0/3025050483 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f76140cc200 con 0x7f7604004d40 2024-04-26T23:19:22.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.102+0000 7f76137fe640 1 -- 172.21.15.47:0/3025050483 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f760400baa0 con 0x7f7604003f60 2024-04-26T23:19:22.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.102+0000 7f76137fe640 1 -- 172.21.15.47:0/3025050483 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f760400bca0 con 0x7f7614156460 2024-04-26T23:19:22.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.102+0000 7f761964d640 1 --2- 172.21.15.47:0/3025050483 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7604004d40 0x7f76040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:22.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.102+0000 7f7618e4c640 1 --2- 172.21.15.47:0/3025050483 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7614156460 0x7f76040039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:22.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.102+0000 7f7613fff640 1 --2- 172.21.15.47:0/3025050483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7604003f60 0x7f7604004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:22.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.103+0000 7f761964d640 1 -- 172.21.15.47:0/3025050483 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7614156460 msgr2=0x7f76040039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:22.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.103+0000 7f761964d640 1 --2- 172.21.15.47:0/3025050483 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7614156460 0x7f76040039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:22.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.103+0000 7f761964d640 1 -- 172.21.15.47:0/3025050483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7604003f60 msgr2=0x7f7604004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:22.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.103+0000 7f761964d640 1 --2- 172.21.15.47:0/3025050483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7604003f60 0x7f7604004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:22.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.103+0000 7f761964d640 1 -- 172.21.15.47:0/3025050483 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7604003520 con 0x7f7604004d40 2024-04-26T23:19:22.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.103+0000 7f761964d640 1 --2- 172.21.15.47:0/3025050483 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7604004d40 0x7f76040051c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f76140d0bb0 tx=0x7f7604005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fe5a2a95a9f7c85 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:22.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.103+0000 7f76137fe640 1 -- 172.21.15.47:0/3025050483 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760c001a70 con 0x7f7604004d40 2024-04-26T23:19:22.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.103+0000 7f76137fe640 1 -- 172.21.15.47:0/3025050483 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f760c02ae10 con 0x7f7604004d40 2024-04-26T23:19:22.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.103+0000 7f76137fe640 1 -- 172.21.15.47:0/3025050483 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760c003bb0 con 0x7f7604004d40 2024-04-26T23:19:22.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.104+0000 7f761b0b0640 1 -- 172.21.15.47:0/3025050483 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7604004d40 msgr2=0x7f76040051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:22.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.104+0000 7f761b0b0640 1 --2- 172.21.15.47:0/3025050483 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7604004d40 0x7f76040051c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f76140d0bb0 tx=0x7f7604005830 comp rx=0 tx=0).stop 2024-04-26T23:19:22.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.104+0000 7f761964d640 1 -- 172.21.15.47:0/3025050483 reap_dead start 2024-04-26T23:19:22.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.104+0000 7f761b0b0640 1 -- 172.21.15.47:0/3025050483 shutdown_connections 2024-04-26T23:19:22.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.104+0000 7f761b0b0640 1 -- 172.21.15.47:0/3025050483 >> 172.21.15.47:0/3025050483 conn(0x7f76140d0be0 msgr2=0x7f76140d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:22.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.104+0000 7f761b0b0640 1 -- 172.21.15.47:0/3025050483 shutdown_connections 2024-04-26T23:19:22.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.104+0000 7f761b0b0640 1 -- 172.21.15.47:0/3025050483 wait complete. 2024-04-26T23:19:22.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.105+0000 7f761b0b0640 1 Processor -- start 2024-04-26T23:19:22.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.105+0000 7f761b0b0640 1 -- start start 2024-04-26T23:19:22.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.106+0000 7f761b0b0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7604004d40 0x7f76140b1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:22.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.106+0000 7f761b0b0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76140b3860 0x7f761415ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:22.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.106+0000 7f761b0b0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f761415c400 0x7f761415c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:22.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.106+0000 7f761b0b0640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f761406e260 con 0x7f761415c400 2024-04-26T23:19:22.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.106+0000 7f761b0b0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7614072700 con 0x7f76140b3860 2024-04-26T23:19:22.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.106+0000 7f761b0b0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f761406e3e0 con 0x7f7604004d40 2024-04-26T23:19:22.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.106+0000 7f761964d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f761415c400 0x7f761415c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:22.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.106+0000 7f761964d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f761415c400 0x7f761415c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54648/0 (socket says 172.21.15.47:54648) 2024-04-26T23:19:22.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.106+0000 7f761964d640 1 -- 172.21.15.47:0/1184975773 learned_addr learned my addr 172.21.15.47:0/1184975773 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:22.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.106+0000 7f7618e4c640 1 --2- 172.21.15.47:0/1184975773 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7604004d40 0x7f76140b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:22.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.106+0000 7f7613fff640 1 --2- 172.21.15.47:0/1184975773 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76140b3860 0x7f761415ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:22.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.106+0000 7f761964d640 1 -- 172.21.15.47:0/1184975773 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7604004d40 msgr2=0x7f76140b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:22.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.107+0000 7f761964d640 1 --2- 172.21.15.47:0/1184975773 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7604004d40 0x7f76140b1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:22.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.107+0000 7f761964d640 1 -- 172.21.15.47:0/1184975773 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76140b3860 msgr2=0x7f761415ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:22.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.107+0000 7f761964d640 1 --2- 172.21.15.47:0/1184975773 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76140b3860 0x7f761415ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:22.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.107+0000 7f761964d640 1 -- 172.21.15.47:0/1184975773 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76140caa50 con 0x7f761415c400 2024-04-26T23:19:22.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.107+0000 7f761964d640 1 --2- 172.21.15.47:0/1184975773 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f761415c400 0x7f761415c860 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f760c001f10 tx=0x7f760c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:22.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.107+0000 7f76117fa640 1 -- 172.21.15.47:0/1184975773 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760c037040 con 0x7f761415c400 2024-04-26T23:19:22.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.107+0000 7f76117fa640 1 -- 172.21.15.47:0/1184975773 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f760c02ac10 con 0x7f761415c400 2024-04-26T23:19:22.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.107+0000 7f761b0b0640 1 -- 172.21.15.47:0/1184975773 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76140b2500 con 0x7f761415c400 2024-04-26T23:19:22.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.107+0000 7f76117fa640 1 -- 172.21.15.47:0/1184975773 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760c001a70 con 0x7f761415c400 2024-04-26T23:19:22.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.107+0000 7f761b0b0640 1 -- 172.21.15.47:0/1184975773 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f761414a780 con 0x7f761415c400 2024-04-26T23:19:22.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.108+0000 7f761b0b0640 1 -- 172.21.15.47:0/1184975773 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76140694f0 con 0x7f761415c400 2024-04-26T23:19:22.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.108+0000 7f76117fa640 1 -- 172.21.15.47:0/1184975773 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f760c042020 con 0x7f761415c400 2024-04-26T23:19:22.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.109+0000 7f76117fa640 1 --2- 172.21.15.47:0/1184975773 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f75f806c8f0 0x7f75f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:22.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.109+0000 7f76117fa640 1 -- 172.21.15.47:0/1184975773 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f760c0ab470 con 0x7f761415c400 2024-04-26T23:19:22.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.109+0000 7f7618e4c640 1 --2- 172.21.15.47:0/1184975773 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f75f806c8f0 0x7f75f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:22.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.110+0000 7f7618e4c640 1 --2- 172.21.15.47:0/1184975773 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f75f806c8f0 0x7f75f806edb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7608009b00 tx=0x7f7608008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:22.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.113+0000 7f76117fa640 1 -- 172.21.15.47:0/1184975773 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f760c0795f0 con 0x7f761415c400 2024-04-26T23:19:22.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.298+0000 7f761b0b0640 1 -- 172.21.15.47:0/1184975773 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f761406de80 con 0x7f761415c400 2024-04-26T23:19:22.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.299+0000 7f76117fa640 1 -- 172.21.15.47:0/1184975773 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f760c078ef0 con 0x7f761415c400 2024-04-26T23:19:22.300 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-26T23:19:22.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.301+0000 7f761b0b0640 1 -- 172.21.15.47:0/1184975773 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f75f806c8f0 msgr2=0x7f75f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:22.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.301+0000 7f761b0b0640 1 --2- 172.21.15.47:0/1184975773 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f75f806c8f0 0x7f75f806edb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7608009b00 tx=0x7f7608008040 comp rx=0 tx=0).stop 2024-04-26T23:19:22.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.301+0000 7f761b0b0640 1 -- 172.21.15.47:0/1184975773 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f761415c400 msgr2=0x7f761415c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:22.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.301+0000 7f761b0b0640 1 --2- 172.21.15.47:0/1184975773 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f761415c400 0x7f761415c860 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f760c001f10 tx=0x7f760c0014a0 comp rx=0 tx=0).stop 2024-04-26T23:19:22.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.301+0000 7f761b0b0640 1 -- 172.21.15.47:0/1184975773 shutdown_connections 2024-04-26T23:19:22.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.301+0000 7f761b0b0640 1 --2- 172.21.15.47:0/1184975773 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f75f806c8f0 0x7f75f806edb0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:22.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.301+0000 7f761b0b0640 1 --2- 172.21.15.47:0/1184975773 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f761415c400 0x7f761415c860 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:22.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.301+0000 7f761b0b0640 1 --2- 172.21.15.47:0/1184975773 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76140b3860 0x7f761415ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:22.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.301+0000 7f761b0b0640 1 --2- 172.21.15.47:0/1184975773 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7604004d40 0x7f76140b1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:22.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.301+0000 7f761b0b0640 1 -- 172.21.15.47:0/1184975773 >> 172.21.15.47:0/1184975773 conn(0x7f76140d0be0 msgr2=0x7f761406e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:22.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.301+0000 7f761b0b0640 1 -- 172.21.15.47:0/1184975773 shutdown_connections 2024-04-26T23:19:22.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.301+0000 7f761b0b0640 1 -- 172.21.15.47:0/1184975773 wait complete. 2024-04-26T23:19:22.313 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T23:19:22.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.414+0000 7fee62eb8640 1 Processor -- start 2024-04-26T23:19:22.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.414+0000 7fee62eb8640 1 -- start start 2024-04-26T23:19:22.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.415+0000 7fee62eb8640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fee5c0d1b30 0x7fee5c0d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:22.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.415+0000 7fee62eb8640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee5c0d10d0 con 0x7fee5c162810 2024-04-26T23:19:22.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.415+0000 7fee62eb8640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee5c0d1370 con 0x7fee5c0b2340 2024-04-26T23:19:22.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.415+0000 7fee62eb8640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fee5c0d0750 con 0x7fee5c0d1b30 2024-04-26T23:19:22.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.415+0000 7fee61455640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fee5c0d1b30 0x7fee5c0d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:22.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.415+0000 7fee60c54640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fee5c162810 0x7fee5c070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:55554/0 (socket says 172.21.15.47:55554) 2024-04-26T23:19:22.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.415+0000 7fee60c54640 1 -- 172.21.15.47:0/26211347 learned_addr learned my addr 172.21.15.47:0/26211347 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:22.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.416+0000 7fee61455640 1 -- 172.21.15.47:0/26211347 >> v1:172.21.15.203:6789/0 conn(0x7fee5c0b2340 legacy=0x7fee5c0cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:22.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.416+0000 7fee61455640 1 -- 172.21.15.47:0/26211347 >> v1:172.21.15.47:6789/0 conn(0x7fee5c162810 legacy=0x7fee5c070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:22.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.416+0000 7fee61455640 1 -- 172.21.15.47:0/26211347 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee5c0d01b0 con 0x7fee5c0d1b30 2024-04-26T23:19:22.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.416+0000 7fee5b7fe640 1 -- 172.21.15.47:0/26211347 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3502441920 0 0) 0x7fee5c0d10d0 con 0x7fee5c162810 2024-04-26T23:19:22.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.416+0000 7fee61455640 1 --2- 172.21.15.47:0/26211347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee5c0d1b30 0x7fee5c0d1fc0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fee4c0097a0 tx=0x7fee4c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbae55bbb6476e2c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:22.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.416+0000 7fee5b7fe640 1 -- 172.21.15.47:0/26211347 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4c037070 con 0x7fee5c0d1b30 2024-04-26T23:19:22.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.416+0000 7fee5b7fe640 1 -- 172.21.15.47:0/26211347 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee4c032440 con 0x7fee5c0d1b30 2024-04-26T23:19:22.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.416+0000 7fee5b7fe640 1 -- 172.21.15.47:0/26211347 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4c032810 con 0x7fee5c0d1b30 2024-04-26T23:19:22.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.417+0000 7fee62eb8640 1 -- 172.21.15.47:0/26211347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee5c0d1b30 msgr2=0x7fee5c0d1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:22.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.417+0000 7fee62eb8640 1 --2- 172.21.15.47:0/26211347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee5c0d1b30 0x7fee5c0d1fc0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fee4c0097a0 tx=0x7fee4c02a1e0 comp rx=0 tx=0).stop 2024-04-26T23:19:22.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.417+0000 7fee62eb8640 1 -- 172.21.15.47:0/26211347 shutdown_connections 2024-04-26T23:19:22.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.417+0000 7fee62eb8640 1 --2- 172.21.15.47:0/26211347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee5c0d1b30 0x7fee5c0d1fc0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:22.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.417+0000 7fee62eb8640 1 -- 172.21.15.47:0/26211347 >> 172.21.15.47:0/26211347 conn(0x7fee5c0d2da0 msgr2=0x7fee5c0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:22.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.417+0000 7fee62eb8640 1 -- 172.21.15.47:0/26211347 shutdown_connections 2024-04-26T23:19:22.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.417+0000 7fee62eb8640 1 -- 172.21.15.47:0/26211347 wait complete. 2024-04-26T23:19:22.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.418+0000 7fee62eb8640 1 Processor -- start 2024-04-26T23:19:22.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.418+0000 7fee62eb8640 1 -- start start 2024-04-26T23:19:22.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.419+0000 7fee62eb8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee5c0b2340 0x7fee5c1f4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:22.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.419+0000 7fee62eb8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee5c0d1b30 0x7fee5c14c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:22.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.419+0000 7fee62eb8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee5c162810 0x7fee5c15bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:22.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.419+0000 7fee62eb8640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fee5c0cdb00 con 0x7fee5c162810 2024-04-26T23:19:22.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.419+0000 7fee62eb8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fee5c0d16a0 con 0x7fee5c0b2340 2024-04-26T23:19:22.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.419+0000 7fee62eb8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fee5c070b50 con 0x7fee5c0d1b30 2024-04-26T23:19:22.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.419+0000 7fee61455640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee5c162810 0x7fee5c15bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:22.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.419+0000 7fee61455640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee5c162810 0x7fee5c15bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54656/0 (socket says 172.21.15.47:54656) 2024-04-26T23:19:22.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.419+0000 7fee61455640 1 -- 172.21.15.47:0/1575046823 learned_addr learned my addr 172.21.15.47:0/1575046823 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:22.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.420+0000 7fee60c54640 1 --2- 172.21.15.47:0/1575046823 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee5c0b2340 0x7fee5c1f4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:22.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.420+0000 7fee5bfff640 1 --2- 172.21.15.47:0/1575046823 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee5c0d1b30 0x7fee5c14c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:22.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.420+0000 7fee61455640 1 -- 172.21.15.47:0/1575046823 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee5c0d1b30 msgr2=0x7fee5c14c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:22.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.420+0000 7fee61455640 1 --2- 172.21.15.47:0/1575046823 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee5c0d1b30 0x7fee5c14c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:22.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.420+0000 7fee61455640 1 -- 172.21.15.47:0/1575046823 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee5c0b2340 msgr2=0x7fee5c1f4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:22.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.420+0000 7fee61455640 1 --2- 172.21.15.47:0/1575046823 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee5c0b2340 0x7fee5c1f4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:22.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.420+0000 7fee61455640 1 -- 172.21.15.47:0/1575046823 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee5c14cd80 con 0x7fee5c162810 2024-04-26T23:19:22.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.420+0000 7fee60c54640 1 --2- 172.21.15.47:0/1575046823 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee5c0b2340 0x7fee5c1f4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:19:22.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.420+0000 7fee61455640 1 --2- 172.21.15.47:0/1575046823 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee5c162810 0x7fee5c15bf70 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fee4c035040 tx=0x7fee4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:22.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.420+0000 7fee597fa640 1 -- 172.21.15.47:0/1575046823 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4c037040 con 0x7fee5c162810 2024-04-26T23:19:22.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.420+0000 7fee597fa640 1 -- 172.21.15.47:0/1575046823 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee4c004230 con 0x7fee5c162810 2024-04-26T23:19:22.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.420+0000 7fee62eb8640 1 -- 172.21.15.47:0/1575046823 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee5c1f5710 con 0x7fee5c162810 2024-04-26T23:19:22.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.420+0000 7fee597fa640 1 -- 172.21.15.47:0/1575046823 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4c032e20 con 0x7fee5c162810 2024-04-26T23:19:22.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.420+0000 7fee62eb8640 1 -- 172.21.15.47:0/1575046823 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee5c1f5cd0 con 0x7fee5c162810 2024-04-26T23:19:22.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.421+0000 7fee62eb8640 1 -- 172.21.15.47:0/1575046823 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee5c0694f0 con 0x7fee5c162810 2024-04-26T23:19:22.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.422+0000 7fee597fa640 1 -- 172.21.15.47:0/1575046823 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fee4c030770 con 0x7fee5c162810 2024-04-26T23:19:22.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.422+0000 7fee597fa640 1 --2- 172.21.15.47:0/1575046823 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fee3c06c8f0 0x7fee3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:22.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.422+0000 7fee597fa640 1 -- 172.21.15.47:0/1575046823 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7fee4c0ad1f0 con 0x7fee5c162810 2024-04-26T23:19:22.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.422+0000 7fee60c54640 1 --2- 172.21.15.47:0/1575046823 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fee3c06c8f0 0x7fee3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:22.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.423+0000 7fee60c54640 1 --2- 172.21.15.47:0/1575046823 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fee3c06c8f0 0x7fee3c06edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fee54007fd0 tx=0x7fee54009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:22.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.427+0000 7fee597fa640 1 -- 172.21.15.47:0/1575046823 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fee5c14cd80 con 0x7fee5c162810 2024-04-26T23:19:22.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.623+0000 7fee62eb8640 1 -- 172.21.15.47:0/1575046823 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fee5c0702e0 con 0x7fee5c162810 2024-04-26T23:19:22.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.624+0000 7fee597fa640 1 -- 172.21.15.47:0/1575046823 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7fee4c07b200 con 0x7fee5c162810 2024-04-26T23:19:22.625 INFO:teuthology.orchestra.run.smithi047.stderr:listed 0 entries 2024-04-26T23:19:22.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.625+0000 7fee62eb8640 1 -- 172.21.15.47:0/1575046823 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fee3c06c8f0 msgr2=0x7fee3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:22.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.626+0000 7fee62eb8640 1 --2- 172.21.15.47:0/1575046823 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fee3c06c8f0 0x7fee3c06edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fee54007fd0 tx=0x7fee54009040 comp rx=0 tx=0).stop 2024-04-26T23:19:22.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.626+0000 7fee62eb8640 1 -- 172.21.15.47:0/1575046823 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee5c162810 msgr2=0x7fee5c15bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:22.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.626+0000 7fee62eb8640 1 --2- 172.21.15.47:0/1575046823 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee5c162810 0x7fee5c15bf70 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fee4c035040 tx=0x7fee4c0046c0 comp rx=0 tx=0).stop 2024-04-26T23:19:22.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.626+0000 7fee62eb8640 1 -- 172.21.15.47:0/1575046823 shutdown_connections 2024-04-26T23:19:22.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.626+0000 7fee62eb8640 1 --2- 172.21.15.47:0/1575046823 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fee3c06c8f0 0x7fee3c06edb0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:22.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.626+0000 7fee62eb8640 1 --2- 172.21.15.47:0/1575046823 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee5c162810 0x7fee5c15bf70 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:22.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.626+0000 7fee62eb8640 1 --2- 172.21.15.47:0/1575046823 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee5c0d1b30 0x7fee5c14c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:22.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.626+0000 7fee62eb8640 1 --2- 172.21.15.47:0/1575046823 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee5c0b2340 0x7fee5c1f4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:22.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.626+0000 7fee62eb8640 1 -- 172.21.15.47:0/1575046823 >> 172.21.15.47:0/1575046823 conn(0x7fee5c0d2da0 msgr2=0x7fee5c0d3180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:22.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.626+0000 7fee62eb8640 1 -- 172.21.15.47:0/1575046823 shutdown_connections 2024-04-26T23:19:22.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.626+0000 7fee62eb8640 1 -- 172.21.15.47:0/1575046823 wait complete. 2024-04-26T23:19:22.637 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T23:19:22.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:22.638 DEBUG:teuthology.orchestra.run.smithi047:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-26T23:19:22.663 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi047.front.sepia.ceph.com 2024-04-26T23:19:22.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:22.663 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T23:19:22.719 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-26T23:19:22.719 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:19:22.738 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-26T23:19:22.738 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:19:22.797 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-26T23:19:22.797 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:19:22.854 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:19:22.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.955+0000 7fdd40d71640 1 Processor -- start 2024-04-26T23:19:22.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.955+0000 7fdd40d71640 1 -- start start 2024-04-26T23:19:22.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.956+0000 7fdd40d71640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdd3c162880 0x7fdd3c0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:22.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.956+0000 7fdd40d71640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd3c0b7e50 con 0x7fdd3c0b5f20 2024-04-26T23:19:22.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.956+0000 7fdd40d71640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd3c0d1800 con 0x7fdd3c0d3bc0 2024-04-26T23:19:22.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.956+0000 7fdd40d71640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fdd3c0d0580 con 0x7fdd3c162880 2024-04-26T23:19:22.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.956+0000 7fdd3a59c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdd3c162880 0x7fdd3c0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:22.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.956+0000 7fdd33fff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fdd3c0b5f20 0x7fdd3c0d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:55556/0 (socket says 172.21.15.47:55556) 2024-04-26T23:19:22.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.956+0000 7fdd3a59c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdd3c162880 0x7fdd3c0d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54670/0 (socket says 172.21.15.47:54670) 2024-04-26T23:19:22.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.956+0000 7fdd33fff640 1 -- 172.21.15.47:0/2940757030 learned_addr learned my addr 172.21.15.47:0/2940757030 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:22.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.957+0000 7fdd3a59c640 1 -- 172.21.15.47:0/2940757030 >> v1:172.21.15.203:6789/0 conn(0x7fdd3c0d3bc0 legacy=0x7fdd3c0b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:22.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.957+0000 7fdd3a59c640 1 -- 172.21.15.47:0/2940757030 >> v1:172.21.15.47:6789/0 conn(0x7fdd3c0b5f20 legacy=0x7fdd3c0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:22.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.957+0000 7fdd3a59c640 1 -- 172.21.15.47:0/2940757030 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd3c0d07d0 con 0x7fdd3c162880 2024-04-26T23:19:22.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.957+0000 7fdd39d9b640 1 -- 172.21.15.47:0/2940757030 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 707936018 0 0) 0x7fdd3c0b7e50 con 0x7fdd3c0b5f20 2024-04-26T23:19:22.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.957+0000 7fdd3a59c640 1 --2- 172.21.15.47:0/2940757030 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd3c162880 0x7fdd3c0d2cb0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c0097a0 tx=0x7fdd2c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ea93f44b84e631d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:22.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.957+0000 7fdd39d9b640 1 -- 172.21.15.47:0/2940757030 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c037070 con 0x7fdd3c162880 2024-04-26T23:19:22.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.958+0000 7fdd39d9b640 1 -- 172.21.15.47:0/2940757030 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd2c032440 con 0x7fdd3c162880 2024-04-26T23:19:22.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.958+0000 7fdd39d9b640 1 -- 172.21.15.47:0/2940757030 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c032810 con 0x7fdd3c162880 2024-04-26T23:19:22.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.958+0000 7fdd40d71640 1 -- 172.21.15.47:0/2940757030 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd3c162880 msgr2=0x7fdd3c0d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:22.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.958+0000 7fdd40d71640 1 --2- 172.21.15.47:0/2940757030 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd3c162880 0x7fdd3c0d2cb0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c0097a0 tx=0x7fdd2c02a1e0 comp rx=0 tx=0).stop 2024-04-26T23:19:22.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.958+0000 7fdd40d71640 1 -- 172.21.15.47:0/2940757030 shutdown_connections 2024-04-26T23:19:22.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.958+0000 7fdd40d71640 1 --2- 172.21.15.47:0/2940757030 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd3c162880 0x7fdd3c0d2cb0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:22.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.958+0000 7fdd40d71640 1 -- 172.21.15.47:0/2940757030 >> 172.21.15.47:0/2940757030 conn(0x7fdd3c0d4f60 msgr2=0x7fdd3c0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:22.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.958+0000 7fdd40d71640 1 -- 172.21.15.47:0/2940757030 shutdown_connections 2024-04-26T23:19:22.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.958+0000 7fdd40d71640 1 -- 172.21.15.47:0/2940757030 wait complete. 2024-04-26T23:19:22.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.959+0000 7fdd40d71640 1 Processor -- start 2024-04-26T23:19:22.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.959+0000 7fdd40d71640 1 -- start start 2024-04-26T23:19:22.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.960+0000 7fdd40d71640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd3c0b5f20 0x7fdd3c1f3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:22.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.960+0000 7fdd40d71640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd3c0d3bc0 0x7fdd3c1f4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:22.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.960+0000 7fdd40d71640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd3c162880 0x7fdd3c1f83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:22.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.960+0000 7fdd40d71640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3c162c60 con 0x7fdd3c0b5f20 2024-04-26T23:19:22.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.960+0000 7fdd40d71640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3c0d0d90 con 0x7fdd3c0d3bc0 2024-04-26T23:19:22.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.960+0000 7fdd40d71640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fdd3c162de0 con 0x7fdd3c162880 2024-04-26T23:19:22.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.960+0000 7fdd3a59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd3c0b5f20 0x7fdd3c1f3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:22.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.960+0000 7fdd3a59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd3c0b5f20 0x7fdd3c1f3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54678/0 (socket says 172.21.15.47:54678) 2024-04-26T23:19:22.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.960+0000 7fdd3a59c640 1 -- 172.21.15.47:0/4045092613 learned_addr learned my addr 172.21.15.47:0/4045092613 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:22.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.960+0000 7fdd33fff640 1 --2- 172.21.15.47:0/4045092613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd3c0d3bc0 0x7fdd3c1f4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:22.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.960+0000 7fdd3ad9d640 1 --2- 172.21.15.47:0/4045092613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd3c162880 0x7fdd3c1f83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:22.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.961+0000 7fdd3a59c640 1 -- 172.21.15.47:0/4045092613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd3c162880 msgr2=0x7fdd3c1f83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:22.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.961+0000 7fdd3a59c640 1 --2- 172.21.15.47:0/4045092613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd3c162880 0x7fdd3c1f83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:22.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.961+0000 7fdd3a59c640 1 -- 172.21.15.47:0/4045092613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd3c0d3bc0 msgr2=0x7fdd3c1f4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:22.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.961+0000 7fdd3a59c640 1 --2- 172.21.15.47:0/4045092613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd3c0d3bc0 0x7fdd3c1f4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:22.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.961+0000 7fdd3a59c640 1 -- 172.21.15.47:0/4045092613 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd3c0ccf20 con 0x7fdd3c0b5f20 2024-04-26T23:19:22.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.961+0000 7fdd3a59c640 1 --2- 172.21.15.47:0/4045092613 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd3c0b5f20 0x7fdd3c1f3ff0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c000c00 tx=0x7fdd2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:22.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.961+0000 7fdd32ffd640 1 -- 172.21.15.47:0/4045092613 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c041070 con 0x7fdd3c0b5f20 2024-04-26T23:19:22.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.961+0000 7fdd32ffd640 1 -- 172.21.15.47:0/4045092613 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd2c03c930 con 0x7fdd3c0b5f20 2024-04-26T23:19:22.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.961+0000 7fdd32ffd640 1 -- 172.21.15.47:0/4045092613 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c037040 con 0x7fdd3c0b5f20 2024-04-26T23:19:22.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.961+0000 7fdd40d71640 1 -- 172.21.15.47:0/4045092613 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd3c0cd200 con 0x7fdd3c0b5f20 2024-04-26T23:19:22.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.961+0000 7fdd40d71640 1 -- 172.21.15.47:0/4045092613 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd3c0cd7c0 con 0x7fdd3c0b5f20 2024-04-26T23:19:22.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.962+0000 7fdd40d71640 1 -- 172.21.15.47:0/4045092613 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd3c0694f0 con 0x7fdd3c0b5f20 2024-04-26T23:19:22.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.962+0000 7fdd32ffd640 1 -- 172.21.15.47:0/4045092613 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdd2c044070 con 0x7fdd3c0b5f20 2024-04-26T23:19:22.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.963+0000 7fdd32ffd640 1 --2- 172.21.15.47:0/4045092613 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdd1c06c920 0x7fdd1c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:22.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.963+0000 7fdd32ffd640 1 -- 172.21.15.47:0/4045092613 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7fdd2c0aca00 con 0x7fdd3c0b5f20 2024-04-26T23:19:22.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.963+0000 7fdd33fff640 1 --2- 172.21.15.47:0/4045092613 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdd1c06c920 0x7fdd1c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:22.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.963+0000 7fdd33fff640 1 --2- 172.21.15.47:0/4045092613 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdd1c06c920 0x7fdd1c06ede0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c0d3b10 tx=0x7fdd24007c00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:22.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:22.967+0000 7fdd32ffd640 1 -- 172.21.15.47:0/4045092613 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdd3c0d1800 con 0x7fdd3c0b5f20 2024-04-26T23:19:23.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.151+0000 7fdd40d71640 1 -- 172.21.15.47:0/4045092613 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdd3c0cb4a0 con 0x7fdd3c0b5f20 2024-04-26T23:19:23.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.152+0000 7fdd32ffd640 1 -- 172.21.15.47:0/4045092613 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18857 (secure 0 0 0) 0x7fdd2c07abc0 con 0x7fdd3c0b5f20 2024-04-26T23:19:23.153 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:19:23.153 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":26,"fsid":"399006d3-4db1-43e6-ba81-a79a276eabfd","created":"2024-04-26T23:18:06.731211+0000","modified":"2024-04-26T23:19:12.124051+0000","last_up_change":"2024-04-26T23:18:17.808736+0000","last_in_change":"2024-04-26T23:18:08.945859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:18:16.937492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:18:29.033575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:18:30.318499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:18:34.014536+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ee091567-fea0-4a14-bf4e-0e16c20bb2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6819","nonce":326520779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6821","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6823","nonce":326520779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6829","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6831","nonce":326520779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6825","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6827","nonce":326520779}]},"public_addr":"172.21.15.47:6819/326520779","cluster_addr":"172.21.15.47:6823/326520779","heartbeat_back_addr":"172.21.15.47:6831/326520779","heartbeat_front_addr":"172.21.15.47:6827/326520779","state":["exists","up"]},{"osd":1,"uuid":"c14440a0-bba5-4931-a6db-b736c08e869b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6809","nonce":1912610931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6811","nonce":1912610931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6815","nonce":1912610931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6813","nonce":1912610931}]},"public_addr":"172.21.15.47:6809/1912610931","cluster_addr":"172.21.15.47:6811/1912610931","heartbeat_back_addr":"172.21.15.47:6815/1912610931","heartbeat_front_addr":"172.21.15.47:6813/1912610931","state":["exists","up"]},{"osd":2,"uuid":"3940f8de-c9b2-4cd3-a911-9c301729eb30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6801","nonce":2569869305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6803","nonce":2569869305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6807","nonce":2569869305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6805","nonce":2569869305}]},"public_addr":"172.21.15.47:6801/2569869305","cluster_addr":"172.21.15.47:6803/2569869305","heartbeat_back_addr":"172.21.15.47:6807/2569869305","heartbeat_front_addr":"172.21.15.47:6805/2569869305","state":["exists","up"]},{"osd":3,"uuid":"1286a67e-8958-426a-8c2e-bc85498c619c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6818","nonce":2791411683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6822","nonce":2791411683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6830","nonce":2791411683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6826","nonce":2791411683}]},"public_addr":"172.21.15.47:6818/2791411683","cluster_addr":"172.21.15.47:6822/2791411683","heartbeat_back_addr":"172.21.15.47:6830/2791411683","heartbeat_front_addr":"172.21.15.47:6826/2791411683","state":["exists","up"]},{"osd":4,"uuid":"d8ee2c20-e5b7-46c2-9999-b234070126cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6809","nonce":441313104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6811","nonce":441313104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6815","nonce":441313104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6813","nonce":441313104}]},"public_addr":"172.21.15.203:6809/441313104","cluster_addr":"172.21.15.203:6811/441313104","heartbeat_back_addr":"172.21.15.203:6815/441313104","heartbeat_front_addr":"172.21.15.203:6813/441313104","state":["exists","up"]},{"osd":5,"uuid":"b453a8f0-f324-4840-9ae0-d67bdadb9f8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6801","nonce":56838444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6803","nonce":56838444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6807","nonce":56838444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6805","nonce":56838444}]},"public_addr":"172.21.15.203:6801/56838444","cluster_addr":"172.21.15.203:6803/56838444","heartbeat_back_addr":"172.21.15.203:6807/56838444","heartbeat_front_addr":"172.21.15.203:6805/56838444","state":["exists","up"]},{"osd":6,"uuid":"a6d31951-2d4b-4e8d-a510-6283b01dc158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6825","nonce":2738517080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6827","nonce":2738517080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6831","nonce":2738517080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6829","nonce":2738517080}]},"public_addr":"172.21.15.203:6825/2738517080","cluster_addr":"172.21.15.203:6827/2738517080","heartbeat_back_addr":"172.21.15.203:6831/2738517080","heartbeat_front_addr":"172.21.15.203:6829/2738517080","state":["exists","up"]},{"osd":7,"uuid":"7119b8c0-fe96-402d-9e75-7b509d9d44d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6817","nonce":1315565058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6819","nonce":1315565058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6823","nonce":1315565058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6821","nonce":1315565058}]},"public_addr":"172.21.15.203:6817/1315565058","cluster_addr":"172.21.15.203:6819/1315565058","heartbeat_back_addr":"172.21.15.203:6823/1315565058","heartbeat_front_addr":"172.21.15.203:6821/1315565058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.121921+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.137494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.084814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.111593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.072430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.125066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.097867+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.057061+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":5,"to":1}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:19:23.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.154+0000 7fdd40d71640 1 -- 172.21.15.47:0/4045092613 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdd1c06c920 msgr2=0x7fdd1c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:23.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.154+0000 7fdd40d71640 1 --2- 172.21.15.47:0/4045092613 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdd1c06c920 0x7fdd1c06ede0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c0d3b10 tx=0x7fdd24007c00 comp rx=0 tx=0).stop 2024-04-26T23:19:23.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.154+0000 7fdd40d71640 1 -- 172.21.15.47:0/4045092613 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd3c0b5f20 msgr2=0x7fdd3c1f3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:23.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.154+0000 7fdd40d71640 1 --2- 172.21.15.47:0/4045092613 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd3c0b5f20 0x7fdd3c1f3ff0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c000c00 tx=0x7fdd2c0046c0 comp rx=0 tx=0).stop 2024-04-26T23:19:23.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.154+0000 7fdd40d71640 1 -- 172.21.15.47:0/4045092613 shutdown_connections 2024-04-26T23:19:23.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.154+0000 7fdd40d71640 1 --2- 172.21.15.47:0/4045092613 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdd1c06c920 0x7fdd1c06ede0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:23.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.154+0000 7fdd40d71640 1 --2- 172.21.15.47:0/4045092613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd3c162880 0x7fdd3c1f83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:23.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.154+0000 7fdd40d71640 1 --2- 172.21.15.47:0/4045092613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd3c0d3bc0 0x7fdd3c1f4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:23.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.154+0000 7fdd40d71640 1 --2- 172.21.15.47:0/4045092613 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd3c0b5f20 0x7fdd3c1f3ff0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:23.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.154+0000 7fdd40d71640 1 -- 172.21.15.47:0/4045092613 >> 172.21.15.47:0/4045092613 conn(0x7fdd3c0d4f60 msgr2=0x7fdd3c160af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:23.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.154+0000 7fdd40d71640 1 -- 172.21.15.47:0/4045092613 shutdown_connections 2024-04-26T23:19:23.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.154+0000 7fdd40d71640 1 -- 172.21.15.47:0/4045092613 wait complete. 2024-04-26T23:19:23.167 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:18:16.937492+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:18:29.033575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:18:30.318499+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T23:18:34.014536+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T23:19:23.167 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:19:23.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.270+0000 7f3f78b0a640 1 Processor -- start 2024-04-26T23:19:23.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.270+0000 7f3f78b0a640 1 -- start start 2024-04-26T23:19:23.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.271+0000 7f3f78b0a640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3f740d3bc0 0x7f3f740d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:23.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.271+0000 7f3f78b0a640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f740d1040 con 0x7f3f7415f550 2024-04-26T23:19:23.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.271+0000 7f3f78b0a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f740c9680 con 0x7f3f7415f170 2024-04-26T23:19:23.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.271+0000 7f3f78b0a640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f3f740c9430 con 0x7f3f740d3bc0 2024-04-26T23:19:23.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.271+0000 7f3f72d9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3f740d3bc0 0x7f3f740d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:23.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.271+0000 7f3f71d9b640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f3f7415f550 0x7f3f740ca440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:55566/0 (socket says 172.21.15.47:55566) 2024-04-26T23:19:23.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.271+0000 7f3f71d9b640 1 -- 172.21.15.47:0/3303999656 learned_addr learned my addr 172.21.15.47:0/3303999656 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:23.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.271+0000 7f3f72d9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3f740d3bc0 0x7f3f740d5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54686/0 (socket says 172.21.15.47:54686) 2024-04-26T23:19:23.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.271+0000 7f3f72d9d640 1 -- 172.21.15.47:0/3303999656 >> v1:172.21.15.203:6789/0 conn(0x7f3f7415f170 legacy=0x7f3f740d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:23.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.271+0000 7f3f72d9d640 1 -- 172.21.15.47:0/3303999656 >> v1:172.21.15.47:6789/0 conn(0x7f3f7415f550 legacy=0x7f3f740ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:23.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.271+0000 7f3f72d9d640 1 -- 172.21.15.47:0/3303999656 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f740c9960 con 0x7f3f740d3bc0 2024-04-26T23:19:23.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.271+0000 7f3f7159a640 1 -- 172.21.15.47:0/3303999656 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3231552068 0 0) 0x7f3f68001110 con 0x7f3f7415f550 2024-04-26T23:19:23.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.272+0000 7f3f72d9d640 1 --2- 172.21.15.47:0/3303999656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f740d3bc0 0x7f3f740d5a80 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3f740c95b0 tx=0x7f3f6c02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c309212f8ca23ec7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:23.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.272+0000 7f3f7159a640 1 -- 172.21.15.47:0/3303999656 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c007740 con 0x7f3f740d3bc0 2024-04-26T23:19:23.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.272+0000 7f3f7159a640 1 -- 172.21.15.47:0/3303999656 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f6c0078e0 con 0x7f3f740d3bc0 2024-04-26T23:19:23.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.272+0000 7f3f7159a640 1 -- 172.21.15.47:0/3303999656 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c007cb0 con 0x7f3f740d3bc0 2024-04-26T23:19:23.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.273+0000 7f3f78b0a640 1 -- 172.21.15.47:0/3303999656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f740d3bc0 msgr2=0x7f3f740d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:23.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.273+0000 7f3f78b0a640 1 --2- 172.21.15.47:0/3303999656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f740d3bc0 0x7f3f740d5a80 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3f740c95b0 tx=0x7f3f6c02a6c0 comp rx=0 tx=0).stop 2024-04-26T23:19:23.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.273+0000 7f3f78b0a640 1 -- 172.21.15.47:0/3303999656 shutdown_connections 2024-04-26T23:19:23.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.273+0000 7f3f78b0a640 1 --2- 172.21.15.47:0/3303999656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f740d3bc0 0x7f3f740d5a80 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:23.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.273+0000 7f3f78b0a640 1 -- 172.21.15.47:0/3303999656 >> 172.21.15.47:0/3303999656 conn(0x7f3f740d4f60 msgr2=0x7f3f740d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:23.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.273+0000 7f3f78b0a640 1 -- 172.21.15.47:0/3303999656 shutdown_connections 2024-04-26T23:19:23.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.273+0000 7f3f78b0a640 1 -- 172.21.15.47:0/3303999656 wait complete. 2024-04-26T23:19:23.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.274+0000 7f3f78b0a640 1 Processor -- start 2024-04-26T23:19:23.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.274+0000 7f3f78b0a640 1 -- start start 2024-04-26T23:19:23.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.275+0000 7f3f78b0a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f740d3bc0 0x7f3f74156d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:23.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.275+0000 7f3f78b0a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f7415f170 0x7f3f74156130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:23.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.275+0000 7f3f78b0a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f74156670 0x7f3f7414c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:23.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.275+0000 7f3f78b0a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3f740d2e30 con 0x7f3f74156670 2024-04-26T23:19:23.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.275+0000 7f3f78b0a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3f740d2cb0 con 0x7f3f740d3bc0 2024-04-26T23:19:23.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.275+0000 7f3f78b0a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3f740d2fb0 con 0x7f3f7415f170 2024-04-26T23:19:23.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.275+0000 7f3f72d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f74156670 0x7f3f7414c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:23.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.275+0000 7f3f72d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f74156670 0x7f3f7414c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54692/0 (socket says 172.21.15.47:54692) 2024-04-26T23:19:23.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.275+0000 7f3f72d9d640 1 -- 172.21.15.47:0/14621117 learned_addr learned my addr 172.21.15.47:0/14621117 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:23.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.275+0000 7f3f7259c640 1 --2- 172.21.15.47:0/14621117 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f740d3bc0 0x7f3f74156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:23.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.275+0000 7f3f71d9b640 1 --2- 172.21.15.47:0/14621117 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f7415f170 0x7f3f74156130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:23.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.275+0000 7f3f72d9d640 1 -- 172.21.15.47:0/14621117 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f7415f170 msgr2=0x7f3f74156130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:23.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.275+0000 7f3f72d9d640 1 --2- 172.21.15.47:0/14621117 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f7415f170 0x7f3f74156130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:23.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.275+0000 7f3f72d9d640 1 -- 172.21.15.47:0/14621117 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f740d3bc0 msgr2=0x7f3f74156d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:23.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.275+0000 7f3f72d9d640 1 --2- 172.21.15.47:0/14621117 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f740d3bc0 0x7f3f74156d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:23.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.275+0000 7f3f72d9d640 1 -- 172.21.15.47:0/14621117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f740ba4d0 con 0x7f3f74156670 2024-04-26T23:19:23.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.275+0000 7f3f71d9b640 1 --2- 172.21.15.47:0/14621117 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f7415f170 0x7f3f74156130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:19:23.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.276+0000 7f3f72d9d640 1 --2- 172.21.15.47:0/14621117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f74156670 0x7f3f7414c530 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c009e80 tx=0x7f3f6c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:23.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.276+0000 7f3f62ffd640 1 -- 172.21.15.47:0/14621117 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c040070 con 0x7f3f74156670 2024-04-26T23:19:23.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.276+0000 7f3f62ffd640 1 -- 172.21.15.47:0/14621117 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f6c031cb0 con 0x7f3f74156670 2024-04-26T23:19:23.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.276+0000 7f3f78b0a640 1 -- 172.21.15.47:0/14621117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f740b9710 con 0x7f3f74156670 2024-04-26T23:19:23.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.276+0000 7f3f62ffd640 1 -- 172.21.15.47:0/14621117 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c031e60 con 0x7f3f74156670 2024-04-26T23:19:23.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.276+0000 7f3f78b0a640 1 -- 172.21.15.47:0/14621117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f740b9cd0 con 0x7f3f74156670 2024-04-26T23:19:23.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.277+0000 7f3f78b0a640 1 -- 172.21.15.47:0/14621117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f740694f0 con 0x7f3f74156670 2024-04-26T23:19:23.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.277+0000 7f3f62ffd640 1 -- 172.21.15.47:0/14621117 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3f6c039040 con 0x7f3f74156670 2024-04-26T23:19:23.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.278+0000 7f3f62ffd640 1 --2- 172.21.15.47:0/14621117 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3f5406c8f0 0x7f3f5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:23.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.278+0000 7f3f62ffd640 1 -- 172.21.15.47:0/14621117 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f3f6c0ad500 con 0x7f3f74156670 2024-04-26T23:19:23.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.278+0000 7f3f7259c640 1 --2- 172.21.15.47:0/14621117 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3f5406c8f0 0x7f3f5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:23.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.279+0000 7f3f7259c640 1 --2- 172.21.15.47:0/14621117 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3f5406c8f0 0x7f3f5406edb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3f740b6c50 tx=0x7f3f64009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:23.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.283+0000 7f3f62ffd640 1 -- 172.21.15.47:0/14621117 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3f6c07af50 con 0x7f3f74156670 2024-04-26T23:19:23.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.465+0000 7f3f78b0a640 1 -- 172.21.15.47:0/14621117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3f740d35e0 con 0x7f3f74156670 2024-04-26T23:19:23.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.466+0000 7f3f62ffd640 1 -- 172.21.15.47:0/14621117 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f3f6c07f250 con 0x7f3f74156670 2024-04-26T23:19:23.466 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-26T23:19:23.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.468+0000 7f3f78b0a640 1 -- 172.21.15.47:0/14621117 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3f5406c8f0 msgr2=0x7f3f5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:23.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.468+0000 7f3f78b0a640 1 --2- 172.21.15.47:0/14621117 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3f5406c8f0 0x7f3f5406edb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3f740b6c50 tx=0x7f3f64009040 comp rx=0 tx=0).stop 2024-04-26T23:19:23.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.468+0000 7f3f78b0a640 1 -- 172.21.15.47:0/14621117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f74156670 msgr2=0x7f3f7414c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:23.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.468+0000 7f3f78b0a640 1 --2- 172.21.15.47:0/14621117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f74156670 0x7f3f7414c530 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c009e80 tx=0x7f3f6c004a40 comp rx=0 tx=0).stop 2024-04-26T23:19:23.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.468+0000 7f3f78b0a640 1 -- 172.21.15.47:0/14621117 shutdown_connections 2024-04-26T23:19:23.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.468+0000 7f3f78b0a640 1 --2- 172.21.15.47:0/14621117 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3f5406c8f0 0x7f3f5406edb0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:23.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.468+0000 7f3f78b0a640 1 --2- 172.21.15.47:0/14621117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f74156670 0x7f3f7414c530 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:23.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.468+0000 7f3f78b0a640 1 --2- 172.21.15.47:0/14621117 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f7415f170 0x7f3f74156130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:23.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.468+0000 7f3f78b0a640 1 --2- 172.21.15.47:0/14621117 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f740d3bc0 0x7f3f74156d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:23.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.468+0000 7f3f78b0a640 1 -- 172.21.15.47:0/14621117 >> 172.21.15.47:0/14621117 conn(0x7f3f740d4f60 msgr2=0x7f3f7406e520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:23.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.468+0000 7f3f78b0a640 1 -- 172.21.15.47:0/14621117 shutdown_connections 2024-04-26T23:19:23.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.468+0000 7f3f78b0a640 1 -- 172.21.15.47:0/14621117 wait complete. 2024-04-26T23:19:23.479 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:19:23.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.582+0000 7f105f1f0640 1 Processor -- start 2024-04-26T23:19:23.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.582+0000 7f105f1f0640 1 -- start start 2024-04-26T23:19:23.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.583+0000 7f105f1f0640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f10580b0b70 0x7f10580b0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:23.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.583+0000 7f105f1f0640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f10580b4d40 0x7f10580b6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:23.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.583+0000 7f105f1f0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10580b6c50 0x7f105815f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:23.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.583+0000 7f105f1f0640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f1058073bc0 con 0x7f10580b0b70 2024-04-26T23:19:23.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.583+0000 7f105f1f0640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f10580b1490 con 0x7f10580b4d40 2024-04-26T23:19:23.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.583+0000 7f105f1f0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f1058072fb0 con 0x7f10580b6c50 2024-04-26T23:19:23.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.583+0000 7f105cf8c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f10580b0b70 0x7f10580b0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:23.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.583+0000 7f105cf8c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f10580b0b70 0x7f10580b0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54694/0 (socket says 172.21.15.47:54694) 2024-04-26T23:19:23.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.583+0000 7f105cf8c640 1 -- 172.21.15.47:0/1599143079 learned_addr learned my addr 172.21.15.47:0/1599143079 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:23.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.584+0000 7f1057fff640 1 --2- 172.21.15.47:0/1599143079 >> v2:172.21.15.203:3300/0 conn(0x7f10580b4d40 0x7f10580b6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:23.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.584+0000 7f105cf8c640 1 -- 172.21.15.47:0/1599143079 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10580b6c50 msgr2=0x7f105815f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:23.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.584+0000 7f105d78d640 1 --2- 172.21.15.47:0/1599143079 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10580b6c50 0x7f105815f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:23.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.584+0000 7f105cf8c640 1 --2- 172.21.15.47:0/1599143079 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10580b6c50 0x7f105815f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:23.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.584+0000 7f105cf8c640 1 -- 172.21.15.47:0/1599143079 >> v2:172.21.15.203:3300/0 conn(0x7f10580b4d40 msgr2=0x7f10580b6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:23.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.584+0000 7f105cf8c640 1 --2- 172.21.15.47:0/1599143079 >> v2:172.21.15.203:3300/0 conn(0x7f10580b4d40 0x7f10580b6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:23.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.584+0000 7f105cf8c640 1 -- 172.21.15.47:0/1599143079 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1058073290 con 0x7f10580b0b70 2024-04-26T23:19:23.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.584+0000 7f105cf8c640 1 --2- 172.21.15.47:0/1599143079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10580b0b70 0x7f10580b0f50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1058074840 tx=0x7f104802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65769ef207a17846 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:23.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.585+0000 7f10577fe640 1 -- 172.21.15.47:0/1599143079 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1048037070 con 0x7f10580b0b70 2024-04-26T23:19:23.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.585+0000 7f10577fe640 1 -- 172.21.15.47:0/1599143079 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1048032440 con 0x7f10580b0b70 2024-04-26T23:19:23.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.585+0000 7f10577fe640 1 -- 172.21.15.47:0/1599143079 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1048032810 con 0x7f10580b0b70 2024-04-26T23:19:23.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.585+0000 7f105f1f0640 1 -- 172.21.15.47:0/1599143079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10580b0b70 msgr2=0x7f10580b0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:23.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.585+0000 7f105f1f0640 1 --2- 172.21.15.47:0/1599143079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10580b0b70 0x7f10580b0f50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1058074840 tx=0x7f104802a1e0 comp rx=0 tx=0).stop 2024-04-26T23:19:23.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.585+0000 7f105f1f0640 1 -- 172.21.15.47:0/1599143079 shutdown_connections 2024-04-26T23:19:23.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.585+0000 7f105f1f0640 1 --2- 172.21.15.47:0/1599143079 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10580b6c50 0x7f105815f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:23.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.585+0000 7f105f1f0640 1 --2- 172.21.15.47:0/1599143079 >> v2:172.21.15.203:3300/0 conn(0x7f10580b4d40 0x7f10580b6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:23.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.585+0000 7f105f1f0640 1 --2- 172.21.15.47:0/1599143079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10580b0b70 0x7f10580b0f50 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:23.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.585+0000 7f105f1f0640 1 -- 172.21.15.47:0/1599143079 >> 172.21.15.47:0/1599143079 conn(0x7f10580b39b0 msgr2=0x7f10580b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:23.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.585+0000 7f105f1f0640 1 -- 172.21.15.47:0/1599143079 shutdown_connections 2024-04-26T23:19:23.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.585+0000 7f105f1f0640 1 -- 172.21.15.47:0/1599143079 wait complete. 2024-04-26T23:19:23.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.586+0000 7f105f1f0640 1 Processor -- start 2024-04-26T23:19:23.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.586+0000 7f105f1f0640 1 -- start start 2024-04-26T23:19:23.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.587+0000 7f105f1f0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10580b0b70 0x7f10581481c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:23.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.587+0000 7f105f1f0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10580b4d40 0x7f10580d16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:23.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.587+0000 7f105f1f0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10580b6c50 0x7f10580d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:23.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.587+0000 7f105f1f0640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f105815f890 con 0x7f10580b4d40 2024-04-26T23:19:23.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.587+0000 7f105f1f0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1058073850 con 0x7f10580b0b70 2024-04-26T23:19:23.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.587+0000 7f105f1f0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f105815fa10 con 0x7f10580b6c50 2024-04-26T23:19:23.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.587+0000 7f1057fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10580b4d40 0x7f10580d16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:23.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.587+0000 7f1057fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10580b4d40 0x7f10580d16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54700/0 (socket says 172.21.15.47:54700) 2024-04-26T23:19:23.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.587+0000 7f1057fff640 1 -- 172.21.15.47:0/2383003855 learned_addr learned my addr 172.21.15.47:0/2383003855 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:23.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.588+0000 7f105d78d640 1 --2- 172.21.15.47:0/2383003855 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10580b6c50 0x7f10580d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:23.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.588+0000 7f105cf8c640 1 --2- 172.21.15.47:0/2383003855 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10580b0b70 0x7f10581481c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:23.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.588+0000 7f1057fff640 1 -- 172.21.15.47:0/2383003855 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10580b6c50 msgr2=0x7f10580d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:23.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.588+0000 7f1057fff640 1 --2- 172.21.15.47:0/2383003855 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10580b6c50 0x7f10580d1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:23.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.588+0000 7f1057fff640 1 -- 172.21.15.47:0/2383003855 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10580b0b70 msgr2=0x7f10581481c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:23.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.588+0000 7f1057fff640 1 --2- 172.21.15.47:0/2383003855 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10580b0b70 0x7f10581481c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:23.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.588+0000 7f1057fff640 1 -- 172.21.15.47:0/2383003855 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10580d2220 con 0x7f10580b4d40 2024-04-26T23:19:23.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.588+0000 7f105cf8c640 1 --2- 172.21.15.47:0/2383003855 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10580b0b70 0x7f10581481c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:19:23.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.588+0000 7f1057fff640 1 --2- 172.21.15.47:0/2383003855 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10580b4d40 0x7f10580d16d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f104c00c7f0 tx=0x7f104c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:23.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.588+0000 7f10557fa640 1 -- 172.21.15.47:0/2383003855 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f104c015070 con 0x7f10580b4d40 2024-04-26T23:19:23.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.588+0000 7f10557fa640 1 -- 172.21.15.47:0/2383003855 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f104c004220 con 0x7f10580b4d40 2024-04-26T23:19:23.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.588+0000 7f105f1f0640 1 -- 172.21.15.47:0/2383003855 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10580d2500 con 0x7f10580b4d40 2024-04-26T23:19:23.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.588+0000 7f10557fa640 1 -- 172.21.15.47:0/2383003855 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f104c007620 con 0x7f10580b4d40 2024-04-26T23:19:23.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.588+0000 7f105f1f0640 1 -- 172.21.15.47:0/2383003855 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10580d2ac0 con 0x7f10580b4d40 2024-04-26T23:19:23.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.589+0000 7f105f1f0640 1 -- 172.21.15.47:0/2383003855 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10580694f0 con 0x7f10580b4d40 2024-04-26T23:19:23.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.590+0000 7f10557fa640 1 -- 172.21.15.47:0/2383003855 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f104c0079d0 con 0x7f10580b4d40 2024-04-26T23:19:23.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.590+0000 7f10557fa640 1 --2- 172.21.15.47:0/2383003855 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f104406c8f0 0x7f104406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:23.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.590+0000 7f10557fa640 1 -- 172.21.15.47:0/2383003855 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f104c08c010 con 0x7f10580b4d40 2024-04-26T23:19:23.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.590+0000 7f105cf8c640 1 --2- 172.21.15.47:0/2383003855 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f104406c8f0 0x7f104406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:23.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.591+0000 7f105cf8c640 1 --2- 172.21.15.47:0/2383003855 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f104406c8f0 0x7f104406edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1048034000 tx=0x7f104802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:23.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.594+0000 7f10557fa640 1 -- 172.21.15.47:0/2383003855 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f104c059f90 con 0x7f10580b4d40 2024-04-26T23:19:23.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.779+0000 7f105f1f0640 1 -- 172.21.15.47:0/2383003855 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f105806dfb0 con 0x7f10580b4d40 2024-04-26T23:19:23.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.779+0000 7f10557fa640 1 -- 172.21.15.47:0/2383003855 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f104c059890 con 0x7f10580b4d40 2024-04-26T23:19:23.780 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-26T23:19:23.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.781+0000 7f105f1f0640 1 -- 172.21.15.47:0/2383003855 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f104406c8f0 msgr2=0x7f104406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:23.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.781+0000 7f105f1f0640 1 --2- 172.21.15.47:0/2383003855 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f104406c8f0 0x7f104406edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1048034000 tx=0x7f104802e000 comp rx=0 tx=0).stop 2024-04-26T23:19:23.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.781+0000 7f105f1f0640 1 -- 172.21.15.47:0/2383003855 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10580b4d40 msgr2=0x7f10580d16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:23.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.781+0000 7f105f1f0640 1 --2- 172.21.15.47:0/2383003855 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10580b4d40 0x7f10580d16d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f104c00c7f0 tx=0x7f104c00ccb0 comp rx=0 tx=0).stop 2024-04-26T23:19:23.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.781+0000 7f105f1f0640 1 -- 172.21.15.47:0/2383003855 shutdown_connections 2024-04-26T23:19:23.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.781+0000 7f105f1f0640 1 --2- 172.21.15.47:0/2383003855 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f104406c8f0 0x7f104406edb0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:23.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.781+0000 7f105f1f0640 1 --2- 172.21.15.47:0/2383003855 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10580b6c50 0x7f10580d1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:23.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.781+0000 7f105f1f0640 1 --2- 172.21.15.47:0/2383003855 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10580b4d40 0x7f10580d16d0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:23.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.781+0000 7f105f1f0640 1 --2- 172.21.15.47:0/2383003855 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10580b0b70 0x7f10581481c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:23.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.781+0000 7f105f1f0640 1 -- 172.21.15.47:0/2383003855 >> 172.21.15.47:0/2383003855 conn(0x7f10580b39b0 msgr2=0x7f10580afc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:23.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.782+0000 7f105f1f0640 1 -- 172.21.15.47:0/2383003855 shutdown_connections 2024-04-26T23:19:23.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.782+0000 7f105f1f0640 1 -- 172.21.15.47:0/2383003855 wait complete. 2024-04-26T23:19:23.793 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:19:23.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.896+0000 7f01955f0640 1 Processor -- start 2024-04-26T23:19:23.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.896+0000 7f01955f0640 1 -- start start 2024-04-26T23:19:23.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.896+0000 7f01955f0640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f019015d250 0x7f01900d0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:23.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.896+0000 7f01955f0640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01900701e0 con 0x7f01900d39f0 2024-04-26T23:19:23.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.897+0000 7f01955f0640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01900ce370 con 0x7f01900d1720 2024-04-26T23:19:23.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.897+0000 7f01955f0640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f019015d630 con 0x7f019015d250 2024-04-26T23:19:23.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.897+0000 7f018effd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f019015d250 0x7f01900d0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:23.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.897+0000 7f018effd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f019015d250 0x7f01900d0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54702/0 (socket says 172.21.15.47:54702) 2024-04-26T23:19:23.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.897+0000 7f018e7fc640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f01900d39f0 0x7f01900d1010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:55572/0 (socket says 172.21.15.47:55572) 2024-04-26T23:19:23.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.897+0000 7f018effd640 1 -- 172.21.15.47:0/1073191234 learned_addr learned my addr 172.21.15.47:0/1073191234 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:23.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.897+0000 7f018effd640 1 -- 172.21.15.47:0/1073191234 >> v1:172.21.15.203:6789/0 conn(0x7f01900d1720 legacy=0x7f0190070970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:23.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.897+0000 7f018effd640 1 -- 172.21.15.47:0/1073191234 >> v1:172.21.15.47:6789/0 conn(0x7f01900d39f0 legacy=0x7f01900d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:23.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.897+0000 7f018dffb640 1 -- 172.21.15.47:0/1073191234 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3713125042 0 0) 0x7f01900701e0 con 0x7f01900d39f0 2024-04-26T23:19:23.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.897+0000 7f018effd640 1 -- 172.21.15.47:0/1073191234 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01900ce5f0 con 0x7f019015d250 2024-04-26T23:19:23.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.898+0000 7f018effd640 1 --2- 172.21.15.47:0/1073191234 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f019015d250 0x7f01900d0950 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0180009870 tx=0x7f018002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e91ee0922fdb6176 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:23.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.898+0000 7f018dffb640 1 -- 172.21.15.47:0/1073191234 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0180037070 con 0x7f019015d250 2024-04-26T23:19:23.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.898+0000 7f018dffb640 1 -- 172.21.15.47:0/1073191234 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0180032440 con 0x7f019015d250 2024-04-26T23:19:23.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.898+0000 7f018dffb640 1 -- 172.21.15.47:0/1073191234 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0180032810 con 0x7f019015d250 2024-04-26T23:19:23.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.898+0000 7f01955f0640 1 -- 172.21.15.47:0/1073191234 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f019015d250 msgr2=0x7f01900d0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:23.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.898+0000 7f01955f0640 1 --2- 172.21.15.47:0/1073191234 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f019015d250 0x7f01900d0950 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0180009870 tx=0x7f018002a270 comp rx=0 tx=0).stop 2024-04-26T23:19:23.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.898+0000 7f01955f0640 1 -- 172.21.15.47:0/1073191234 shutdown_connections 2024-04-26T23:19:23.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.898+0000 7f01955f0640 1 --2- 172.21.15.47:0/1073191234 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f019015d250 0x7f01900d0950 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:23.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.898+0000 7f01955f0640 1 -- 172.21.15.47:0/1073191234 >> 172.21.15.47:0/1073191234 conn(0x7f01900d2d80 msgr2=0x7f01900d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:23.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.898+0000 7f01955f0640 1 -- 172.21.15.47:0/1073191234 shutdown_connections 2024-04-26T23:19:23.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.899+0000 7f01955f0640 1 -- 172.21.15.47:0/1073191234 wait complete. 2024-04-26T23:19:23.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.900+0000 7f01955f0640 1 Processor -- start 2024-04-26T23:19:23.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.900+0000 7f01955f0640 1 -- start start 2024-04-26T23:19:23.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.900+0000 7f01955f0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01900d1720 0x7f01900c9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:23.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.901+0000 7f01955f0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01900d39f0 0x7f01901e5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:23.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.901+0000 7f01955f0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f019015d250 0x7f01901f2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:23.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.901+0000 7f01955f0640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f01900cebb0 con 0x7f01900d1720 2024-04-26T23:19:23.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.901+0000 7f01955f0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f01900704e0 con 0x7f01900d39f0 2024-04-26T23:19:23.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.901+0000 7f01955f0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f0190071080 con 0x7f019015d250 2024-04-26T23:19:23.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.901+0000 7f018effd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01900d1720 0x7f01900c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:23.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.901+0000 7f018effd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01900d1720 0x7f01900c9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54712/0 (socket says 172.21.15.47:54712) 2024-04-26T23:19:23.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.901+0000 7f018effd640 1 -- 172.21.15.47:0/2346549458 learned_addr learned my addr 172.21.15.47:0/2346549458 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:23.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.901+0000 7f018f7fe640 1 --2- 172.21.15.47:0/2346549458 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f019015d250 0x7f01901f2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:23.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.901+0000 7f018e7fc640 1 --2- 172.21.15.47:0/2346549458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01900d39f0 0x7f01901e5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:23.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.901+0000 7f018effd640 1 -- 172.21.15.47:0/2346549458 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f019015d250 msgr2=0x7f01901f2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:23.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.901+0000 7f018effd640 1 --2- 172.21.15.47:0/2346549458 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f019015d250 0x7f01901f2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:23.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.901+0000 7f018effd640 1 -- 172.21.15.47:0/2346549458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01900d39f0 msgr2=0x7f01901e5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:23.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.901+0000 7f018effd640 1 --2- 172.21.15.47:0/2346549458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01900d39f0 0x7f01901e5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:23.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.901+0000 7f018effd640 1 -- 172.21.15.47:0/2346549458 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f019015c080 con 0x7f01900d1720 2024-04-26T23:19:23.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.901+0000 7f018f7fe640 1 --2- 172.21.15.47:0/2346549458 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f019015d250 0x7f01901f2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:19:23.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.901+0000 7f018e7fc640 1 --2- 172.21.15.47:0/2346549458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01900d39f0 0x7f01901e5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:19:23.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.901+0000 7f018effd640 1 --2- 172.21.15.47:0/2346549458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01900d1720 0x7f01900c9480 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0180009870 tx=0x7f01800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:23.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.901+0000 7f01777fe640 1 -- 172.21.15.47:0/2346549458 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0180041070 con 0x7f01900d1720 2024-04-26T23:19:23.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.901+0000 7f01777fe640 1 -- 172.21.15.47:0/2346549458 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f018003c870 con 0x7f01900d1720 2024-04-26T23:19:23.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.901+0000 7f01955f0640 1 -- 172.21.15.47:0/2346549458 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01901e6380 con 0x7f01900d1720 2024-04-26T23:19:23.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.901+0000 7f01777fe640 1 -- 172.21.15.47:0/2346549458 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0180037040 con 0x7f01900d1720 2024-04-26T23:19:23.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.901+0000 7f01955f0640 1 -- 172.21.15.47:0/2346549458 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01901e6940 con 0x7f01900d1720 2024-04-26T23:19:23.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.902+0000 7f01955f0640 1 -- 172.21.15.47:0/2346549458 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01900694f0 con 0x7f01900d1720 2024-04-26T23:19:23.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.903+0000 7f01777fe640 1 -- 172.21.15.47:0/2346549458 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0180044070 con 0x7f01900d1720 2024-04-26T23:19:23.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.903+0000 7f01777fe640 1 --2- 172.21.15.47:0/2346549458 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f017c06c8f0 0x7f017c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:23.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.903+0000 7f01777fe640 1 -- 172.21.15.47:0/2346549458 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f01800accb0 con 0x7f01900d1720 2024-04-26T23:19:23.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.903+0000 7f018e7fc640 1 --2- 172.21.15.47:0/2346549458 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f017c06c8f0 0x7f017c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:23.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.904+0000 7f018e7fc640 1 --2- 172.21.15.47:0/2346549458 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f017c06c8f0 0x7f017c06edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f01900c8430 tx=0x7f0184009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:23.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:23.908+0000 7f01777fe640 1 -- 172.21.15.47:0/2346549458 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f018007ade0 con 0x7f01900d1720 2024-04-26T23:19:24.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.097+0000 7f01955f0640 1 -- 172.21.15.47:0/2346549458 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0190070a10 con 0x7f01900d1720 2024-04-26T23:19:24.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.097+0000 7f01777fe640 1 -- 172.21.15.47:0/2346549458 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f018007a6e0 con 0x7f01900d1720 2024-04-26T23:19:24.098 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-26T23:19:24.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.099+0000 7f01955f0640 1 -- 172.21.15.47:0/2346549458 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f017c06c8f0 msgr2=0x7f017c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:24.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.099+0000 7f01955f0640 1 --2- 172.21.15.47:0/2346549458 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f017c06c8f0 0x7f017c06edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f01900c8430 tx=0x7f0184009040 comp rx=0 tx=0).stop 2024-04-26T23:19:24.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.099+0000 7f01955f0640 1 -- 172.21.15.47:0/2346549458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01900d1720 msgr2=0x7f01900c9480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:24.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.099+0000 7f01955f0640 1 --2- 172.21.15.47:0/2346549458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01900d1720 0x7f01900c9480 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0180009870 tx=0x7f01800046c0 comp rx=0 tx=0).stop 2024-04-26T23:19:24.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.099+0000 7f01955f0640 1 -- 172.21.15.47:0/2346549458 shutdown_connections 2024-04-26T23:19:24.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.099+0000 7f01955f0640 1 --2- 172.21.15.47:0/2346549458 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f017c06c8f0 0x7f017c06edb0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:24.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.099+0000 7f01955f0640 1 --2- 172.21.15.47:0/2346549458 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f019015d250 0x7f01901f2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:24.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.099+0000 7f01955f0640 1 --2- 172.21.15.47:0/2346549458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01900d39f0 0x7f01901e5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:24.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.099+0000 7f01955f0640 1 --2- 172.21.15.47:0/2346549458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01900d1720 0x7f01900c9480 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:24.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.099+0000 7f01955f0640 1 -- 172.21.15.47:0/2346549458 >> 172.21.15.47:0/2346549458 conn(0x7f01900d2d80 msgr2=0x7f019015e150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:24.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.099+0000 7f01955f0640 1 -- 172.21.15.47:0/2346549458 shutdown_connections 2024-04-26T23:19:24.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.099+0000 7f01955f0640 1 -- 172.21.15.47:0/2346549458 wait complete. 2024-04-26T23:19:24.111 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T23:19:24.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.214+0000 7fc76427b640 1 Processor -- start 2024-04-26T23:19:24.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.214+0000 7fc76427b640 1 -- start start 2024-04-26T23:19:24.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.215+0000 7fc76427b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc75c15f550 0x7fc75c0d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:24.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.215+0000 7fc76427b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc75c0d3bc0 0x7fc75c0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:24.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.215+0000 7fc76427b640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc75c0d1040 con 0x7fc75c15f170 2024-04-26T23:19:24.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.215+0000 7fc76427b640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fc75c0c9430 con 0x7fc75c0d3bc0 2024-04-26T23:19:24.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.215+0000 7fc76427b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc75c0c9680 con 0x7fc75c15f550 2024-04-26T23:19:24.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.215+0000 7fc762818640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc75c0d3bc0 0x7fc75c0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:24.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.215+0000 7fc762017640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc75c15f170 0x7fc75c0d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:55576/0 (socket says 172.21.15.47:55576) 2024-04-26T23:19:24.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.215+0000 7fc762017640 1 -- 172.21.15.47:0/1743150070 learned_addr learned my addr 172.21.15.47:0/1743150070 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:24.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.215+0000 7fc762818640 1 -- 172.21.15.47:0/1743150070 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc75c15f550 msgr2=0x7fc75c0d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:24.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.215+0000 7fc761816640 1 --2- 172.21.15.47:0/1743150070 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc75c15f550 0x7fc75c0d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:24.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.215+0000 7fc761015640 1 -- 172.21.15.47:0/1743150070 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1762978297 0 0) 0x7fc754001110 con 0x7fc75c15f170 2024-04-26T23:19:24.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.215+0000 7fc762818640 1 --2- 172.21.15.47:0/1743150070 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc75c15f550 0x7fc75c0d5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:24.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.215+0000 7fc762818640 1 -- 172.21.15.47:0/1743150070 >> v1:172.21.15.47:6789/0 conn(0x7fc75c15f170 legacy=0x7fc75c0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:24.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.215+0000 7fc762818640 1 -- 172.21.15.47:0/1743150070 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc75c0c9960 con 0x7fc75c0d3bc0 2024-04-26T23:19:24.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.216+0000 7fc762818640 1 --2- 172.21.15.47:0/1743150070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc75c0d3bc0 0x7fc75c0d2cb0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc74c009d90 tx=0x7fc74c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aaf7e448457ea53 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:24.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.216+0000 7fc761015640 1 -- 172.21.15.47:0/1743150070 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74c007740 con 0x7fc75c0d3bc0 2024-04-26T23:19:24.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.216+0000 7fc761015640 1 -- 172.21.15.47:0/1743150070 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc74c0078e0 con 0x7fc75c0d3bc0 2024-04-26T23:19:24.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.216+0000 7fc761015640 1 -- 172.21.15.47:0/1743150070 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74c007cb0 con 0x7fc75c0d3bc0 2024-04-26T23:19:24.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.217+0000 7fc76427b640 1 -- 172.21.15.47:0/1743150070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc75c0d3bc0 msgr2=0x7fc75c0d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:24.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.217+0000 7fc76427b640 1 --2- 172.21.15.47:0/1743150070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc75c0d3bc0 0x7fc75c0d2cb0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc74c009d90 tx=0x7fc74c02a660 comp rx=0 tx=0).stop 2024-04-26T23:19:24.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.217+0000 7fc76427b640 1 -- 172.21.15.47:0/1743150070 shutdown_connections 2024-04-26T23:19:24.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.217+0000 7fc76427b640 1 --2- 172.21.15.47:0/1743150070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc75c0d3bc0 0x7fc75c0d2cb0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:24.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.217+0000 7fc76427b640 1 --2- 172.21.15.47:0/1743150070 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc75c15f550 0x7fc75c0d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:24.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.217+0000 7fc76427b640 1 -- 172.21.15.47:0/1743150070 >> 172.21.15.47:0/1743150070 conn(0x7fc75c0d4f60 msgr2=0x7fc75c0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:24.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.217+0000 7fc76427b640 1 -- 172.21.15.47:0/1743150070 shutdown_connections 2024-04-26T23:19:24.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.217+0000 7fc76427b640 1 -- 172.21.15.47:0/1743150070 wait complete. 2024-04-26T23:19:24.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.218+0000 7fc76427b640 1 Processor -- start 2024-04-26T23:19:24.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.218+0000 7fc76427b640 1 -- start start 2024-04-26T23:19:24.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.219+0000 7fc76427b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc75c0d3bc0 0x7fc75c0b9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:24.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.219+0000 7fc76427b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc75c15f170 0x7fc75c0ba0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:24.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.219+0000 7fc76427b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc75c0ce170 0x7fc75c0ce620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:24.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.219+0000 7fc76427b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c0ca5c0 con 0x7fc75c0ce170 2024-04-26T23:19:24.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.219+0000 7fc76427b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c0ca440 con 0x7fc75c0d3bc0 2024-04-26T23:19:24.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.219+0000 7fc76427b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc75c0ca740 con 0x7fc75c15f170 2024-04-26T23:19:24.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.219+0000 7fc762818640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc75c0ce170 0x7fc75c0ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:24.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.219+0000 7fc762818640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc75c0ce170 0x7fc75c0ce620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54726/0 (socket says 172.21.15.47:54726) 2024-04-26T23:19:24.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.219+0000 7fc762818640 1 -- 172.21.15.47:0/1643119044 learned_addr learned my addr 172.21.15.47:0/1643119044 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:24.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.219+0000 7fc761816640 1 --2- 172.21.15.47:0/1643119044 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc75c15f170 0x7fc75c0ba0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:24.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.219+0000 7fc762017640 1 --2- 172.21.15.47:0/1643119044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc75c0d3bc0 0x7fc75c0b9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:24.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.219+0000 7fc762818640 1 -- 172.21.15.47:0/1643119044 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc75c15f170 msgr2=0x7fc75c0ba0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:24.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.219+0000 7fc762818640 1 --2- 172.21.15.47:0/1643119044 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc75c15f170 0x7fc75c0ba0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:24.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.219+0000 7fc762818640 1 -- 172.21.15.47:0/1643119044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc75c0d3bc0 msgr2=0x7fc75c0b9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:24.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.219+0000 7fc762818640 1 --2- 172.21.15.47:0/1643119044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc75c0d3bc0 0x7fc75c0b9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:24.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.219+0000 7fc762818640 1 -- 172.21.15.47:0/1643119044 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc75c1f8070 con 0x7fc75c0ce170 2024-04-26T23:19:24.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.219+0000 7fc762818640 1 --2- 172.21.15.47:0/1643119044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc75c0ce170 0x7fc75c0ce620 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc74c009eb0 tx=0x7fc74c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:24.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.220+0000 7fc75affd640 1 -- 172.21.15.47:0/1643119044 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74c040070 con 0x7fc75c0ce170 2024-04-26T23:19:24.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.220+0000 7fc75affd640 1 -- 172.21.15.47:0/1643119044 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc74c031c90 con 0x7fc75c0ce170 2024-04-26T23:19:24.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.220+0000 7fc76427b640 1 -- 172.21.15.47:0/1643119044 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc75c1f8350 con 0x7fc75c0ce170 2024-04-26T23:19:24.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.220+0000 7fc75affd640 1 -- 172.21.15.47:0/1643119044 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74c045680 con 0x7fc75c0ce170 2024-04-26T23:19:24.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.220+0000 7fc76427b640 1 -- 172.21.15.47:0/1643119044 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc75c0ceb60 con 0x7fc75c0ce170 2024-04-26T23:19:24.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.221+0000 7fc76427b640 1 -- 172.21.15.47:0/1643119044 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc75c0694f0 con 0x7fc75c0ce170 2024-04-26T23:19:24.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.221+0000 7fc75affd640 1 -- 172.21.15.47:0/1643119044 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc74c039040 con 0x7fc75c0ce170 2024-04-26T23:19:24.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.222+0000 7fc75affd640 1 --2- 172.21.15.47:0/1643119044 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fc74806c8f0 0x7fc74806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:24.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.222+0000 7fc75affd640 1 -- 172.21.15.47:0/1643119044 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7fc74c0ad7c0 con 0x7fc75c0ce170 2024-04-26T23:19:24.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.222+0000 7fc762017640 1 --2- 172.21.15.47:0/1643119044 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fc74806c8f0 0x7fc74806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:24.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.222+0000 7fc762017640 1 --2- 172.21.15.47:0/1643119044 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fc74806c8f0 0x7fc74806edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc754000f60 tx=0x7fc754007b50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:24.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.226+0000 7fc75affd640 1 -- 172.21.15.47:0/1643119044 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc74c07b230 con 0x7fc75c0ce170 2024-04-26T23:19:24.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.415+0000 7fc76427b640 1 -- 172.21.15.47:0/1643119044 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fc75c074590 con 0x7fc75c0ce170 2024-04-26T23:19:24.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.415+0000 7fc75affd640 1 -- 172.21.15.47:0/1643119044 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fc74c07f530 con 0x7fc75c0ce170 2024-04-26T23:19:24.418 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-26T23:19:24.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.417+0000 7fc76427b640 1 -- 172.21.15.47:0/1643119044 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fc74806c8f0 msgr2=0x7fc74806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:24.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.417+0000 7fc76427b640 1 --2- 172.21.15.47:0/1643119044 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fc74806c8f0 0x7fc74806edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc754000f60 tx=0x7fc754007b50 comp rx=0 tx=0).stop 2024-04-26T23:19:24.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.417+0000 7fc76427b640 1 -- 172.21.15.47:0/1643119044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc75c0ce170 msgr2=0x7fc75c0ce620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:24.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.417+0000 7fc76427b640 1 --2- 172.21.15.47:0/1643119044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc75c0ce170 0x7fc75c0ce620 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc74c009eb0 tx=0x7fc74c004a40 comp rx=0 tx=0).stop 2024-04-26T23:19:24.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.417+0000 7fc76427b640 1 -- 172.21.15.47:0/1643119044 shutdown_connections 2024-04-26T23:19:24.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.417+0000 7fc76427b640 1 --2- 172.21.15.47:0/1643119044 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fc74806c8f0 0x7fc74806edb0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:24.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.417+0000 7fc76427b640 1 --2- 172.21.15.47:0/1643119044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc75c0ce170 0x7fc75c0ce620 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:24.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.417+0000 7fc76427b640 1 --2- 172.21.15.47:0/1643119044 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc75c15f170 0x7fc75c0ba0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:24.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.417+0000 7fc76427b640 1 --2- 172.21.15.47:0/1643119044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc75c0d3bc0 0x7fc75c0b9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:24.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.417+0000 7fc76427b640 1 -- 172.21.15.47:0/1643119044 >> 172.21.15.47:0/1643119044 conn(0x7fc75c0d4f60 msgr2=0x7fc75c0d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:24.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.417+0000 7fc76427b640 1 -- 172.21.15.47:0/1643119044 shutdown_connections 2024-04-26T23:19:24.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.417+0000 7fc76427b640 1 -- 172.21.15.47:0/1643119044 wait complete. 2024-04-26T23:19:24.429 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T23:19:24.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.530+0000 7fdb055e4640 1 Processor -- start 2024-04-26T23:19:24.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.530+0000 7fdb055e4640 1 -- start start 2024-04-26T23:19:24.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.531+0000 7fdb055e4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb001608f0 0x7fdb000d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:24.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.531+0000 7fdb055e4640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb000cde20 con 0x7fdb000d0a90 2024-04-26T23:19:24.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.531+0000 7fdb055e4640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb000ce100 con 0x7fdb000cffa0 2024-04-26T23:19:24.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.531+0000 7fdb055e4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fdb000cea30 con 0x7fdb001608f0 2024-04-26T23:19:24.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.531+0000 7fdaf7fff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fdb000d0a90 0x7fdb000cf890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:55590/0 (socket says 172.21.15.47:55590) 2024-04-26T23:19:24.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.531+0000 7fdaf7fff640 1 -- 172.21.15.47:0/3997816513 learned_addr learned my addr 172.21.15.47:0/3997816513 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:24.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.531+0000 7fdafe7fc640 1 -- 172.21.15.47:0/3997816513 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 28750310 0 0) 0x7fdb000cde20 con 0x7fdb000d0a90 2024-04-26T23:19:24.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.531+0000 7fdafe7fc640 1 -- 172.21.15.47:0/3997816513 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdaec003520 con 0x7fdb000d0a90 2024-04-26T23:19:24.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.531+0000 7fdafeffd640 1 --2- 172.21.15.47:0/3997816513 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb001608f0 0x7fdb000d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:24.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.532+0000 7fdafe7fc640 1 -- 172.21.15.47:0/3997816513 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 442590211 0 0) 0x7fdaec003520 con 0x7fdb000d0a90 2024-04-26T23:19:24.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.532+0000 7fdafe7fc640 1 -- 172.21.15.47:0/3997816513 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb001608f0 msgr2=0x7fdb000d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:24.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.532+0000 7fdafe7fc640 1 --2- 172.21.15.47:0/3997816513 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb001608f0 0x7fdb000d1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:24.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.532+0000 7fdafe7fc640 1 -- 172.21.15.47:0/3997816513 >> v1:172.21.15.203:6789/0 conn(0x7fdb000cffa0 legacy=0x7fdb000b2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:24.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.532+0000 7fdafe7fc640 1 -- 172.21.15.47:0/3997816513 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb000ce3e0 con 0x7fdb000d0a90 2024-04-26T23:19:24.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.532+0000 7fdafe7fc640 1 -- 172.21.15.47:0/3997816513 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7fdae8001670 con 0x7fdb000d0a90 2024-04-26T23:19:24.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.532+0000 7fdafe7fc640 1 -- 172.21.15.47:0/3997816513 >> v1:172.21.15.47:6789/0 conn(0x7fdb000d0a90 legacy=0x7fdb000cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:24.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.532+0000 7fdafe7fc640 1 --2- 172.21.15.47:0/3997816513 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb001608f0 0x7fdaec0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:24.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.532+0000 7fdafe7fc640 1 --2- 172.21.15.47:0/3997816513 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdaec003f60 0x7fdaec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:24.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.532+0000 7fdafe7fc640 1 --2- 172.21.15.47:0/3997816513 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdaec004d40 0x7fdaec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:24.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.532+0000 7fdafe7fc640 1 -- 172.21.15.47:0/3997816513 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fdb000cea30 con 0x7fdaec003f60 2024-04-26T23:19:24.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.532+0000 7fdafe7fc640 1 -- 172.21.15.47:0/3997816513 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdaec00baa0 con 0x7fdaec004d40 2024-04-26T23:19:24.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.532+0000 7fdafe7fc640 1 -- 172.21.15.47:0/3997816513 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fdaec00bca0 con 0x7fdb001608f0 2024-04-26T23:19:24.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.532+0000 7fdaf7fff640 1 --2- 172.21.15.47:0/3997816513 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdaec003f60 0x7fdaec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:24.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.532+0000 7fdafeffd640 1 --2- 172.21.15.47:0/3997816513 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb001608f0 0x7fdaec0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:24.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.532+0000 7fdaff7fe640 1 --2- 172.21.15.47:0/3997816513 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdaec004d40 0x7fdaec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:24.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.532+0000 7fdaf7fff640 1 -- 172.21.15.47:0/3997816513 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb001608f0 msgr2=0x7fdaec0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:24.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.532+0000 7fdaf7fff640 1 --2- 172.21.15.47:0/3997816513 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb001608f0 0x7fdaec0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:24.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.532+0000 7fdaf7fff640 1 -- 172.21.15.47:0/3997816513 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdaec004d40 msgr2=0x7fdaec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:24.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.532+0000 7fdaf7fff640 1 --2- 172.21.15.47:0/3997816513 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdaec004d40 0x7fdaec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:24.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.532+0000 7fdaf7fff640 1 -- 172.21.15.47:0/3997816513 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaec003520 con 0x7fdaec003f60 2024-04-26T23:19:24.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.533+0000 7fdaf7fff640 1 --2- 172.21.15.47:0/3997816513 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdaec003f60 0x7fdaec004800 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fdae800c210 tx=0x7fdaec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1911c11754f132a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:24.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.533+0000 7fdafe7fc640 1 -- 172.21.15.47:0/3997816513 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae8007d30 con 0x7fdaec003f60 2024-04-26T23:19:24.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.533+0000 7fdafe7fc640 1 -- 172.21.15.47:0/3997816513 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdae8005af0 con 0x7fdaec003f60 2024-04-26T23:19:24.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.533+0000 7fdafe7fc640 1 -- 172.21.15.47:0/3997816513 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae8007070 con 0x7fdaec003f60 2024-04-26T23:19:24.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.534+0000 7fdb055e4640 1 -- 172.21.15.47:0/3997816513 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdaec003f60 msgr2=0x7fdaec004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:24.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.534+0000 7fdb055e4640 1 --2- 172.21.15.47:0/3997816513 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdaec003f60 0x7fdaec004800 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fdae800c210 tx=0x7fdaec005830 comp rx=0 tx=0).stop 2024-04-26T23:19:24.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.534+0000 7fdaff7fe640 1 -- 172.21.15.47:0/3997816513 reap_dead start 2024-04-26T23:19:24.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.534+0000 7fdb055e4640 1 -- 172.21.15.47:0/3997816513 shutdown_connections 2024-04-26T23:19:24.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.534+0000 7fdb055e4640 1 -- 172.21.15.47:0/3997816513 >> 172.21.15.47:0/3997816513 conn(0x7fdb000d2ec0 msgr2=0x7fdb000d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:24.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.534+0000 7fdb055e4640 1 -- 172.21.15.47:0/3997816513 shutdown_connections 2024-04-26T23:19:24.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.534+0000 7fdb055e4640 1 -- 172.21.15.47:0/3997816513 wait complete. 2024-04-26T23:19:24.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.535+0000 7fdb055e4640 1 Processor -- start 2024-04-26T23:19:24.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.535+0000 7fdb055e4640 1 -- start start 2024-04-26T23:19:24.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.536+0000 7fdb055e4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdaec003f60 0x7fdb000ca200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:24.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.536+0000 7fdb055e4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb0006d350 0x7fdb0006d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:24.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.536+0000 7fdb055e4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb00157b20 0x7fdb00157fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:24.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.536+0000 7fdb055e4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fdb000ceec0 con 0x7fdaec003f60 2024-04-26T23:19:24.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.536+0000 7fdb055e4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdb000d0ef0 con 0x7fdb0006d350 2024-04-26T23:19:24.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.536+0000 7fdb055e4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fdb000cda40 con 0x7fdb00157b20 2024-04-26T23:19:24.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.536+0000 7fdafeffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdaec003f60 0x7fdb000ca200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:24.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.536+0000 7fdafeffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdaec003f60 0x7fdb000ca200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54748/0 (socket says 172.21.15.47:54748) 2024-04-26T23:19:24.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.536+0000 7fdafeffd640 1 -- 172.21.15.47:0/1773684104 learned_addr learned my addr 172.21.15.47:0/1773684104 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:24.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.536+0000 7fdafeffd640 1 -- 172.21.15.47:0/1773684104 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb00157b20 msgr2=0x7fdb00157fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:24.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.536+0000 7fdafeffd640 1 --2- 172.21.15.47:0/1773684104 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb00157b20 0x7fdb00157fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:24.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.536+0000 7fdafeffd640 1 -- 172.21.15.47:0/1773684104 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb0006d350 msgr2=0x7fdb0006d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:24.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.536+0000 7fdafeffd640 1 --2- 172.21.15.47:0/1773684104 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb0006d350 0x7fdb0006d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:24.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.536+0000 7fdafeffd640 1 -- 172.21.15.47:0/1773684104 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb0006de10 con 0x7fdaec003f60 2024-04-26T23:19:24.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.536+0000 7fdafeffd640 1 --2- 172.21.15.47:0/1773684104 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdaec003f60 0x7fdb000ca200 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdaf00119a0 tx=0x7fdaf0011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:24.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.536+0000 7fdaf77fe640 1 -- 172.21.15.47:0/1773684104 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf000fa70 con 0x7fdaec003f60 2024-04-26T23:19:24.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.536+0000 7fdaf77fe640 1 -- 172.21.15.47:0/1773684104 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaf000fc10 con 0x7fdaec003f60 2024-04-26T23:19:24.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.536+0000 7fdb055e4640 1 -- 172.21.15.47:0/1773684104 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb0006e0f0 con 0x7fdaec003f60 2024-04-26T23:19:24.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.536+0000 7fdaf77fe640 1 -- 172.21.15.47:0/1773684104 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0014440 con 0x7fdaec003f60 2024-04-26T23:19:24.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.536+0000 7fdb055e4640 1 -- 172.21.15.47:0/1773684104 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb0006e6b0 con 0x7fdaec003f60 2024-04-26T23:19:24.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.537+0000 7fdb055e4640 1 -- 172.21.15.47:0/1773684104 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb000b6e10 con 0x7fdaec003f60 2024-04-26T23:19:24.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.538+0000 7fdaf77fe640 1 -- 172.21.15.47:0/1773684104 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdaf0012040 con 0x7fdaec003f60 2024-04-26T23:19:24.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.538+0000 7fdaf77fe640 1 --2- 172.21.15.47:0/1773684104 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdae006c920 0x7fdae006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:24.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.538+0000 7fdaf77fe640 1 -- 172.21.15.47:0/1773684104 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7fdaf008c6b0 con 0x7fdaec003f60 2024-04-26T23:19:24.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.539+0000 7fdaf7fff640 1 --2- 172.21.15.47:0/1773684104 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdae006c920 0x7fdae006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:24.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.539+0000 7fdaf7fff640 1 --2- 172.21.15.47:0/1773684104 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdae006c920 0x7fdae006ede0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdae8033000 tx=0x7fdae8009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:24.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.543+0000 7fdaf77fe640 1 -- 172.21.15.47:0/1773684104 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdb000b6e10 con 0x7fdaec003f60 2024-04-26T23:19:24.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.738+0000 7fdb055e4640 1 -- 172.21.15.47:0/1773684104 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdb000694f0 con 0x7fdaec003f60 2024-04-26T23:19:24.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.739+0000 7fdaf77fe640 1 -- 172.21.15.47:0/1773684104 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7fdaf0019030 con 0x7fdaec003f60 2024-04-26T23:19:24.740 INFO:teuthology.orchestra.run.smithi047.stderr:listed 0 entries 2024-04-26T23:19:24.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.741+0000 7fdb055e4640 1 -- 172.21.15.47:0/1773684104 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdae006c920 msgr2=0x7fdae006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:24.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.741+0000 7fdb055e4640 1 --2- 172.21.15.47:0/1773684104 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdae006c920 0x7fdae006ede0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdae8033000 tx=0x7fdae8009030 comp rx=0 tx=0).stop 2024-04-26T23:19:24.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.741+0000 7fdb055e4640 1 -- 172.21.15.47:0/1773684104 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdaec003f60 msgr2=0x7fdb000ca200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:24.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.741+0000 7fdb055e4640 1 --2- 172.21.15.47:0/1773684104 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdaec003f60 0x7fdb000ca200 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdaf00119a0 tx=0x7fdaf0011e60 comp rx=0 tx=0).stop 2024-04-26T23:19:24.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.741+0000 7fdb055e4640 1 -- 172.21.15.47:0/1773684104 shutdown_connections 2024-04-26T23:19:24.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.741+0000 7fdb055e4640 1 --2- 172.21.15.47:0/1773684104 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdae006c920 0x7fdae006ede0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:24.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.741+0000 7fdb055e4640 1 --2- 172.21.15.47:0/1773684104 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb00157b20 0x7fdb00157fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:24.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.741+0000 7fdb055e4640 1 --2- 172.21.15.47:0/1773684104 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb0006d350 0x7fdb0006d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:24.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.741+0000 7fdb055e4640 1 --2- 172.21.15.47:0/1773684104 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdaec003f60 0x7fdb000ca200 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:24.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.741+0000 7fdb055e4640 1 -- 172.21.15.47:0/1773684104 >> 172.21.15.47:0/1773684104 conn(0x7fdb000d2ec0 msgr2=0x7fdb000b6700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:24.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.741+0000 7fdb055e4640 1 -- 172.21.15.47:0/1773684104 shutdown_connections 2024-04-26T23:19:24.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:24.741+0000 7fdb055e4640 1 -- 172.21.15.47:0/1773684104 wait complete. 2024-04-26T23:19:24.753 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-26T23:19:24.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:24.753 DEBUG:teuthology.orchestra.run.smithi203:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-04-26T23:19:24.784 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2024-04-26T23:19:24.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:24.785 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-26T23:19:24.851 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:19:24.851 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:19:24.867 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:19:24.867 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:19:24.922 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:19:24.922 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:19:24.978 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:19:25.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.118+0000 7f4dd3d89640 1 Processor -- start 2024-04-26T23:19:25.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.118+0000 7f4dd3d89640 1 -- start start 2024-04-26T23:19:25.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.119+0000 7f4dd3d89640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4dcc162810 0x7f4dcc070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:25.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.119+0000 7f4dd3d89640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4dcc0b2340 0x7f4dcc0cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:25.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.119+0000 7f4dd3d89640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4dcc0d10d0 con 0x7f4dcc0cd930 2024-04-26T23:19:25.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.119+0000 7f4dd3d89640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f4dcc0d0750 con 0x7f4dcc0b2340 2024-04-26T23:19:25.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.119+0000 7f4dd3d89640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f4dcc0d16a0 con 0x7f4dcc162810 2024-04-26T23:19:25.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.119+0000 7f4dd1324640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4dcc0b2340 0x7f4dcc0cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:25.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.119+0000 7f4dd2326640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4dcc0cd930 0x7f4dcc0d1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:55606/0 (socket says 172.21.15.47:55606) 2024-04-26T23:19:25.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.119+0000 7f4dd1324640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4dcc0b2340 0x7f4dcc0cd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54760/0 (socket says 172.21.15.47:54760) 2024-04-26T23:19:25.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.119+0000 7f4dd2326640 1 -- 172.21.15.47:0/1961523645 learned_addr learned my addr 172.21.15.47:0/1961523645 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:25.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.119+0000 7f4dd1324640 1 -- 172.21.15.47:0/1961523645 >> v2:172.21.15.203:3300/0 conn(0x7f4dcc162810 msgr2=0x7f4dcc070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:25.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.119+0000 7f4dd1b25640 1 --2- 172.21.15.47:0/1961523645 >> v2:172.21.15.203:3300/0 conn(0x7f4dcc162810 0x7f4dcc070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:25.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.119+0000 7f4dd1324640 1 --2- 172.21.15.47:0/1961523645 >> v2:172.21.15.203:3300/0 conn(0x7f4dcc162810 0x7f4dcc070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:25.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.120+0000 7f4dd1324640 1 -- 172.21.15.47:0/1961523645 >> v1:172.21.15.47:6789/0 conn(0x7f4dcc0cd930 legacy=0x7f4dcc0d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:25.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.120+0000 7f4dd1324640 1 -- 172.21.15.47:0/1961523645 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dcc0d1370 con 0x7f4dcc0b2340 2024-04-26T23:19:25.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.119+0000 7f4dd0b23640 1 -- 172.21.15.47:0/1961523645 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2299325535 0 0) 0x7f4dcc0d10d0 con 0x7f4dcc0cd930 2024-04-26T23:19:25.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.120+0000 7f4dd1324640 1 --2- 172.21.15.47:0/1961523645 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4dcc0b2340 0x7f4dcc0cd3f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc073d90 tx=0x7f4dc002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f425e55f7757f8ad server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:25.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.120+0000 7f4dd0b23640 1 -- 172.21.15.47:0/1961523645 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0037070 con 0x7f4dcc0b2340 2024-04-26T23:19:25.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.120+0000 7f4dd0b23640 1 -- 172.21.15.47:0/1961523645 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dc0032440 con 0x7f4dcc0b2340 2024-04-26T23:19:25.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.120+0000 7f4dd0b23640 1 -- 172.21.15.47:0/1961523645 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0032810 con 0x7f4dcc0b2340 2024-04-26T23:19:25.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.121+0000 7f4dd3d89640 1 -- 172.21.15.47:0/1961523645 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4dcc0b2340 msgr2=0x7f4dcc0cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:25.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.121+0000 7f4dd3d89640 1 --2- 172.21.15.47:0/1961523645 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4dcc0b2340 0x7f4dcc0cd3f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc073d90 tx=0x7f4dc002a1a0 comp rx=0 tx=0).stop 2024-04-26T23:19:25.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.121+0000 7f4dd3d89640 1 -- 172.21.15.47:0/1961523645 shutdown_connections 2024-04-26T23:19:25.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.121+0000 7f4dd3d89640 1 --2- 172.21.15.47:0/1961523645 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4dcc0b2340 0x7f4dcc0cd3f0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:25.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.121+0000 7f4dd3d89640 1 --2- 172.21.15.47:0/1961523645 >> v2:172.21.15.203:3300/0 conn(0x7f4dcc162810 0x7f4dcc070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:25.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.121+0000 7f4dd3d89640 1 -- 172.21.15.47:0/1961523645 >> 172.21.15.47:0/1961523645 conn(0x7f4dcc0d2da0 msgr2=0x7f4dcc0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:25.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.121+0000 7f4dd3d89640 1 -- 172.21.15.47:0/1961523645 shutdown_connections 2024-04-26T23:19:25.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.121+0000 7f4dd3d89640 1 -- 172.21.15.47:0/1961523645 wait complete. 2024-04-26T23:19:25.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.122+0000 7f4dd3d89640 1 Processor -- start 2024-04-26T23:19:25.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.122+0000 7f4dd3d89640 1 -- start start 2024-04-26T23:19:25.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.123+0000 7f4dd3d89640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4dcc0b2340 0x7f4dcc0c8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:25.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.123+0000 7f4dd3d89640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4dcc0cd930 0x7f4dcc0c6a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:25.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.123+0000 7f4dd3d89640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4dcc162810 0x7f4dcc1f8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:25.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.123+0000 7f4dd3d89640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4dcc070890 con 0x7f4dcc162810 2024-04-26T23:19:25.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.123+0000 7f4dd3d89640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4dcc0d0470 con 0x7f4dcc0cd930 2024-04-26T23:19:25.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.123+0000 7f4dd3d89640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f4dcc0d2240 con 0x7f4dcc0b2340 2024-04-26T23:19:25.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.123+0000 7f4dd2326640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4dcc162810 0x7f4dcc1f8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:25.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.123+0000 7f4dd2326640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4dcc162810 0x7f4dcc1f8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54770/0 (socket says 172.21.15.47:54770) 2024-04-26T23:19:25.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.123+0000 7f4dd2326640 1 -- 172.21.15.47:0/514900113 learned_addr learned my addr 172.21.15.47:0/514900113 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:25.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.123+0000 7f4dd1324640 1 --2- 172.21.15.47:0/514900113 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4dcc0cd930 0x7f4dcc0c6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:25.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.123+0000 7f4dd1b25640 1 --2- 172.21.15.47:0/514900113 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4dcc0b2340 0x7f4dcc0c8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:25.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.123+0000 7f4dd2326640 1 -- 172.21.15.47:0/514900113 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4dcc0b2340 msgr2=0x7f4dcc0c8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:25.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.123+0000 7f4dd2326640 1 --2- 172.21.15.47:0/514900113 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4dcc0b2340 0x7f4dcc0c8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:25.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.123+0000 7f4dd2326640 1 -- 172.21.15.47:0/514900113 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4dcc0cd930 msgr2=0x7f4dcc0c6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:25.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.123+0000 7f4dd2326640 1 --2- 172.21.15.47:0/514900113 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4dcc0cd930 0x7f4dcc0c6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:25.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.123+0000 7f4dd2326640 1 -- 172.21.15.47:0/514900113 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dcc0c6fd0 con 0x7f4dcc162810 2024-04-26T23:19:25.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.123+0000 7f4dd1324640 1 --2- 172.21.15.47:0/514900113 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4dcc0cd930 0x7f4dcc0c6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:19:25.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.123+0000 7f4dd2326640 1 --2- 172.21.15.47:0/514900113 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4dcc162810 0x7f4dcc1f8400 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc00c760 tx=0x7f4dbc00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:25.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.124+0000 7f4dca7fc640 1 -- 172.21.15.47:0/514900113 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc018070 con 0x7f4dcc162810 2024-04-26T23:19:25.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.124+0000 7f4dca7fc640 1 -- 172.21.15.47:0/514900113 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dbc00a640 con 0x7f4dcc162810 2024-04-26T23:19:25.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.124+0000 7f4dd3d89640 1 -- 172.21.15.47:0/514900113 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dcc0c9bd0 con 0x7f4dcc162810 2024-04-26T23:19:25.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.124+0000 7f4dca7fc640 1 -- 172.21.15.47:0/514900113 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc00aa30 con 0x7f4dcc162810 2024-04-26T23:19:25.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.124+0000 7f4dd3d89640 1 -- 172.21.15.47:0/514900113 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dcc0c98f0 con 0x7f4dcc162810 2024-04-26T23:19:25.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.125+0000 7f4dd3d89640 1 -- 172.21.15.47:0/514900113 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dcc0694f0 con 0x7f4dcc162810 2024-04-26T23:19:25.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.125+0000 7f4dca7fc640 1 -- 172.21.15.47:0/514900113 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4dbc01b070 con 0x7f4dcc162810 2024-04-26T23:19:25.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.125+0000 7f4dca7fc640 1 --2- 172.21.15.47:0/514900113 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4db806c8f0 0x7f4db806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:25.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.126+0000 7f4dca7fc640 1 -- 172.21.15.47:0/514900113 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f4dbc08b4e0 con 0x7f4dcc162810 2024-04-26T23:19:25.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.126+0000 7f4dd1b25640 1 --2- 172.21.15.47:0/514900113 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4db806c8f0 0x7f4db806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:25.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.126+0000 7f4dd1b25640 1 --2- 172.21.15.47:0/514900113 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4db806c8f0 0x7f4db806edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4dc400b7c0 tx=0x7f4dc4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:25.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.130+0000 7f4dca7fc640 1 -- 172.21.15.47:0/514900113 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4dbc0594f0 con 0x7f4dcc162810 2024-04-26T23:19:25.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.318+0000 7f4dd3d89640 1 -- 172.21.15.47:0/514900113 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4dcc06ce60 con 0x7f4dcc162810 2024-04-26T23:19:25.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.320+0000 7f4dca7fc640 1 -- 172.21.15.47:0/514900113 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18857 (secure 0 0 0) 0x7f4dbc058df0 con 0x7f4dcc162810 2024-04-26T23:19:25.321 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:19:25.321 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":26,"fsid":"399006d3-4db1-43e6-ba81-a79a276eabfd","created":"2024-04-26T23:18:06.731211+0000","modified":"2024-04-26T23:19:12.124051+0000","last_up_change":"2024-04-26T23:18:17.808736+0000","last_in_change":"2024-04-26T23:18:08.945859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:18:16.937492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:18:29.033575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:18:30.318499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:18:34.014536+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ee091567-fea0-4a14-bf4e-0e16c20bb2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6819","nonce":326520779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6821","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6823","nonce":326520779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6829","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6831","nonce":326520779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6825","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6827","nonce":326520779}]},"public_addr":"172.21.15.47:6819/326520779","cluster_addr":"172.21.15.47:6823/326520779","heartbeat_back_addr":"172.21.15.47:6831/326520779","heartbeat_front_addr":"172.21.15.47:6827/326520779","state":["exists","up"]},{"osd":1,"uuid":"c14440a0-bba5-4931-a6db-b736c08e869b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6809","nonce":1912610931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6811","nonce":1912610931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6815","nonce":1912610931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6813","nonce":1912610931}]},"public_addr":"172.21.15.47:6809/1912610931","cluster_addr":"172.21.15.47:6811/1912610931","heartbeat_back_addr":"172.21.15.47:6815/1912610931","heartbeat_front_addr":"172.21.15.47:6813/1912610931","state":["exists","up"]},{"osd":2,"uuid":"3940f8de-c9b2-4cd3-a911-9c301729eb30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6801","nonce":2569869305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6803","nonce":2569869305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6807","nonce":2569869305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6805","nonce":2569869305}]},"public_addr":"172.21.15.47:6801/2569869305","cluster_addr":"172.21.15.47:6803/2569869305","heartbeat_back_addr":"172.21.15.47:6807/2569869305","heartbeat_front_addr":"172.21.15.47:6805/2569869305","state":["exists","up"]},{"osd":3,"uuid":"1286a67e-8958-426a-8c2e-bc85498c619c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6818","nonce":2791411683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6822","nonce":2791411683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6830","nonce":2791411683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6826","nonce":2791411683}]},"public_addr":"172.21.15.47:6818/2791411683","cluster_addr":"172.21.15.47:6822/2791411683","heartbeat_back_addr":"172.21.15.47:6830/2791411683","heartbeat_front_addr":"172.21.15.47:6826/2791411683","state":["exists","up"]},{"osd":4,"uuid":"d8ee2c20-e5b7-46c2-9999-b234070126cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6809","nonce":441313104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6811","nonce":441313104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6815","nonce":441313104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6813","nonce":441313104}]},"public_addr":"172.21.15.203:6809/441313104","cluster_addr":"172.21.15.203:6811/441313104","heartbeat_back_addr":"172.21.15.203:6815/441313104","heartbeat_front_addr":"172.21.15.203:6813/441313104","state":["exists","up"]},{"osd":5,"uuid":"b453a8f0-f324-4840-9ae0-d67bdadb9f8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6801","nonce":56838444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6803","nonce":56838444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6807","nonce":56838444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6805","nonce":56838444}]},"public_addr":"172.21.15.203:6801/56838444","cluster_addr":"172.21.15.203:6803/56838444","heartbeat_back_addr":"172.21.15.203:6807/56838444","heartbeat_front_addr":"172.21.15.203:6805/56838444","state":["exists","up"]},{"osd":6,"uuid":"a6d31951-2d4b-4e8d-a510-6283b01dc158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6825","nonce":2738517080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6827","nonce":2738517080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6831","nonce":2738517080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6829","nonce":2738517080}]},"public_addr":"172.21.15.203:6825/2738517080","cluster_addr":"172.21.15.203:6827/2738517080","heartbeat_back_addr":"172.21.15.203:6831/2738517080","heartbeat_front_addr":"172.21.15.203:6829/2738517080","state":["exists","up"]},{"osd":7,"uuid":"7119b8c0-fe96-402d-9e75-7b509d9d44d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6817","nonce":1315565058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6819","nonce":1315565058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6823","nonce":1315565058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6821","nonce":1315565058}]},"public_addr":"172.21.15.203:6817/1315565058","cluster_addr":"172.21.15.203:6819/1315565058","heartbeat_back_addr":"172.21.15.203:6823/1315565058","heartbeat_front_addr":"172.21.15.203:6821/1315565058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.121921+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.137494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.084814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.111593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.072430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.125066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.097867+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.057061+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":5,"to":1}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:19:25.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.321+0000 7f4dd3d89640 1 -- 172.21.15.47:0/514900113 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4db806c8f0 msgr2=0x7f4db806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:25.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.321+0000 7f4dd3d89640 1 --2- 172.21.15.47:0/514900113 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4db806c8f0 0x7f4db806edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4dc400b7c0 tx=0x7f4dc4008040 comp rx=0 tx=0).stop 2024-04-26T23:19:25.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.321+0000 7f4dd3d89640 1 -- 172.21.15.47:0/514900113 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4dcc162810 msgr2=0x7f4dcc1f8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:25.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.321+0000 7f4dd3d89640 1 --2- 172.21.15.47:0/514900113 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4dcc162810 0x7f4dcc1f8400 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc00c760 tx=0x7f4dbc00cc20 comp rx=0 tx=0).stop 2024-04-26T23:19:25.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.322+0000 7f4dd3d89640 1 -- 172.21.15.47:0/514900113 shutdown_connections 2024-04-26T23:19:25.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.322+0000 7f4dd3d89640 1 --2- 172.21.15.47:0/514900113 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4db806c8f0 0x7f4db806edb0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:25.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.322+0000 7f4dd3d89640 1 --2- 172.21.15.47:0/514900113 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4dcc162810 0x7f4dcc1f8400 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:25.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.322+0000 7f4dd3d89640 1 --2- 172.21.15.47:0/514900113 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4dcc0cd930 0x7f4dcc0c6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:25.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.322+0000 7f4dd3d89640 1 --2- 172.21.15.47:0/514900113 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4dcc0b2340 0x7f4dcc0c8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:25.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.322+0000 7f4dd3d89640 1 -- 172.21.15.47:0/514900113 >> 172.21.15.47:0/514900113 conn(0x7f4dcc0d2da0 msgr2=0x7f4dcc0b2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:25.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.322+0000 7f4dd3d89640 1 -- 172.21.15.47:0/514900113 shutdown_connections 2024-04-26T23:19:25.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.322+0000 7f4dd3d89640 1 -- 172.21.15.47:0/514900113 wait complete. 2024-04-26T23:19:25.333 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:18:16.937492+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:18:29.033575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:18:30.318499+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T23:18:34.014536+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T23:19:25.334 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:19:25.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.437+0000 7fed2a408640 1 Processor -- start 2024-04-26T23:19:25.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.437+0000 7fed2a408640 1 -- start start 2024-04-26T23:19:25.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.437+0000 7fed2a408640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed24071260 0x7fed2415e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:25.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.437+0000 7fed2a408640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed240d2390 con 0x7fed24070350 2024-04-26T23:19:25.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.437+0000 7fed2a408640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed2406d3f0 con 0x7fed240cfde0 2024-04-26T23:19:25.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.437+0000 7fed2a408640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fed240afab0 con 0x7fed24071260 2024-04-26T23:19:25.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.437+0000 7fed237fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fed24070350 0x7fed24070b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:55620/0 (socket says 172.21.15.47:55620) 2024-04-26T23:19:25.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.437+0000 7fed237fe640 1 -- 172.21.15.47:0/2066365083 learned_addr learned my addr 172.21.15.47:0/2066365083 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:25.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.438+0000 7fed289a5640 1 --2- 172.21.15.47:0/2066365083 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed24071260 0x7fed2415e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:25.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.438+0000 7fed22ffd640 1 -- 172.21.15.47:0/2066365083 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2808677387 0 0) 0x7fed240d2390 con 0x7fed24070350 2024-04-26T23:19:25.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.438+0000 7fed22ffd640 1 -- 172.21.15.47:0/2066365083 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed0c003520 con 0x7fed24070350 2024-04-26T23:19:25.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.438+0000 7fed22ffd640 1 -- 172.21.15.47:0/2066365083 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4093436176 0 0) 0x7fed2406d3f0 con 0x7fed240cfde0 2024-04-26T23:19:25.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.438+0000 7fed22ffd640 1 -- 172.21.15.47:0/2066365083 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed240d2390 con 0x7fed240cfde0 2024-04-26T23:19:25.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.438+0000 7fed22ffd640 1 -- 172.21.15.47:0/2066365083 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2094736284 0 0) 0x7fed0c003520 con 0x7fed24070350 2024-04-26T23:19:25.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.438+0000 7fed22ffd640 1 -- 172.21.15.47:0/2066365083 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed24071260 msgr2=0x7fed2415e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:25.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.438+0000 7fed22ffd640 1 --2- 172.21.15.47:0/2066365083 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed24071260 0x7fed2415e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:25.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.438+0000 7fed22ffd640 1 -- 172.21.15.47:0/2066365083 >> v1:172.21.15.203:6789/0 conn(0x7fed240cfde0 legacy=0x7fed240d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:25.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.438+0000 7fed22ffd640 1 -- 172.21.15.47:0/2066365083 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed2406d6d0 con 0x7fed24070350 2024-04-26T23:19:25.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.438+0000 7fed22ffd640 1 -- 172.21.15.47:0/2066365083 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7fed18001670 con 0x7fed24070350 2024-04-26T23:19:25.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.438+0000 7fed22ffd640 1 -- 172.21.15.47:0/2066365083 >> v1:172.21.15.47:6789/0 conn(0x7fed24070350 legacy=0x7fed24070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:25.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.438+0000 7fed22ffd640 1 --2- 172.21.15.47:0/2066365083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed0c003b70 0x7fed0c003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:25.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.438+0000 7fed22ffd640 1 --2- 172.21.15.47:0/2066365083 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed0c004800 0x7fed0c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:25.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.438+0000 7fed22ffd640 1 --2- 172.21.15.47:0/2066365083 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed0c0051c0 0x7fed0c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:25.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.438+0000 7fed22ffd640 1 -- 172.21.15.47:0/2066365083 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fed0c00c090 con 0x7fed0c004800 2024-04-26T23:19:25.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.438+0000 7fed22ffd640 1 -- 172.21.15.47:0/2066365083 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fed0c00c210 con 0x7fed0c003b70 2024-04-26T23:19:25.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.438+0000 7fed22ffd640 1 -- 172.21.15.47:0/2066365083 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fed0c00c410 con 0x7fed0c0051c0 2024-04-26T23:19:25.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.438+0000 7fed289a5640 1 --2- 172.21.15.47:0/2066365083 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed24071260 0x7fed2415e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:19:25.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.438+0000 7fed237fe640 1 --2- 172.21.15.47:0/2066365083 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed0c004800 0x7fed0c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:25.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.438+0000 7fed23fff640 1 --2- 172.21.15.47:0/2066365083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed0c003b70 0x7fed0c003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:25.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.439+0000 7fed289a5640 1 --2- 172.21.15.47:0/2066365083 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed0c0051c0 0x7fed0c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:25.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.439+0000 7fed237fe640 1 -- 172.21.15.47:0/2066365083 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed0c0051c0 msgr2=0x7fed0c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:25.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.439+0000 7fed237fe640 1 --2- 172.21.15.47:0/2066365083 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed0c0051c0 0x7fed0c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:25.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.439+0000 7fed237fe640 1 -- 172.21.15.47:0/2066365083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed0c003b70 msgr2=0x7fed0c003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:25.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.439+0000 7fed237fe640 1 --2- 172.21.15.47:0/2066365083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed0c003b70 0x7fed0c003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:25.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.439+0000 7fed237fe640 1 -- 172.21.15.47:0/2066365083 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed0c003520 con 0x7fed0c004800 2024-04-26T23:19:25.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.439+0000 7fed23fff640 1 --2- 172.21.15.47:0/2066365083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed0c003b70 0x7fed0c003f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:19:25.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.439+0000 7fed237fe640 1 --2- 172.21.15.47:0/2066365083 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed0c004800 0x7fed0c004c80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fed1800c0e0 tx=0x7fed0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=271dcebe8f69e9fd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:25.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.439+0000 7fed22ffd640 1 -- 172.21.15.47:0/2066365083 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed18007c40 con 0x7fed0c004800 2024-04-26T23:19:25.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.439+0000 7fed22ffd640 1 -- 172.21.15.47:0/2066365083 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed18005ad0 con 0x7fed0c004800 2024-04-26T23:19:25.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.439+0000 7fed22ffd640 1 -- 172.21.15.47:0/2066365083 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed18007070 con 0x7fed0c004800 2024-04-26T23:19:25.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.440+0000 7fed2a408640 1 -- 172.21.15.47:0/2066365083 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed0c004800 msgr2=0x7fed0c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:25.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.440+0000 7fed2a408640 1 --2- 172.21.15.47:0/2066365083 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed0c004800 0x7fed0c004c80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fed1800c0e0 tx=0x7fed0c005830 comp rx=0 tx=0).stop 2024-04-26T23:19:25.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.440+0000 7fed289a5640 1 -- 172.21.15.47:0/2066365083 reap_dead start 2024-04-26T23:19:25.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.440+0000 7fed2a408640 1 -- 172.21.15.47:0/2066365083 shutdown_connections 2024-04-26T23:19:25.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.440+0000 7fed2a408640 1 -- 172.21.15.47:0/2066365083 >> 172.21.15.47:0/2066365083 conn(0x7fed240d2d80 msgr2=0x7fed240d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:25.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.440+0000 7fed2a408640 1 -- 172.21.15.47:0/2066365083 shutdown_connections 2024-04-26T23:19:25.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.440+0000 7fed2a408640 1 -- 172.21.15.47:0/2066365083 wait complete. 2024-04-26T23:19:25.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.441+0000 7fed2a408640 1 Processor -- start 2024-04-26T23:19:25.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.441+0000 7fed2a408640 1 -- start start 2024-04-26T23:19:25.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.442+0000 7fed2a408640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed0c004800 0x7fed24155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:25.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.442+0000 7fed2a408640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed24159f30 0x7fed2415a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:25.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.442+0000 7fed2a408640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed24156360 0x7fed241567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:25.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.442+0000 7fed2a408640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fed2406f1c0 con 0x7fed0c004800 2024-04-26T23:19:25.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.442+0000 7fed2a408640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fed2406dd20 con 0x7fed24159f30 2024-04-26T23:19:25.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.442+0000 7fed2a408640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fed2406f340 con 0x7fed24156360 2024-04-26T23:19:25.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.442+0000 7fed23fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed0c004800 0x7fed24155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:25.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.442+0000 7fed23fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed0c004800 0x7fed24155bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54790/0 (socket says 172.21.15.47:54790) 2024-04-26T23:19:25.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.442+0000 7fed23fff640 1 -- 172.21.15.47:0/3724396075 learned_addr learned my addr 172.21.15.47:0/3724396075 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:25.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.442+0000 7fed289a5640 1 --2- 172.21.15.47:0/3724396075 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed24156360 0x7fed241567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:25.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.442+0000 7fed237fe640 1 --2- 172.21.15.47:0/3724396075 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed24159f30 0x7fed2415a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:25.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.443+0000 7fed23fff640 1 -- 172.21.15.47:0/3724396075 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed24156360 msgr2=0x7fed241567e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:25.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.443+0000 7fed23fff640 1 --2- 172.21.15.47:0/3724396075 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed24156360 0x7fed241567e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:25.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.443+0000 7fed23fff640 1 -- 172.21.15.47:0/3724396075 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed24159f30 msgr2=0x7fed2415a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:25.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.443+0000 7fed23fff640 1 --2- 172.21.15.47:0/3724396075 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed24159f30 0x7fed2415a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:25.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.443+0000 7fed23fff640 1 -- 172.21.15.47:0/3724396075 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed24156110 con 0x7fed0c004800 2024-04-26T23:19:25.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.443+0000 7fed289a5640 1 --2- 172.21.15.47:0/3724396075 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed24156360 0x7fed241567e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:19:25.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.443+0000 7fed237fe640 1 --2- 172.21.15.47:0/3724396075 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed24159f30 0x7fed2415a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:19:25.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.443+0000 7fed23fff640 1 --2- 172.21.15.47:0/3724396075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed0c004800 0x7fed24155bd0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fed1400e550 tx=0x7fed1400ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:25.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.443+0000 7fed20ff9640 1 -- 172.21.15.47:0/3724396075 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed140052e0 con 0x7fed0c004800 2024-04-26T23:19:25.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.443+0000 7fed20ff9640 1 -- 172.21.15.47:0/3724396075 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed14018070 con 0x7fed0c004800 2024-04-26T23:19:25.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.443+0000 7fed2a408640 1 -- 172.21.15.47:0/3724396075 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed2415a9b0 con 0x7fed0c004800 2024-04-26T23:19:25.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.443+0000 7fed20ff9640 1 -- 172.21.15.47:0/3724396075 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed14003810 con 0x7fed0c004800 2024-04-26T23:19:25.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.443+0000 7fed2a408640 1 -- 172.21.15.47:0/3724396075 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed2415af70 con 0x7fed0c004800 2024-04-26T23:19:25.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.444+0000 7fed2a408640 1 -- 172.21.15.47:0/3724396075 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed240694f0 con 0x7fed0c004800 2024-04-26T23:19:25.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.444+0000 7fed20ff9640 1 -- 172.21.15.47:0/3724396075 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fed140039b0 con 0x7fed0c004800 2024-04-26T23:19:25.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.445+0000 7fed20ff9640 1 --2- 172.21.15.47:0/3724396075 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fed1006c8f0 0x7fed1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:25.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.445+0000 7fed20ff9640 1 -- 172.21.15.47:0/3724396075 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7fed1408bcf0 con 0x7fed0c004800 2024-04-26T23:19:25.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.445+0000 7fed237fe640 1 --2- 172.21.15.47:0/3724396075 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fed1006c8f0 0x7fed1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:25.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.446+0000 7fed237fe640 1 --2- 172.21.15.47:0/3724396075 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fed1006c8f0 0x7fed1006edb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fed1800c0e0 tx=0x7fed1802d000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:25.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.450+0000 7fed20ff9640 1 -- 172.21.15.47:0/3724396075 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fed14090070 con 0x7fed0c004800 2024-04-26T23:19:25.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.633+0000 7fed2a408640 1 -- 172.21.15.47:0/3724396075 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fed240cf1e0 con 0x7fed0c004800 2024-04-26T23:19:25.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.633+0000 7fed20ff9640 1 -- 172.21.15.47:0/3724396075 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7fed14059d00 con 0x7fed0c004800 2024-04-26T23:19:25.634 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-26T23:19:25.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.635+0000 7fed2a408640 1 -- 172.21.15.47:0/3724396075 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fed1006c8f0 msgr2=0x7fed1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:25.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.635+0000 7fed2a408640 1 --2- 172.21.15.47:0/3724396075 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fed1006c8f0 0x7fed1006edb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fed1800c0e0 tx=0x7fed1802d000 comp rx=0 tx=0).stop 2024-04-26T23:19:25.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.635+0000 7fed2a408640 1 -- 172.21.15.47:0/3724396075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed0c004800 msgr2=0x7fed24155bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:25.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.635+0000 7fed2a408640 1 --2- 172.21.15.47:0/3724396075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed0c004800 0x7fed24155bd0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fed1400e550 tx=0x7fed1400ea10 comp rx=0 tx=0).stop 2024-04-26T23:19:25.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.635+0000 7fed2a408640 1 -- 172.21.15.47:0/3724396075 shutdown_connections 2024-04-26T23:19:25.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.635+0000 7fed2a408640 1 --2- 172.21.15.47:0/3724396075 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fed1006c8f0 0x7fed1006edb0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:25.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.635+0000 7fed2a408640 1 --2- 172.21.15.47:0/3724396075 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed24156360 0x7fed241567e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:25.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.635+0000 7fed2a408640 1 --2- 172.21.15.47:0/3724396075 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed24159f30 0x7fed2415a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:25.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.635+0000 7fed2a408640 1 --2- 172.21.15.47:0/3724396075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed0c004800 0x7fed24155bd0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:25.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.635+0000 7fed2a408640 1 -- 172.21.15.47:0/3724396075 >> 172.21.15.47:0/3724396075 conn(0x7fed240d2d80 msgr2=0x7fed240c6ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:25.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.635+0000 7fed2a408640 1 -- 172.21.15.47:0/3724396075 shutdown_connections 2024-04-26T23:19:25.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.635+0000 7fed2a408640 1 -- 172.21.15.47:0/3724396075 wait complete. 2024-04-26T23:19:25.647 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:19:25.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.751+0000 7fdf256d1640 1 Processor -- start 2024-04-26T23:19:25.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.751+0000 7fdf256d1640 1 -- start start 2024-04-26T23:19:25.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.752+0000 7fdf256d1640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdf200cfde0 0x7fdf200d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:25.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.752+0000 7fdf256d1640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdf20070350 0x7fdf20070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:25.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.752+0000 7fdf256d1640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf200d2390 con 0x7fdf20071090 2024-04-26T23:19:25.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.752+0000 7fdf256d1640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fdf200afab0 con 0x7fdf20070350 2024-04-26T23:19:25.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.752+0000 7fdf256d1640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fdf2006dd20 con 0x7fdf200cfde0 2024-04-26T23:19:25.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.752+0000 7fdf17fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdf20070350 0x7fdf20070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:25.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.752+0000 7fdf17fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdf20070350 0x7fdf20070b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54796/0 (socket says 172.21.15.47:54796) 2024-04-26T23:19:25.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.752+0000 7fdf17fff640 1 -- 172.21.15.47:0/3417594437 learned_addr learned my addr 172.21.15.47:0/3417594437 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:25.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.752+0000 7fdf1effd640 1 --2- 172.21.15.47:0/3417594437 >> v2:172.21.15.203:3300/0 conn(0x7fdf200cfde0 0x7fdf200d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:25.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.752+0000 7fdf17fff640 1 -- 172.21.15.47:0/3417594437 >> v2:172.21.15.203:3300/0 conn(0x7fdf200cfde0 msgr2=0x7fdf200d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:25.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.752+0000 7fdf17fff640 1 --2- 172.21.15.47:0/3417594437 >> v2:172.21.15.203:3300/0 conn(0x7fdf200cfde0 0x7fdf200d1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:25.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.752+0000 7fdf17fff640 1 -- 172.21.15.47:0/3417594437 >> v1:172.21.15.203:6789/0 conn(0x7fdf20071090 legacy=0x7fdf2015e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:25.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.752+0000 7fdf17fff640 1 -- 172.21.15.47:0/3417594437 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf2006d3f0 con 0x7fdf20070350 2024-04-26T23:19:25.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.753+0000 7fdf17fff640 1 --2- 172.21.15.47:0/3417594437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf20070350 0x7fdf20070b50 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fdf200762b0 tx=0x7fdf0802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37393ca4409470ff server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:25.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.753+0000 7fdf1e7fc640 1 -- 172.21.15.47:0/3417594437 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf08037070 con 0x7fdf20070350 2024-04-26T23:19:25.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.753+0000 7fdf1e7fc640 1 -- 172.21.15.47:0/3417594437 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf08032440 con 0x7fdf20070350 2024-04-26T23:19:25.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.753+0000 7fdf1e7fc640 1 -- 172.21.15.47:0/3417594437 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf08032810 con 0x7fdf20070350 2024-04-26T23:19:25.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.754+0000 7fdf256d1640 1 -- 172.21.15.47:0/3417594437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf20070350 msgr2=0x7fdf20070b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:25.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.754+0000 7fdf256d1640 1 --2- 172.21.15.47:0/3417594437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf20070350 0x7fdf20070b50 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fdf200762b0 tx=0x7fdf0802a1e0 comp rx=0 tx=0).stop 2024-04-26T23:19:25.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.754+0000 7fdf256d1640 1 -- 172.21.15.47:0/3417594437 shutdown_connections 2024-04-26T23:19:25.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.754+0000 7fdf256d1640 1 --2- 172.21.15.47:0/3417594437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf20070350 0x7fdf20070b50 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:25.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.754+0000 7fdf256d1640 1 --2- 172.21.15.47:0/3417594437 >> v2:172.21.15.203:3300/0 conn(0x7fdf200cfde0 0x7fdf200d1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:25.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.754+0000 7fdf256d1640 1 -- 172.21.15.47:0/3417594437 >> 172.21.15.47:0/3417594437 conn(0x7fdf200d2d80 msgr2=0x7fdf200d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:25.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.754+0000 7fdf256d1640 1 -- 172.21.15.47:0/3417594437 shutdown_connections 2024-04-26T23:19:25.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.754+0000 7fdf256d1640 1 -- 172.21.15.47:0/3417594437 wait complete. 2024-04-26T23:19:25.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.755+0000 7fdf256d1640 1 Processor -- start 2024-04-26T23:19:25.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.755+0000 7fdf256d1640 1 -- start start 2024-04-26T23:19:25.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.756+0000 7fdf256d1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf20070350 0x7fdf200c9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:25.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.756+0000 7fdf256d1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf20071090 0x7fdf2014ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:25.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.756+0000 7fdf256d1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf200cfde0 0x7fdf200afc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:25.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.756+0000 7fdf256d1640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2006d9b0 con 0x7fdf20071090 2024-04-26T23:19:25.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.756+0000 7fdf256d1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdf200714c0 con 0x7fdf200cfde0 2024-04-26T23:19:25.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.756+0000 7fdf256d1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fdf200d20a0 con 0x7fdf20070350 2024-04-26T23:19:25.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.756+0000 7fdf17fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf20071090 0x7fdf2014ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:25.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.756+0000 7fdf17fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf20071090 0x7fdf2014ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54804/0 (socket says 172.21.15.47:54804) 2024-04-26T23:19:25.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.756+0000 7fdf17fff640 1 -- 172.21.15.47:0/1361664238 learned_addr learned my addr 172.21.15.47:0/1361664238 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:25.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.756+0000 7fdf1f7fe640 1 --2- 172.21.15.47:0/1361664238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf200cfde0 0x7fdf200afc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:25.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.756+0000 7fdf1effd640 1 --2- 172.21.15.47:0/1361664238 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf20070350 0x7fdf200c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:25.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.756+0000 7fdf17fff640 1 -- 172.21.15.47:0/1361664238 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf20070350 msgr2=0x7fdf200c9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:25.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.756+0000 7fdf17fff640 1 --2- 172.21.15.47:0/1361664238 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf20070350 0x7fdf200c9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:25.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.756+0000 7fdf17fff640 1 -- 172.21.15.47:0/1361664238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf200cfde0 msgr2=0x7fdf200afc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:25.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.756+0000 7fdf17fff640 1 --2- 172.21.15.47:0/1361664238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf200cfde0 0x7fdf200afc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:25.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.756+0000 7fdf17fff640 1 -- 172.21.15.47:0/1361664238 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf200c9ba0 con 0x7fdf20071090 2024-04-26T23:19:25.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.756+0000 7fdf1f7fe640 1 --2- 172.21.15.47:0/1361664238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf200cfde0 0x7fdf200afc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:19:25.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.756+0000 7fdf17fff640 1 --2- 172.21.15.47:0/1361664238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf20071090 0x7fdf2014ab80 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fdf0802a220 tx=0x7fdf080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:25.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.756+0000 7fdf177fe640 1 -- 172.21.15.47:0/1361664238 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf08041020 con 0x7fdf20071090 2024-04-26T23:19:25.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.756+0000 7fdf177fe640 1 -- 172.21.15.47:0/1361664238 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf080306c0 con 0x7fdf20071090 2024-04-26T23:19:25.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.757+0000 7fdf256d1640 1 -- 172.21.15.47:0/1361664238 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf2014b530 con 0x7fdf20071090 2024-04-26T23:19:25.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.757+0000 7fdf177fe640 1 -- 172.21.15.47:0/1361664238 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf08037040 con 0x7fdf20071090 2024-04-26T23:19:25.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.757+0000 7fdf256d1640 1 -- 172.21.15.47:0/1361664238 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf2014baf0 con 0x7fdf20071090 2024-04-26T23:19:25.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.757+0000 7fdf256d1640 1 -- 172.21.15.47:0/1361664238 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf200d02c0 con 0x7fdf20071090 2024-04-26T23:19:25.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.758+0000 7fdf177fe640 1 -- 172.21.15.47:0/1361664238 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdf08044020 con 0x7fdf20071090 2024-04-26T23:19:25.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.758+0000 7fdf177fe640 1 --2- 172.21.15.47:0/1361664238 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdf0006c8f0 0x7fdf0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:25.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.759+0000 7fdf177fe640 1 -- 172.21.15.47:0/1361664238 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7fdf080ac7c0 con 0x7fdf20071090 2024-04-26T23:19:25.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.759+0000 7fdf1effd640 1 --2- 172.21.15.47:0/1361664238 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdf0006c8f0 0x7fdf0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:25.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.759+0000 7fdf1effd640 1 --2- 172.21.15.47:0/1361664238 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdf0006c8f0 0x7fdf0006edb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fdf10006fd0 tx=0x7fdf10008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:25.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.763+0000 7fdf177fe640 1 -- 172.21.15.47:0/1361664238 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdf200d02c0 con 0x7fdf20071090 2024-04-26T23:19:25.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.951+0000 7fdf256d1640 1 -- 172.21.15.47:0/1361664238 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdf200b8c40 con 0x7fdf20071090 2024-04-26T23:19:25.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.952+0000 7fdf177fe640 1 -- 172.21.15.47:0/1361664238 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fdf0804b030 con 0x7fdf20071090 2024-04-26T23:19:25.953 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-26T23:19:25.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.954+0000 7fdf256d1640 1 -- 172.21.15.47:0/1361664238 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdf0006c8f0 msgr2=0x7fdf0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:25.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.954+0000 7fdf256d1640 1 --2- 172.21.15.47:0/1361664238 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdf0006c8f0 0x7fdf0006edb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fdf10006fd0 tx=0x7fdf10008040 comp rx=0 tx=0).stop 2024-04-26T23:19:25.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.954+0000 7fdf256d1640 1 -- 172.21.15.47:0/1361664238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf20071090 msgr2=0x7fdf2014ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:25.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.954+0000 7fdf256d1640 1 --2- 172.21.15.47:0/1361664238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf20071090 0x7fdf2014ab80 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fdf0802a220 tx=0x7fdf080046c0 comp rx=0 tx=0).stop 2024-04-26T23:19:25.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.954+0000 7fdf256d1640 1 -- 172.21.15.47:0/1361664238 shutdown_connections 2024-04-26T23:19:25.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.954+0000 7fdf256d1640 1 --2- 172.21.15.47:0/1361664238 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdf0006c8f0 0x7fdf0006edb0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:25.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.954+0000 7fdf256d1640 1 --2- 172.21.15.47:0/1361664238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf200cfde0 0x7fdf200afc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:25.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.954+0000 7fdf256d1640 1 --2- 172.21.15.47:0/1361664238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf20071090 0x7fdf2014ab80 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:25.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.954+0000 7fdf256d1640 1 --2- 172.21.15.47:0/1361664238 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf20070350 0x7fdf200c9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:25.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.954+0000 7fdf256d1640 1 -- 172.21.15.47:0/1361664238 >> 172.21.15.47:0/1361664238 conn(0x7fdf200d2d80 msgr2=0x7fdf2015fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:25.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.954+0000 7fdf256d1640 1 -- 172.21.15.47:0/1361664238 shutdown_connections 2024-04-26T23:19:25.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:25.954+0000 7fdf256d1640 1 -- 172.21.15.47:0/1361664238 wait complete. 2024-04-26T23:19:25.966 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:19:26.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.069+0000 7f484a252640 1 Processor -- start 2024-04-26T23:19:26.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.069+0000 7f484a252640 1 -- start start 2024-04-26T23:19:26.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.070+0000 7f484a252640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f48440d3bc0 0x7f48440d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:26.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.070+0000 7f484a252640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48440d1040 con 0x7f484415f170 2024-04-26T23:19:26.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.070+0000 7f484a252640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48440c9680 con 0x7f484415f550 2024-04-26T23:19:26.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.070+0000 7f484a252640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f48440c9430 con 0x7f48440d3bc0 2024-04-26T23:19:26.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.070+0000 7f48437fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f484415f170 0x7f48440d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:55634/0 (socket says 172.21.15.47:55634) 2024-04-26T23:19:26.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.070+0000 7f48437fe640 1 -- 172.21.15.47:0/1356269617 learned_addr learned my addr 172.21.15.47:0/1356269617 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:26.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.070+0000 7f4843fff640 1 --2- 172.21.15.47:0/1356269617 >> v2:172.21.15.203:3300/0 conn(0x7f48440d3bc0 0x7f48440d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:26.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.070+0000 7f48427fc640 1 -- 172.21.15.47:0/1356269617 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3013681272 0 0) 0x7f4834001110 con 0x7f484415f170 2024-04-26T23:19:26.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.070+0000 7f48427fc640 1 -- 172.21.15.47:0/1356269617 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f482c003520 con 0x7f484415f170 2024-04-26T23:19:26.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.071+0000 7f48427fc640 1 -- 172.21.15.47:0/1356269617 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2415225671 0 0) 0x7f48440c9680 con 0x7f484415f550 2024-04-26T23:19:26.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.071+0000 7f48427fc640 1 -- 172.21.15.47:0/1356269617 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4834001110 con 0x7f484415f550 2024-04-26T23:19:26.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.071+0000 7f4843fff640 1 -- 172.21.15.47:0/1356269617 >> v1:172.21.15.203:6789/0 conn(0x7f484415f550 legacy=0x7f48440ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:26.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.071+0000 7f4843fff640 1 -- 172.21.15.47:0/1356269617 >> v1:172.21.15.47:6789/0 conn(0x7f484415f170 legacy=0x7f48440d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:26.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.071+0000 7f48427fc640 1 -- 172.21.15.47:0/1356269617 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2682676478 0 0) 0x7f482c003520 con 0x7f484415f170 2024-04-26T23:19:26.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.071+0000 7f4843fff640 1 -- 172.21.15.47:0/1356269617 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48440c9960 con 0x7f48440d3bc0 2024-04-26T23:19:26.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.071+0000 7f4843fff640 1 --2- 172.21.15.47:0/1356269617 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48440d3bc0 0x7f48440d5a80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f48440c95b0 tx=0x7f483c02a6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a457196c933294a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:26.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.072+0000 7f48427fc640 1 -- 172.21.15.47:0/1356269617 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4834001840 con 0x7f48440d3bc0 2024-04-26T23:19:26.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.072+0000 7f48427fc640 1 -- 172.21.15.47:0/1356269617 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f483c007540 con 0x7f48440d3bc0 2024-04-26T23:19:26.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.072+0000 7f48427fc640 1 -- 172.21.15.47:0/1356269617 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f483c007710 con 0x7f48440d3bc0 2024-04-26T23:19:26.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.072+0000 7f484a252640 1 -- 172.21.15.47:0/1356269617 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48440d3bc0 msgr2=0x7f48440d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:26.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.072+0000 7f484a252640 1 --2- 172.21.15.47:0/1356269617 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48440d3bc0 0x7f48440d5a80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f48440c95b0 tx=0x7f483c02a6a0 comp rx=0 tx=0).stop 2024-04-26T23:19:26.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.072+0000 7f484a252640 1 -- 172.21.15.47:0/1356269617 shutdown_connections 2024-04-26T23:19:26.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.072+0000 7f484a252640 1 --2- 172.21.15.47:0/1356269617 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48440d3bc0 0x7f48440d5a80 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:26.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.072+0000 7f484a252640 1 -- 172.21.15.47:0/1356269617 >> 172.21.15.47:0/1356269617 conn(0x7f48440d4f60 msgr2=0x7f48440d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:26.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.072+0000 7f484a252640 1 -- 172.21.15.47:0/1356269617 shutdown_connections 2024-04-26T23:19:26.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.072+0000 7f484a252640 1 -- 172.21.15.47:0/1356269617 wait complete. 2024-04-26T23:19:26.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.073+0000 7f484a252640 1 Processor -- start 2024-04-26T23:19:26.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.074+0000 7f484a252640 1 -- start start 2024-04-26T23:19:26.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.074+0000 7f484a252640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48440d3bc0 0x7f48441548d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:26.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.074+0000 7f484a252640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f484415f170 0x7f48440b9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:26.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.074+0000 7f484a252640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48440ba340 0x7f48441527b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:26.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.074+0000 7f484a252640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f48440d2e30 con 0x7f484415f170 2024-04-26T23:19:26.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.074+0000 7f484a252640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f48440d2cb0 con 0x7f48440d3bc0 2024-04-26T23:19:26.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.074+0000 7f484a252640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f48440d2fb0 con 0x7f48440ba340 2024-04-26T23:19:26.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.075+0000 7f4842ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f484415f170 0x7f48440b9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:26.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.075+0000 7f4842ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f484415f170 0x7f48440b9e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54816/0 (socket says 172.21.15.47:54816) 2024-04-26T23:19:26.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.075+0000 7f4842ffd640 1 -- 172.21.15.47:0/3537726043 learned_addr learned my addr 172.21.15.47:0/3537726043 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:26.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.075+0000 7f4843fff640 1 --2- 172.21.15.47:0/3537726043 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48440ba340 0x7f48441527b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:26.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.075+0000 7f48437fe640 1 --2- 172.21.15.47:0/3537726043 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48440d3bc0 0x7f48441548d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:26.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.075+0000 7f4842ffd640 1 -- 172.21.15.47:0/3537726043 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48440ba340 msgr2=0x7f48441527b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:26.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.075+0000 7f4842ffd640 1 --2- 172.21.15.47:0/3537726043 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48440ba340 0x7f48441527b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:26.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.075+0000 7f4842ffd640 1 -- 172.21.15.47:0/3537726043 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48440d3bc0 msgr2=0x7f48441548d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:26.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.075+0000 7f4842ffd640 1 --2- 172.21.15.47:0/3537726043 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48440d3bc0 0x7f48441548d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:26.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.075+0000 7f4842ffd640 1 -- 172.21.15.47:0/3537726043 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4844151710 con 0x7f484415f170 2024-04-26T23:19:26.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.075+0000 7f48437fe640 1 --2- 172.21.15.47:0/3537726043 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48440d3bc0 0x7f48441548d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:19:26.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.075+0000 7f4842ffd640 1 --2- 172.21.15.47:0/3537726043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f484415f170 0x7f48440b9e00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f483800c390 tx=0x7f483800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:26.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.075+0000 7f482bfff640 1 -- 172.21.15.47:0/3537726043 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4838017070 con 0x7f484415f170 2024-04-26T23:19:26.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.075+0000 7f482bfff640 1 -- 172.21.15.47:0/3537726043 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48380052f0 con 0x7f484415f170 2024-04-26T23:19:26.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.075+0000 7f484a252640 1 -- 172.21.15.47:0/3537726043 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48441519f0 con 0x7f484415f170 2024-04-26T23:19:26.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.075+0000 7f482bfff640 1 -- 172.21.15.47:0/3537726043 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4838003830 con 0x7f484415f170 2024-04-26T23:19:26.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.075+0000 7f484a252640 1 -- 172.21.15.47:0/3537726043 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4844151fb0 con 0x7f484415f170 2024-04-26T23:19:26.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.076+0000 7f484a252640 1 -- 172.21.15.47:0/3537726043 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4844068eb0 con 0x7f484415f170 2024-04-26T23:19:26.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.077+0000 7f482bfff640 1 -- 172.21.15.47:0/3537726043 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f483801a070 con 0x7f484415f170 2024-04-26T23:19:26.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.077+0000 7f482bfff640 1 --2- 172.21.15.47:0/3537726043 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f483006c920 0x7f483006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:26.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.077+0000 7f482bfff640 1 -- 172.21.15.47:0/3537726043 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f483808b300 con 0x7f484415f170 2024-04-26T23:19:26.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.078+0000 7f48437fe640 1 --2- 172.21.15.47:0/3537726043 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f483006c920 0x7f483006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:26.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.078+0000 7f48437fe640 1 --2- 172.21.15.47:0/3537726043 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f483006c920 0x7f483006ede0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f48340010e0 tx=0x7f4834009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:26.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.082+0000 7f482bfff640 1 -- 172.21.15.47:0/3537726043 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f48380593d0 con 0x7f484415f170 2024-04-26T23:19:26.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.269+0000 7f484a252640 1 -- 172.21.15.47:0/3537726043 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f48440694f0 con 0x7f484415f170 2024-04-26T23:19:26.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.270+0000 7f482bfff640 1 -- 172.21.15.47:0/3537726043 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f4838058cd0 con 0x7f484415f170 2024-04-26T23:19:26.271 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-26T23:19:26.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.272+0000 7f484a252640 1 -- 172.21.15.47:0/3537726043 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f483006c920 msgr2=0x7f483006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:26.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.272+0000 7f484a252640 1 --2- 172.21.15.47:0/3537726043 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f483006c920 0x7f483006ede0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f48340010e0 tx=0x7f4834009040 comp rx=0 tx=0).stop 2024-04-26T23:19:26.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.272+0000 7f484a252640 1 -- 172.21.15.47:0/3537726043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f484415f170 msgr2=0x7f48440b9e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:26.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.272+0000 7f484a252640 1 --2- 172.21.15.47:0/3537726043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f484415f170 0x7f48440b9e00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f483800c390 tx=0x7f483800c850 comp rx=0 tx=0).stop 2024-04-26T23:19:26.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.272+0000 7f484a252640 1 -- 172.21.15.47:0/3537726043 shutdown_connections 2024-04-26T23:19:26.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.272+0000 7f484a252640 1 --2- 172.21.15.47:0/3537726043 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f483006c920 0x7f483006ede0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:26.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.272+0000 7f484a252640 1 --2- 172.21.15.47:0/3537726043 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48440ba340 0x7f48441527b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:26.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.272+0000 7f484a252640 1 --2- 172.21.15.47:0/3537726043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f484415f170 0x7f48440b9e00 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:26.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.272+0000 7f484a252640 1 --2- 172.21.15.47:0/3537726043 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48440d3bc0 0x7f48441548d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:26.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.272+0000 7f484a252640 1 -- 172.21.15.47:0/3537726043 >> 172.21.15.47:0/3537726043 conn(0x7f48440d4f60 msgr2=0x7f48440d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:26.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.272+0000 7f484a252640 1 -- 172.21.15.47:0/3537726043 shutdown_connections 2024-04-26T23:19:26.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.272+0000 7f484a252640 1 -- 172.21.15.47:0/3537726043 wait complete. 2024-04-26T23:19:26.284 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T23:19:26.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.388+0000 7f4656693640 1 Processor -- start 2024-04-26T23:19:26.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.388+0000 7f4656693640 1 -- start start 2024-04-26T23:19:26.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.389+0000 7f4656693640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4650071e60 0x7f465006f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:26.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.389+0000 7f4656693640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f465006fe20 0x7f4650072240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:26.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.389+0000 7f4656693640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f465015d7c0 con 0x7f4650072780 2024-04-26T23:19:26.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.389+0000 7f4656693640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f465015daa0 con 0x7f465006fe20 2024-04-26T23:19:26.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.389+0000 7f4656693640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f465015dd80 con 0x7f4650071e60 2024-04-26T23:19:26.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.389+0000 7f4654c30640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4650072780 0x7f46500cb470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:55646/0 (socket says 172.21.15.47:55646) 2024-04-26T23:19:26.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.389+0000 7f4654c30640 1 -- 172.21.15.47:0/2872599613 learned_addr learned my addr 172.21.15.47:0/2872599613 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:26.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.389+0000 7f464f7fe640 1 --2- 172.21.15.47:0/2872599613 >> v2:172.21.15.203:3300/0 conn(0x7f465006fe20 0x7f4650072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:26.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.389+0000 7f464ffff640 1 --2- 172.21.15.47:0/2872599613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4650071e60 0x7f465006f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:26.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.389+0000 7f464effd640 1 -- 172.21.15.47:0/2872599613 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 130540995 0 0) 0x7f465015d7c0 con 0x7f4650072780 2024-04-26T23:19:26.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.389+0000 7f464effd640 1 -- 172.21.15.47:0/2872599613 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4638003520 con 0x7f4650072780 2024-04-26T23:19:26.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.389+0000 7f464effd640 1 -- 172.21.15.47:0/2872599613 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2441127550 0 0) 0x7f4638003520 con 0x7f4650072780 2024-04-26T23:19:26.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.389+0000 7f464effd640 1 -- 172.21.15.47:0/2872599613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4650071e60 msgr2=0x7f465006f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:26.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.389+0000 7f464effd640 1 --2- 172.21.15.47:0/2872599613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4650071e60 0x7f465006f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:26.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.389+0000 7f464effd640 1 -- 172.21.15.47:0/2872599613 >> v2:172.21.15.203:3300/0 conn(0x7f465006fe20 msgr2=0x7f4650072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:26.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.389+0000 7f464effd640 1 --2- 172.21.15.47:0/2872599613 >> v2:172.21.15.203:3300/0 conn(0x7f465006fe20 0x7f4650072240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:26.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.389+0000 7f464effd640 1 -- 172.21.15.47:0/2872599613 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f465015e060 con 0x7f4650072780 2024-04-26T23:19:26.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.389+0000 7f464effd640 1 -- 172.21.15.47:0/2872599613 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7f46480019f0 con 0x7f4650072780 2024-04-26T23:19:26.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.389+0000 7f464effd640 1 -- 172.21.15.47:0/2872599613 >> v1:172.21.15.47:6789/0 conn(0x7f4650072780 legacy=0x7f46500cb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:26.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.389+0000 7f464effd640 1 --2- 172.21.15.47:0/2872599613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4650071e60 0x7f46380039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:26.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.389+0000 7f464effd640 1 --2- 172.21.15.47:0/2872599613 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4638003f30 0x7f4638004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:26.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.389+0000 7f464effd640 1 --2- 172.21.15.47:0/2872599613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4638004d40 0x7f46380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:26.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.389+0000 7f464effd640 1 -- 172.21.15.47:0/2872599613 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f465015daa0 con 0x7f4638003f30 2024-04-26T23:19:26.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.389+0000 7f464effd640 1 -- 172.21.15.47:0/2872599613 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f465015dd80 con 0x7f4638004d40 2024-04-26T23:19:26.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.389+0000 7f464effd640 1 -- 172.21.15.47:0/2872599613 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f463800b9c0 con 0x7f4650071e60 2024-04-26T23:19:26.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.390+0000 7f464f7fe640 1 --2- 172.21.15.47:0/2872599613 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4638003f30 0x7f4638004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:26.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.390+0000 7f464ffff640 1 --2- 172.21.15.47:0/2872599613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4650071e60 0x7f46380039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:26.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.390+0000 7f4654c30640 1 --2- 172.21.15.47:0/2872599613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4638004d40 0x7f46380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:26.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.390+0000 7f464f7fe640 1 -- 172.21.15.47:0/2872599613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4650071e60 msgr2=0x7f46380039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:26.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.390+0000 7f464f7fe640 1 --2- 172.21.15.47:0/2872599613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4650071e60 0x7f46380039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:26.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.390+0000 7f464f7fe640 1 -- 172.21.15.47:0/2872599613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4638004d40 msgr2=0x7f46380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:26.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.390+0000 7f464f7fe640 1 --2- 172.21.15.47:0/2872599613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4638004d40 0x7f46380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:26.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.390+0000 7f464f7fe640 1 -- 172.21.15.47:0/2872599613 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4638003520 con 0x7f4638003f30 2024-04-26T23:19:26.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.390+0000 7f464f7fe640 1 --2- 172.21.15.47:0/2872599613 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4638003f30 0x7f4638004800 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4644009ac0 tx=0x7f4638005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49652838fad41d5c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:26.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.391+0000 7f464effd640 1 -- 172.21.15.47:0/2872599613 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4644036070 con 0x7f4638003f30 2024-04-26T23:19:26.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.391+0000 7f464effd640 1 -- 172.21.15.47:0/2872599613 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4644031400 con 0x7f4638003f30 2024-04-26T23:19:26.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.391+0000 7f464effd640 1 -- 172.21.15.47:0/2872599613 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4644009690 con 0x7f4638003f30 2024-04-26T23:19:26.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.391+0000 7f4656693640 1 -- 172.21.15.47:0/2872599613 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4638003f30 msgr2=0x7f4638004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:26.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.391+0000 7f4656693640 1 --2- 172.21.15.47:0/2872599613 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4638003f30 0x7f4638004800 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4644009ac0 tx=0x7f4638005830 comp rx=0 tx=0).stop 2024-04-26T23:19:26.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.391+0000 7f4654c30640 1 -- 172.21.15.47:0/2872599613 reap_dead start 2024-04-26T23:19:26.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.391+0000 7f4656693640 1 -- 172.21.15.47:0/2872599613 shutdown_connections 2024-04-26T23:19:26.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.391+0000 7f4656693640 1 -- 172.21.15.47:0/2872599613 >> 172.21.15.47:0/2872599613 conn(0x7f46500d0be0 msgr2=0x7f46500d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:26.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.391+0000 7f4656693640 1 -- 172.21.15.47:0/2872599613 shutdown_connections 2024-04-26T23:19:26.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.391+0000 7f4656693640 1 -- 172.21.15.47:0/2872599613 wait complete. 2024-04-26T23:19:26.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.392+0000 7f4656693640 1 Processor -- start 2024-04-26T23:19:26.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.393+0000 7f4656693640 1 -- start start 2024-04-26T23:19:26.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.393+0000 7f4656693640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4638003f30 0x7f46500b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:26.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.393+0000 7f4656693640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f465014fa00 0x7f465015c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:26.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.393+0000 7f4656693640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46501501c0 0x7f4650150640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:26.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.393+0000 7f4656693640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f465015d3f0 con 0x7f46501501c0 2024-04-26T23:19:26.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.393+0000 7f4656693640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4650072c10 con 0x7f465014fa00 2024-04-26T23:19:26.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.393+0000 7f4656693640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f465015d570 con 0x7f4638003f30 2024-04-26T23:19:26.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.394+0000 7f4654c30640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46501501c0 0x7f4650150640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:26.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.394+0000 7f4654c30640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46501501c0 0x7f4650150640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54822/0 (socket says 172.21.15.47:54822) 2024-04-26T23:19:26.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.394+0000 7f4654c30640 1 -- 172.21.15.47:0/3996510479 learned_addr learned my addr 172.21.15.47:0/3996510479 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:26.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.394+0000 7f464f7fe640 1 --2- 172.21.15.47:0/3996510479 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f465014fa00 0x7f465015c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:26.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.394+0000 7f464ffff640 1 --2- 172.21.15.47:0/3996510479 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4638003f30 0x7f46500b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:26.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.394+0000 7f4654c30640 1 -- 172.21.15.47:0/3996510479 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4638003f30 msgr2=0x7f46500b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:26.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.394+0000 7f4654c30640 1 --2- 172.21.15.47:0/3996510479 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4638003f30 0x7f46500b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:26.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.394+0000 7f4654c30640 1 -- 172.21.15.47:0/3996510479 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f465014fa00 msgr2=0x7f465015c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:26.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.394+0000 7f4654c30640 1 --2- 172.21.15.47:0/3996510479 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f465014fa00 0x7f465015c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:26.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.394+0000 7f4654c30640 1 -- 172.21.15.47:0/3996510479 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46500b5090 con 0x7f46501501c0 2024-04-26T23:19:26.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.394+0000 7f464f7fe640 1 --2- 172.21.15.47:0/3996510479 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f465014fa00 0x7f465015c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:19:26.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.394+0000 7f4654c30640 1 --2- 172.21.15.47:0/3996510479 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46501501c0 0x7f4650150640 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4650072b60 tx=0x7f4648007b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:26.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.394+0000 7f464cff9640 1 -- 172.21.15.47:0/3996510479 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4648017020 con 0x7f46501501c0 2024-04-26T23:19:26.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.394+0000 7f464cff9640 1 -- 172.21.15.47:0/3996510479 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f464800abc0 con 0x7f46501501c0 2024-04-26T23:19:26.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.394+0000 7f4656693640 1 -- 172.21.15.47:0/3996510479 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46500b5370 con 0x7f46501501c0 2024-04-26T23:19:26.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.394+0000 7f464cff9640 1 -- 172.21.15.47:0/3996510479 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46480133f0 con 0x7f46501501c0 2024-04-26T23:19:26.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.394+0000 7f4656693640 1 -- 172.21.15.47:0/3996510479 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46500b5930 con 0x7f46501501c0 2024-04-26T23:19:26.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.395+0000 7f4656693640 1 -- 172.21.15.47:0/3996510479 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46500694f0 con 0x7f46501501c0 2024-04-26T23:19:26.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.396+0000 7f464cff9640 1 -- 172.21.15.47:0/3996510479 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f464801a020 con 0x7f46501501c0 2024-04-26T23:19:26.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.396+0000 7f464cff9640 1 --2- 172.21.15.47:0/3996510479 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f463c06c920 0x7f463c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:26.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.396+0000 7f464cff9640 1 -- 172.21.15.47:0/3996510479 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f464808c750 con 0x7f46501501c0 2024-04-26T23:19:26.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.397+0000 7f464ffff640 1 --2- 172.21.15.47:0/3996510479 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f463c06c920 0x7f463c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:26.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.397+0000 7f464ffff640 1 --2- 172.21.15.47:0/3996510479 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f463c06c920 0x7f463c06ede0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4640009a20 tx=0x7f4640008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:26.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.401+0000 7f464cff9640 1 -- 172.21.15.47:0/3996510479 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4648021020 con 0x7f46501501c0 2024-04-26T23:19:26.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.585+0000 7f4656693640 1 -- 172.21.15.47:0/3996510479 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f46500cc700 con 0x7f46501501c0 2024-04-26T23:19:26.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.586+0000 7f464cff9640 1 -- 172.21.15.47:0/3996510479 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f464805a7f0 con 0x7f46501501c0 2024-04-26T23:19:26.587 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-26T23:19:26.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.587+0000 7f4656693640 1 -- 172.21.15.47:0/3996510479 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f463c06c920 msgr2=0x7f463c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:26.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.587+0000 7f4656693640 1 --2- 172.21.15.47:0/3996510479 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f463c06c920 0x7f463c06ede0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4640009a20 tx=0x7f4640008040 comp rx=0 tx=0).stop 2024-04-26T23:19:26.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.587+0000 7f4656693640 1 -- 172.21.15.47:0/3996510479 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46501501c0 msgr2=0x7f4650150640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:26.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.588+0000 7f4656693640 1 --2- 172.21.15.47:0/3996510479 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46501501c0 0x7f4650150640 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4650072b60 tx=0x7f4648007b20 comp rx=0 tx=0).stop 2024-04-26T23:19:26.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.588+0000 7f4656693640 1 -- 172.21.15.47:0/3996510479 shutdown_connections 2024-04-26T23:19:26.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.588+0000 7f4656693640 1 --2- 172.21.15.47:0/3996510479 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f463c06c920 0x7f463c06ede0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:26.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.588+0000 7f4656693640 1 --2- 172.21.15.47:0/3996510479 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46501501c0 0x7f4650150640 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:26.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.588+0000 7f4656693640 1 --2- 172.21.15.47:0/3996510479 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f465014fa00 0x7f465015c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:26.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.588+0000 7f4656693640 1 --2- 172.21.15.47:0/3996510479 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4638003f30 0x7f46500b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:26.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.588+0000 7f4656693640 1 -- 172.21.15.47:0/3996510479 >> 172.21.15.47:0/3996510479 conn(0x7f46500d0be0 msgr2=0x7f46500b06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:26.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.588+0000 7f4656693640 1 -- 172.21.15.47:0/3996510479 shutdown_connections 2024-04-26T23:19:26.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.588+0000 7f4656693640 1 -- 172.21.15.47:0/3996510479 wait complete. 2024-04-26T23:19:26.599 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T23:19:26.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.700+0000 7fdc0c4d9640 1 Processor -- start 2024-04-26T23:19:26.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.700+0000 7fdc0c4d9640 1 -- start start 2024-04-26T23:19:26.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.701+0000 7fdc0c4d9640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdc0415a9f0 0x7fdc040b4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:26.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.701+0000 7fdc0c4d9640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdc040726e0 0x7fdc04072b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:26.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.701+0000 7fdc0c4d9640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc040ce060 con 0x7fdc04161510 2024-04-26T23:19:26.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.701+0000 7fdc0c4d9640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fdc040ce620 con 0x7fdc0415a9f0 2024-04-26T23:19:26.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.701+0000 7fdc0c4d9640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fdc040ce340 con 0x7fdc040726e0 2024-04-26T23:19:26.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.701+0000 7fdc0a275640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdc0415a9f0 0x7fdc040b4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:26.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.701+0000 7fdc0a275640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdc0415a9f0 0x7fdc040b4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54824/0 (socket says 172.21.15.47:54824) 2024-04-26T23:19:26.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.701+0000 7fdc0a275640 1 -- 172.21.15.47:0/3844141293 learned_addr learned my addr 172.21.15.47:0/3844141293 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:26.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.701+0000 7fdc0aa76640 1 --2- 172.21.15.47:0/3844141293 >> v2:172.21.15.203:3300/0 conn(0x7fdc040726e0 0x7fdc04072b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:26.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.701+0000 7fdc0a275640 1 -- 172.21.15.47:0/3844141293 >> v2:172.21.15.203:3300/0 conn(0x7fdc040726e0 msgr2=0x7fdc04072b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:26.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.701+0000 7fdc0a275640 1 --2- 172.21.15.47:0/3844141293 >> v2:172.21.15.203:3300/0 conn(0x7fdc040726e0 0x7fdc04072b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:26.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.701+0000 7fdc0a275640 1 -- 172.21.15.47:0/3844141293 >> v1:172.21.15.203:6789/0 conn(0x7fdc04161510 legacy=0x7fdc0415dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:26.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.701+0000 7fdc0a275640 1 -- 172.21.15.47:0/3844141293 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc0415d6b0 con 0x7fdc0415a9f0 2024-04-26T23:19:26.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.702+0000 7fdc0a275640 1 --2- 172.21.15.47:0/3844141293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc0415a9f0 0x7fdc040b4480 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fdc040d1480 tx=0x7fdc0002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=915e597bcd12b280 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:26.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.702+0000 7fdc09273640 1 -- 172.21.15.47:0/3844141293 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc00037070 con 0x7fdc0415a9f0 2024-04-26T23:19:26.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.702+0000 7fdc09273640 1 -- 172.21.15.47:0/3844141293 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc00032440 con 0x7fdc0415a9f0 2024-04-26T23:19:26.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.702+0000 7fdc09273640 1 -- 172.21.15.47:0/3844141293 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc00032810 con 0x7fdc0415a9f0 2024-04-26T23:19:26.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.703+0000 7fdc0c4d9640 1 -- 172.21.15.47:0/3844141293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc0415a9f0 msgr2=0x7fdc040b4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:26.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.703+0000 7fdc0c4d9640 1 --2- 172.21.15.47:0/3844141293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc0415a9f0 0x7fdc040b4480 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fdc040d1480 tx=0x7fdc0002a270 comp rx=0 tx=0).stop 2024-04-26T23:19:26.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.703+0000 7fdc0c4d9640 1 -- 172.21.15.47:0/3844141293 shutdown_connections 2024-04-26T23:19:26.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.703+0000 7fdc0c4d9640 1 --2- 172.21.15.47:0/3844141293 >> v2:172.21.15.203:3300/0 conn(0x7fdc040726e0 0x7fdc04072b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:26.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.703+0000 7fdc0c4d9640 1 --2- 172.21.15.47:0/3844141293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc0415a9f0 0x7fdc040b4480 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:26.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.703+0000 7fdc0c4d9640 1 -- 172.21.15.47:0/3844141293 >> 172.21.15.47:0/3844141293 conn(0x7fdc040b3180 msgr2=0x7fdc040b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:26.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.703+0000 7fdc0c4d9640 1 -- 172.21.15.47:0/3844141293 shutdown_connections 2024-04-26T23:19:26.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.703+0000 7fdc0c4d9640 1 -- 172.21.15.47:0/3844141293 wait complete. 2024-04-26T23:19:26.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.704+0000 7fdc0c4d9640 1 Processor -- start 2024-04-26T23:19:26.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.704+0000 7fdc0c4d9640 1 -- start start 2024-04-26T23:19:26.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.705+0000 7fdc0c4d9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc040726e0 0x7fdc04151ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:26.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.705+0000 7fdc0c4d9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc0415a9f0 0x7fdc041533d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:26.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.705+0000 7fdc0c4d9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc04161510 0x7fdc04147fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:26.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.705+0000 7fdc0c4d9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0415e2e0 con 0x7fdc040726e0 2024-04-26T23:19:26.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.705+0000 7fdc0c4d9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0415add0 con 0x7fdc04161510 2024-04-26T23:19:26.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.705+0000 7fdc0c4d9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fdc0415e460 con 0x7fdc0415a9f0 2024-04-26T23:19:26.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.705+0000 7fdc0a275640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc040726e0 0x7fdc04151ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:26.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.705+0000 7fdc0a275640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc040726e0 0x7fdc04151ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54836/0 (socket says 172.21.15.47:54836) 2024-04-26T23:19:26.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.705+0000 7fdc0a275640 1 -- 172.21.15.47:0/1989464929 learned_addr learned my addr 172.21.15.47:0/1989464929 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:26.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.705+0000 7fdc0aa76640 1 --2- 172.21.15.47:0/1989464929 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc04161510 0x7fdc04147fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:26.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.705+0000 7fdc09a74640 1 --2- 172.21.15.47:0/1989464929 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc0415a9f0 0x7fdc041533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:26.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.705+0000 7fdc0a275640 1 -- 172.21.15.47:0/1989464929 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc0415a9f0 msgr2=0x7fdc041533d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:26.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.705+0000 7fdc0a275640 1 --2- 172.21.15.47:0/1989464929 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc0415a9f0 0x7fdc041533d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:26.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.705+0000 7fdc0a275640 1 -- 172.21.15.47:0/1989464929 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc04161510 msgr2=0x7fdc04147fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:26.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.705+0000 7fdc0a275640 1 --2- 172.21.15.47:0/1989464929 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc04161510 0x7fdc04147fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:26.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.705+0000 7fdc0a275640 1 -- 172.21.15.47:0/1989464929 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc04158000 con 0x7fdc040726e0 2024-04-26T23:19:26.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.706+0000 7fdc0a275640 1 --2- 172.21.15.47:0/1989464929 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc040726e0 0x7fdc04151ec0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fdc0002a2b0 tx=0x7fdc000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:26.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.706+0000 7fdbfaffd640 1 -- 172.21.15.47:0/1989464929 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc00041020 con 0x7fdc040726e0 2024-04-26T23:19:26.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.706+0000 7fdbfaffd640 1 -- 172.21.15.47:0/1989464929 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc000306c0 con 0x7fdc040726e0 2024-04-26T23:19:26.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.706+0000 7fdc0c4d9640 1 -- 172.21.15.47:0/1989464929 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc04157010 con 0x7fdc040726e0 2024-04-26T23:19:26.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.706+0000 7fdbfaffd640 1 -- 172.21.15.47:0/1989464929 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc00037040 con 0x7fdc040726e0 2024-04-26T23:19:26.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.706+0000 7fdc0c4d9640 1 -- 172.21.15.47:0/1989464929 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc04156d30 con 0x7fdc040726e0 2024-04-26T23:19:26.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.707+0000 7fdc0c4d9640 1 -- 172.21.15.47:0/1989464929 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc04068fc0 con 0x7fdc040726e0 2024-04-26T23:19:26.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.707+0000 7fdbfaffd640 1 -- 172.21.15.47:0/1989464929 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdc00044020 con 0x7fdc040726e0 2024-04-26T23:19:26.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.708+0000 7fdbfaffd640 1 --2- 172.21.15.47:0/1989464929 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdbe806c8f0 0x7fdbe806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:26.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.708+0000 7fdbfaffd640 1 -- 172.21.15.47:0/1989464929 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7fdc000ac5d0 con 0x7fdc040726e0 2024-04-26T23:19:26.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.708+0000 7fdc09a74640 1 --2- 172.21.15.47:0/1989464929 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdbe806c8f0 0x7fdbe806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:26.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.709+0000 7fdc09a74640 1 --2- 172.21.15.47:0/1989464929 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdbe806c8f0 0x7fdbe806edb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdc040cd8b0 tx=0x7fdbf4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:26.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.712+0000 7fdbfaffd640 1 -- 172.21.15.47:0/1989464929 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdc0004b030 con 0x7fdc040726e0 2024-04-26T23:19:26.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.909+0000 7fdc0c4d9640 1 -- 172.21.15.47:0/1989464929 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdc040694f0 con 0x7fdc040726e0 2024-04-26T23:19:26.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.910+0000 7fdbfaffd640 1 -- 172.21.15.47:0/1989464929 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7fdc0007a020 con 0x7fdc040726e0 2024-04-26T23:19:26.911 INFO:teuthology.orchestra.run.smithi047.stderr:listed 0 entries 2024-04-26T23:19:26.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.911+0000 7fdc0c4d9640 1 -- 172.21.15.47:0/1989464929 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdbe806c8f0 msgr2=0x7fdbe806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:26.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.911+0000 7fdc0c4d9640 1 --2- 172.21.15.47:0/1989464929 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdbe806c8f0 0x7fdbe806edb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdc040cd8b0 tx=0x7fdbf4009040 comp rx=0 tx=0).stop 2024-04-26T23:19:26.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.912+0000 7fdc0c4d9640 1 -- 172.21.15.47:0/1989464929 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc040726e0 msgr2=0x7fdc04151ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:26.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.912+0000 7fdc0c4d9640 1 --2- 172.21.15.47:0/1989464929 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc040726e0 0x7fdc04151ec0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fdc0002a2b0 tx=0x7fdc000046c0 comp rx=0 tx=0).stop 2024-04-26T23:19:26.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.912+0000 7fdc0c4d9640 1 -- 172.21.15.47:0/1989464929 shutdown_connections 2024-04-26T23:19:26.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.912+0000 7fdc0c4d9640 1 --2- 172.21.15.47:0/1989464929 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdbe806c8f0 0x7fdbe806edb0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:26.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.912+0000 7fdc0c4d9640 1 --2- 172.21.15.47:0/1989464929 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc04161510 0x7fdc04147fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:26.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.912+0000 7fdc0c4d9640 1 --2- 172.21.15.47:0/1989464929 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc0415a9f0 0x7fdc041533d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:26.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.912+0000 7fdc0c4d9640 1 --2- 172.21.15.47:0/1989464929 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc040726e0 0x7fdc04151ec0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:26.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.912+0000 7fdc0c4d9640 1 -- 172.21.15.47:0/1989464929 >> 172.21.15.47:0/1989464929 conn(0x7fdc040b3180 msgr2=0x7fdc040d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:26.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.912+0000 7fdc0c4d9640 1 -- 172.21.15.47:0/1989464929 shutdown_connections 2024-04-26T23:19:26.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:26.912+0000 7fdc0c4d9640 1 -- 172.21.15.47:0/1989464929 wait complete. 2024-04-26T23:19:26.923 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-04-26T23:19:26.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:26.923 DEBUG:teuthology.orchestra.run.smithi047:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-04-26T23:19:26.950 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi047.front.sepia.ceph.com 2024-04-26T23:19:26.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:26.950 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-26T23:19:27.006 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-26T23:19:27.006 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:19:27.025 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-26T23:19:27.025 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:19:27.083 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-26T23:19:27.083 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:19:27.140 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:19:27.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.242+0000 7f082b26d640 1 Processor -- start 2024-04-26T23:19:27.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.242+0000 7f082b26d640 1 -- start start 2024-04-26T23:19:27.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.243+0000 7f082b26d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0824156460 0x7f08240cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:27.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.243+0000 7f082b26d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08240cba10 0x7f082406f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:27.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.243+0000 7f082b26d640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0824156840 con 0x7f082406fcd0 2024-04-26T23:19:27.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.243+0000 7f082b26d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f08240cbf20 con 0x7f0824156460 2024-04-26T23:19:27.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.243+0000 7f082b26d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f08240cc200 con 0x7f08240cba10 2024-04-26T23:19:27.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.243+0000 7f0828808640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08240cba10 0x7f082406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:27.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.243+0000 7f0829009640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0824156460 0x7f08240cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:27.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.243+0000 7f082980a640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f082406fcd0 0x7f082415f980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.47:56104/0 (socket says 172.21.15.47:56104) 2024-04-26T23:19:27.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.243+0000 7f082980a640 1 -- 172.21.15.47:0/2468675086 learned_addr learned my addr 172.21.15.47:0/2468675086 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:27.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.244+0000 7f0829009640 1 -- 172.21.15.47:0/2468675086 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08240cba10 msgr2=0x7f082406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:27.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.244+0000 7f0829009640 1 --2- 172.21.15.47:0/2468675086 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08240cba10 0x7f082406f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:27.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.244+0000 7f0829009640 1 -- 172.21.15.47:0/2468675086 >> v1:172.21.15.203:6789/0 conn(0x7f082406fcd0 legacy=0x7f082415f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:27.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.244+0000 7f081bfff640 1 -- 172.21.15.47:0/2468675086 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1519652165 0 0) 0x7f0824156840 con 0x7f082406fcd0 2024-04-26T23:19:27.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.244+0000 7f0829009640 1 -- 172.21.15.47:0/2468675086 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08240cc4e0 con 0x7f0824156460 2024-04-26T23:19:27.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.244+0000 7f0829009640 1 --2- 172.21.15.47:0/2468675086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0824156460 0x7f08240cb4d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f08200097a0 tx=0x7f082002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5013c2d342a18a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:27.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.245+0000 7f081bfff640 1 -- 172.21.15.47:0/2468675086 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0820037070 con 0x7f0824156460 2024-04-26T23:19:27.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.245+0000 7f081bfff640 1 -- 172.21.15.47:0/2468675086 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0820032440 con 0x7f0824156460 2024-04-26T23:19:27.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.245+0000 7f081bfff640 1 -- 172.21.15.47:0/2468675086 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0820032810 con 0x7f0824156460 2024-04-26T23:19:27.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.245+0000 7f082b26d640 1 -- 172.21.15.47:0/2468675086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0824156460 msgr2=0x7f08240cb4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:27.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.245+0000 7f082b26d640 1 --2- 172.21.15.47:0/2468675086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0824156460 0x7f08240cb4d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f08200097a0 tx=0x7f082002a180 comp rx=0 tx=0).stop 2024-04-26T23:19:27.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.245+0000 7f082b26d640 1 -- 172.21.15.47:0/2468675086 shutdown_connections 2024-04-26T23:19:27.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.245+0000 7f082b26d640 1 --2- 172.21.15.47:0/2468675086 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08240cba10 0x7f082406f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:27.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.245+0000 7f082b26d640 1 --2- 172.21.15.47:0/2468675086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0824156460 0x7f08240cb4d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:27.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.245+0000 7f082b26d640 1 -- 172.21.15.47:0/2468675086 >> 172.21.15.47:0/2468675086 conn(0x7f08240d0be0 msgr2=0x7f08240d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:27.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.245+0000 7f082b26d640 1 -- 172.21.15.47:0/2468675086 shutdown_connections 2024-04-26T23:19:27.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.245+0000 7f082b26d640 1 -- 172.21.15.47:0/2468675086 wait complete. 2024-04-26T23:19:27.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.246+0000 7f082b26d640 1 Processor -- start 2024-04-26T23:19:27.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.246+0000 7f082b26d640 1 -- start start 2024-04-26T23:19:27.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.247+0000 7f082b26d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f082406fcd0 0x7f08241e6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:27.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.247+0000 7f082b26d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08240cba10 0x7f0824152190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:27.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.247+0000 7f082b26d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0824156460 0x7f08240c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:27.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.247+0000 7f082b26d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f082406e260 con 0x7f08240cba10 2024-04-26T23:19:27.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.247+0000 7f082b26d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0824072700 con 0x7f0824156460 2024-04-26T23:19:27.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.247+0000 7f082b26d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f082406e3e0 con 0x7f082406fcd0 2024-04-26T23:19:27.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.247+0000 7f0828808640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08240cba10 0x7f0824152190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:27.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.247+0000 7f0828808640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08240cba10 0x7f0824152190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54850/0 (socket says 172.21.15.47:54850) 2024-04-26T23:19:27.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.247+0000 7f0828808640 1 -- 172.21.15.47:0/3092854955 learned_addr learned my addr 172.21.15.47:0/3092854955 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:27.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.248+0000 7f082980a640 1 --2- 172.21.15.47:0/3092854955 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0824156460 0x7f08240c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:27.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.248+0000 7f0829009640 1 --2- 172.21.15.47:0/3092854955 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f082406fcd0 0x7f08241e6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:27.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.248+0000 7f0828808640 1 -- 172.21.15.47:0/3092854955 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f082406fcd0 msgr2=0x7f08241e6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:27.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.248+0000 7f0828808640 1 --2- 172.21.15.47:0/3092854955 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f082406fcd0 0x7f08241e6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:27.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.248+0000 7f0828808640 1 -- 172.21.15.47:0/3092854955 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0824156460 msgr2=0x7f08240c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:27.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.248+0000 7f0828808640 1 --2- 172.21.15.47:0/3092854955 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0824156460 0x7f08240c7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:27.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.248+0000 7f0828808640 1 -- 172.21.15.47:0/3092854955 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08240c81f0 con 0x7f08240cba10 2024-04-26T23:19:27.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.248+0000 7f0828808640 1 --2- 172.21.15.47:0/3092854955 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08240cba10 0x7f0824152190 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f081400e980 tx=0x7f081400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:27.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.248+0000 7f0819ffb640 1 -- 172.21.15.47:0/3092854955 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0814017070 con 0x7f08240cba10 2024-04-26T23:19:27.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.248+0000 7f0819ffb640 1 -- 172.21.15.47:0/3092854955 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f081400ca00 con 0x7f08240cba10 2024-04-26T23:19:27.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.248+0000 7f0819ffb640 1 -- 172.21.15.47:0/3092854955 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081400cdd0 con 0x7f08240cba10 2024-04-26T23:19:27.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.248+0000 7f082b26d640 1 -- 172.21.15.47:0/3092854955 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08241e6b20 con 0x7f08240cba10 2024-04-26T23:19:27.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.248+0000 7f082b26d640 1 -- 172.21.15.47:0/3092854955 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08241e70e0 con 0x7f08240cba10 2024-04-26T23:19:27.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.249+0000 7f082b26d640 1 -- 172.21.15.47:0/3092854955 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08240694f0 con 0x7f08240cba10 2024-04-26T23:19:27.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.250+0000 7f0819ffb640 1 -- 172.21.15.47:0/3092854955 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f081401a070 con 0x7f08240cba10 2024-04-26T23:19:27.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.250+0000 7f0819ffb640 1 --2- 172.21.15.47:0/3092854955 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f080806c8f0 0x7f080806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:27.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.250+0000 7f0819ffb640 1 -- 172.21.15.47:0/3092854955 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f081408bf70 con 0x7f08240cba10 2024-04-26T23:19:27.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.250+0000 7f0829009640 1 --2- 172.21.15.47:0/3092854955 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f080806c8f0 0x7f080806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:27.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.251+0000 7f0829009640 1 --2- 172.21.15.47:0/3092854955 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f080806c8f0 0x7f080806edb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0820034000 tx=0x7f082002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:27.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.254+0000 7f0819ffb640 1 -- 172.21.15.47:0/3092854955 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0814059f90 con 0x7f08240cba10 2024-04-26T23:19:27.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.440+0000 7f082b26d640 1 -- 172.21.15.47:0/3092854955 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f082406de80 con 0x7f08240cba10 2024-04-26T23:19:27.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.441+0000 7f0819ffb640 1 -- 172.21.15.47:0/3092854955 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18857 (secure 0 0 0) 0x7f0814059890 con 0x7f08240cba10 2024-04-26T23:19:27.442 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:19:27.442 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":26,"fsid":"399006d3-4db1-43e6-ba81-a79a276eabfd","created":"2024-04-26T23:18:06.731211+0000","modified":"2024-04-26T23:19:12.124051+0000","last_up_change":"2024-04-26T23:18:17.808736+0000","last_in_change":"2024-04-26T23:18:08.945859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:18:16.937492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:18:29.033575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:18:30.318499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:18:34.014536+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ee091567-fea0-4a14-bf4e-0e16c20bb2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6819","nonce":326520779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6821","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6823","nonce":326520779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6829","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6831","nonce":326520779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6825","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6827","nonce":326520779}]},"public_addr":"172.21.15.47:6819/326520779","cluster_addr":"172.21.15.47:6823/326520779","heartbeat_back_addr":"172.21.15.47:6831/326520779","heartbeat_front_addr":"172.21.15.47:6827/326520779","state":["exists","up"]},{"osd":1,"uuid":"c14440a0-bba5-4931-a6db-b736c08e869b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6809","nonce":1912610931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6811","nonce":1912610931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6815","nonce":1912610931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6813","nonce":1912610931}]},"public_addr":"172.21.15.47:6809/1912610931","cluster_addr":"172.21.15.47:6811/1912610931","heartbeat_back_addr":"172.21.15.47:6815/1912610931","heartbeat_front_addr":"172.21.15.47:6813/1912610931","state":["exists","up"]},{"osd":2,"uuid":"3940f8de-c9b2-4cd3-a911-9c301729eb30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6801","nonce":2569869305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6803","nonce":2569869305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6807","nonce":2569869305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6805","nonce":2569869305}]},"public_addr":"172.21.15.47:6801/2569869305","cluster_addr":"172.21.15.47:6803/2569869305","heartbeat_back_addr":"172.21.15.47:6807/2569869305","heartbeat_front_addr":"172.21.15.47:6805/2569869305","state":["exists","up"]},{"osd":3,"uuid":"1286a67e-8958-426a-8c2e-bc85498c619c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6818","nonce":2791411683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6822","nonce":2791411683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6830","nonce":2791411683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6826","nonce":2791411683}]},"public_addr":"172.21.15.47:6818/2791411683","cluster_addr":"172.21.15.47:6822/2791411683","heartbeat_back_addr":"172.21.15.47:6830/2791411683","heartbeat_front_addr":"172.21.15.47:6826/2791411683","state":["exists","up"]},{"osd":4,"uuid":"d8ee2c20-e5b7-46c2-9999-b234070126cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6809","nonce":441313104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6811","nonce":441313104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6815","nonce":441313104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6813","nonce":441313104}]},"public_addr":"172.21.15.203:6809/441313104","cluster_addr":"172.21.15.203:6811/441313104","heartbeat_back_addr":"172.21.15.203:6815/441313104","heartbeat_front_addr":"172.21.15.203:6813/441313104","state":["exists","up"]},{"osd":5,"uuid":"b453a8f0-f324-4840-9ae0-d67bdadb9f8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6801","nonce":56838444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6803","nonce":56838444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6807","nonce":56838444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6805","nonce":56838444}]},"public_addr":"172.21.15.203:6801/56838444","cluster_addr":"172.21.15.203:6803/56838444","heartbeat_back_addr":"172.21.15.203:6807/56838444","heartbeat_front_addr":"172.21.15.203:6805/56838444","state":["exists","up"]},{"osd":6,"uuid":"a6d31951-2d4b-4e8d-a510-6283b01dc158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6825","nonce":2738517080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6827","nonce":2738517080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6831","nonce":2738517080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6829","nonce":2738517080}]},"public_addr":"172.21.15.203:6825/2738517080","cluster_addr":"172.21.15.203:6827/2738517080","heartbeat_back_addr":"172.21.15.203:6831/2738517080","heartbeat_front_addr":"172.21.15.203:6829/2738517080","state":["exists","up"]},{"osd":7,"uuid":"7119b8c0-fe96-402d-9e75-7b509d9d44d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6817","nonce":1315565058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6819","nonce":1315565058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6823","nonce":1315565058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6821","nonce":1315565058}]},"public_addr":"172.21.15.203:6817/1315565058","cluster_addr":"172.21.15.203:6819/1315565058","heartbeat_back_addr":"172.21.15.203:6823/1315565058","heartbeat_front_addr":"172.21.15.203:6821/1315565058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.121921+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.137494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.084814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.111593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.072430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.125066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.097867+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.057061+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":5,"to":1}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:19:27.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.443+0000 7f082b26d640 1 -- 172.21.15.47:0/3092854955 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f080806c8f0 msgr2=0x7f080806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:27.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.443+0000 7f082b26d640 1 --2- 172.21.15.47:0/3092854955 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f080806c8f0 0x7f080806edb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0820034000 tx=0x7f082002e000 comp rx=0 tx=0).stop 2024-04-26T23:19:27.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.443+0000 7f082b26d640 1 -- 172.21.15.47:0/3092854955 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08240cba10 msgr2=0x7f0824152190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:27.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.443+0000 7f082b26d640 1 --2- 172.21.15.47:0/3092854955 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08240cba10 0x7f0824152190 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f081400e980 tx=0x7f081400ee40 comp rx=0 tx=0).stop 2024-04-26T23:19:27.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.443+0000 7f082b26d640 1 -- 172.21.15.47:0/3092854955 shutdown_connections 2024-04-26T23:19:27.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.443+0000 7f082b26d640 1 --2- 172.21.15.47:0/3092854955 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f080806c8f0 0x7f080806edb0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:27.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.443+0000 7f082b26d640 1 --2- 172.21.15.47:0/3092854955 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0824156460 0x7f08240c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:27.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.443+0000 7f082b26d640 1 --2- 172.21.15.47:0/3092854955 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08240cba10 0x7f0824152190 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:27.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.443+0000 7f082b26d640 1 --2- 172.21.15.47:0/3092854955 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f082406fcd0 0x7f08241e6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:27.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.443+0000 7f082b26d640 1 -- 172.21.15.47:0/3092854955 >> 172.21.15.47:0/3092854955 conn(0x7f08240d0be0 msgr2=0x7f08241550c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:27.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.443+0000 7f082b26d640 1 -- 172.21.15.47:0/3092854955 shutdown_connections 2024-04-26T23:19:27.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.443+0000 7f082b26d640 1 -- 172.21.15.47:0/3092854955 wait complete. 2024-04-26T23:19:27.455 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:18:16.937492+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:18:29.033575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:18:30.318499+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T23:18:34.014536+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T23:19:27.456 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:19:27.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.561+0000 7eff3aafc640 1 Processor -- start 2024-04-26T23:19:27.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.561+0000 7eff3aafc640 1 -- start start 2024-04-26T23:19:27.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.562+0000 7eff3aafc640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7eff340cf5e0 0x7eff340ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:27.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.562+0000 7eff3aafc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff340cbbd0 0x7eff340cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:27.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.562+0000 7eff3aafc640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff3415b9e0 con 0x7eff3415b600 2024-04-26T23:19:27.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.562+0000 7eff3aafc640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7eff340cb9b0 con 0x7eff340cf5e0 2024-04-26T23:19:27.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.562+0000 7eff3aafc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7eff340d0740 con 0x7eff340cbbd0 2024-04-26T23:19:27.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.562+0000 7eff33fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7eff340cf5e0 0x7eff340ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:27.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.562+0000 7eff33fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7eff340cf5e0 0x7eff340ce9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54858/0 (socket says 172.21.15.47:54858) 2024-04-26T23:19:27.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.562+0000 7eff33fff640 1 -- 172.21.15.47:0/4169789118 learned_addr learned my addr 172.21.15.47:0/4169789118 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:27.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.562+0000 7eff39099640 1 --2- 172.21.15.47:0/4169789118 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff340cbbd0 0x7eff340cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:27.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.562+0000 7eff33fff640 1 -- 172.21.15.47:0/4169789118 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff340cbbd0 msgr2=0x7eff340cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:27.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.562+0000 7eff33fff640 1 --2- 172.21.15.47:0/4169789118 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff340cbbd0 0x7eff340cef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:27.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.562+0000 7eff33fff640 1 -- 172.21.15.47:0/4169789118 >> v1:172.21.15.203:6789/0 conn(0x7eff3415b600 legacy=0x7eff34071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:27.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.562+0000 7eff33fff640 1 -- 172.21.15.47:0/4169789118 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff340cb360 con 0x7eff340cf5e0 2024-04-26T23:19:27.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.563+0000 7eff33fff640 1 --2- 172.21.15.47:0/4169789118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff340cf5e0 0x7eff340ce9d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7eff2000bb10 tx=0x7eff2002c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bffb374d0c119f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:27.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.564+0000 7eff337fe640 1 -- 172.21.15.47:0/4169789118 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff20032790 con 0x7eff340cf5e0 2024-04-26T23:19:27.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.564+0000 7eff337fe640 1 -- 172.21.15.47:0/4169789118 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff20038070 con 0x7eff340cf5e0 2024-04-26T23:19:27.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.564+0000 7eff337fe640 1 -- 172.21.15.47:0/4169789118 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff20032b50 con 0x7eff340cf5e0 2024-04-26T23:19:27.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.564+0000 7eff3aafc640 1 -- 172.21.15.47:0/4169789118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff340cf5e0 msgr2=0x7eff340ce9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:27.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.564+0000 7eff3aafc640 1 --2- 172.21.15.47:0/4169789118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff340cf5e0 0x7eff340ce9d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7eff2000bb10 tx=0x7eff2002c8d0 comp rx=0 tx=0).stop 2024-04-26T23:19:27.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.564+0000 7eff3aafc640 1 -- 172.21.15.47:0/4169789118 shutdown_connections 2024-04-26T23:19:27.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.564+0000 7eff3aafc640 1 --2- 172.21.15.47:0/4169789118 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff340cbbd0 0x7eff340cef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:27.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.564+0000 7eff3aafc640 1 --2- 172.21.15.47:0/4169789118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff340cf5e0 0x7eff340ce9d0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:27.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.564+0000 7eff3aafc640 1 -- 172.21.15.47:0/4169789118 >> 172.21.15.47:0/4169789118 conn(0x7eff340d0be0 msgr2=0x7eff340d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:27.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.564+0000 7eff3aafc640 1 -- 172.21.15.47:0/4169789118 shutdown_connections 2024-04-26T23:19:27.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.564+0000 7eff3aafc640 1 -- 172.21.15.47:0/4169789118 wait complete. 2024-04-26T23:19:27.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.565+0000 7eff3aafc640 1 Processor -- start 2024-04-26T23:19:27.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.566+0000 7eff3aafc640 1 -- start start 2024-04-26T23:19:27.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.566+0000 7eff3aafc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff340cbbd0 0x7eff340c5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:27.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.566+0000 7eff3aafc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff340cf5e0 0x7eff34156260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:27.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.566+0000 7eff3aafc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff3415b600 0x7eff340c6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:27.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.566+0000 7eff3aafc640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7eff340d0470 con 0x7eff340cbbd0 2024-04-26T23:19:27.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.566+0000 7eff3aafc640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7eff340d02f0 con 0x7eff3415b600 2024-04-26T23:19:27.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.566+0000 7eff3aafc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7eff3406ddf0 con 0x7eff340cf5e0 2024-04-26T23:19:27.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.567+0000 7eff38898640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff340cbbd0 0x7eff340c5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:27.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.567+0000 7eff38898640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff340cbbd0 0x7eff340c5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54874/0 (socket says 172.21.15.47:54874) 2024-04-26T23:19:27.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.567+0000 7eff38898640 1 -- 172.21.15.47:0/4287831505 learned_addr learned my addr 172.21.15.47:0/4287831505 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:27.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.567+0000 7eff39099640 1 --2- 172.21.15.47:0/4287831505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff3415b600 0x7eff340c6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:27.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.567+0000 7eff38898640 1 -- 172.21.15.47:0/4287831505 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff340cf5e0 msgr2=0x7eff34156260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:27.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.567+0000 7eff33fff640 1 --2- 172.21.15.47:0/4287831505 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff340cf5e0 0x7eff34156260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:27.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.567+0000 7eff38898640 1 --2- 172.21.15.47:0/4287831505 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff340cf5e0 0x7eff34156260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:27.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.567+0000 7eff38898640 1 -- 172.21.15.47:0/4287831505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff3415b600 msgr2=0x7eff340c6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:27.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.567+0000 7eff38898640 1 --2- 172.21.15.47:0/4287831505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff3415b600 0x7eff340c6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:27.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.567+0000 7eff38898640 1 -- 172.21.15.47:0/4287831505 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff3415bc80 con 0x7eff340cbbd0 2024-04-26T23:19:27.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.567+0000 7eff39099640 1 --2- 172.21.15.47:0/4287831505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff3415b600 0x7eff340c6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:19:27.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.567+0000 7eff38898640 1 --2- 172.21.15.47:0/4287831505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff340cbbd0 0x7eff340c5600 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7eff2800b3f0 tx=0x7eff2800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:27.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.567+0000 7eff317fa640 1 -- 172.21.15.47:0/4287831505 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff28015020 con 0x7eff340cbbd0 2024-04-26T23:19:27.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.567+0000 7eff317fa640 1 -- 172.21.15.47:0/4287831505 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff28011830 con 0x7eff340cbbd0 2024-04-26T23:19:27.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.567+0000 7eff3aafc640 1 -- 172.21.15.47:0/4287831505 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff340c5b40 con 0x7eff340cbbd0 2024-04-26T23:19:27.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.567+0000 7eff317fa640 1 -- 172.21.15.47:0/4287831505 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff28011c20 con 0x7eff340cbbd0 2024-04-26T23:19:27.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.567+0000 7eff3aafc640 1 -- 172.21.15.47:0/4287831505 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff3415bef0 con 0x7eff340cbbd0 2024-04-26T23:19:27.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.569+0000 7eff3aafc640 1 -- 172.21.15.47:0/4287831505 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff340694f0 con 0x7eff340cbbd0 2024-04-26T23:19:27.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.569+0000 7eff317fa640 1 -- 172.21.15.47:0/4287831505 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7eff28018020 con 0x7eff340cbbd0 2024-04-26T23:19:27.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.569+0000 7eff317fa640 1 --2- 172.21.15.47:0/4287831505 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7eff1806c8f0 0x7eff1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:27.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.569+0000 7eff317fa640 1 -- 172.21.15.47:0/4287831505 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7eff2808bec0 con 0x7eff340cbbd0 2024-04-26T23:19:27.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.569+0000 7eff33fff640 1 --2- 172.21.15.47:0/4287831505 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7eff1806c8f0 0x7eff1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:27.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.570+0000 7eff33fff640 1 --2- 172.21.15.47:0/4287831505 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7eff1806c8f0 0x7eff1806edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7eff20037000 tx=0x7eff20008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:27.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.574+0000 7eff317fa640 1 -- 172.21.15.47:0/4287831505 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7eff2801f020 con 0x7eff340cbbd0 2024-04-26T23:19:27.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.758+0000 7eff3aafc640 1 -- 172.21.15.47:0/4287831505 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7eff340ce7b0 con 0x7eff340cbbd0 2024-04-26T23:19:27.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.758+0000 7eff317fa640 1 -- 172.21.15.47:0/4287831505 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7eff280172c0 con 0x7eff340cbbd0 2024-04-26T23:19:27.759 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-26T23:19:27.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.761+0000 7eff3aafc640 1 -- 172.21.15.47:0/4287831505 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7eff1806c8f0 msgr2=0x7eff1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:27.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.761+0000 7eff3aafc640 1 --2- 172.21.15.47:0/4287831505 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7eff1806c8f0 0x7eff1806edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7eff20037000 tx=0x7eff20008040 comp rx=0 tx=0).stop 2024-04-26T23:19:27.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.761+0000 7eff3aafc640 1 -- 172.21.15.47:0/4287831505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff340cbbd0 msgr2=0x7eff340c5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:27.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.761+0000 7eff3aafc640 1 --2- 172.21.15.47:0/4287831505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff340cbbd0 0x7eff340c5600 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7eff2800b3f0 tx=0x7eff2800b8b0 comp rx=0 tx=0).stop 2024-04-26T23:19:27.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.761+0000 7eff3aafc640 1 -- 172.21.15.47:0/4287831505 shutdown_connections 2024-04-26T23:19:27.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.761+0000 7eff3aafc640 1 --2- 172.21.15.47:0/4287831505 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7eff1806c8f0 0x7eff1806edb0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:27.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.761+0000 7eff3aafc640 1 --2- 172.21.15.47:0/4287831505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff3415b600 0x7eff340c6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:27.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.761+0000 7eff3aafc640 1 --2- 172.21.15.47:0/4287831505 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff340cf5e0 0x7eff34156260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:27.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.761+0000 7eff3aafc640 1 --2- 172.21.15.47:0/4287831505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff340cbbd0 0x7eff340c5600 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:27.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.761+0000 7eff3aafc640 1 -- 172.21.15.47:0/4287831505 >> 172.21.15.47:0/4287831505 conn(0x7eff340d0be0 msgr2=0x7eff340b06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:27.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.761+0000 7eff3aafc640 1 -- 172.21.15.47:0/4287831505 shutdown_connections 2024-04-26T23:19:27.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.761+0000 7eff3aafc640 1 -- 172.21.15.47:0/4287831505 wait complete. 2024-04-26T23:19:27.777 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:19:27.786 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:19:27.822 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:19:27.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.884+0000 7f53d2e25640 1 Processor -- start 2024-04-26T23:19:27.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.884+0000 7f53d2e25640 1 -- start start 2024-04-26T23:19:27.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.885+0000 7f53d2e25640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f53cc160630 0x7f53cc0d0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:27.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.885+0000 7f53d2e25640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f53cc160a10 0x7f53cc0cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:27.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.885+0000 7f53d2e25640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53cc0d10e0 con 0x7f53cc0cf930 2024-04-26T23:19:27.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.885+0000 7f53d2e25640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f53cc0d0570 con 0x7f53cc160630 2024-04-26T23:19:27.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.885+0000 7f53d2e25640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f53cc0b41d0 con 0x7f53cc160a10 2024-04-26T23:19:27.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.885+0000 7f53d0bc1640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f53cc160630 0x7f53cc0d0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:27.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.885+0000 7f53d0bc1640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f53cc160630 0x7f53cc0d0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54882/0 (socket says 172.21.15.47:54882) 2024-04-26T23:19:27.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.885+0000 7f53d13c2640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f53cc0cf930 0x7f53cc0cd8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:55656/0 (socket says 172.21.15.47:55656) 2024-04-26T23:19:27.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.885+0000 7f53d0bc1640 1 -- 172.21.15.47:0/3242688304 learned_addr learned my addr 172.21.15.47:0/3242688304 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:27.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.886+0000 7f53cbfff640 1 --2- 172.21.15.47:0/3242688304 >> v2:172.21.15.203:3300/0 conn(0x7f53cc160a10 0x7f53cc0cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:27.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.886+0000 7f53cb7fe640 1 -- 172.21.15.47:0/3242688304 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 960283626 0 0) 0x7f53cc0d10e0 con 0x7f53cc0cf930 2024-04-26T23:19:27.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.886+0000 7f53cb7fe640 1 -- 172.21.15.47:0/3242688304 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f53b8003520 con 0x7f53cc0cf930 2024-04-26T23:19:27.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.886+0000 7f53d0bc1640 1 -- 172.21.15.47:0/3242688304 >> v2:172.21.15.203:3300/0 conn(0x7f53cc160a10 msgr2=0x7f53cc0cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:27.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.886+0000 7f53d0bc1640 1 --2- 172.21.15.47:0/3242688304 >> v2:172.21.15.203:3300/0 conn(0x7f53cc160a10 0x7f53cc0cf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:27.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.886+0000 7f53d0bc1640 1 -- 172.21.15.47:0/3242688304 >> v1:172.21.15.47:6789/0 conn(0x7f53cc0cf930 legacy=0x7f53cc0cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:27.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.886+0000 7f53d0bc1640 1 -- 172.21.15.47:0/3242688304 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53cc0b44b0 con 0x7f53cc160630 2024-04-26T23:19:27.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.886+0000 7f53cbfff640 1 --2- 172.21.15.47:0/3242688304 >> v2:172.21.15.203:3300/0 conn(0x7f53cc160a10 0x7f53cc0cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:19:27.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.886+0000 7f53d0bc1640 1 --2- 172.21.15.47:0/3242688304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53cc160630 0x7f53cc0d0ba0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f53cc073cc0 tx=0x7f53c402e600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4063829188bec4a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:27.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.887+0000 7f53cb7fe640 1 -- 172.21.15.47:0/3242688304 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c403a070 con 0x7f53cc160630 2024-04-26T23:19:27.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.887+0000 7f53cb7fe640 1 -- 172.21.15.47:0/3242688304 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53c4033710 con 0x7f53cc160630 2024-04-26T23:19:27.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.887+0000 7f53cb7fe640 1 -- 172.21.15.47:0/3242688304 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c4033ae0 con 0x7f53cc160630 2024-04-26T23:19:27.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.887+0000 7f53d2e25640 1 -- 172.21.15.47:0/3242688304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53cc160630 msgr2=0x7f53cc0d0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:27.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.887+0000 7f53d2e25640 1 --2- 172.21.15.47:0/3242688304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53cc160630 0x7f53cc0d0ba0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f53cc073cc0 tx=0x7f53c402e600 comp rx=0 tx=0).stop 2024-04-26T23:19:27.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.887+0000 7f53d2e25640 1 -- 172.21.15.47:0/3242688304 shutdown_connections 2024-04-26T23:19:27.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.887+0000 7f53d2e25640 1 --2- 172.21.15.47:0/3242688304 >> v2:172.21.15.203:3300/0 conn(0x7f53cc160a10 0x7f53cc0cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:27.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.887+0000 7f53d2e25640 1 --2- 172.21.15.47:0/3242688304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53cc160630 0x7f53cc0d0ba0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:27.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.887+0000 7f53d2e25640 1 -- 172.21.15.47:0/3242688304 >> 172.21.15.47:0/3242688304 conn(0x7f53cc0d2dc0 msgr2=0x7f53cc0d31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:27.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.887+0000 7f53d2e25640 1 -- 172.21.15.47:0/3242688304 shutdown_connections 2024-04-26T23:19:27.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.887+0000 7f53d2e25640 1 -- 172.21.15.47:0/3242688304 wait complete. 2024-04-26T23:19:27.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.888+0000 7f53d2e25640 1 Processor -- start 2024-04-26T23:19:27.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.889+0000 7f53d2e25640 1 -- start start 2024-04-26T23:19:27.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.889+0000 7f53d2e25640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53cc0cf930 0x7f53cc15b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:27.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.889+0000 7f53d2e25640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53cc160630 0x7f53cc152120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:27.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.889+0000 7f53d2e25640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53cc160a10 0x7f53cc06e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:27.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.889+0000 7f53d2e25640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc0d3b10 con 0x7f53cc160630 2024-04-26T23:19:27.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.889+0000 7f53d2e25640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc0b3e80 con 0x7f53cc0cf930 2024-04-26T23:19:27.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.889+0000 7f53d2e25640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f53cc0cfd40 con 0x7f53cc160a10 2024-04-26T23:19:27.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.889+0000 7f53cbfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53cc160630 0x7f53cc152120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:27.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.890+0000 7f53cbfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53cc160630 0x7f53cc152120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54898/0 (socket says 172.21.15.47:54898) 2024-04-26T23:19:27.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.890+0000 7f53cbfff640 1 -- 172.21.15.47:0/833021843 learned_addr learned my addr 172.21.15.47:0/833021843 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:27.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.890+0000 7f53d0bc1640 1 --2- 172.21.15.47:0/833021843 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53cc0cf930 0x7f53cc15b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:27.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.890+0000 7f53d13c2640 1 --2- 172.21.15.47:0/833021843 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53cc160a10 0x7f53cc06e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:27.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.890+0000 7f53cbfff640 1 -- 172.21.15.47:0/833021843 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53cc160a10 msgr2=0x7f53cc06e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:27.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.890+0000 7f53cbfff640 1 --2- 172.21.15.47:0/833021843 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53cc160a10 0x7f53cc06e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:27.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.890+0000 7f53cbfff640 1 -- 172.21.15.47:0/833021843 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53cc0cf930 msgr2=0x7f53cc15b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:27.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.890+0000 7f53cbfff640 1 --2- 172.21.15.47:0/833021843 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53cc0cf930 0x7f53cc15b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:27.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.890+0000 7f53cbfff640 1 -- 172.21.15.47:0/833021843 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53cc152660 con 0x7f53cc160630 2024-04-26T23:19:27.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.890+0000 7f53d0bc1640 1 --2- 172.21.15.47:0/833021843 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53cc0cf930 0x7f53cc15b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:19:27.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.890+0000 7f53cbfff640 1 --2- 172.21.15.47:0/833021843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53cc160630 0x7f53cc152120 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f53bc007ef0 tx=0x7f53bc00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:27.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.890+0000 7f53c97fa640 1 -- 172.21.15.47:0/833021843 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53bc01a070 con 0x7f53cc160630 2024-04-26T23:19:27.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.890+0000 7f53c97fa640 1 -- 172.21.15.47:0/833021843 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53bc003b90 con 0x7f53cc160630 2024-04-26T23:19:27.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.890+0000 7f53d2e25640 1 -- 172.21.15.47:0/833021843 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53cc152a20 con 0x7f53cc160630 2024-04-26T23:19:27.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.890+0000 7f53c97fa640 1 -- 172.21.15.47:0/833021843 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53bc004030 con 0x7f53cc160630 2024-04-26T23:19:27.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.890+0000 7f53d2e25640 1 -- 172.21.15.47:0/833021843 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53cc159c50 con 0x7f53cc160630 2024-04-26T23:19:27.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.891+0000 7f53d2e25640 1 -- 172.21.15.47:0/833021843 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53cc0694f0 con 0x7f53cc160630 2024-04-26T23:19:27.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.892+0000 7f53c97fa640 1 -- 172.21.15.47:0/833021843 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f53bc01d070 con 0x7f53cc160630 2024-04-26T23:19:27.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.892+0000 7f53c97fa640 1 --2- 172.21.15.47:0/833021843 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f53ac06c920 0x7f53ac06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:27.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.892+0000 7f53c97fa640 1 -- 172.21.15.47:0/833021843 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f53bc08c020 con 0x7f53cc160630 2024-04-26T23:19:27.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.893+0000 7f53d0bc1640 1 --2- 172.21.15.47:0/833021843 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f53ac06c920 0x7f53ac06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:27.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.893+0000 7f53d0bc1640 1 --2- 172.21.15.47:0/833021843 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f53ac06c920 0x7f53ac06ede0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f53c4037000 tx=0x7f53c4008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:27.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:27.897+0000 7f53c97fa640 1 -- 172.21.15.47:0/833021843 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f53cc152a20 con 0x7f53cc160630 2024-04-26T23:19:28.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.086+0000 7f53d2e25640 1 -- 172.21.15.47:0/833021843 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f53cc0ce7c0 con 0x7f53cc160630 2024-04-26T23:19:28.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.086+0000 7f53c97fa640 1 -- 172.21.15.47:0/833021843 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f53bc010070 con 0x7f53cc160630 2024-04-26T23:19:28.087 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-26T23:19:28.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.088+0000 7f53d2e25640 1 -- 172.21.15.47:0/833021843 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f53ac06c920 msgr2=0x7f53ac06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:28.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.088+0000 7f53d2e25640 1 --2- 172.21.15.47:0/833021843 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f53ac06c920 0x7f53ac06ede0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f53c4037000 tx=0x7f53c4008030 comp rx=0 tx=0).stop 2024-04-26T23:19:28.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.088+0000 7f53d2e25640 1 -- 172.21.15.47:0/833021843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53cc160630 msgr2=0x7f53cc152120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:28.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.088+0000 7f53d2e25640 1 --2- 172.21.15.47:0/833021843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53cc160630 0x7f53cc152120 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f53bc007ef0 tx=0x7f53bc00d4a0 comp rx=0 tx=0).stop 2024-04-26T23:19:28.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.088+0000 7f53d2e25640 1 -- 172.21.15.47:0/833021843 shutdown_connections 2024-04-26T23:19:28.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.088+0000 7f53d2e25640 1 --2- 172.21.15.47:0/833021843 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f53ac06c920 0x7f53ac06ede0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:28.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.088+0000 7f53d2e25640 1 --2- 172.21.15.47:0/833021843 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53cc160a10 0x7f53cc06e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:28.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.088+0000 7f53d2e25640 1 --2- 172.21.15.47:0/833021843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53cc160630 0x7f53cc152120 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:28.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.088+0000 7f53d2e25640 1 --2- 172.21.15.47:0/833021843 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53cc0cf930 0x7f53cc15b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:28.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.088+0000 7f53d2e25640 1 -- 172.21.15.47:0/833021843 >> 172.21.15.47:0/833021843 conn(0x7f53cc0d2dc0 msgr2=0x7f53cc0d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:28.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.088+0000 7f53d2e25640 1 -- 172.21.15.47:0/833021843 shutdown_connections 2024-04-26T23:19:28.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.088+0000 7f53d2e25640 1 -- 172.21.15.47:0/833021843 wait complete. 2024-04-26T23:19:28.100 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:19:28.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.245+0000 7f126fa85640 1 Processor -- start 2024-04-26T23:19:28.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.245+0000 7f126fa85640 1 -- start start 2024-04-26T23:19:28.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.246+0000 7f126fa85640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f12680b5b10 0x7f126816a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:28.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.246+0000 7f126fa85640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12680d0a10 con 0x7f12680d1a00 2024-04-26T23:19:28.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.246+0000 7f126fa85640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12680d0cf0 con 0x7f12680b40d0 2024-04-26T23:19:28.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.246+0000 7f126fa85640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f12680d0fd0 con 0x7f12680b5b10 2024-04-26T23:19:28.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.246+0000 7f126d821640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f12680d1a00 0x7f12680d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:55660/0 (socket says 172.21.15.47:55660) 2024-04-26T23:19:28.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.246+0000 7f126d821640 1 -- 172.21.15.47:0/1784187473 learned_addr learned my addr 172.21.15.47:0/1784187473 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:28.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.246+0000 7f126c81f640 1 -- 172.21.15.47:0/1784187473 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2186654257 0 0) 0x7f12680d0a10 con 0x7f12680d1a00 2024-04-26T23:19:28.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.246+0000 7f126c81f640 1 -- 172.21.15.47:0/1784187473 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1254003520 con 0x7f12680d1a00 2024-04-26T23:19:28.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.246+0000 7f126e022640 1 --2- 172.21.15.47:0/1784187473 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f12680b5b10 0x7f126816a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:28.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.247+0000 7f126c81f640 1 -- 172.21.15.47:0/1784187473 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3156647048 0 0) 0x7f1254003520 con 0x7f12680d1a00 2024-04-26T23:19:28.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.247+0000 7f126c81f640 1 -- 172.21.15.47:0/1784187473 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f12680b5b10 msgr2=0x7f126816a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:28.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.247+0000 7f126c81f640 1 --2- 172.21.15.47:0/1784187473 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f12680b5b10 0x7f126816a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:28.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.247+0000 7f126c81f640 1 -- 172.21.15.47:0/1784187473 >> v1:172.21.15.203:6789/0 conn(0x7f12680b40d0 legacy=0x7f12681625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:28.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.247+0000 7f126c81f640 1 -- 172.21.15.47:0/1784187473 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12680d40a0 con 0x7f12680d1a00 2024-04-26T23:19:28.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.247+0000 7f126c81f640 1 -- 172.21.15.47:0/1784187473 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7f1264001670 con 0x7f12680d1a00 2024-04-26T23:19:28.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.247+0000 7f126c81f640 1 -- 172.21.15.47:0/1784187473 >> v1:172.21.15.47:6789/0 conn(0x7f12680d1a00 legacy=0x7f12680d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:28.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.247+0000 7f126c81f640 1 --2- 172.21.15.47:0/1784187473 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12540039e0 0x7f1254003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:28.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.247+0000 7f126c81f640 1 --2- 172.21.15.47:0/1784187473 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1254004390 0x7f1254004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:28.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.247+0000 7f126c81f640 1 --2- 172.21.15.47:0/1784187473 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f12680b5b10 0x7f1254005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:28.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.247+0000 7f126c81f640 1 -- 172.21.15.47:0/1784187473 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f12680d0fd0 con 0x7f1254004390 2024-04-26T23:19:28.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.247+0000 7f126c81f640 1 -- 172.21.15.47:0/1784187473 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f125400baa0 con 0x7f12540039e0 2024-04-26T23:19:28.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.247+0000 7f126c81f640 1 -- 172.21.15.47:0/1784187473 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f125400bca0 con 0x7f12680b5b10 2024-04-26T23:19:28.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.247+0000 7f126d020640 1 --2- 172.21.15.47:0/1784187473 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1254004390 0x7f1254004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:28.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.247+0000 7f126d821640 1 --2- 172.21.15.47:0/1784187473 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12540039e0 0x7f1254003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:28.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.247+0000 7f126e022640 1 --2- 172.21.15.47:0/1784187473 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f12680b5b10 0x7f1254005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:28.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.247+0000 7f126d020640 1 -- 172.21.15.47:0/1784187473 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f12680b5b10 msgr2=0x7f1254005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:28.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.247+0000 7f126d020640 1 --2- 172.21.15.47:0/1784187473 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f12680b5b10 0x7f1254005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:28.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.247+0000 7f126d020640 1 -- 172.21.15.47:0/1784187473 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12540039e0 msgr2=0x7f1254003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:28.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.247+0000 7f126d020640 1 --2- 172.21.15.47:0/1784187473 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12540039e0 0x7f1254003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:28.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.247+0000 7f126d020640 1 -- 172.21.15.47:0/1784187473 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1254003520 con 0x7f1254004390 2024-04-26T23:19:28.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.248+0000 7f126d020640 1 --2- 172.21.15.47:0/1784187473 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1254004390 0x7f1254004860 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f125c001860 tx=0x7f1254005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0038d2ed248f063 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:28.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.248+0000 7f126c81f640 1 -- 172.21.15.47:0/1784187473 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125c037070 con 0x7f1254004390 2024-04-26T23:19:28.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.248+0000 7f126c81f640 1 -- 172.21.15.47:0/1784187473 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f125c02abf0 con 0x7f1254004390 2024-04-26T23:19:28.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.248+0000 7f126c81f640 1 -- 172.21.15.47:0/1784187473 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125c001430 con 0x7f1254004390 2024-04-26T23:19:28.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.249+0000 7f126fa85640 1 -- 172.21.15.47:0/1784187473 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1254004390 msgr2=0x7f1254004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:28.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.249+0000 7f126fa85640 1 --2- 172.21.15.47:0/1784187473 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1254004390 0x7f1254004860 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f125c001860 tx=0x7f1254005830 comp rx=0 tx=0).stop 2024-04-26T23:19:28.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.249+0000 7f126e022640 1 -- 172.21.15.47:0/1784187473 reap_dead start 2024-04-26T23:19:28.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.249+0000 7f126fa85640 1 -- 172.21.15.47:0/1784187473 shutdown_connections 2024-04-26T23:19:28.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.249+0000 7f126fa85640 1 -- 172.21.15.47:0/1784187473 >> 172.21.15.47:0/1784187473 conn(0x7f12680d4fd0 msgr2=0x7f12680d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:28.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.249+0000 7f126fa85640 1 -- 172.21.15.47:0/1784187473 shutdown_connections 2024-04-26T23:19:28.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.249+0000 7f126fa85640 1 -- 172.21.15.47:0/1784187473 wait complete. 2024-04-26T23:19:28.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.250+0000 7f126fa85640 1 Processor -- start 2024-04-26T23:19:28.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.250+0000 7f126fa85640 1 -- start start 2024-04-26T23:19:28.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.251+0000 7f126fa85640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1254004390 0x7f12680ba190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:28.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.251+0000 7f126fa85640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1268154b30 0x7f126814e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:28.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.251+0000 7f126fa85640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f126814e7d0 0x7f1268151270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:28.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.251+0000 7f126fa85640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1268162e80 con 0x7f1268154b30 2024-04-26T23:19:28.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.251+0000 7f126fa85640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1268162d00 con 0x7f1254004390 2024-04-26T23:19:28.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.251+0000 7f126fa85640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f1268163000 con 0x7f126814e7d0 2024-04-26T23:19:28.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.251+0000 7f126d020640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1268154b30 0x7f126814e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:28.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.251+0000 7f126d020640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1268154b30 0x7f126814e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54916/0 (socket says 172.21.15.47:54916) 2024-04-26T23:19:28.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.251+0000 7f126d020640 1 -- 172.21.15.47:0/4187753721 learned_addr learned my addr 172.21.15.47:0/4187753721 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:28.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.251+0000 7f126e022640 1 --2- 172.21.15.47:0/4187753721 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f126814e7d0 0x7f1268151270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:28.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.251+0000 7f126d821640 1 --2- 172.21.15.47:0/4187753721 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1254004390 0x7f12680ba190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:28.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.251+0000 7f126d020640 1 -- 172.21.15.47:0/4187753721 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f126814e7d0 msgr2=0x7f1268151270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:28.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.251+0000 7f126d020640 1 --2- 172.21.15.47:0/4187753721 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f126814e7d0 0x7f1268151270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:28.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.251+0000 7f126d020640 1 -- 172.21.15.47:0/4187753721 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1254004390 msgr2=0x7f12680ba190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:28.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.251+0000 7f126d020640 1 --2- 172.21.15.47:0/4187753721 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1254004390 0x7f12680ba190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:28.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.251+0000 7f126d020640 1 -- 172.21.15.47:0/4187753721 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12680b96f0 con 0x7f1268154b30 2024-04-26T23:19:28.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.251+0000 7f126d821640 1 --2- 172.21.15.47:0/4187753721 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1254004390 0x7f12680ba190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:19:28.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.251+0000 7f126d020640 1 --2- 172.21.15.47:0/4187753721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1268154b30 0x7f126814e290 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f125c02a6c0 tx=0x7f125c001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:28.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.251+0000 7f125a7fc640 1 -- 172.21.15.47:0/4187753721 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125c03f020 con 0x7f1268154b30 2024-04-26T23:19:28.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.251+0000 7f125a7fc640 1 -- 172.21.15.47:0/4187753721 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f125c005270 con 0x7f1268154b30 2024-04-26T23:19:28.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.251+0000 7f126fa85640 1 -- 172.21.15.47:0/4187753721 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f126814ec50 con 0x7f1268154b30 2024-04-26T23:19:28.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.251+0000 7f125a7fc640 1 -- 172.21.15.47:0/4187753721 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125c037040 con 0x7f1268154b30 2024-04-26T23:19:28.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.252+0000 7f126fa85640 1 -- 172.21.15.47:0/4187753721 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12680b9990 con 0x7f1268154b30 2024-04-26T23:19:28.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.253+0000 7f126fa85640 1 -- 172.21.15.47:0/4187753721 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12680694f0 con 0x7f1268154b30 2024-04-26T23:19:28.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.253+0000 7f125a7fc640 1 -- 172.21.15.47:0/4187753721 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f125c042020 con 0x7f1268154b30 2024-04-26T23:19:28.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.253+0000 7f125a7fc640 1 --2- 172.21.15.47:0/4187753721 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f124c06c920 0x7f124c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:28.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.254+0000 7f125a7fc640 1 -- 172.21.15.47:0/4187753721 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f125c0abcc0 con 0x7f1268154b30 2024-04-26T23:19:28.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.254+0000 7f126d821640 1 --2- 172.21.15.47:0/4187753721 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f124c06c920 0x7f124c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:28.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.254+0000 7f126d821640 1 --2- 172.21.15.47:0/4187753721 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f124c06c920 0x7f124c06ede0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f12640011e0 tx=0x7f1264009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:28.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.258+0000 7f125a7fc640 1 -- 172.21.15.47:0/4187753721 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f125c049030 con 0x7f1268154b30 2024-04-26T23:19:28.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.442+0000 7f126fa85640 1 -- 172.21.15.47:0/4187753721 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f12680d53b0 con 0x7f1268154b30 2024-04-26T23:19:28.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.442+0000 7f125a7fc640 1 -- 172.21.15.47:0/4187753721 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f125c079cd0 con 0x7f1268154b30 2024-04-26T23:19:28.443 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-26T23:19:28.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.444+0000 7f126fa85640 1 -- 172.21.15.47:0/4187753721 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f124c06c920 msgr2=0x7f124c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:28.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.444+0000 7f126fa85640 1 --2- 172.21.15.47:0/4187753721 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f124c06c920 0x7f124c06ede0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f12640011e0 tx=0x7f1264009040 comp rx=0 tx=0).stop 2024-04-26T23:19:28.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.444+0000 7f126fa85640 1 -- 172.21.15.47:0/4187753721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1268154b30 msgr2=0x7f126814e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:28.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.444+0000 7f126fa85640 1 --2- 172.21.15.47:0/4187753721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1268154b30 0x7f126814e290 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f125c02a6c0 tx=0x7f125c001230 comp rx=0 tx=0).stop 2024-04-26T23:19:28.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.444+0000 7f126fa85640 1 -- 172.21.15.47:0/4187753721 shutdown_connections 2024-04-26T23:19:28.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.444+0000 7f126fa85640 1 --2- 172.21.15.47:0/4187753721 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f124c06c920 0x7f124c06ede0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:28.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.444+0000 7f126fa85640 1 --2- 172.21.15.47:0/4187753721 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f126814e7d0 0x7f1268151270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:28.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.444+0000 7f126fa85640 1 --2- 172.21.15.47:0/4187753721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1268154b30 0x7f126814e290 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:28.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.444+0000 7f126fa85640 1 --2- 172.21.15.47:0/4187753721 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1254004390 0x7f12680ba190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:28.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.444+0000 7f126fa85640 1 -- 172.21.15.47:0/4187753721 >> 172.21.15.47:0/4187753721 conn(0x7f12680d4fd0 msgr2=0x7f126806cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:28.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.444+0000 7f126fa85640 1 -- 172.21.15.47:0/4187753721 shutdown_connections 2024-04-26T23:19:28.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.444+0000 7f126fa85640 1 -- 172.21.15.47:0/4187753721 wait complete. 2024-04-26T23:19:28.456 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T23:19:28.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.560+0000 7fb122d0a640 1 Processor -- start 2024-04-26T23:19:28.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.560+0000 7fb122d0a640 1 -- start start 2024-04-26T23:19:28.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.561+0000 7fb122d0a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb11c0d0d60 0x7fb11c0b2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:28.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.561+0000 7fb122d0a640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb11c0ce5e0 con 0x7fb11c0b4360 2024-04-26T23:19:28.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.561+0000 7fb122d0a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb11c0ce8c0 con 0x7fb11c0d1240 2024-04-26T23:19:28.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.561+0000 7fb122d0a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb11c0cf210 con 0x7fb11c0d0d60 2024-04-26T23:19:28.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.561+0000 7fb11bfff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb11c0b4360 0x7fb11c0b2600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:55674/0 (socket says 172.21.15.47:55674) 2024-04-26T23:19:28.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.561+0000 7fb11bfff640 1 -- 172.21.15.47:0/2516823801 learned_addr learned my addr 172.21.15.47:0/2516823801 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:28.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.562+0000 7fb1212a7640 1 --2- 172.21.15.47:0/2516823801 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb11c0d0d60 0x7fb11c0b2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:28.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.562+0000 7fb11b7fe640 1 -- 172.21.15.47:0/2516823801 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1510992421 0 0) 0x7fb11c0ce5e0 con 0x7fb11c0b4360 2024-04-26T23:19:28.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.562+0000 7fb11b7fe640 1 -- 172.21.15.47:0/2516823801 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb10c003520 con 0x7fb11c0b4360 2024-04-26T23:19:28.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.562+0000 7fb11b7fe640 1 -- 172.21.15.47:0/2516823801 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3077456020 0 0) 0x7fb11c0ce8c0 con 0x7fb11c0d1240 2024-04-26T23:19:28.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.562+0000 7fb11b7fe640 1 -- 172.21.15.47:0/2516823801 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb11c0ce5e0 con 0x7fb11c0d1240 2024-04-26T23:19:28.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.562+0000 7fb1212a7640 1 -- 172.21.15.47:0/2516823801 >> v1:172.21.15.203:6789/0 conn(0x7fb11c0d1240 legacy=0x7fb11c0d1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:28.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.562+0000 7fb1212a7640 1 -- 172.21.15.47:0/2516823801 >> v1:172.21.15.47:6789/0 conn(0x7fb11c0b4360 legacy=0x7fb11c0b2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:28.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.562+0000 7fb1212a7640 1 -- 172.21.15.47:0/2516823801 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb11c160ab0 con 0x7fb11c0d0d60 2024-04-26T23:19:28.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.563+0000 7fb1212a7640 1 --2- 172.21.15.47:0/2516823801 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb11c0d0d60 0x7fb11c0b2d10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb114009dc0 tx=0x7fb11402a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7128150cb85d376d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:28.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.563+0000 7fb11b7fe640 1 -- 172.21.15.47:0/2516823801 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb114007740 con 0x7fb11c0d0d60 2024-04-26T23:19:28.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.563+0000 7fb11b7fe640 1 -- 172.21.15.47:0/2516823801 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1140078e0 con 0x7fb11c0d0d60 2024-04-26T23:19:28.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.563+0000 7fb11b7fe640 1 -- 172.21.15.47:0/2516823801 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb114007c90 con 0x7fb11c0d0d60 2024-04-26T23:19:28.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.563+0000 7fb122d0a640 1 -- 172.21.15.47:0/2516823801 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb11c0d0d60 msgr2=0x7fb11c0b2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:28.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.563+0000 7fb122d0a640 1 --2- 172.21.15.47:0/2516823801 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb11c0d0d60 0x7fb11c0b2d10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb114009dc0 tx=0x7fb11402a6d0 comp rx=0 tx=0).stop 2024-04-26T23:19:28.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.564+0000 7fb122d0a640 1 -- 172.21.15.47:0/2516823801 shutdown_connections 2024-04-26T23:19:28.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.564+0000 7fb122d0a640 1 --2- 172.21.15.47:0/2516823801 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb11c0d0d60 0x7fb11c0b2d10 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:28.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.564+0000 7fb122d0a640 1 -- 172.21.15.47:0/2516823801 >> 172.21.15.47:0/2516823801 conn(0x7fb11c0d2e60 msgr2=0x7fb11c0d3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:28.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.564+0000 7fb122d0a640 1 -- 172.21.15.47:0/2516823801 shutdown_connections 2024-04-26T23:19:28.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.564+0000 7fb122d0a640 1 -- 172.21.15.47:0/2516823801 wait complete. 2024-04-26T23:19:28.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.565+0000 7fb122d0a640 1 Processor -- start 2024-04-26T23:19:28.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.565+0000 7fb122d0a640 1 -- start start 2024-04-26T23:19:28.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.566+0000 7fb122d0a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb11c0d0d60 0x7fb11c15b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:28.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.566+0000 7fb122d0a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb11c0d1240 0x7fb11c06e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:28.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.566+0000 7fb122d0a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb11c0c95d0 0x7fb11c0c80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:28.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.566+0000 7fb122d0a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb11c0c6790 con 0x7fb11c0d0d60 2024-04-26T23:19:28.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.566+0000 7fb122d0a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb11c0c6610 con 0x7fb11c0d1240 2024-04-26T23:19:28.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.566+0000 7fb122d0a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb11c0b3250 con 0x7fb11c0c95d0 2024-04-26T23:19:28.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.566+0000 7fb120aa6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb11c0d0d60 0x7fb11c15b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:28.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.566+0000 7fb120aa6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb11c0d0d60 0x7fb11c15b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54926/0 (socket says 172.21.15.47:54926) 2024-04-26T23:19:28.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.566+0000 7fb120aa6640 1 -- 172.21.15.47:0/255031492 learned_addr learned my addr 172.21.15.47:0/255031492 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:28.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.566+0000 7fb11bfff640 1 --2- 172.21.15.47:0/255031492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb11c0d1240 0x7fb11c06e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:28.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.566+0000 7fb1212a7640 1 --2- 172.21.15.47:0/255031492 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb11c0c95d0 0x7fb11c0c80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:28.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.566+0000 7fb120aa6640 1 -- 172.21.15.47:0/255031492 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb11c0c95d0 msgr2=0x7fb11c0c80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:28.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.566+0000 7fb120aa6640 1 --2- 172.21.15.47:0/255031492 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb11c0c95d0 0x7fb11c0c80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:28.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.566+0000 7fb120aa6640 1 -- 172.21.15.47:0/255031492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb11c0d1240 msgr2=0x7fb11c06e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:28.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.566+0000 7fb120aa6640 1 --2- 172.21.15.47:0/255031492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb11c0d1240 0x7fb11c06e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:28.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.566+0000 7fb120aa6640 1 -- 172.21.15.47:0/255031492 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb11c15bce0 con 0x7fb11c0d0d60 2024-04-26T23:19:28.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.566+0000 7fb120aa6640 1 --2- 172.21.15.47:0/255031492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb11c0d0d60 0x7fb11c15b7a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fb11000c460 tx=0x7fb11000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:28.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.566+0000 7fb1197fa640 1 -- 172.21.15.47:0/255031492 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb110017070 con 0x7fb11c0d0d60 2024-04-26T23:19:28.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.566+0000 7fb1197fa640 1 -- 172.21.15.47:0/255031492 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1100052f0 con 0x7fb11c0d0d60 2024-04-26T23:19:28.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.566+0000 7fb122d0a640 1 -- 172.21.15.47:0/255031492 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb11c06efd0 con 0x7fb11c0d0d60 2024-04-26T23:19:28.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.566+0000 7fb1197fa640 1 -- 172.21.15.47:0/255031492 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb110003830 con 0x7fb11c0d0d60 2024-04-26T23:19:28.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.566+0000 7fb122d0a640 1 -- 172.21.15.47:0/255031492 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb11c0c7610 con 0x7fb11c0d0d60 2024-04-26T23:19:28.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.567+0000 7fb122d0a640 1 -- 172.21.15.47:0/255031492 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb11c0b5760 con 0x7fb11c0d0d60 2024-04-26T23:19:28.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.568+0000 7fb1197fa640 1 -- 172.21.15.47:0/255031492 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb11001a070 con 0x7fb11c0d0d60 2024-04-26T23:19:28.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.568+0000 7fb1197fa640 1 --2- 172.21.15.47:0/255031492 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb10006c8f0 0x7fb10006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:28.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.568+0000 7fb1197fa640 1 -- 172.21.15.47:0/255031492 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7fb11008b220 con 0x7fb11c0d0d60 2024-04-26T23:19:28.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.569+0000 7fb11bfff640 1 --2- 172.21.15.47:0/255031492 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb10006c8f0 0x7fb10006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:28.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.569+0000 7fb11bfff640 1 --2- 172.21.15.47:0/255031492 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb10006c8f0 0x7fb10006edb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb11c06e5c0 tx=0x7fb108007d30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:28.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.573+0000 7fb1197fa640 1 -- 172.21.15.47:0/255031492 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb11c0b5760 con 0x7fb11c0d0d60 2024-04-26T23:19:28.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.773+0000 7fb122d0a640 1 -- 172.21.15.47:0/255031492 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fb11c0694f0 con 0x7fb11c0d0d60 2024-04-26T23:19:28.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.774+0000 7fb1197fa640 1 -- 172.21.15.47:0/255031492 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fb110058c70 con 0x7fb11c0d0d60 2024-04-26T23:19:28.775 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-26T23:19:28.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.776+0000 7fb122d0a640 1 -- 172.21.15.47:0/255031492 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb10006c8f0 msgr2=0x7fb10006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:28.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.776+0000 7fb122d0a640 1 --2- 172.21.15.47:0/255031492 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb10006c8f0 0x7fb10006edb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb11c06e5c0 tx=0x7fb108007d30 comp rx=0 tx=0).stop 2024-04-26T23:19:28.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.776+0000 7fb122d0a640 1 -- 172.21.15.47:0/255031492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb11c0d0d60 msgr2=0x7fb11c15b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:28.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.776+0000 7fb122d0a640 1 --2- 172.21.15.47:0/255031492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb11c0d0d60 0x7fb11c15b7a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fb11000c460 tx=0x7fb11000c920 comp rx=0 tx=0).stop 2024-04-26T23:19:28.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.776+0000 7fb122d0a640 1 -- 172.21.15.47:0/255031492 shutdown_connections 2024-04-26T23:19:28.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.776+0000 7fb122d0a640 1 --2- 172.21.15.47:0/255031492 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb10006c8f0 0x7fb10006edb0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:28.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.776+0000 7fb122d0a640 1 --2- 172.21.15.47:0/255031492 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb11c0c95d0 0x7fb11c0c80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:28.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.776+0000 7fb122d0a640 1 --2- 172.21.15.47:0/255031492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb11c0d1240 0x7fb11c06e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:28.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.776+0000 7fb122d0a640 1 --2- 172.21.15.47:0/255031492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb11c0d0d60 0x7fb11c15b7a0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:28.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.776+0000 7fb122d0a640 1 -- 172.21.15.47:0/255031492 >> 172.21.15.47:0/255031492 conn(0x7fb11c0d2e60 msgr2=0x7fb11c0b5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:28.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.776+0000 7fb122d0a640 1 -- 172.21.15.47:0/255031492 shutdown_connections 2024-04-26T23:19:28.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.776+0000 7fb122d0a640 1 -- 172.21.15.47:0/255031492 wait complete. 2024-04-26T23:19:28.788 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T23:19:28.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.888+0000 7f26d3fe2640 1 Processor -- start 2024-04-26T23:19:28.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.889+0000 7f26d3fe2640 1 -- start start 2024-04-26T23:19:28.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.889+0000 7f26d3fe2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26cc0d06f0 0x7f26cc0d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:28.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.889+0000 7f26d3fe2640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f26cc0b2850 0x7f26cc0b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:28.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.889+0000 7f26d3fe2640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26cc0d0b70 con 0x7f26cc15f3a0 2024-04-26T23:19:28.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.889+0000 7f26d3fe2640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f26cc0d0e20 con 0x7f26cc0b2850 2024-04-26T23:19:28.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.889+0000 7f26d3fe2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f26cc0b1d70 con 0x7f26cc0d06f0 2024-04-26T23:19:28.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.890+0000 7f26d157d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26cc0d06f0 0x7f26cc0d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:28.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.890+0000 7f26d1d7e640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f26cc15f3a0 0x7f26cc0d1770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.47:56124/0 (socket says 172.21.15.47:56124) 2024-04-26T23:19:28.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.890+0000 7f26d257f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f26cc0b2850 0x7f26cc0b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:28.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.890+0000 7f26d1d7e640 1 -- 172.21.15.47:0/2901173981 learned_addr learned my addr 172.21.15.47:0/2901173981 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:28.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.890+0000 7f26d0d7c640 1 -- 172.21.15.47:0/2901173981 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 982246417 0 0) 0x7f26cc0d0b70 con 0x7f26cc15f3a0 2024-04-26T23:19:28.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.890+0000 7f26d0d7c640 1 -- 172.21.15.47:0/2901173981 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f26b4003520 con 0x7f26cc15f3a0 2024-04-26T23:19:28.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.890+0000 7f26d157d640 1 -- 172.21.15.47:0/2901173981 >> v2:172.21.15.203:3300/0 conn(0x7f26cc0b2850 msgr2=0x7f26cc0b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:28.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.890+0000 7f26d157d640 1 --2- 172.21.15.47:0/2901173981 >> v2:172.21.15.203:3300/0 conn(0x7f26cc0b2850 0x7f26cc0b2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:28.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.890+0000 7f26d157d640 1 -- 172.21.15.47:0/2901173981 >> v1:172.21.15.203:6789/0 conn(0x7f26cc15f3a0 legacy=0x7f26cc0d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:28.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.890+0000 7f26d157d640 1 -- 172.21.15.47:0/2901173981 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26cc0b2050 con 0x7f26cc0d06f0 2024-04-26T23:19:28.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.890+0000 7f26d257f640 1 --2- 172.21.15.47:0/2901173981 >> v2:172.21.15.203:3300/0 conn(0x7f26cc0b2850 0x7f26cc0b2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:19:28.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.891+0000 7f26d157d640 1 --2- 172.21.15.47:0/2901173981 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26cc0d06f0 0x7f26cc0d1e80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f26c00097a0 tx=0x7f26c002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b84513aa9d0ab8fd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:28.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.891+0000 7f26d0d7c640 1 -- 172.21.15.47:0/2901173981 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c0037070 con 0x7f26cc0d06f0 2024-04-26T23:19:28.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.891+0000 7f26d0d7c640 1 -- 172.21.15.47:0/2901173981 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26c0032440 con 0x7f26cc0d06f0 2024-04-26T23:19:28.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.891+0000 7f26d0d7c640 1 -- 172.21.15.47:0/2901173981 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c0032810 con 0x7f26cc0d06f0 2024-04-26T23:19:28.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.892+0000 7f26d3fe2640 1 -- 172.21.15.47:0/2901173981 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26cc0d06f0 msgr2=0x7f26cc0d1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:28.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.892+0000 7f26d3fe2640 1 --2- 172.21.15.47:0/2901173981 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26cc0d06f0 0x7f26cc0d1e80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f26c00097a0 tx=0x7f26c002a180 comp rx=0 tx=0).stop 2024-04-26T23:19:28.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.892+0000 7f26d3fe2640 1 -- 172.21.15.47:0/2901173981 shutdown_connections 2024-04-26T23:19:28.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.892+0000 7f26d3fe2640 1 --2- 172.21.15.47:0/2901173981 >> v2:172.21.15.203:3300/0 conn(0x7f26cc0b2850 0x7f26cc0b2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:28.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.892+0000 7f26d3fe2640 1 --2- 172.21.15.47:0/2901173981 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26cc0d06f0 0x7f26cc0d1e80 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:28.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.892+0000 7f26d3fe2640 1 -- 172.21.15.47:0/2901173981 >> 172.21.15.47:0/2901173981 conn(0x7f26cc0d4f90 msgr2=0x7f26cc0d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:28.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.892+0000 7f26d3fe2640 1 -- 172.21.15.47:0/2901173981 shutdown_connections 2024-04-26T23:19:28.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.892+0000 7f26d3fe2640 1 -- 172.21.15.47:0/2901173981 wait complete. 2024-04-26T23:19:28.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.893+0000 7f26d3fe2640 1 Processor -- start 2024-04-26T23:19:28.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.893+0000 7f26d3fe2640 1 -- start start 2024-04-26T23:19:28.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.894+0000 7f26d3fe2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26cc0b2850 0x7f26cc14f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:28.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.894+0000 7f26d3fe2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26cc0d06f0 0x7f26cc14eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:28.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.894+0000 7f26d3fe2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26cc15f3a0 0x7f26cc0b8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:28.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.894+0000 7f26d3fe2640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f26cc15f900 con 0x7f26cc0b2850 2024-04-26T23:19:28.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.894+0000 7f26d3fe2640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f26cc15f780 con 0x7f26cc0d06f0 2024-04-26T23:19:28.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.894+0000 7f26d3fe2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f26cc0d23c0 con 0x7f26cc15f3a0 2024-04-26T23:19:28.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.894+0000 7f26d1d7e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26cc0b2850 0x7f26cc14f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:28.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.894+0000 7f26d1d7e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26cc0b2850 0x7f26cc14f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54936/0 (socket says 172.21.15.47:54936) 2024-04-26T23:19:28.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.894+0000 7f26d1d7e640 1 -- 172.21.15.47:0/1538088493 learned_addr learned my addr 172.21.15.47:0/1538088493 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:28.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.894+0000 7f26d257f640 1 --2- 172.21.15.47:0/1538088493 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26cc15f3a0 0x7f26cc0b8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:28.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.894+0000 7f26d157d640 1 --2- 172.21.15.47:0/1538088493 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26cc0d06f0 0x7f26cc14eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:28.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.894+0000 7f26d1d7e640 1 -- 172.21.15.47:0/1538088493 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26cc15f3a0 msgr2=0x7f26cc0b8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:28.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.894+0000 7f26d1d7e640 1 --2- 172.21.15.47:0/1538088493 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26cc15f3a0 0x7f26cc0b8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:28.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.894+0000 7f26d1d7e640 1 -- 172.21.15.47:0/1538088493 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26cc0d06f0 msgr2=0x7f26cc14eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:28.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.894+0000 7f26d1d7e640 1 --2- 172.21.15.47:0/1538088493 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26cc0d06f0 0x7f26cc14eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:28.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.894+0000 7f26d1d7e640 1 -- 172.21.15.47:0/1538088493 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26cc0cc440 con 0x7f26cc0b2850 2024-04-26T23:19:28.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.894+0000 7f26d157d640 1 --2- 172.21.15.47:0/1538088493 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26cc0d06f0 0x7f26cc14eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:19:28.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.894+0000 7f26d1d7e640 1 --2- 172.21.15.47:0/1538088493 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26cc0b2850 0x7f26cc14f3f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f26c400c430 tx=0x7f26c400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:28.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.895+0000 7f26ca7fc640 1 -- 172.21.15.47:0/1538088493 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c4017070 con 0x7f26cc0b2850 2024-04-26T23:19:28.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.895+0000 7f26ca7fc640 1 -- 172.21.15.47:0/1538088493 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26c4005ce0 con 0x7f26cc0b2850 2024-04-26T23:19:28.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.895+0000 7f26d3fe2640 1 -- 172.21.15.47:0/1538088493 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26cc0b8380 con 0x7f26cc0b2850 2024-04-26T23:19:28.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.895+0000 7f26ca7fc640 1 -- 172.21.15.47:0/1538088493 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c40072f0 con 0x7f26cc0b2850 2024-04-26T23:19:28.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.895+0000 7f26d3fe2640 1 -- 172.21.15.47:0/1538088493 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26cc14f020 con 0x7f26cc0b2850 2024-04-26T23:19:28.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.896+0000 7f26d3fe2640 1 -- 172.21.15.47:0/1538088493 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26cc0d48c0 con 0x7f26cc0b2850 2024-04-26T23:19:28.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.896+0000 7f26ca7fc640 1 -- 172.21.15.47:0/1538088493 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f26c401a070 con 0x7f26cc0b2850 2024-04-26T23:19:28.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.897+0000 7f26ca7fc640 1 --2- 172.21.15.47:0/1538088493 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f26b806c8f0 0x7f26b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:28.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.897+0000 7f26ca7fc640 1 -- 172.21.15.47:0/1538088493 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f26c408b2b0 con 0x7f26cc0b2850 2024-04-26T23:19:28.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.897+0000 7f26d157d640 1 --2- 172.21.15.47:0/1538088493 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f26b806c8f0 0x7f26b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:28.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.898+0000 7f26d157d640 1 --2- 172.21.15.47:0/1538088493 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f26b806c8f0 0x7f26b806edb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f26c0034000 tx=0x7f26c002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:28.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:28.901+0000 7f26ca7fc640 1 -- 172.21.15.47:0/1538088493 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f26cc0d48c0 con 0x7f26cc0b2850 2024-04-26T23:19:29.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.099+0000 7f26d3fe2640 1 -- 172.21.15.47:0/1538088493 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f26cc0694f0 con 0x7f26cc0b2850 2024-04-26T23:19:29.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.100+0000 7f26ca7fc640 1 -- 172.21.15.47:0/1538088493 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f26c408b0d0 con 0x7f26cc0b2850 2024-04-26T23:19:29.302 INFO:teuthology.orchestra.run.smithi047.stderr:listed 0 entries 2024-04-26T23:19:29.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.102+0000 7f26d3fe2640 1 -- 172.21.15.47:0/1538088493 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f26b806c8f0 msgr2=0x7f26b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:29.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.102+0000 7f26d3fe2640 1 --2- 172.21.15.47:0/1538088493 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f26b806c8f0 0x7f26b806edb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f26c0034000 tx=0x7f26c002e000 comp rx=0 tx=0).stop 2024-04-26T23:19:29.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.102+0000 7f26d3fe2640 1 -- 172.21.15.47:0/1538088493 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26cc0b2850 msgr2=0x7f26cc14f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:29.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.102+0000 7f26d3fe2640 1 --2- 172.21.15.47:0/1538088493 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26cc0b2850 0x7f26cc14f3f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f26c400c430 tx=0x7f26c400c830 comp rx=0 tx=0).stop 2024-04-26T23:19:29.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.102+0000 7f26d3fe2640 1 -- 172.21.15.47:0/1538088493 shutdown_connections 2024-04-26T23:19:29.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.102+0000 7f26d3fe2640 1 --2- 172.21.15.47:0/1538088493 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f26b806c8f0 0x7f26b806edb0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:29.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.102+0000 7f26d3fe2640 1 --2- 172.21.15.47:0/1538088493 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26cc15f3a0 0x7f26cc0b8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:29.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.102+0000 7f26d3fe2640 1 --2- 172.21.15.47:0/1538088493 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26cc0d06f0 0x7f26cc14eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:29.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.102+0000 7f26d3fe2640 1 --2- 172.21.15.47:0/1538088493 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26cc0b2850 0x7f26cc14f3f0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:29.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.102+0000 7f26d3fe2640 1 -- 172.21.15.47:0/1538088493 >> 172.21.15.47:0/1538088493 conn(0x7f26cc0d4f90 msgr2=0x7f26cc0d1720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:29.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.102+0000 7f26d3fe2640 1 -- 172.21.15.47:0/1538088493 shutdown_connections 2024-04-26T23:19:29.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.102+0000 7f26d3fe2640 1 -- 172.21.15.47:0/1538088493 wait complete. 2024-04-26T23:19:29.305 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-04-26T23:19:29.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:29.305 DEBUG:teuthology.orchestra.run.smithi203:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-04-26T23:19:29.333 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2024-04-26T23:19:29.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:29.333 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-26T23:19:29.392 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-26T23:19:29.393 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:19:29.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.499+0000 7f30f0df0640 1 Processor -- start 2024-04-26T23:19:29.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.499+0000 7f30f0df0640 1 -- start start 2024-04-26T23:19:29.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.500+0000 7f30f0df0640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f30ec0b5f20 0x7f30ec0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:29.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.500+0000 7f30f0df0640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30ec0b7e50 con 0x7f30ec0d3bc0 2024-04-26T23:19:29.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.500+0000 7f30f0df0640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30ec0d1800 con 0x7f30ec162880 2024-04-26T23:19:29.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.500+0000 7f30f0df0640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f30ec0d0580 con 0x7f30ec0b5f20 2024-04-26T23:19:29.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.500+0000 7f30e3fff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f30ec0d3bc0 0x7f30ec0b8300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:55682/0 (socket says 172.21.15.47:55682) 2024-04-26T23:19:29.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.500+0000 7f30e3fff640 1 -- 172.21.15.47:0/1048695429 learned_addr learned my addr 172.21.15.47:0/1048695429 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:29.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.500+0000 7f30ead9d640 1 --2- 172.21.15.47:0/1048695429 >> v2:172.21.15.203:3300/0 conn(0x7f30ec0b5f20 0x7f30ec0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:29.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.500+0000 7f30e9d9b640 1 -- 172.21.15.47:0/1048695429 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 573068036 0 0) 0x7f30ec0b7e50 con 0x7f30ec0d3bc0 2024-04-26T23:19:29.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.500+0000 7f30e9d9b640 1 -- 172.21.15.47:0/1048695429 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f30d8003520 con 0x7f30ec0d3bc0 2024-04-26T23:19:29.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.500+0000 7f30e9d9b640 1 -- 172.21.15.47:0/1048695429 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1839435482 0 0) 0x7f30dc001110 con 0x7f30ec162880 2024-04-26T23:19:29.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.500+0000 7f30e9d9b640 1 -- 172.21.15.47:0/1048695429 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f30ec0b7e50 con 0x7f30ec162880 2024-04-26T23:19:29.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.500+0000 7f30ead9d640 1 -- 172.21.15.47:0/1048695429 >> v1:172.21.15.203:6789/0 conn(0x7f30ec162880 legacy=0x7f30ec0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:29.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.500+0000 7f30ead9d640 1 -- 172.21.15.47:0/1048695429 >> v1:172.21.15.47:6789/0 conn(0x7f30ec0d3bc0 legacy=0x7f30ec0b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:29.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.500+0000 7f30ead9d640 1 -- 172.21.15.47:0/1048695429 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30ec0d07d0 con 0x7f30ec0b5f20 2024-04-26T23:19:29.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.501+0000 7f30ead9d640 1 --2- 172.21.15.47:0/1048695429 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30ec0b5f20 0x7f30ec0d2cb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f30e4009d90 tx=0x7f30e402a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57fb0852e3d441d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:29.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.501+0000 7f30e9d9b640 1 -- 172.21.15.47:0/1048695429 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e4007740 con 0x7f30ec0b5f20 2024-04-26T23:19:29.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.501+0000 7f30e9d9b640 1 -- 172.21.15.47:0/1048695429 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30e40078e0 con 0x7f30ec0b5f20 2024-04-26T23:19:29.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.501+0000 7f30e9d9b640 1 -- 172.21.15.47:0/1048695429 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e4007cb0 con 0x7f30ec0b5f20 2024-04-26T23:19:29.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.502+0000 7f30f0df0640 1 -- 172.21.15.47:0/1048695429 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30ec0b5f20 msgr2=0x7f30ec0d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:29.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.502+0000 7f30f0df0640 1 --2- 172.21.15.47:0/1048695429 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30ec0b5f20 0x7f30ec0d2cb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f30e4009d90 tx=0x7f30e402a6f0 comp rx=0 tx=0).stop 2024-04-26T23:19:29.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.502+0000 7f30f0df0640 1 -- 172.21.15.47:0/1048695429 shutdown_connections 2024-04-26T23:19:29.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.502+0000 7f30f0df0640 1 --2- 172.21.15.47:0/1048695429 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30ec0b5f20 0x7f30ec0d2cb0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:29.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.502+0000 7f30f0df0640 1 -- 172.21.15.47:0/1048695429 >> 172.21.15.47:0/1048695429 conn(0x7f30ec0d4f60 msgr2=0x7f30ec0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:29.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.502+0000 7f30f0df0640 1 -- 172.21.15.47:0/1048695429 shutdown_connections 2024-04-26T23:19:29.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.502+0000 7f30f0df0640 1 -- 172.21.15.47:0/1048695429 wait complete. 2024-04-26T23:19:29.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.503+0000 7f30f0df0640 1 Processor -- start 2024-04-26T23:19:29.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.503+0000 7f30f0df0640 1 -- start start 2024-04-26T23:19:29.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.504+0000 7f30f0df0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30ec0b5f20 0x7f30ec06dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:29.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.504+0000 7f30f0df0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f30ec0d3bc0 0x7f30ec09f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:29.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.504+0000 7f30f0df0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30ec162880 0x7f30ec09f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:29.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.504+0000 7f30f0df0640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec162c60 con 0x7f30ec0b5f20 2024-04-26T23:19:29.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.504+0000 7f30f0df0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec0d0d90 con 0x7f30ec162880 2024-04-26T23:19:29.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.504+0000 7f30f0df0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f30ec162de0 con 0x7f30ec0d3bc0 2024-04-26T23:19:29.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.504+0000 7f30ea59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30ec0b5f20 0x7f30ec06dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:29.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.504+0000 7f30ea59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30ec0b5f20 0x7f30ec06dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54938/0 (socket says 172.21.15.47:54938) 2024-04-26T23:19:29.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.504+0000 7f30ea59c640 1 -- 172.21.15.47:0/5997819 learned_addr learned my addr 172.21.15.47:0/5997819 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:29.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.504+0000 7f30ead9d640 1 --2- 172.21.15.47:0/5997819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30ec162880 0x7f30ec09f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:29.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.504+0000 7f30e3fff640 1 --2- 172.21.15.47:0/5997819 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f30ec0d3bc0 0x7f30ec09f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:29.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.504+0000 7f30ea59c640 1 -- 172.21.15.47:0/5997819 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f30ec0d3bc0 msgr2=0x7f30ec09f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:29.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.504+0000 7f30ea59c640 1 --2- 172.21.15.47:0/5997819 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f30ec0d3bc0 0x7f30ec09f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:29.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.504+0000 7f30ea59c640 1 -- 172.21.15.47:0/5997819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30ec162880 msgr2=0x7f30ec09f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:29.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.504+0000 7f30ea59c640 1 --2- 172.21.15.47:0/5997819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30ec162880 0x7f30ec09f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:29.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.504+0000 7f30ea59c640 1 -- 172.21.15.47:0/5997819 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30ec1f4c90 con 0x7f30ec0b5f20 2024-04-26T23:19:29.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.504+0000 7f30ea59c640 1 --2- 172.21.15.47:0/5997819 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30ec0b5f20 0x7f30ec06dad0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f30dc00c700 tx=0x7f30dc00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:29.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.505+0000 7f30e2ffd640 1 -- 172.21.15.47:0/5997819 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30dc0052d0 con 0x7f30ec0b5f20 2024-04-26T23:19:29.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.505+0000 7f30e2ffd640 1 -- 172.21.15.47:0/5997819 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30dc005490 con 0x7f30ec0b5f20 2024-04-26T23:19:29.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.505+0000 7f30f0df0640 1 -- 172.21.15.47:0/5997819 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30ec1f4f30 con 0x7f30ec0b5f20 2024-04-26T23:19:29.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.505+0000 7f30e2ffd640 1 -- 172.21.15.47:0/5997819 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30dc00a880 con 0x7f30ec0b5f20 2024-04-26T23:19:29.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.505+0000 7f30f0df0640 1 -- 172.21.15.47:0/5997819 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30ec1f54f0 con 0x7f30ec0b5f20 2024-04-26T23:19:29.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.506+0000 7f30f0df0640 1 -- 172.21.15.47:0/5997819 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30ec0694f0 con 0x7f30ec0b5f20 2024-04-26T23:19:29.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.506+0000 7f30e2ffd640 1 -- 172.21.15.47:0/5997819 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f30dc00aa20 con 0x7f30ec0b5f20 2024-04-26T23:19:29.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.506+0000 7f30e2ffd640 1 --2- 172.21.15.47:0/5997819 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f30cc06c8f0 0x7f30cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:29.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.507+0000 7f30e2ffd640 1 -- 172.21.15.47:0/5997819 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f30dc08b9e0 con 0x7f30ec0b5f20 2024-04-26T23:19:29.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.507+0000 7f30e3fff640 1 --2- 172.21.15.47:0/5997819 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f30cc06c8f0 0x7f30cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:29.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.507+0000 7f30e3fff640 1 --2- 172.21.15.47:0/5997819 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f30cc06c8f0 0x7f30cc06edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f30ec0d15d0 tx=0x7f30d4007d50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:29.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.511+0000 7f30e2ffd640 1 -- 172.21.15.47:0/5997819 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f30ec0d1800 con 0x7f30ec0b5f20 2024-04-26T23:19:29.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.734+0000 7f30f0df0640 1 -- 172.21.15.47:0/5997819 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30ec06e7d0 con 0x7f30ec0b5f20 2024-04-26T23:19:29.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.735+0000 7f30e2ffd640 1 -- 172.21.15.47:0/5997819 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5180 (secure 0 0 0) 0x7f30dc0593a0 con 0x7f30ec0b5f20 2024-04-26T23:19:29.735 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:19:29.736 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/1553696401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1553696401},{"type":"v1","addr":"172.21.15.47:6834","nonce":1553696401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/3792767564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3792767564},{"type":"v1","addr":"172.21.15.203:6836","nonce":3792767564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4088100766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":4088100766},{"type":"v1","addr":"172.21.15.47:6835","nonce":4088100766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:18:31.367292+0000","modified":"2024-04-26T23:18:53.430267+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/997115011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":997115011},{"type":"v1","addr":"172.21.15.203:6837","nonce":997115011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4282,"qdb_cluster":[4282]},"id":1}]} 2024-04-26T23:19:29.736 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 9 2024-04-26T23:19:29.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.737+0000 7f30f0df0640 1 -- 172.21.15.47:0/5997819 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f30cc06c8f0 msgr2=0x7f30cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:29.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.737+0000 7f30f0df0640 1 --2- 172.21.15.47:0/5997819 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f30cc06c8f0 0x7f30cc06edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f30ec0d15d0 tx=0x7f30d4007d50 comp rx=0 tx=0).stop 2024-04-26T23:19:29.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.737+0000 7f30f0df0640 1 -- 172.21.15.47:0/5997819 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30ec0b5f20 msgr2=0x7f30ec06dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:29.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.737+0000 7f30f0df0640 1 --2- 172.21.15.47:0/5997819 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30ec0b5f20 0x7f30ec06dad0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f30dc00c700 tx=0x7f30dc00cbc0 comp rx=0 tx=0).stop 2024-04-26T23:19:29.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.737+0000 7f30f0df0640 1 -- 172.21.15.47:0/5997819 shutdown_connections 2024-04-26T23:19:29.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.737+0000 7f30f0df0640 1 --2- 172.21.15.47:0/5997819 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f30cc06c8f0 0x7f30cc06edb0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:29.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.737+0000 7f30f0df0640 1 --2- 172.21.15.47:0/5997819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30ec162880 0x7f30ec09f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:29.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.737+0000 7f30f0df0640 1 --2- 172.21.15.47:0/5997819 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f30ec0d3bc0 0x7f30ec09f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:29.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.737+0000 7f30f0df0640 1 --2- 172.21.15.47:0/5997819 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30ec0b5f20 0x7f30ec06dad0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:29.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.737+0000 7f30f0df0640 1 -- 172.21.15.47:0/5997819 >> 172.21.15.47:0/5997819 conn(0x7f30ec0d4f60 msgr2=0x7f30ec0d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:29.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.737+0000 7f30f0df0640 1 -- 172.21.15.47:0/5997819 shutdown_connections 2024-04-26T23:19:29.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.737+0000 7f30f0df0640 1 -- 172.21.15.47:0/5997819 wait complete. 2024-04-26T23:19:29.749 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:19:29.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.851+0000 7f0022209640 1 Processor -- start 2024-04-26T23:19:29.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.851+0000 7f0022209640 1 -- start start 2024-04-26T23:19:29.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.851+0000 7f0022209640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f001c15d1e0 0x7f001c0d09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:29.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.851+0000 7f0022209640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f001c0d0f20 0x7f001c0d1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:29.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.851+0000 7f0022209640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f001c0d18c0 0x7f001c0cf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:29.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.852+0000 7f0022209640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f001c0cede0 con 0x7f001c15d1e0 2024-04-26T23:19:29.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.852+0000 7f0022209640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f001c070650 con 0x7f001c0d18c0 2024-04-26T23:19:29.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.852+0000 7f0022209640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f001c0ce840 con 0x7f001c0d0f20 2024-04-26T23:19:29.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.852+0000 7f001b7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f001c15d1e0 0x7f001c0d09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:29.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.852+0000 7f001b7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f001c15d1e0 0x7f001c0d09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33756/0 (socket says 172.21.15.47:33756) 2024-04-26T23:19:29.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.852+0000 7f001b7fe640 1 -- 172.21.15.47:0/2503069671 learned_addr learned my addr 172.21.15.47:0/2503069671 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:29.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.852+0000 7f001bfff640 1 --2- 172.21.15.47:0/2503069671 >> v2:172.21.15.203:3300/0 conn(0x7f001c0d18c0 0x7f001c0cf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:29.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.852+0000 7f001affd640 1 --2- 172.21.15.47:0/2503069671 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f001c0d0f20 0x7f001c0d1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:29.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.852+0000 7f001b7fe640 1 -- 172.21.15.47:0/2503069671 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f001c0d0f20 msgr2=0x7f001c0d1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:29.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.852+0000 7f001b7fe640 1 --2- 172.21.15.47:0/2503069671 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f001c0d0f20 0x7f001c0d1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:29.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.852+0000 7f001b7fe640 1 -- 172.21.15.47:0/2503069671 >> v2:172.21.15.203:3300/0 conn(0x7f001c0d18c0 msgr2=0x7f001c0cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:29.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.852+0000 7f001b7fe640 1 --2- 172.21.15.47:0/2503069671 >> v2:172.21.15.203:3300/0 conn(0x7f001c0d18c0 0x7f001c0cf5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:29.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.852+0000 7f001b7fe640 1 -- 172.21.15.47:0/2503069671 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f001c0ceb00 con 0x7f001c15d1e0 2024-04-26T23:19:29.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.853+0000 7f001b7fe640 1 --2- 172.21.15.47:0/2503069671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f001c15d1e0 0x7f001c0d09e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f001c0c6720 tx=0x7f000c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8267ec86e04098a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:29.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.853+0000 7f001a7fc640 1 -- 172.21.15.47:0/2503069671 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000c037070 con 0x7f001c15d1e0 2024-04-26T23:19:29.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.853+0000 7f001a7fc640 1 -- 172.21.15.47:0/2503069671 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f000c032440 con 0x7f001c15d1e0 2024-04-26T23:19:29.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.853+0000 7f001a7fc640 1 -- 172.21.15.47:0/2503069671 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000c032810 con 0x7f001c15d1e0 2024-04-26T23:19:29.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.853+0000 7f0022209640 1 -- 172.21.15.47:0/2503069671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f001c15d1e0 msgr2=0x7f001c0d09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:29.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.853+0000 7f0022209640 1 --2- 172.21.15.47:0/2503069671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f001c15d1e0 0x7f001c0d09e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f001c0c6720 tx=0x7f000c02a1e0 comp rx=0 tx=0).stop 2024-04-26T23:19:29.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.853+0000 7f0022209640 1 -- 172.21.15.47:0/2503069671 shutdown_connections 2024-04-26T23:19:29.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.853+0000 7f0022209640 1 --2- 172.21.15.47:0/2503069671 >> v2:172.21.15.203:3300/0 conn(0x7f001c0d18c0 0x7f001c0cf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:29.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.853+0000 7f0022209640 1 --2- 172.21.15.47:0/2503069671 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f001c0d0f20 0x7f001c0d1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:29.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.853+0000 7f0022209640 1 --2- 172.21.15.47:0/2503069671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f001c15d1e0 0x7f001c0d09e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:29.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.853+0000 7f0022209640 1 -- 172.21.15.47:0/2503069671 >> 172.21.15.47:0/2503069671 conn(0x7f001c0d2da0 msgr2=0x7f001c0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:29.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.853+0000 7f0022209640 1 -- 172.21.15.47:0/2503069671 shutdown_connections 2024-04-26T23:19:29.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.853+0000 7f0022209640 1 -- 172.21.15.47:0/2503069671 wait complete. 2024-04-26T23:19:29.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.855+0000 7f0022209640 1 Processor -- start 2024-04-26T23:19:29.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.855+0000 7f0022209640 1 -- start start 2024-04-26T23:19:29.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.856+0000 7f0022209640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f001c0d0f20 0x7f001c0c7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:29.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.856+0000 7f0022209640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f001c0d18c0 0x7f001c1f4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:29.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.856+0000 7f0022209640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f001c15d1e0 0x7f001c151290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:29.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.856+0000 7f0022209640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f001c0cfb20 con 0x7f001c0d0f20 2024-04-26T23:19:29.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.856+0000 7f0022209640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f001c0ce4c0 con 0x7f001c15d1e0 2024-04-26T23:19:29.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.856+0000 7f0022209640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f001c0cfca0 con 0x7f001c0d18c0 2024-04-26T23:19:29.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.856+0000 7f001b7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f001c0d0f20 0x7f001c0c7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:29.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.856+0000 7f001b7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f001c0d0f20 0x7f001c0c7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33762/0 (socket says 172.21.15.47:33762) 2024-04-26T23:19:29.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.856+0000 7f001b7fe640 1 -- 172.21.15.47:0/4125127702 learned_addr learned my addr 172.21.15.47:0/4125127702 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:29.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.856+0000 7f001bfff640 1 --2- 172.21.15.47:0/4125127702 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f001c15d1e0 0x7f001c151290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:29.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.856+0000 7f001affd640 1 --2- 172.21.15.47:0/4125127702 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f001c0d18c0 0x7f001c1f4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:29.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.856+0000 7f001b7fe640 1 -- 172.21.15.47:0/4125127702 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f001c0d18c0 msgr2=0x7f001c1f4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:29.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.856+0000 7f001b7fe640 1 --2- 172.21.15.47:0/4125127702 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f001c0d18c0 0x7f001c1f4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:29.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.856+0000 7f001b7fe640 1 -- 172.21.15.47:0/4125127702 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f001c15d1e0 msgr2=0x7f001c151290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:29.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.856+0000 7f001b7fe640 1 --2- 172.21.15.47:0/4125127702 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f001c15d1e0 0x7f001c151290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:29.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.856+0000 7f001b7fe640 1 -- 172.21.15.47:0/4125127702 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f001c1f5690 con 0x7f001c0d0f20 2024-04-26T23:19:29.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.856+0000 7f001b7fe640 1 --2- 172.21.15.47:0/4125127702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f001c0d0f20 0x7f001c0c7e00 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f000c02a220 tx=0x7f000c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:29.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.856+0000 7f0003fff640 1 -- 172.21.15.47:0/4125127702 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000c041020 con 0x7f001c0d0f20 2024-04-26T23:19:29.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.856+0000 7f0003fff640 1 -- 172.21.15.47:0/4125127702 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f000c03d620 con 0x7f001c0d0f20 2024-04-26T23:19:29.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.856+0000 7f0022209640 1 -- 172.21.15.47:0/4125127702 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f001c1f5970 con 0x7f001c0d0f20 2024-04-26T23:19:29.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.856+0000 7f0003fff640 1 -- 172.21.15.47:0/4125127702 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000c037040 con 0x7f001c0d0f20 2024-04-26T23:19:29.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.856+0000 7f0022209640 1 -- 172.21.15.47:0/4125127702 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f001c0ca490 con 0x7f001c0d0f20 2024-04-26T23:19:29.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.857+0000 7f0022209640 1 -- 172.21.15.47:0/4125127702 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f001c0694f0 con 0x7f001c0d0f20 2024-04-26T23:19:29.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.858+0000 7f0003fff640 1 -- 172.21.15.47:0/4125127702 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f000c044020 con 0x7f001c0d0f20 2024-04-26T23:19:29.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.858+0000 7f0003fff640 1 --2- 172.21.15.47:0/4125127702 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f000806c8f0 0x7f000806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:29.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.859+0000 7f0003fff640 1 -- 172.21.15.47:0/4125127702 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f000c0ac450 con 0x7f001c0d0f20 2024-04-26T23:19:29.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.859+0000 7f001affd640 1 --2- 172.21.15.47:0/4125127702 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f000806c8f0 0x7f000806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:29.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.859+0000 7f001affd640 1 --2- 172.21.15.47:0/4125127702 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f000806c8f0 0x7f000806edb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f001c06e230 tx=0x7f0010009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:29.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:29.863+0000 7f0003fff640 1 -- 172.21.15.47:0/4125127702 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f000c04b030 con 0x7f001c0d0f20 2024-04-26T23:19:30.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.049+0000 7f0022209640 1 -- 172.21.15.47:0/4125127702 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f001c0c8340 con 0x7f001c0d0f20 2024-04-26T23:19:30.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.050+0000 7f0003fff640 1 -- 172.21.15.47:0/4125127702 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18857 (secure 0 0 0) 0x7f000c079ea0 con 0x7f001c0d0f20 2024-04-26T23:19:30.051 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:19:30.051 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":26,"fsid":"399006d3-4db1-43e6-ba81-a79a276eabfd","created":"2024-04-26T23:18:06.731211+0000","modified":"2024-04-26T23:19:12.124051+0000","last_up_change":"2024-04-26T23:18:17.808736+0000","last_in_change":"2024-04-26T23:18:08.945859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:18:16.937492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:18:29.033575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:18:30.318499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:18:34.014536+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ee091567-fea0-4a14-bf4e-0e16c20bb2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6819","nonce":326520779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6821","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6823","nonce":326520779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6829","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6831","nonce":326520779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6825","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6827","nonce":326520779}]},"public_addr":"172.21.15.47:6819/326520779","cluster_addr":"172.21.15.47:6823/326520779","heartbeat_back_addr":"172.21.15.47:6831/326520779","heartbeat_front_addr":"172.21.15.47:6827/326520779","state":["exists","up"]},{"osd":1,"uuid":"c14440a0-bba5-4931-a6db-b736c08e869b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6809","nonce":1912610931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6811","nonce":1912610931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6815","nonce":1912610931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6813","nonce":1912610931}]},"public_addr":"172.21.15.47:6809/1912610931","cluster_addr":"172.21.15.47:6811/1912610931","heartbeat_back_addr":"172.21.15.47:6815/1912610931","heartbeat_front_addr":"172.21.15.47:6813/1912610931","state":["exists","up"]},{"osd":2,"uuid":"3940f8de-c9b2-4cd3-a911-9c301729eb30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6801","nonce":2569869305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6803","nonce":2569869305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6807","nonce":2569869305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6805","nonce":2569869305}]},"public_addr":"172.21.15.47:6801/2569869305","cluster_addr":"172.21.15.47:6803/2569869305","heartbeat_back_addr":"172.21.15.47:6807/2569869305","heartbeat_front_addr":"172.21.15.47:6805/2569869305","state":["exists","up"]},{"osd":3,"uuid":"1286a67e-8958-426a-8c2e-bc85498c619c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6818","nonce":2791411683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6822","nonce":2791411683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6830","nonce":2791411683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6826","nonce":2791411683}]},"public_addr":"172.21.15.47:6818/2791411683","cluster_addr":"172.21.15.47:6822/2791411683","heartbeat_back_addr":"172.21.15.47:6830/2791411683","heartbeat_front_addr":"172.21.15.47:6826/2791411683","state":["exists","up"]},{"osd":4,"uuid":"d8ee2c20-e5b7-46c2-9999-b234070126cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6809","nonce":441313104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6811","nonce":441313104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6815","nonce":441313104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6813","nonce":441313104}]},"public_addr":"172.21.15.203:6809/441313104","cluster_addr":"172.21.15.203:6811/441313104","heartbeat_back_addr":"172.21.15.203:6815/441313104","heartbeat_front_addr":"172.21.15.203:6813/441313104","state":["exists","up"]},{"osd":5,"uuid":"b453a8f0-f324-4840-9ae0-d67bdadb9f8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6801","nonce":56838444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6803","nonce":56838444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6807","nonce":56838444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6805","nonce":56838444}]},"public_addr":"172.21.15.203:6801/56838444","cluster_addr":"172.21.15.203:6803/56838444","heartbeat_back_addr":"172.21.15.203:6807/56838444","heartbeat_front_addr":"172.21.15.203:6805/56838444","state":["exists","up"]},{"osd":6,"uuid":"a6d31951-2d4b-4e8d-a510-6283b01dc158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6825","nonce":2738517080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6827","nonce":2738517080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6831","nonce":2738517080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6829","nonce":2738517080}]},"public_addr":"172.21.15.203:6825/2738517080","cluster_addr":"172.21.15.203:6827/2738517080","heartbeat_back_addr":"172.21.15.203:6831/2738517080","heartbeat_front_addr":"172.21.15.203:6829/2738517080","state":["exists","up"]},{"osd":7,"uuid":"7119b8c0-fe96-402d-9e75-7b509d9d44d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6817","nonce":1315565058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6819","nonce":1315565058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6823","nonce":1315565058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6821","nonce":1315565058}]},"public_addr":"172.21.15.203:6817/1315565058","cluster_addr":"172.21.15.203:6819/1315565058","heartbeat_back_addr":"172.21.15.203:6823/1315565058","heartbeat_front_addr":"172.21.15.203:6821/1315565058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.121921+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.137494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.084814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.111593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.072430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.125066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.097867+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.057061+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":5,"to":1}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:19:30.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.052+0000 7f0022209640 1 -- 172.21.15.47:0/4125127702 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f000806c8f0 msgr2=0x7f000806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:30.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.052+0000 7f0022209640 1 --2- 172.21.15.47:0/4125127702 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f000806c8f0 0x7f000806edb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f001c06e230 tx=0x7f0010009190 comp rx=0 tx=0).stop 2024-04-26T23:19:30.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.052+0000 7f0022209640 1 -- 172.21.15.47:0/4125127702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f001c0d0f20 msgr2=0x7f001c0c7e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:30.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.052+0000 7f0022209640 1 --2- 172.21.15.47:0/4125127702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f001c0d0f20 0x7f001c0c7e00 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f000c02a220 tx=0x7f000c0046c0 comp rx=0 tx=0).stop 2024-04-26T23:19:30.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.052+0000 7f0022209640 1 -- 172.21.15.47:0/4125127702 shutdown_connections 2024-04-26T23:19:30.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.052+0000 7f0022209640 1 --2- 172.21.15.47:0/4125127702 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f000806c8f0 0x7f000806edb0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:30.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.052+0000 7f0022209640 1 --2- 172.21.15.47:0/4125127702 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f001c15d1e0 0x7f001c151290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:30.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.052+0000 7f0022209640 1 --2- 172.21.15.47:0/4125127702 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f001c0d18c0 0x7f001c1f4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:30.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.052+0000 7f0022209640 1 --2- 172.21.15.47:0/4125127702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f001c0d0f20 0x7f001c0c7e00 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:30.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.052+0000 7f0022209640 1 -- 172.21.15.47:0/4125127702 >> 172.21.15.47:0/4125127702 conn(0x7f001c0d2da0 msgr2=0x7f001c0b3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:30.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.052+0000 7f0022209640 1 -- 172.21.15.47:0/4125127702 shutdown_connections 2024-04-26T23:19:30.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.052+0000 7f0022209640 1 -- 172.21.15.47:0/4125127702 wait complete. 2024-04-26T23:19:30.064 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:18:16.937492+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:18:29.033575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:18:30.318499+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T23:18:34.014536+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T23:19:30.064 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:19:30.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.167+0000 7f2bae5da640 1 Processor -- start 2024-04-26T23:19:30.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.167+0000 7f2bae5da640 1 -- start start 2024-04-26T23:19:30.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.168+0000 7f2bae5da640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2ba815f1d0 0x7f2ba806f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:30.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.168+0000 7f2bae5da640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2ba80afd60 0x7f2ba806c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:30.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.168+0000 7f2bae5da640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ba80b4480 con 0x7f2ba806c240 2024-04-26T23:19:30.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.168+0000 7f2bae5da640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f2ba806cbe0 con 0x7f2ba80afd60 2024-04-26T23:19:30.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.168+0000 7f2bae5da640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f2ba80ae8c0 con 0x7f2ba815f1d0 2024-04-26T23:19:30.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.168+0000 7f2bacb77640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2ba80afd60 0x7f2ba806c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:30.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.168+0000 7f2bacb77640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2ba80afd60 0x7f2ba806c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33770/0 (socket says 172.21.15.47:33770) 2024-04-26T23:19:30.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.168+0000 7f2bacb77640 1 -- 172.21.15.47:0/274871112 learned_addr learned my addr 172.21.15.47:0/274871112 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:30.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.168+0000 7f2ba7fff640 1 --2- 172.21.15.47:0/274871112 >> v2:172.21.15.203:3300/0 conn(0x7f2ba815f1d0 0x7f2ba806f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:30.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.168+0000 7f2bacb77640 1 -- 172.21.15.47:0/274871112 >> v2:172.21.15.203:3300/0 conn(0x7f2ba815f1d0 msgr2=0x7f2ba806f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:30.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.168+0000 7f2bacb77640 1 --2- 172.21.15.47:0/274871112 >> v2:172.21.15.203:3300/0 conn(0x7f2ba815f1d0 0x7f2ba806f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:30.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.168+0000 7f2bacb77640 1 -- 172.21.15.47:0/274871112 >> v1:172.21.15.47:6789/0 conn(0x7f2ba806c240 legacy=0x7f2ba80726e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:30.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.168+0000 7f2bacb77640 1 -- 172.21.15.47:0/274871112 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ba80b4720 con 0x7f2ba80afd60 2024-04-26T23:19:30.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.168+0000 7f2ba6ffd640 1 -- 172.21.15.47:0/274871112 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1592784473 0 0) 0x7f2ba80b4480 con 0x7f2ba806c240 2024-04-26T23:19:30.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.168+0000 7f2ba7fff640 1 --2- 172.21.15.47:0/274871112 >> v2:172.21.15.203:3300/0 conn(0x7f2ba815f1d0 0x7f2ba806f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:19:30.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.169+0000 7f2bacb77640 1 --2- 172.21.15.47:0/274871112 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ba80afd60 0x7f2ba806c6a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2ba806cfb0 tx=0x7f2ba002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=353ec5b0834f306f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:30.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.169+0000 7f2ba6ffd640 1 -- 172.21.15.47:0/274871112 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba0036070 con 0x7f2ba80afd60 2024-04-26T23:19:30.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.169+0000 7f2ba6ffd640 1 -- 172.21.15.47:0/274871112 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ba0031600 con 0x7f2ba80afd60 2024-04-26T23:19:30.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.169+0000 7f2ba6ffd640 1 -- 172.21.15.47:0/274871112 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba00319d0 con 0x7f2ba80afd60 2024-04-26T23:19:30.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.169+0000 7f2bae5da640 1 -- 172.21.15.47:0/274871112 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ba80afd60 msgr2=0x7f2ba806c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:30.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.169+0000 7f2bae5da640 1 --2- 172.21.15.47:0/274871112 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ba80afd60 0x7f2ba806c6a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2ba806cfb0 tx=0x7f2ba002a730 comp rx=0 tx=0).stop 2024-04-26T23:19:30.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.170+0000 7f2bae5da640 1 -- 172.21.15.47:0/274871112 shutdown_connections 2024-04-26T23:19:30.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.170+0000 7f2bae5da640 1 --2- 172.21.15.47:0/274871112 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ba80afd60 0x7f2ba806c6a0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:30.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.170+0000 7f2bae5da640 1 --2- 172.21.15.47:0/274871112 >> v2:172.21.15.203:3300/0 conn(0x7f2ba815f1d0 0x7f2ba806f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:30.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.170+0000 7f2bae5da640 1 -- 172.21.15.47:0/274871112 >> 172.21.15.47:0/274871112 conn(0x7f2ba80b3180 msgr2=0x7f2ba80b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:30.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.170+0000 7f2bae5da640 1 -- 172.21.15.47:0/274871112 shutdown_connections 2024-04-26T23:19:30.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.170+0000 7f2bae5da640 1 -- 172.21.15.47:0/274871112 wait complete. 2024-04-26T23:19:30.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.171+0000 7f2bae5da640 1 Processor -- start 2024-04-26T23:19:30.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.171+0000 7f2bae5da640 1 -- start start 2024-04-26T23:19:30.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.172+0000 7f2bae5da640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ba806c240 0x7f2ba81614c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:30.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.172+0000 7f2bae5da640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ba80afd60 0x7f2ba8161a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:30.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.172+0000 7f2bae5da640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ba815f1d0 0x7f2ba814d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:30.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.172+0000 7f2bae5da640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba80b4fc0 con 0x7f2ba80afd60 2024-04-26T23:19:30.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.172+0000 7f2bae5da640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba80b4ce0 con 0x7f2ba806c240 2024-04-26T23:19:30.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.172+0000 7f2bae5da640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f2ba806f6b0 con 0x7f2ba815f1d0 2024-04-26T23:19:30.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.172+0000 7f2ba77fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ba80afd60 0x7f2ba8161a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:30.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.172+0000 7f2ba77fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ba80afd60 0x7f2ba8161a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33772/0 (socket says 172.21.15.47:33772) 2024-04-26T23:19:30.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.172+0000 7f2ba77fe640 1 -- 172.21.15.47:0/2836428133 learned_addr learned my addr 172.21.15.47:0/2836428133 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:30.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.172+0000 7f2ba7fff640 1 --2- 172.21.15.47:0/2836428133 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ba806c240 0x7f2ba81614c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:30.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.172+0000 7f2bacb77640 1 --2- 172.21.15.47:0/2836428133 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ba815f1d0 0x7f2ba814d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:30.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.172+0000 7f2ba77fe640 1 -- 172.21.15.47:0/2836428133 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ba815f1d0 msgr2=0x7f2ba814d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:30.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.172+0000 7f2ba77fe640 1 --2- 172.21.15.47:0/2836428133 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ba815f1d0 0x7f2ba814d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:30.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.172+0000 7f2ba77fe640 1 -- 172.21.15.47:0/2836428133 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ba806c240 msgr2=0x7f2ba81614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:30.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.172+0000 7f2ba77fe640 1 --2- 172.21.15.47:0/2836428133 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ba806c240 0x7f2ba81614c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:30.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.172+0000 7f2ba77fe640 1 -- 172.21.15.47:0/2836428133 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ba8161f40 con 0x7f2ba80afd60 2024-04-26T23:19:30.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.172+0000 7f2ba77fe640 1 --2- 172.21.15.47:0/2836428133 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ba80afd60 0x7f2ba8161a00 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2b9800c410 tx=0x7f2b9800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:30.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.172+0000 7f2ba4ff9640 1 -- 172.21.15.47:0/2836428133 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b98017070 con 0x7f2ba80afd60 2024-04-26T23:19:30.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.172+0000 7f2ba4ff9640 1 -- 172.21.15.47:0/2836428133 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b980035f0 con 0x7f2ba80afd60 2024-04-26T23:19:30.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.172+0000 7f2bae5da640 1 -- 172.21.15.47:0/2836428133 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ba8162210 con 0x7f2ba80afd60 2024-04-26T23:19:30.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.172+0000 7f2ba4ff9640 1 -- 172.21.15.47:0/2836428133 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b980039e0 con 0x7f2ba80afd60 2024-04-26T23:19:30.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.172+0000 7f2bae5da640 1 -- 172.21.15.47:0/2836428133 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ba814bc10 con 0x7f2ba80afd60 2024-04-26T23:19:30.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.173+0000 7f2bae5da640 1 -- 172.21.15.47:0/2836428133 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ba8068fc0 con 0x7f2ba80afd60 2024-04-26T23:19:30.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.174+0000 7f2ba4ff9640 1 -- 172.21.15.47:0/2836428133 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2b9801a070 con 0x7f2ba80afd60 2024-04-26T23:19:30.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.174+0000 7f2ba4ff9640 1 --2- 172.21.15.47:0/2836428133 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f2b8806c8f0 0x7f2b8806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:30.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.174+0000 7f2ba4ff9640 1 -- 172.21.15.47:0/2836428133 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f2b9808b400 con 0x7f2ba80afd60 2024-04-26T23:19:30.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.174+0000 7f2ba7fff640 1 --2- 172.21.15.47:0/2836428133 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f2b8806c8f0 0x7f2b8806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:30.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.175+0000 7f2ba7fff640 1 --2- 172.21.15.47:0/2836428133 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f2b8806c8f0 0x7f2b8806edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2b90006fd0 tx=0x7f2b90008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:30.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.178+0000 7f2ba4ff9640 1 -- 172.21.15.47:0/2836428133 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2b98059410 con 0x7f2ba80afd60 2024-04-26T23:19:30.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.366+0000 7f2bae5da640 1 -- 172.21.15.47:0/2836428133 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2ba80694f0 con 0x7f2ba80afd60 2024-04-26T23:19:30.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.366+0000 7f2ba4ff9640 1 -- 172.21.15.47:0/2836428133 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f2b98058d10 con 0x7f2ba80afd60 2024-04-26T23:19:30.367 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-26T23:19:30.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.368+0000 7f2bae5da640 1 -- 172.21.15.47:0/2836428133 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f2b8806c8f0 msgr2=0x7f2b8806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:30.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.368+0000 7f2bae5da640 1 --2- 172.21.15.47:0/2836428133 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f2b8806c8f0 0x7f2b8806edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2b90006fd0 tx=0x7f2b90008040 comp rx=0 tx=0).stop 2024-04-26T23:19:30.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.368+0000 7f2bae5da640 1 -- 172.21.15.47:0/2836428133 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ba80afd60 msgr2=0x7f2ba8161a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:30.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.368+0000 7f2bae5da640 1 --2- 172.21.15.47:0/2836428133 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ba80afd60 0x7f2ba8161a00 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2b9800c410 tx=0x7f2b9800c8d0 comp rx=0 tx=0).stop 2024-04-26T23:19:30.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.368+0000 7f2bae5da640 1 -- 172.21.15.47:0/2836428133 shutdown_connections 2024-04-26T23:19:30.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.368+0000 7f2bae5da640 1 --2- 172.21.15.47:0/2836428133 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f2b8806c8f0 0x7f2b8806edb0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:30.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.368+0000 7f2bae5da640 1 --2- 172.21.15.47:0/2836428133 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ba815f1d0 0x7f2ba814d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:30.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.368+0000 7f2bae5da640 1 --2- 172.21.15.47:0/2836428133 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ba80afd60 0x7f2ba8161a00 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:30.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.368+0000 7f2bae5da640 1 --2- 172.21.15.47:0/2836428133 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ba806c240 0x7f2ba81614c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:30.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.368+0000 7f2bae5da640 1 -- 172.21.15.47:0/2836428133 >> 172.21.15.47:0/2836428133 conn(0x7f2ba80b3180 msgr2=0x7f2ba80b14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:30.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.368+0000 7f2bae5da640 1 -- 172.21.15.47:0/2836428133 shutdown_connections 2024-04-26T23:19:30.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.368+0000 7f2bae5da640 1 -- 172.21.15.47:0/2836428133 wait complete. 2024-04-26T23:19:30.379 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:19:30.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.482+0000 7fbf98eb0640 1 Processor -- start 2024-04-26T23:19:30.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.483+0000 7fbf98eb0640 1 -- start start 2024-04-26T23:19:30.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.483+0000 7fbf98eb0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf94156460 0x7fbf940cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:30.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.483+0000 7fbf98eb0640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbf940cba10 0x7fbf9406f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:30.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.483+0000 7fbf98eb0640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbf9406fcd0 0x7fbf9415f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:30.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.483+0000 7fbf98eb0640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fbf94156840 con 0x7fbf9406fcd0 2024-04-26T23:19:30.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.483+0000 7fbf98eb0640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fbf940cbf20 con 0x7fbf940cba10 2024-04-26T23:19:30.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.483+0000 7fbf98eb0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fbf940cc200 con 0x7fbf94156460 2024-04-26T23:19:30.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.484+0000 7fbf92d9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbf9406fcd0 0x7fbf9415f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:30.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.484+0000 7fbf92d9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbf9406fcd0 0x7fbf9415f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33786/0 (socket says 172.21.15.47:33786) 2024-04-26T23:19:30.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.484+0000 7fbf92d9d640 1 -- 172.21.15.47:0/2786541320 learned_addr learned my addr 172.21.15.47:0/2786541320 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:30.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.484+0000 7fbf91d9b640 1 --2- 172.21.15.47:0/2786541320 >> v2:172.21.15.203:3300/0 conn(0x7fbf940cba10 0x7fbf9406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:30.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.484+0000 7fbf9259c640 1 --2- 172.21.15.47:0/2786541320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf94156460 0x7fbf940cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:30.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.484+0000 7fbf92d9d640 1 -- 172.21.15.47:0/2786541320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf94156460 msgr2=0x7fbf940cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:30.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.484+0000 7fbf92d9d640 1 --2- 172.21.15.47:0/2786541320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf94156460 0x7fbf940cb4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:30.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.484+0000 7fbf92d9d640 1 -- 172.21.15.47:0/2786541320 >> v2:172.21.15.203:3300/0 conn(0x7fbf940cba10 msgr2=0x7fbf9406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:30.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.484+0000 7fbf92d9d640 1 --2- 172.21.15.47:0/2786541320 >> v2:172.21.15.203:3300/0 conn(0x7fbf940cba10 0x7fbf9406f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:30.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.484+0000 7fbf92d9d640 1 -- 172.21.15.47:0/2786541320 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf940cc4e0 con 0x7fbf9406fcd0 2024-04-26T23:19:30.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.484+0000 7fbf91d9b640 1 --2- 172.21.15.47:0/2786541320 >> v2:172.21.15.203:3300/0 conn(0x7fbf940cba10 0x7fbf9406f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:19:30.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.485+0000 7fbf92d9d640 1 --2- 172.21.15.47:0/2786541320 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf9406fcd0 0x7fbf9415f980 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fbf94070320 tx=0x7fbf8c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=159b8b1857241666 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:30.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.485+0000 7fbf9159a640 1 -- 172.21.15.47:0/2786541320 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c036070 con 0x7fbf9406fcd0 2024-04-26T23:19:30.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.485+0000 7fbf9159a640 1 -- 172.21.15.47:0/2786541320 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf8c031600 con 0x7fbf9406fcd0 2024-04-26T23:19:30.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.485+0000 7fbf9159a640 1 -- 172.21.15.47:0/2786541320 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c0319d0 con 0x7fbf9406fcd0 2024-04-26T23:19:30.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.485+0000 7fbf98eb0640 1 -- 172.21.15.47:0/2786541320 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf9406fcd0 msgr2=0x7fbf9415f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:30.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.485+0000 7fbf98eb0640 1 --2- 172.21.15.47:0/2786541320 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf9406fcd0 0x7fbf9415f980 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fbf94070320 tx=0x7fbf8c02a6a0 comp rx=0 tx=0).stop 2024-04-26T23:19:30.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.485+0000 7fbf98eb0640 1 -- 172.21.15.47:0/2786541320 shutdown_connections 2024-04-26T23:19:30.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.485+0000 7fbf98eb0640 1 --2- 172.21.15.47:0/2786541320 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf9406fcd0 0x7fbf9415f980 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:30.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.485+0000 7fbf98eb0640 1 --2- 172.21.15.47:0/2786541320 >> v2:172.21.15.203:3300/0 conn(0x7fbf940cba10 0x7fbf9406f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:30.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.485+0000 7fbf98eb0640 1 --2- 172.21.15.47:0/2786541320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf94156460 0x7fbf940cb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:30.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.485+0000 7fbf98eb0640 1 -- 172.21.15.47:0/2786541320 >> 172.21.15.47:0/2786541320 conn(0x7fbf940d0be0 msgr2=0x7fbf940d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:30.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.486+0000 7fbf98eb0640 1 -- 172.21.15.47:0/2786541320 shutdown_connections 2024-04-26T23:19:30.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.486+0000 7fbf98eb0640 1 -- 172.21.15.47:0/2786541320 wait complete. 2024-04-26T23:19:30.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.487+0000 7fbf98eb0640 1 Processor -- start 2024-04-26T23:19:30.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.487+0000 7fbf98eb0640 1 -- start start 2024-04-26T23:19:30.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.487+0000 7fbf98eb0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf9406fcd0 0x7fbf9414e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:30.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.487+0000 7fbf98eb0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf940cba10 0x7fbf94155550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:30.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.487+0000 7fbf98eb0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf94156460 0x7fbf9406e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:30.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.488+0000 7fbf98eb0640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9406e260 con 0x7fbf940cba10 2024-04-26T23:19:30.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.488+0000 7fbf98eb0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbf94072700 con 0x7fbf9406fcd0 2024-04-26T23:19:30.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.488+0000 7fbf98eb0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fbf9406e3e0 con 0x7fbf94156460 2024-04-26T23:19:30.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.488+0000 7fbf91d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf940cba10 0x7fbf94155550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:30.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.488+0000 7fbf91d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf940cba10 0x7fbf94155550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33790/0 (socket says 172.21.15.47:33790) 2024-04-26T23:19:30.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.488+0000 7fbf91d9b640 1 -- 172.21.15.47:0/1329584293 learned_addr learned my addr 172.21.15.47:0/1329584293 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:30.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.488+0000 7fbf92d9d640 1 --2- 172.21.15.47:0/1329584293 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf94156460 0x7fbf9406e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:30.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.488+0000 7fbf9259c640 1 --2- 172.21.15.47:0/1329584293 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf9406fcd0 0x7fbf9414e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:30.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.488+0000 7fbf91d9b640 1 -- 172.21.15.47:0/1329584293 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf94156460 msgr2=0x7fbf9406e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:30.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.488+0000 7fbf91d9b640 1 --2- 172.21.15.47:0/1329584293 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf94156460 0x7fbf9406e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:30.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.488+0000 7fbf91d9b640 1 -- 172.21.15.47:0/1329584293 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf9406fcd0 msgr2=0x7fbf9414e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:30.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.488+0000 7fbf91d9b640 1 --2- 172.21.15.47:0/1329584293 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf9406fcd0 0x7fbf9414e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:30.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.488+0000 7fbf91d9b640 1 -- 172.21.15.47:0/1329584293 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf94153750 con 0x7fbf940cba10 2024-04-26T23:19:30.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.488+0000 7fbf92d9d640 1 --2- 172.21.15.47:0/1329584293 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf94156460 0x7fbf9406e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:19:30.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.488+0000 7fbf91d9b640 1 --2- 172.21.15.47:0/1329584293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf940cba10 0x7fbf94155550 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fbf8800e7e0 tx=0x7fbf8800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:30.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.488+0000 7fbf82ffd640 1 -- 172.21.15.47:0/1329584293 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8800f040 con 0x7fbf940cba10 2024-04-26T23:19:30.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.488+0000 7fbf82ffd640 1 -- 172.21.15.47:0/1329584293 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf88004030 con 0x7fbf940cba10 2024-04-26T23:19:30.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.488+0000 7fbf98eb0640 1 -- 172.21.15.47:0/1329584293 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf94153a30 con 0x7fbf940cba10 2024-04-26T23:19:30.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.488+0000 7fbf82ffd640 1 -- 172.21.15.47:0/1329584293 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf880043e0 con 0x7fbf940cba10 2024-04-26T23:19:30.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.488+0000 7fbf98eb0640 1 -- 172.21.15.47:0/1329584293 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf94153ff0 con 0x7fbf940cba10 2024-04-26T23:19:30.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.489+0000 7fbf98eb0640 1 -- 172.21.15.47:0/1329584293 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf940694f0 con 0x7fbf940cba10 2024-04-26T23:19:30.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.490+0000 7fbf82ffd640 1 -- 172.21.15.47:0/1329584293 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbf8800c6d0 con 0x7fbf940cba10 2024-04-26T23:19:30.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.490+0000 7fbf82ffd640 1 --2- 172.21.15.47:0/1329584293 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fbf7406c8f0 0x7fbf7406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:30.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.490+0000 7fbf82ffd640 1 -- 172.21.15.47:0/1329584293 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7fbf8808bac0 con 0x7fbf940cba10 2024-04-26T23:19:30.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.490+0000 7fbf9259c640 1 --2- 172.21.15.47:0/1329584293 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fbf7406c8f0 0x7fbf7406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:30.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.491+0000 7fbf9259c640 1 --2- 172.21.15.47:0/1329584293 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fbf7406c8f0 0x7fbf7406edb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbf840079a0 tx=0x7fbf84008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:30.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.494+0000 7fbf82ffd640 1 -- 172.21.15.47:0/1329584293 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbf88017020 con 0x7fbf940cba10 2024-04-26T23:19:30.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.678+0000 7fbf98eb0640 1 -- 172.21.15.47:0/1329584293 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbf94073ee0 con 0x7fbf940cba10 2024-04-26T23:19:30.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.679+0000 7fbf82ffd640 1 -- 172.21.15.47:0/1329584293 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fbf88059510 con 0x7fbf940cba10 2024-04-26T23:19:30.680 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-26T23:19:30.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.680+0000 7fbf98eb0640 1 -- 172.21.15.47:0/1329584293 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fbf7406c8f0 msgr2=0x7fbf7406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:30.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.680+0000 7fbf98eb0640 1 --2- 172.21.15.47:0/1329584293 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fbf7406c8f0 0x7fbf7406edb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbf840079a0 tx=0x7fbf84008040 comp rx=0 tx=0).stop 2024-04-26T23:19:30.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.680+0000 7fbf98eb0640 1 -- 172.21.15.47:0/1329584293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf940cba10 msgr2=0x7fbf94155550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:30.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.680+0000 7fbf98eb0640 1 --2- 172.21.15.47:0/1329584293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf940cba10 0x7fbf94155550 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fbf8800e7e0 tx=0x7fbf8800eca0 comp rx=0 tx=0).stop 2024-04-26T23:19:30.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.680+0000 7fbf98eb0640 1 -- 172.21.15.47:0/1329584293 shutdown_connections 2024-04-26T23:19:30.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.680+0000 7fbf98eb0640 1 --2- 172.21.15.47:0/1329584293 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fbf7406c8f0 0x7fbf7406edb0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:30.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.680+0000 7fbf98eb0640 1 --2- 172.21.15.47:0/1329584293 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf94156460 0x7fbf9406e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:30.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.680+0000 7fbf98eb0640 1 --2- 172.21.15.47:0/1329584293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf940cba10 0x7fbf94155550 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:30.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.680+0000 7fbf98eb0640 1 --2- 172.21.15.47:0/1329584293 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf9406fcd0 0x7fbf9414e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:30.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.680+0000 7fbf98eb0640 1 -- 172.21.15.47:0/1329584293 >> 172.21.15.47:0/1329584293 conn(0x7fbf940d0be0 msgr2=0x7fbf940b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:30.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.681+0000 7fbf98eb0640 1 -- 172.21.15.47:0/1329584293 shutdown_connections 2024-04-26T23:19:30.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.681+0000 7fbf98eb0640 1 -- 172.21.15.47:0/1329584293 wait complete. 2024-04-26T23:19:30.692 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:19:30.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.795+0000 7f8b252a9640 1 Processor -- start 2024-04-26T23:19:30.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.795+0000 7f8b252a9640 1 -- start start 2024-04-26T23:19:30.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.796+0000 7f8b252a9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8b200d1a00 0x7f8b200d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:30.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.796+0000 7f8b252a9640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8b200d2320 0x7f8b201625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:30.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.796+0000 7f8b252a9640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b200d0a10 con 0x7f8b200b40d0 2024-04-26T23:19:30.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.796+0000 7f8b252a9640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f8b200d0cf0 con 0x7f8b200d2320 2024-04-26T23:19:30.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.796+0000 7f8b252a9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f8b200d0fd0 con 0x7f8b200d1a00 2024-04-26T23:19:30.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.796+0000 7f8b1f7fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f8b200b40d0 0x7f8b200d2ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40910/0 (socket says 172.21.15.47:40910) 2024-04-26T23:19:30.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.796+0000 7f8b1f7fe640 1 -- 172.21.15.47:0/2290650460 learned_addr learned my addr 172.21.15.47:0/2290650460 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:30.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.796+0000 7f8b1e7fc640 1 --2- 172.21.15.47:0/2290650460 >> v2:172.21.15.203:3300/0 conn(0x7f8b200d2320 0x7f8b201625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:30.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.796+0000 7f8b1effd640 1 --2- 172.21.15.47:0/2290650460 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8b200d1a00 0x7f8b200d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:30.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.796+0000 7f8b1dffb640 1 -- 172.21.15.47:0/2290650460 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4218302448 0 0) 0x7f8b200d0a10 con 0x7f8b200b40d0 2024-04-26T23:19:30.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.796+0000 7f8b1dffb640 1 -- 172.21.15.47:0/2290650460 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8b08003520 con 0x7f8b200b40d0 2024-04-26T23:19:30.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.796+0000 7f8b1dffb640 1 -- 172.21.15.47:0/2290650460 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3820678933 0 0) 0x7f8b08003520 con 0x7f8b200b40d0 2024-04-26T23:19:30.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.796+0000 7f8b1dffb640 1 -- 172.21.15.47:0/2290650460 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8b200d1a00 msgr2=0x7f8b200d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:30.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.796+0000 7f8b1dffb640 1 --2- 172.21.15.47:0/2290650460 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8b200d1a00 0x7f8b200d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:30.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.796+0000 7f8b1dffb640 1 -- 172.21.15.47:0/2290650460 >> v2:172.21.15.203:3300/0 conn(0x7f8b200d2320 msgr2=0x7f8b201625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:30.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.796+0000 7f8b1dffb640 1 --2- 172.21.15.47:0/2290650460 >> v2:172.21.15.203:3300/0 conn(0x7f8b200d2320 0x7f8b201625f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:30.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.796+0000 7f8b1dffb640 1 -- 172.21.15.47:0/2290650460 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b200d40a0 con 0x7f8b200b40d0 2024-04-26T23:19:30.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.796+0000 7f8b1dffb640 1 -- 172.21.15.47:0/2290650460 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7f8b180019f0 con 0x7f8b200b40d0 2024-04-26T23:19:30.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.796+0000 7f8b1e7fc640 1 --2- 172.21.15.47:0/2290650460 >> v2:172.21.15.203:3300/0 conn(0x7f8b200d2320 0x7f8b201625f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:19:30.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.796+0000 7f8b1effd640 1 --2- 172.21.15.47:0/2290650460 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8b200d1a00 0x7f8b200d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:19:30.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.797+0000 7f8b1dffb640 1 -- 172.21.15.47:0/2290650460 >> v1:172.21.15.47:6789/0 conn(0x7f8b200b40d0 legacy=0x7f8b200d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:30.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.797+0000 7f8b1dffb640 1 --2- 172.21.15.47:0/2290650460 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8b200d1a00 0x7f8b080039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:30.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.797+0000 7f8b1dffb640 1 --2- 172.21.15.47:0/2290650460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b08003ef0 0x7f8b08004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:30.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.797+0000 7f8b1dffb640 1 --2- 172.21.15.47:0/2290650460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b08004d40 0x7f8b080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:30.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.797+0000 7f8b1dffb640 1 -- 172.21.15.47:0/2290650460 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8b200d0cf0 con 0x7f8b08004d40 2024-04-26T23:19:30.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.797+0000 7f8b1dffb640 1 -- 172.21.15.47:0/2290650460 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8b200d0fd0 con 0x7f8b08003ef0 2024-04-26T23:19:30.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.797+0000 7f8b1dffb640 1 -- 172.21.15.47:0/2290650460 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f8b0800b9c0 con 0x7f8b200d1a00 2024-04-26T23:19:30.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.797+0000 7f8b1f7fe640 1 --2- 172.21.15.47:0/2290650460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b08004d40 0x7f8b080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:30.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.797+0000 7f8b1e7fc640 1 --2- 172.21.15.47:0/2290650460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b08003ef0 0x7f8b08004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:30.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.797+0000 7f8b1effd640 1 --2- 172.21.15.47:0/2290650460 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8b200d1a00 0x7f8b080039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:30.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.797+0000 7f8b1f7fe640 1 -- 172.21.15.47:0/2290650460 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8b200d1a00 msgr2=0x7f8b080039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:30.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.797+0000 7f8b1f7fe640 1 --2- 172.21.15.47:0/2290650460 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8b200d1a00 0x7f8b080039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:30.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.797+0000 7f8b1f7fe640 1 -- 172.21.15.47:0/2290650460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b08003ef0 msgr2=0x7f8b08004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:30.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.797+0000 7f8b1f7fe640 1 --2- 172.21.15.47:0/2290650460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b08003ef0 0x7f8b08004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:30.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.797+0000 7f8b1f7fe640 1 -- 172.21.15.47:0/2290650460 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b08003520 con 0x7f8b08004d40 2024-04-26T23:19:30.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.798+0000 7f8b1f7fe640 1 --2- 172.21.15.47:0/2290650460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b08004d40 0x7f8b080051c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8b1800a8a0 tx=0x7f8b08005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c77518f913a4000f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:30.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.798+0000 7f8b1dffb640 1 -- 172.21.15.47:0/2290650460 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b1800a4c0 con 0x7f8b08004d40 2024-04-26T23:19:30.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.798+0000 7f8b1dffb640 1 -- 172.21.15.47:0/2290650460 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b1802f070 con 0x7f8b08004d40 2024-04-26T23:19:30.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.798+0000 7f8b1dffb640 1 -- 172.21.15.47:0/2290650460 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b1802bd70 con 0x7f8b08004d40 2024-04-26T23:19:30.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.798+0000 7f8b252a9640 1 -- 172.21.15.47:0/2290650460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b08004d40 msgr2=0x7f8b080051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:30.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.798+0000 7f8b252a9640 1 --2- 172.21.15.47:0/2290650460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b08004d40 0x7f8b080051c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8b1800a8a0 tx=0x7f8b08005830 comp rx=0 tx=0).stop 2024-04-26T23:19:30.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.798+0000 7f8b1f7fe640 1 -- 172.21.15.47:0/2290650460 reap_dead start 2024-04-26T23:19:30.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.798+0000 7f8b252a9640 1 -- 172.21.15.47:0/2290650460 shutdown_connections 2024-04-26T23:19:30.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.798+0000 7f8b252a9640 1 -- 172.21.15.47:0/2290650460 >> 172.21.15.47:0/2290650460 conn(0x7f8b200d4fd0 msgr2=0x7f8b200d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:30.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.798+0000 7f8b252a9640 1 -- 172.21.15.47:0/2290650460 shutdown_connections 2024-04-26T23:19:30.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.798+0000 7f8b252a9640 1 -- 172.21.15.47:0/2290650460 wait complete. 2024-04-26T23:19:30.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.800+0000 7f8b252a9640 1 Processor -- start 2024-04-26T23:19:30.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.800+0000 7f8b252a9640 1 -- start start 2024-04-26T23:19:30.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.800+0000 7f8b252a9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8b201f4db0 0x7f8b20160ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:30.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.800+0000 7f8b252a9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b20160ff0 0x7f8b200cc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:30.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.800+0000 7f8b252a9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b200cc5e0 0x7f8b200c9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:30.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.800+0000 7f8b252a9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8b200d1de0 con 0x7f8b20160ff0 2024-04-26T23:19:30.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.800+0000 7f8b252a9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8b200b5d20 con 0x7f8b200cc5e0 2024-04-26T23:19:30.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.800+0000 7f8b252a9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f8b200d1f60 con 0x7f8b201f4db0 2024-04-26T23:19:30.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.801+0000 7f8b1e7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b20160ff0 0x7f8b200cc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:30.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.801+0000 7f8b1e7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b20160ff0 0x7f8b200cc0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33808/0 (socket says 172.21.15.47:33808) 2024-04-26T23:19:30.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.801+0000 7f8b1e7fc640 1 -- 172.21.15.47:0/1139070949 learned_addr learned my addr 172.21.15.47:0/1139070949 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:30.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.801+0000 7f8b1f7fe640 1 --2- 172.21.15.47:0/1139070949 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b200cc5e0 0x7f8b200c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:30.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.801+0000 7f8b1effd640 1 --2- 172.21.15.47:0/1139070949 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8b201f4db0 0x7f8b20160ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:30.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.801+0000 7f8b1e7fc640 1 -- 172.21.15.47:0/1139070949 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8b201f4db0 msgr2=0x7f8b20160ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:30.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.801+0000 7f8b1e7fc640 1 --2- 172.21.15.47:0/1139070949 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8b201f4db0 0x7f8b20160ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:30.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.801+0000 7f8b1e7fc640 1 -- 172.21.15.47:0/1139070949 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b200cc5e0 msgr2=0x7f8b200c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:30.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.801+0000 7f8b1e7fc640 1 --2- 172.21.15.47:0/1139070949 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b200cc5e0 0x7f8b200c9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:30.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.801+0000 7f8b1e7fc640 1 -- 172.21.15.47:0/1139070949 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b200ccd90 con 0x7f8b20160ff0 2024-04-26T23:19:30.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.801+0000 7f8b1f7fe640 1 --2- 172.21.15.47:0/1139070949 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b200cc5e0 0x7f8b200c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:19:30.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.801+0000 7f8b1e7fc640 1 --2- 172.21.15.47:0/1139070949 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b20160ff0 0x7f8b200cc0a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8b14011d00 tx=0x7f8b1400f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:30.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.801+0000 7f8b077fe640 1 -- 172.21.15.47:0/1139070949 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b1400fd20 con 0x7f8b20160ff0 2024-04-26T23:19:30.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.801+0000 7f8b077fe640 1 -- 172.21.15.47:0/1139070949 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b14004030 con 0x7f8b20160ff0 2024-04-26T23:19:30.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.801+0000 7f8b252a9640 1 -- 172.21.15.47:0/1139070949 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b200cd070 con 0x7f8b20160ff0 2024-04-26T23:19:30.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.801+0000 7f8b077fe640 1 -- 172.21.15.47:0/1139070949 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b14004420 con 0x7f8b20160ff0 2024-04-26T23:19:30.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.801+0000 7f8b252a9640 1 -- 172.21.15.47:0/1139070949 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b200cd630 con 0x7f8b20160ff0 2024-04-26T23:19:30.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.802+0000 7f8b252a9640 1 -- 172.21.15.47:0/1139070949 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b20161470 con 0x7f8b20160ff0 2024-04-26T23:19:30.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.803+0000 7f8b077fe640 1 -- 172.21.15.47:0/1139070949 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8b14012040 con 0x7f8b20160ff0 2024-04-26T23:19:30.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.803+0000 7f8b077fe640 1 --2- 172.21.15.47:0/1139070949 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f8b0c06c920 0x7f8b0c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:30.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.804+0000 7f8b077fe640 1 -- 172.21.15.47:0/1139070949 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f8b1408c4a0 con 0x7f8b20160ff0 2024-04-26T23:19:30.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.804+0000 7f8b1effd640 1 --2- 172.21.15.47:0/1139070949 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f8b0c06c920 0x7f8b0c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:30.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.804+0000 7f8b1effd640 1 --2- 172.21.15.47:0/1139070949 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f8b0c06c920 0x7f8b0c06ede0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8b100099f0 tx=0x7f8b10008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:30.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.808+0000 7f8b077fe640 1 -- 172.21.15.47:0/1139070949 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8b14019070 con 0x7f8b20160ff0 2024-04-26T23:19:30.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.991+0000 7f8b252a9640 1 -- 172.21.15.47:0/1139070949 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8b200694f0 con 0x7f8b20160ff0 2024-04-26T23:19:30.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.992+0000 7f8b077fe640 1 -- 172.21.15.47:0/1139070949 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f8b1405a4b0 con 0x7f8b20160ff0 2024-04-26T23:19:30.993 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-26T23:19:30.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.994+0000 7f8b252a9640 1 -- 172.21.15.47:0/1139070949 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f8b0c06c920 msgr2=0x7f8b0c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:30.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.994+0000 7f8b252a9640 1 --2- 172.21.15.47:0/1139070949 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f8b0c06c920 0x7f8b0c06ede0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8b100099f0 tx=0x7f8b10008040 comp rx=0 tx=0).stop 2024-04-26T23:19:30.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.994+0000 7f8b252a9640 1 -- 172.21.15.47:0/1139070949 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b20160ff0 msgr2=0x7f8b200cc0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:30.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.994+0000 7f8b252a9640 1 --2- 172.21.15.47:0/1139070949 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b20160ff0 0x7f8b200cc0a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8b14011d00 tx=0x7f8b1400f610 comp rx=0 tx=0).stop 2024-04-26T23:19:30.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.994+0000 7f8b252a9640 1 -- 172.21.15.47:0/1139070949 shutdown_connections 2024-04-26T23:19:30.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.994+0000 7f8b252a9640 1 --2- 172.21.15.47:0/1139070949 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f8b0c06c920 0x7f8b0c06ede0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:30.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.994+0000 7f8b252a9640 1 --2- 172.21.15.47:0/1139070949 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b200cc5e0 0x7f8b200c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:30.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.994+0000 7f8b252a9640 1 --2- 172.21.15.47:0/1139070949 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b20160ff0 0x7f8b200cc0a0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:30.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.994+0000 7f8b252a9640 1 --2- 172.21.15.47:0/1139070949 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8b201f4db0 0x7f8b20160ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:30.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.994+0000 7f8b252a9640 1 -- 172.21.15.47:0/1139070949 >> 172.21.15.47:0/1139070949 conn(0x7f8b200d4fd0 msgr2=0x7f8b200d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:30.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.994+0000 7f8b252a9640 1 -- 172.21.15.47:0/1139070949 shutdown_connections 2024-04-26T23:19:30.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:30.994+0000 7f8b252a9640 1 -- 172.21.15.47:0/1139070949 wait complete. 2024-04-26T23:19:31.005 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T23:19:31.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.108+0000 7fedf455f640 1 Processor -- start 2024-04-26T23:19:31.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.108+0000 7fedf455f640 1 -- start start 2024-04-26T23:19:31.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.109+0000 7fedf455f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedec15f170 0x7fedec0d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:31.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.109+0000 7fedf455f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fedec15f550 0x7fedec0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:31.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.109+0000 7fedf455f640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fedec0d34b0 0x7fedec0d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:31.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.109+0000 7fedf455f640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fedec0c9430 con 0x7fedec0d34b0 2024-04-26T23:19:31.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.109+0000 7fedf455f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fedec0d3e20 con 0x7fedec15f550 2024-04-26T23:19:31.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.109+0000 7fedf455f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fedec0d1040 con 0x7fedec15f170 2024-04-26T23:19:31.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.109+0000 7fedf2afc640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fedec0d34b0 0x7fedec0d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:31.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.109+0000 7fedf2afc640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fedec0d34b0 0x7fedec0d38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33822/0 (socket says 172.21.15.47:33822) 2024-04-26T23:19:31.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.109+0000 7fedf2afc640 1 -- 172.21.15.47:0/2579788141 learned_addr learned my addr 172.21.15.47:0/2579788141 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:31.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.110+0000 7fedf22fb640 1 --2- 172.21.15.47:0/2579788141 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedec15f170 0x7fedec0d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:31.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.110+0000 7fedf1afa640 1 --2- 172.21.15.47:0/2579788141 >> v2:172.21.15.203:3300/0 conn(0x7fedec15f550 0x7fedec0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:31.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.110+0000 7fedf2afc640 1 -- 172.21.15.47:0/2579788141 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedec15f170 msgr2=0x7fedec0d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:31.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.110+0000 7fedf2afc640 1 --2- 172.21.15.47:0/2579788141 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedec15f170 0x7fedec0d5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:31.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.110+0000 7fedf2afc640 1 -- 172.21.15.47:0/2579788141 >> v2:172.21.15.203:3300/0 conn(0x7fedec15f550 msgr2=0x7fedec0d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:31.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.110+0000 7fedf2afc640 1 --2- 172.21.15.47:0/2579788141 >> v2:172.21.15.203:3300/0 conn(0x7fedec15f550 0x7fedec0d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:31.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.110+0000 7fedf2afc640 1 -- 172.21.15.47:0/2579788141 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedec0c9680 con 0x7fedec0d34b0 2024-04-26T23:19:31.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.110+0000 7fedf1afa640 1 --2- 172.21.15.47:0/2579788141 >> v2:172.21.15.203:3300/0 conn(0x7fedec15f550 0x7fedec0d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:19:31.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.110+0000 7fedf22fb640 1 --2- 172.21.15.47:0/2579788141 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedec15f170 0x7fedec0d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:19:31.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.110+0000 7fedf2afc640 1 --2- 172.21.15.47:0/2579788141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fedec0d34b0 0x7fedec0d38e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fedec08e960 tx=0x7fede802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c985769d0c49ff95 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:31.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.110+0000 7fedf12f9640 1 -- 172.21.15.47:0/2579788141 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede8036070 con 0x7fedec0d34b0 2024-04-26T23:19:31.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.111+0000 7fedf12f9640 1 -- 172.21.15.47:0/2579788141 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fede8031600 con 0x7fedec0d34b0 2024-04-26T23:19:31.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.111+0000 7fedf12f9640 1 -- 172.21.15.47:0/2579788141 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede80319d0 con 0x7fedec0d34b0 2024-04-26T23:19:31.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.111+0000 7fedf455f640 1 -- 172.21.15.47:0/2579788141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fedec0d34b0 msgr2=0x7fedec0d38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:31.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.111+0000 7fedf455f640 1 --2- 172.21.15.47:0/2579788141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fedec0d34b0 0x7fedec0d38e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fedec08e960 tx=0x7fede802a6a0 comp rx=0 tx=0).stop 2024-04-26T23:19:31.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.111+0000 7fedf455f640 1 -- 172.21.15.47:0/2579788141 shutdown_connections 2024-04-26T23:19:31.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.111+0000 7fedf455f640 1 --2- 172.21.15.47:0/2579788141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fedec0d34b0 0x7fedec0d38e0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:31.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.111+0000 7fedf455f640 1 --2- 172.21.15.47:0/2579788141 >> v2:172.21.15.203:3300/0 conn(0x7fedec15f550 0x7fedec0d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:31.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.111+0000 7fedf455f640 1 --2- 172.21.15.47:0/2579788141 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedec15f170 0x7fedec0d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:31.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.111+0000 7fedf455f640 1 -- 172.21.15.47:0/2579788141 >> 172.21.15.47:0/2579788141 conn(0x7fedec0d4f60 msgr2=0x7fedec0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:31.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.111+0000 7fedf455f640 1 -- 172.21.15.47:0/2579788141 shutdown_connections 2024-04-26T23:19:31.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.111+0000 7fedf455f640 1 -- 172.21.15.47:0/2579788141 wait complete. 2024-04-26T23:19:31.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.112+0000 7fedf455f640 1 Processor -- start 2024-04-26T23:19:31.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.112+0000 7fedf455f640 1 -- start start 2024-04-26T23:19:31.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.113+0000 7fedf455f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fedec0d34b0 0x7fedec0b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:31.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.113+0000 7fedf455f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fedec15f170 0x7fedec0cda10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:31.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.113+0000 7fedf455f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedec151220 0x7fedec0cdf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:31.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.113+0000 7fedf455f640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fedec0cfc50 con 0x7fedec15f170 2024-04-26T23:19:31.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.113+0000 7fedf455f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fedec0c9c40 con 0x7fedec0d34b0 2024-04-26T23:19:31.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.113+0000 7fedf455f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fedec0cfdd0 con 0x7fedec151220 2024-04-26T23:19:31.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.113+0000 7fedf1afa640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fedec15f170 0x7fedec0cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:31.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.113+0000 7fedf1afa640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fedec15f170 0x7fedec0cda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33838/0 (socket says 172.21.15.47:33838) 2024-04-26T23:19:31.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.113+0000 7fedf1afa640 1 -- 172.21.15.47:0/351268778 learned_addr learned my addr 172.21.15.47:0/351268778 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:31.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.113+0000 7fedf22fb640 1 --2- 172.21.15.47:0/351268778 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fedec0d34b0 0x7fedec0b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:31.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.113+0000 7fedf2afc640 1 --2- 172.21.15.47:0/351268778 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedec151220 0x7fedec0cdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:31.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.114+0000 7fedf1afa640 1 -- 172.21.15.47:0/351268778 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedec151220 msgr2=0x7fedec0cdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:31.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.114+0000 7fedf1afa640 1 --2- 172.21.15.47:0/351268778 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedec151220 0x7fedec0cdf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:31.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.114+0000 7fedf1afa640 1 -- 172.21.15.47:0/351268778 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fedec0d34b0 msgr2=0x7fedec0b1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:31.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.114+0000 7fedf1afa640 1 --2- 172.21.15.47:0/351268778 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fedec0d34b0 0x7fedec0b1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:31.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.114+0000 7fedf1afa640 1 -- 172.21.15.47:0/351268778 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedec0b2090 con 0x7fedec15f170 2024-04-26T23:19:31.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.114+0000 7fedf1afa640 1 --2- 172.21.15.47:0/351268778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fedec15f170 0x7fedec0cda10 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7feddc00c7f0 tx=0x7feddc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:31.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.114+0000 7fede2ffd640 1 -- 172.21.15.47:0/351268778 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feddc015070 con 0x7fedec15f170 2024-04-26T23:19:31.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.114+0000 7fede2ffd640 1 -- 172.21.15.47:0/351268778 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feddc007820 con 0x7fedec15f170 2024-04-26T23:19:31.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.114+0000 7fedf455f640 1 -- 172.21.15.47:0/351268778 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedec09f9c0 con 0x7fedec15f170 2024-04-26T23:19:31.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.114+0000 7fede2ffd640 1 -- 172.21.15.47:0/351268778 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feddc007bd0 con 0x7fedec15f170 2024-04-26T23:19:31.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.114+0000 7fedf455f640 1 -- 172.21.15.47:0/351268778 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedec151bd0 con 0x7fedec15f170 2024-04-26T23:19:31.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.115+0000 7fedf455f640 1 -- 172.21.15.47:0/351268778 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedec0694f0 con 0x7fedec15f170 2024-04-26T23:19:31.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.116+0000 7fede2ffd640 1 -- 172.21.15.47:0/351268778 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feddc018070 con 0x7fedec15f170 2024-04-26T23:19:31.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.116+0000 7fede2ffd640 1 --2- 172.21.15.47:0/351268778 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fedd006c8f0 0x7fedd006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:31.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.116+0000 7fede2ffd640 1 -- 172.21.15.47:0/351268778 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7feddc08bff0 con 0x7fedec15f170 2024-04-26T23:19:31.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.116+0000 7fedf22fb640 1 --2- 172.21.15.47:0/351268778 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fedd006c8f0 0x7fedd006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:31.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.117+0000 7fedf22fb640 1 --2- 172.21.15.47:0/351268778 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fedd006c8f0 0x7fedd006edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fede4006b10 tx=0x7fede4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:31.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.121+0000 7fede2ffd640 1 -- 172.21.15.47:0/351268778 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7feddc05a090 con 0x7fedec15f170 2024-04-26T23:19:31.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.309+0000 7fedf455f640 1 -- 172.21.15.47:0/351268778 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fedec0b6e20 con 0x7fedec15f170 2024-04-26T23:19:31.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.310+0000 7fede2ffd640 1 -- 172.21.15.47:0/351268778 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7feddc059990 con 0x7fedec15f170 2024-04-26T23:19:31.311 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-26T23:19:31.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.311+0000 7fedf455f640 1 -- 172.21.15.47:0/351268778 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fedd006c8f0 msgr2=0x7fedd006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:31.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.311+0000 7fedf455f640 1 --2- 172.21.15.47:0/351268778 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fedd006c8f0 0x7fedd006edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fede4006b10 tx=0x7fede4008040 comp rx=0 tx=0).stop 2024-04-26T23:19:31.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.311+0000 7fedf455f640 1 -- 172.21.15.47:0/351268778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fedec15f170 msgr2=0x7fedec0cda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:31.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.311+0000 7fedf455f640 1 --2- 172.21.15.47:0/351268778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fedec15f170 0x7fedec0cda10 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7feddc00c7f0 tx=0x7feddc00ccb0 comp rx=0 tx=0).stop 2024-04-26T23:19:31.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.311+0000 7fedf455f640 1 -- 172.21.15.47:0/351268778 shutdown_connections 2024-04-26T23:19:31.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.311+0000 7fedf455f640 1 --2- 172.21.15.47:0/351268778 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fedd006c8f0 0x7fedd006edb0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:31.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.311+0000 7fedf455f640 1 --2- 172.21.15.47:0/351268778 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedec151220 0x7fedec0cdf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:31.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.311+0000 7fedf455f640 1 --2- 172.21.15.47:0/351268778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fedec15f170 0x7fedec0cda10 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:31.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.311+0000 7fedf455f640 1 --2- 172.21.15.47:0/351268778 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fedec0d34b0 0x7fedec0b1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:31.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.311+0000 7fedf455f640 1 -- 172.21.15.47:0/351268778 >> 172.21.15.47:0/351268778 conn(0x7fedec0d4f60 msgr2=0x7fedec06d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:31.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.311+0000 7fedf455f640 1 -- 172.21.15.47:0/351268778 shutdown_connections 2024-04-26T23:19:31.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.311+0000 7fedf455f640 1 -- 172.21.15.47:0/351268778 wait complete. 2024-04-26T23:19:31.323 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:19:31.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.422+0000 7f5b1562a640 1 Processor -- start 2024-04-26T23:19:31.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.422+0000 7f5b1562a640 1 -- start start 2024-04-26T23:19:31.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.423+0000 7f5b1562a640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5b1015f5a0 0x7f5b100d5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:31.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.423+0000 7f5b1562a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b10163610 0x7f5b100cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:31.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.423+0000 7f5b1562a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b100d1a70 con 0x7f5b100d2830 2024-04-26T23:19:31.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.423+0000 7f5b1562a640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f5b10163d30 con 0x7f5b1015f5a0 2024-04-26T23:19:31.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.423+0000 7f5b1562a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f5b100d1d50 con 0x7f5b10163610 2024-04-26T23:19:31.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.423+0000 7f5b0effd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5b1015f5a0 0x7f5b100d5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:31.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.423+0000 7f5b0effd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5b1015f5a0 0x7f5b100d5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33852/0 (socket says 172.21.15.47:33852) 2024-04-26T23:19:31.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.423+0000 7f5b0effd640 1 -- 172.21.15.47:0/1102876978 learned_addr learned my addr 172.21.15.47:0/1102876978 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:31.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.424+0000 7f5b0effd640 1 -- 172.21.15.47:0/1102876978 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b10163610 msgr2=0x7f5b100cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:31.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.424+0000 7f5b0effd640 1 --2- 172.21.15.47:0/1102876978 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b10163610 0x7f5b100cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:31.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.424+0000 7f5b0effd640 1 -- 172.21.15.47:0/1102876978 >> v1:172.21.15.203:6789/0 conn(0x7f5b100d2830 legacy=0x7f5b10162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:31.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.424+0000 7f5b0effd640 1 -- 172.21.15.47:0/1102876978 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b100d2030 con 0x7f5b1015f5a0 2024-04-26T23:19:31.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.424+0000 7f5b0effd640 1 --2- 172.21.15.47:0/1102876978 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b1015f5a0 0x7f5b100d5cb0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f5b08009870 tx=0x7f5b0802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93d715654c218308 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:31.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.425+0000 7f5b0dffb640 1 -- 172.21.15.47:0/1102876978 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b08037070 con 0x7f5b1015f5a0 2024-04-26T23:19:31.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.425+0000 7f5b0dffb640 1 -- 172.21.15.47:0/1102876978 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b08032440 con 0x7f5b1015f5a0 2024-04-26T23:19:31.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.425+0000 7f5b0dffb640 1 -- 172.21.15.47:0/1102876978 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b08032810 con 0x7f5b1015f5a0 2024-04-26T23:19:31.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.425+0000 7f5b1562a640 1 -- 172.21.15.47:0/1102876978 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b1015f5a0 msgr2=0x7f5b100d5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:31.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.425+0000 7f5b1562a640 1 --2- 172.21.15.47:0/1102876978 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b1015f5a0 0x7f5b100d5cb0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f5b08009870 tx=0x7f5b0802a180 comp rx=0 tx=0).stop 2024-04-26T23:19:31.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.425+0000 7f5b1562a640 1 -- 172.21.15.47:0/1102876978 shutdown_connections 2024-04-26T23:19:31.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.425+0000 7f5b1562a640 1 --2- 172.21.15.47:0/1102876978 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b10163610 0x7f5b100cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:31.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.425+0000 7f5b1562a640 1 --2- 172.21.15.47:0/1102876978 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b1015f5a0 0x7f5b100d5cb0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:31.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.425+0000 7f5b1562a640 1 -- 172.21.15.47:0/1102876978 >> 172.21.15.47:0/1102876978 conn(0x7f5b100d5190 msgr2=0x7f5b100d55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:31.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.425+0000 7f5b1562a640 1 -- 172.21.15.47:0/1102876978 shutdown_connections 2024-04-26T23:19:31.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.425+0000 7f5b1562a640 1 -- 172.21.15.47:0/1102876978 wait complete. 2024-04-26T23:19:31.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.426+0000 7f5b1562a640 1 Processor -- start 2024-04-26T23:19:31.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.426+0000 7f5b1562a640 1 -- start start 2024-04-26T23:19:31.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.427+0000 7f5b1562a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b100d2830 0x7f5b1014fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:31.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.427+0000 7f5b1562a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b10163610 0x7f5b1014e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:31.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.427+0000 7f5b1562a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b101511f0 0x7f5b100a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:31.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.427+0000 7f5b1562a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5b100d3c20 con 0x7f5b101511f0 2024-04-26T23:19:31.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.427+0000 7f5b1562a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5b100d4190 con 0x7f5b10163610 2024-04-26T23:19:31.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.427+0000 7f5b1562a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f5b100d3da0 con 0x7f5b100d2830 2024-04-26T23:19:31.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.427+0000 7f5b0f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b101511f0 0x7f5b100a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:31.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.427+0000 7f5b0f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b101511f0 0x7f5b100a2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33856/0 (socket says 172.21.15.47:33856) 2024-04-26T23:19:31.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.427+0000 7f5b0f7fe640 1 -- 172.21.15.47:0/1652580236 learned_addr learned my addr 172.21.15.47:0/1652580236 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:31.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.428+0000 7f5b0effd640 1 --2- 172.21.15.47:0/1652580236 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b100d2830 0x7f5b1014fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:31.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.428+0000 7f5b0e7fc640 1 --2- 172.21.15.47:0/1652580236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b10163610 0x7f5b1014e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:31.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.428+0000 7f5b0f7fe640 1 -- 172.21.15.47:0/1652580236 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b100d2830 msgr2=0x7f5b1014fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:31.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.428+0000 7f5b0f7fe640 1 --2- 172.21.15.47:0/1652580236 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b100d2830 0x7f5b1014fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:31.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.428+0000 7f5b0f7fe640 1 -- 172.21.15.47:0/1652580236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b10163610 msgr2=0x7f5b1014e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:31.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.428+0000 7f5b0f7fe640 1 --2- 172.21.15.47:0/1652580236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b10163610 0x7f5b1014e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:31.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.428+0000 7f5b0f7fe640 1 -- 172.21.15.47:0/1652580236 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b1014eda0 con 0x7f5b101511f0 2024-04-26T23:19:31.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.428+0000 7f5b0effd640 1 --2- 172.21.15.47:0/1652580236 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b100d2830 0x7f5b1014fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:19:31.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.428+0000 7f5b0e7fc640 1 --2- 172.21.15.47:0/1652580236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b10163610 0x7f5b1014e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:19:31.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.428+0000 7f5b0f7fe640 1 --2- 172.21.15.47:0/1652580236 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b101511f0 0x7f5b100a2610 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5b0000ba10 tx=0x7f5b0000bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:31.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.428+0000 7f5af77fe640 1 -- 172.21.15.47:0/1652580236 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b00019070 con 0x7f5b101511f0 2024-04-26T23:19:31.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.428+0000 7f5af77fe640 1 -- 172.21.15.47:0/1652580236 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b0000cb60 con 0x7f5b101511f0 2024-04-26T23:19:31.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.428+0000 7f5b1562a640 1 -- 172.21.15.47:0/1652580236 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b1014f080 con 0x7f5b101511f0 2024-04-26T23:19:31.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.428+0000 7f5af77fe640 1 -- 172.21.15.47:0/1652580236 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b00014440 con 0x7f5b101511f0 2024-04-26T23:19:31.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.428+0000 7f5b1562a640 1 -- 172.21.15.47:0/1652580236 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b1014f640 con 0x7f5b101511f0 2024-04-26T23:19:31.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.429+0000 7f5b1562a640 1 -- 172.21.15.47:0/1652580236 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b100694f0 con 0x7f5b101511f0 2024-04-26T23:19:31.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.430+0000 7f5af77fe640 1 -- 172.21.15.47:0/1652580236 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5b0001c070 con 0x7f5b101511f0 2024-04-26T23:19:31.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.430+0000 7f5af77fe640 1 --2- 172.21.15.47:0/1652580236 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f5afc06c8f0 0x7f5afc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:31.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.430+0000 7f5af77fe640 1 -- 172.21.15.47:0/1652580236 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f5b0008bc40 con 0x7f5b101511f0 2024-04-26T23:19:31.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.430+0000 7f5b0effd640 1 --2- 172.21.15.47:0/1652580236 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f5afc06c8f0 0x7f5afc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:31.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.431+0000 7f5b0effd640 1 --2- 172.21.15.47:0/1652580236 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f5afc06c8f0 0x7f5afc06edb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5b08034000 tx=0x7f5b0802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:31.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.435+0000 7f5af77fe640 1 -- 172.21.15.47:0/1652580236 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5b00059c50 con 0x7f5b101511f0 2024-04-26T23:19:31.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.662+0000 7f5b1562a640 1 -- 172.21.15.47:0/1652580236 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b100bb110 con 0x7f5b101511f0 2024-04-26T23:19:31.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.662+0000 7f5af77fe640 1 -- 172.21.15.47:0/1652580236 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5180 (secure 0 0 0) 0x7f5b00059550 con 0x7f5b101511f0 2024-04-26T23:19:31.663 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:19:31.663 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/1553696401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1553696401},{"type":"v1","addr":"172.21.15.47:6834","nonce":1553696401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/3792767564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3792767564},{"type":"v1","addr":"172.21.15.203:6836","nonce":3792767564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4088100766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":4088100766},{"type":"v1","addr":"172.21.15.47:6835","nonce":4088100766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:18:31.367292+0000","modified":"2024-04-26T23:18:53.430267+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/997115011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":997115011},{"type":"v1","addr":"172.21.15.203:6837","nonce":997115011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4282,"qdb_cluster":[4282]},"id":1}]} 2024-04-26T23:19:31.664 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 9 2024-04-26T23:19:31.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.664+0000 7f5b1562a640 1 -- 172.21.15.47:0/1652580236 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f5afc06c8f0 msgr2=0x7f5afc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:31.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.664+0000 7f5b1562a640 1 --2- 172.21.15.47:0/1652580236 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f5afc06c8f0 0x7f5afc06edb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5b08034000 tx=0x7f5b0802e000 comp rx=0 tx=0).stop 2024-04-26T23:19:31.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.664+0000 7f5b1562a640 1 -- 172.21.15.47:0/1652580236 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b101511f0 msgr2=0x7f5b100a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:31.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.664+0000 7f5b1562a640 1 --2- 172.21.15.47:0/1652580236 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b101511f0 0x7f5b100a2610 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5b0000ba10 tx=0x7f5b0000bed0 comp rx=0 tx=0).stop 2024-04-26T23:19:31.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.665+0000 7f5b1562a640 1 -- 172.21.15.47:0/1652580236 shutdown_connections 2024-04-26T23:19:31.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.665+0000 7f5b1562a640 1 --2- 172.21.15.47:0/1652580236 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f5afc06c8f0 0x7f5afc06edb0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:31.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.665+0000 7f5b1562a640 1 --2- 172.21.15.47:0/1652580236 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b101511f0 0x7f5b100a2610 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:31.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.665+0000 7f5b1562a640 1 --2- 172.21.15.47:0/1652580236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b10163610 0x7f5b1014e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:31.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.665+0000 7f5b1562a640 1 --2- 172.21.15.47:0/1652580236 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b100d2830 0x7f5b1014fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:31.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.665+0000 7f5b1562a640 1 -- 172.21.15.47:0/1652580236 >> 172.21.15.47:0/1652580236 conn(0x7f5b100d5190 msgr2=0x7f5b10160f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:31.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.665+0000 7f5b1562a640 1 -- 172.21.15.47:0/1652580236 shutdown_connections 2024-04-26T23:19:31.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.665+0000 7f5b1562a640 1 -- 172.21.15.47:0/1652580236 wait complete. 2024-04-26T23:19:31.676 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:19:31.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.777+0000 7f513d9af640 1 Processor -- start 2024-04-26T23:19:31.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.777+0000 7f513d9af640 1 -- start start 2024-04-26T23:19:31.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.778+0000 7f513d9af640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f51380b2340 0x7f51380cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:31.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.778+0000 7f513d9af640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51380d10d0 con 0x7f51380cd930 2024-04-26T23:19:31.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.778+0000 7f513d9af640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51380d1370 con 0x7f5138162810 2024-04-26T23:19:31.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.778+0000 7f513d9af640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f51380d0750 con 0x7f51380b2340 2024-04-26T23:19:31.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.778+0000 7f51377fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f51380cd930 0x7f51380d1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40916/0 (socket says 172.21.15.47:40916) 2024-04-26T23:19:31.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.778+0000 7f51377fe640 1 -- 172.21.15.47:0/794250438 learned_addr learned my addr 172.21.15.47:0/794250438 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:31.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.778+0000 7f51367fc640 1 -- 172.21.15.47:0/794250438 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4158162563 0 0) 0x7f51380d10d0 con 0x7f51380cd930 2024-04-26T23:19:31.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.778+0000 7f512ffff640 1 --2- 172.21.15.47:0/794250438 >> v2:172.21.15.203:3300/0 conn(0x7f51380b2340 0x7f51380cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:31.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.778+0000 7f51367fc640 1 -- 172.21.15.47:0/794250438 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5124003520 con 0x7f51380cd930 2024-04-26T23:19:31.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.778+0000 7f51367fc640 1 -- 172.21.15.47:0/794250438 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1774466485 0 0) 0x7f5124003520 con 0x7f51380cd930 2024-04-26T23:19:31.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.778+0000 7f51367fc640 1 -- 172.21.15.47:0/794250438 >> v2:172.21.15.203:3300/0 conn(0x7f51380b2340 msgr2=0x7f51380cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:31.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.778+0000 7f51367fc640 1 --2- 172.21.15.47:0/794250438 >> v2:172.21.15.203:3300/0 conn(0x7f51380b2340 0x7f51380cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:31.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.778+0000 7f51367fc640 1 -- 172.21.15.47:0/794250438 >> v1:172.21.15.203:6789/0 conn(0x7f5138162810 legacy=0x7f5138070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:31.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.778+0000 7f51367fc640 1 -- 172.21.15.47:0/794250438 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51380d01b0 con 0x7f51380cd930 2024-04-26T23:19:31.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.778+0000 7f51367fc640 1 -- 172.21.15.47:0/794250438 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7f51300019f0 con 0x7f51380cd930 2024-04-26T23:19:31.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.778+0000 7f51367fc640 1 -- 172.21.15.47:0/794250438 >> v1:172.21.15.47:6789/0 conn(0x7f51380cd930 legacy=0x7f51380d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:31.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.778+0000 7f51367fc640 1 --2- 172.21.15.47:0/794250438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51240039b0 0x7f5124003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:31.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.778+0000 7f51367fc640 1 --2- 172.21.15.47:0/794250438 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5124004800 0x7f5124004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:31.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.779+0000 7f51367fc640 1 --2- 172.21.15.47:0/794250438 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51240051c0 0x7f512400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:31.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.779+0000 7f51367fc640 1 -- 172.21.15.47:0/794250438 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f51380d0750 con 0x7f51240039b0 2024-04-26T23:19:31.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.779+0000 7f51367fc640 1 -- 172.21.15.47:0/794250438 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f512400c090 con 0x7f5124004800 2024-04-26T23:19:31.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.779+0000 7f51367fc640 1 -- 172.21.15.47:0/794250438 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f512400c290 con 0x7f51240051c0 2024-04-26T23:19:31.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.779+0000 7f5136ffd640 1 --2- 172.21.15.47:0/794250438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51240039b0 0x7f5124003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:31.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.779+0000 7f512ffff640 1 --2- 172.21.15.47:0/794250438 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5124004800 0x7f5124004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:31.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.779+0000 7f51377fe640 1 --2- 172.21.15.47:0/794250438 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51240051c0 0x7f512400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:31.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.779+0000 7f5136ffd640 1 -- 172.21.15.47:0/794250438 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51240051c0 msgr2=0x7f512400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:31.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.779+0000 7f5136ffd640 1 --2- 172.21.15.47:0/794250438 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51240051c0 0x7f512400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:31.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.779+0000 7f5136ffd640 1 -- 172.21.15.47:0/794250438 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5124004800 msgr2=0x7f5124004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:31.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.779+0000 7f5136ffd640 1 --2- 172.21.15.47:0/794250438 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5124004800 0x7f5124004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:31.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.779+0000 7f5136ffd640 1 -- 172.21.15.47:0/794250438 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5124003520 con 0x7f51240039b0 2024-04-26T23:19:31.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.779+0000 7f51377fe640 1 --2- 172.21.15.47:0/794250438 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51240051c0 0x7f512400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:19:31.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.779+0000 7f51377fe640 1 -- 172.21.15.47:0/794250438 reap_dead start 2024-04-26T23:19:31.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.780+0000 7f5136ffd640 1 --2- 172.21.15.47:0/794250438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51240039b0 0x7f5124003df0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5128001750 tx=0x7f5124005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2a397fa3a1296e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:31.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.780+0000 7f51367fc640 1 -- 172.21.15.47:0/794250438 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5128001370 con 0x7f51240039b0 2024-04-26T23:19:31.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.780+0000 7f51367fc640 1 -- 172.21.15.47:0/794250438 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51280052d0 con 0x7f51240039b0 2024-04-26T23:19:31.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.780+0000 7f51367fc640 1 -- 172.21.15.47:0/794250438 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f512802abc0 con 0x7f51240039b0 2024-04-26T23:19:31.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.780+0000 7f513d9af640 1 -- 172.21.15.47:0/794250438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51240039b0 msgr2=0x7f5124003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:31.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.780+0000 7f513d9af640 1 --2- 172.21.15.47:0/794250438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51240039b0 0x7f5124003df0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5128001750 tx=0x7f5124005830 comp rx=0 tx=0).stop 2024-04-26T23:19:31.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.780+0000 7f513d9af640 1 -- 172.21.15.47:0/794250438 shutdown_connections 2024-04-26T23:19:31.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.780+0000 7f513d9af640 1 --2- 172.21.15.47:0/794250438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51240039b0 0x7f5124003df0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:31.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.780+0000 7f513d9af640 1 -- 172.21.15.47:0/794250438 >> 172.21.15.47:0/794250438 conn(0x7f51380d2da0 msgr2=0x7f51380d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:31.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.780+0000 7f513d9af640 1 -- 172.21.15.47:0/794250438 shutdown_connections 2024-04-26T23:19:31.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.780+0000 7f513d9af640 1 -- 172.21.15.47:0/794250438 wait complete. 2024-04-26T23:19:31.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.782+0000 7f513d9af640 1 Processor -- start 2024-04-26T23:19:31.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.782+0000 7f513d9af640 1 -- start start 2024-04-26T23:19:31.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.782+0000 7f513d9af640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51240039b0 0x7f51380cc000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:31.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.782+0000 7f513d9af640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5138151d20 0x7f51381521a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:31.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.782+0000 7f513d9af640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51380c9fc0 0x7f51380ca400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:31.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.782+0000 7f513d9af640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f51380cd3f0 con 0x7f5138151d20 2024-04-26T23:19:31.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.782+0000 7f513d9af640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f51380d16a0 con 0x7f51240039b0 2024-04-26T23:19:31.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.782+0000 7f513d9af640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f51380cd570 con 0x7f51380c9fc0 2024-04-26T23:19:31.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.782+0000 7f512ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5138151d20 0x7f51381521a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:31.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.782+0000 7f512ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5138151d20 0x7f51381521a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33870/0 (socket says 172.21.15.47:33870) 2024-04-26T23:19:31.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.782+0000 7f512ffff640 1 -- 172.21.15.47:0/757913437 learned_addr learned my addr 172.21.15.47:0/757913437 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:31.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.783+0000 7f512ffff640 1 -- 172.21.15.47:0/757913437 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51380c9fc0 msgr2=0x7f51380ca400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:31.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.783+0000 7f512ffff640 1 --2- 172.21.15.47:0/757913437 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51380c9fc0 0x7f51380ca400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:31.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.783+0000 7f512ffff640 1 -- 172.21.15.47:0/757913437 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51240039b0 msgr2=0x7f51380cc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:31.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.783+0000 7f512ffff640 1 --2- 172.21.15.47:0/757913437 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51240039b0 0x7f51380cc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:31.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.783+0000 7f512ffff640 1 -- 172.21.15.47:0/757913437 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51380cacf0 con 0x7f5138151d20 2024-04-26T23:19:31.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.783+0000 7f512ffff640 1 --2- 172.21.15.47:0/757913437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5138151d20 0x7f51381521a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5120011a70 tx=0x7f5120011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:31.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.783+0000 7f5134ff9640 1 -- 172.21.15.47:0/757913437 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f512000fad0 con 0x7f5138151d20 2024-04-26T23:19:31.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.783+0000 7f5134ff9640 1 -- 172.21.15.47:0/757913437 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f512000fc70 con 0x7f5138151d20 2024-04-26T23:19:31.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.783+0000 7f5134ff9640 1 -- 172.21.15.47:0/757913437 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5120014440 con 0x7f5138151d20 2024-04-26T23:19:31.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.783+0000 7f513d9af640 1 -- 172.21.15.47:0/757913437 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51380cafd0 con 0x7f5138151d20 2024-04-26T23:19:31.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.783+0000 7f513d9af640 1 -- 172.21.15.47:0/757913437 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51380cb590 con 0x7f5138151d20 2024-04-26T23:19:31.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.784+0000 7f513d9af640 1 -- 172.21.15.47:0/757913437 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51380694f0 con 0x7f5138151d20 2024-04-26T23:19:31.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.785+0000 7f5134ff9640 1 -- 172.21.15.47:0/757913437 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5120012040 con 0x7f5138151d20 2024-04-26T23:19:31.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.785+0000 7f5134ff9640 1 --2- 172.21.15.47:0/757913437 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f511806c8f0 0x7f511806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:31.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.785+0000 7f5134ff9640 1 -- 172.21.15.47:0/757913437 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f512008c570 con 0x7f5138151d20 2024-04-26T23:19:31.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.785+0000 7f5136ffd640 1 --2- 172.21.15.47:0/757913437 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f511806c8f0 0x7f511806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:31.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.786+0000 7f5136ffd640 1 --2- 172.21.15.47:0/757913437 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f511806c8f0 0x7f511806edb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5128001750 tx=0x7f512802c000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:31.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.790+0000 7f5134ff9640 1 -- 172.21.15.47:0/757913437 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5120019030 con 0x7f5138151d20 2024-04-26T23:19:31.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.974+0000 7f513d9af640 1 -- 172.21.15.47:0/757913437 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f51380b2850 con 0x7f5138151d20 2024-04-26T23:19:31.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.975+0000 7f5134ff9640 1 -- 172.21.15.47:0/757913437 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18857 (secure 0 0 0) 0x7f512005a580 con 0x7f5138151d20 2024-04-26T23:19:31.976 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:19:31.976 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":26,"fsid":"399006d3-4db1-43e6-ba81-a79a276eabfd","created":"2024-04-26T23:18:06.731211+0000","modified":"2024-04-26T23:19:12.124051+0000","last_up_change":"2024-04-26T23:18:17.808736+0000","last_in_change":"2024-04-26T23:18:08.945859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:18:16.937492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:18:29.033575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:18:30.318499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:18:34.014536+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ee091567-fea0-4a14-bf4e-0e16c20bb2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6819","nonce":326520779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6821","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6823","nonce":326520779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6829","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6831","nonce":326520779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6825","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6827","nonce":326520779}]},"public_addr":"172.21.15.47:6819/326520779","cluster_addr":"172.21.15.47:6823/326520779","heartbeat_back_addr":"172.21.15.47:6831/326520779","heartbeat_front_addr":"172.21.15.47:6827/326520779","state":["exists","up"]},{"osd":1,"uuid":"c14440a0-bba5-4931-a6db-b736c08e869b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6809","nonce":1912610931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6811","nonce":1912610931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6815","nonce":1912610931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6813","nonce":1912610931}]},"public_addr":"172.21.15.47:6809/1912610931","cluster_addr":"172.21.15.47:6811/1912610931","heartbeat_back_addr":"172.21.15.47:6815/1912610931","heartbeat_front_addr":"172.21.15.47:6813/1912610931","state":["exists","up"]},{"osd":2,"uuid":"3940f8de-c9b2-4cd3-a911-9c301729eb30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6801","nonce":2569869305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6803","nonce":2569869305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6807","nonce":2569869305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6805","nonce":2569869305}]},"public_addr":"172.21.15.47:6801/2569869305","cluster_addr":"172.21.15.47:6803/2569869305","heartbeat_back_addr":"172.21.15.47:6807/2569869305","heartbeat_front_addr":"172.21.15.47:6805/2569869305","state":["exists","up"]},{"osd":3,"uuid":"1286a67e-8958-426a-8c2e-bc85498c619c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6818","nonce":2791411683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6822","nonce":2791411683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6830","nonce":2791411683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6826","nonce":2791411683}]},"public_addr":"172.21.15.47:6818/2791411683","cluster_addr":"172.21.15.47:6822/2791411683","heartbeat_back_addr":"172.21.15.47:6830/2791411683","heartbeat_front_addr":"172.21.15.47:6826/2791411683","state":["exists","up"]},{"osd":4,"uuid":"d8ee2c20-e5b7-46c2-9999-b234070126cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6809","nonce":441313104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6811","nonce":441313104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6815","nonce":441313104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6813","nonce":441313104}]},"public_addr":"172.21.15.203:6809/441313104","cluster_addr":"172.21.15.203:6811/441313104","heartbeat_back_addr":"172.21.15.203:6815/441313104","heartbeat_front_addr":"172.21.15.203:6813/441313104","state":["exists","up"]},{"osd":5,"uuid":"b453a8f0-f324-4840-9ae0-d67bdadb9f8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6801","nonce":56838444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6803","nonce":56838444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6807","nonce":56838444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6805","nonce":56838444}]},"public_addr":"172.21.15.203:6801/56838444","cluster_addr":"172.21.15.203:6803/56838444","heartbeat_back_addr":"172.21.15.203:6807/56838444","heartbeat_front_addr":"172.21.15.203:6805/56838444","state":["exists","up"]},{"osd":6,"uuid":"a6d31951-2d4b-4e8d-a510-6283b01dc158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6825","nonce":2738517080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6827","nonce":2738517080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6831","nonce":2738517080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6829","nonce":2738517080}]},"public_addr":"172.21.15.203:6825/2738517080","cluster_addr":"172.21.15.203:6827/2738517080","heartbeat_back_addr":"172.21.15.203:6831/2738517080","heartbeat_front_addr":"172.21.15.203:6829/2738517080","state":["exists","up"]},{"osd":7,"uuid":"7119b8c0-fe96-402d-9e75-7b509d9d44d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6817","nonce":1315565058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6819","nonce":1315565058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6823","nonce":1315565058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6821","nonce":1315565058}]},"public_addr":"172.21.15.203:6817/1315565058","cluster_addr":"172.21.15.203:6819/1315565058","heartbeat_back_addr":"172.21.15.203:6823/1315565058","heartbeat_front_addr":"172.21.15.203:6821/1315565058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.121921+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.137494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.084814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.111593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.072430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.125066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.097867+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.057061+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":5,"to":1}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:19:31.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.977+0000 7f513d9af640 1 -- 172.21.15.47:0/757913437 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f511806c8f0 msgr2=0x7f511806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:31.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.977+0000 7f513d9af640 1 --2- 172.21.15.47:0/757913437 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f511806c8f0 0x7f511806edb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5128001750 tx=0x7f512802c000 comp rx=0 tx=0).stop 2024-04-26T23:19:31.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.977+0000 7f513d9af640 1 -- 172.21.15.47:0/757913437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5138151d20 msgr2=0x7f51381521a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:31.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.977+0000 7f513d9af640 1 --2- 172.21.15.47:0/757913437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5138151d20 0x7f51381521a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5120011a70 tx=0x7f5120011f30 comp rx=0 tx=0).stop 2024-04-26T23:19:31.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.977+0000 7f513d9af640 1 -- 172.21.15.47:0/757913437 shutdown_connections 2024-04-26T23:19:31.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.977+0000 7f513d9af640 1 --2- 172.21.15.47:0/757913437 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f511806c8f0 0x7f511806edb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:31.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.977+0000 7f513d9af640 1 --2- 172.21.15.47:0/757913437 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51380c9fc0 0x7f51380ca400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:31.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.977+0000 7f513d9af640 1 --2- 172.21.15.47:0/757913437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5138151d20 0x7f51381521a0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:31.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.977+0000 7f513d9af640 1 --2- 172.21.15.47:0/757913437 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51240039b0 0x7f51380cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:31.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.977+0000 7f513d9af640 1 -- 172.21.15.47:0/757913437 >> 172.21.15.47:0/757913437 conn(0x7f51380d2da0 msgr2=0x7f513806d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:31.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.977+0000 7f513d9af640 1 -- 172.21.15.47:0/757913437 shutdown_connections 2024-04-26T23:19:31.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:31.977+0000 7f513d9af640 1 -- 172.21.15.47:0/757913437 wait complete. 2024-04-26T23:19:31.989 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T23:19:31.989 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:19:32.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.092+0000 7effb4df0640 1 Processor -- start 2024-04-26T23:19:32.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.092+0000 7effb4df0640 1 -- start start 2024-04-26T23:19:32.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.093+0000 7effb4df0640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7effb015ab60 0x7effb006c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:32.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.093+0000 7effb4df0640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7effb01617f0 0x7effb006c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:32.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.093+0000 7effb4df0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7effb015d5f0 0x7effb015daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:32.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.093+0000 7effb4df0640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7effb015af40 con 0x7effb015ab60 2024-04-26T23:19:32.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.093+0000 7effb4df0640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7effb015e320 con 0x7effb01617f0 2024-04-26T23:19:32.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.093+0000 7effb4df0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7effb0160d10 con 0x7effb015d5f0 2024-04-26T23:19:32.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.093+0000 7effae59c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7effb015ab60 0x7effb006c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:32.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.093+0000 7effae59c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7effb015ab60 0x7effb006c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33882/0 (socket says 172.21.15.47:33882) 2024-04-26T23:19:32.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.093+0000 7effae59c640 1 -- 172.21.15.47:0/18076031 learned_addr learned my addr 172.21.15.47:0/18076031 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:32.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.093+0000 7effaed9d640 1 --2- 172.21.15.47:0/18076031 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7effb015d5f0 0x7effb015daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:32.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.093+0000 7effa7fff640 1 --2- 172.21.15.47:0/18076031 >> v2:172.21.15.203:3300/0 conn(0x7effb01617f0 0x7effb006c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:32.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.093+0000 7effae59c640 1 -- 172.21.15.47:0/18076031 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7effb015d5f0 msgr2=0x7effb015daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:32.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.093+0000 7effae59c640 1 --2- 172.21.15.47:0/18076031 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7effb015d5f0 0x7effb015daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:32.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.093+0000 7effae59c640 1 -- 172.21.15.47:0/18076031 >> v2:172.21.15.203:3300/0 conn(0x7effb01617f0 msgr2=0x7effb006c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:32.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.093+0000 7effae59c640 1 --2- 172.21.15.47:0/18076031 >> v2:172.21.15.203:3300/0 conn(0x7effb01617f0 0x7effb006c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:32.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.093+0000 7effae59c640 1 -- 172.21.15.47:0/18076031 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effb0160ff0 con 0x7effb015ab60 2024-04-26T23:19:32.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.093+0000 7effa7fff640 1 --2- 172.21.15.47:0/18076031 >> v2:172.21.15.203:3300/0 conn(0x7effb01617f0 0x7effb006c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:19:32.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.094+0000 7effae59c640 1 --2- 172.21.15.47:0/18076031 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effb015ab60 0x7effb006c240 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7effb00b48c0 tx=0x7effa002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37113147373090f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:32.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.094+0000 7effadd9b640 1 -- 172.21.15.47:0/18076031 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa0037070 con 0x7effb015ab60 2024-04-26T23:19:32.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.094+0000 7effadd9b640 1 -- 172.21.15.47:0/18076031 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effa0032440 con 0x7effb015ab60 2024-04-26T23:19:32.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.094+0000 7effadd9b640 1 -- 172.21.15.47:0/18076031 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa0032810 con 0x7effb015ab60 2024-04-26T23:19:32.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.095+0000 7effb4df0640 1 -- 172.21.15.47:0/18076031 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effb015ab60 msgr2=0x7effb006c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:32.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.095+0000 7effb4df0640 1 --2- 172.21.15.47:0/18076031 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effb015ab60 0x7effb006c240 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7effb00b48c0 tx=0x7effa002a1e0 comp rx=0 tx=0).stop 2024-04-26T23:19:32.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.095+0000 7effb4df0640 1 -- 172.21.15.47:0/18076031 shutdown_connections 2024-04-26T23:19:32.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.095+0000 7effb4df0640 1 --2- 172.21.15.47:0/18076031 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7effb015d5f0 0x7effb015daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:32.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.095+0000 7effb4df0640 1 --2- 172.21.15.47:0/18076031 >> v2:172.21.15.203:3300/0 conn(0x7effb01617f0 0x7effb006c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:32.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.095+0000 7effb4df0640 1 --2- 172.21.15.47:0/18076031 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effb015ab60 0x7effb006c240 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:32.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.095+0000 7effb4df0640 1 -- 172.21.15.47:0/18076031 >> 172.21.15.47:0/18076031 conn(0x7effb00d1cf0 msgr2=0x7effb00d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:32.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.095+0000 7effb4df0640 1 -- 172.21.15.47:0/18076031 shutdown_connections 2024-04-26T23:19:32.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.095+0000 7effb4df0640 1 -- 172.21.15.47:0/18076031 wait complete. 2024-04-26T23:19:32.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.096+0000 7effb4df0640 1 Processor -- start 2024-04-26T23:19:32.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.096+0000 7effb4df0640 1 -- start start 2024-04-26T23:19:32.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.097+0000 7effb4df0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7effb015ab60 0x7effb00cd550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:32.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.097+0000 7effb4df0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effb015d5f0 0x7effb00cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:32.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.097+0000 7effb4df0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7effb01617f0 0x7effb00b4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:32.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.097+0000 7effb4df0640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7effb0072f70 con 0x7effb015d5f0 2024-04-26T23:19:32.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.097+0000 7effb4df0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7effb015dfe0 con 0x7effb01617f0 2024-04-26T23:19:32.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.097+0000 7effb4df0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7effb00730f0 con 0x7effb015ab60 2024-04-26T23:19:32.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.097+0000 7effa7fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effb015d5f0 0x7effb00cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:32.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.097+0000 7effa7fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effb015d5f0 0x7effb00cdd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33888/0 (socket says 172.21.15.47:33888) 2024-04-26T23:19:32.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.097+0000 7effa7fff640 1 -- 172.21.15.47:0/3541818805 learned_addr learned my addr 172.21.15.47:0/3541818805 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:32.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.097+0000 7effae59c640 1 --2- 172.21.15.47:0/3541818805 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7effb015ab60 0x7effb00cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:32.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.097+0000 7effaed9d640 1 --2- 172.21.15.47:0/3541818805 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7effb01617f0 0x7effb00b4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:32.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.097+0000 7effa7fff640 1 -- 172.21.15.47:0/3541818805 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7effb015ab60 msgr2=0x7effb00cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:32.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.097+0000 7effa7fff640 1 --2- 172.21.15.47:0/3541818805 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7effb015ab60 0x7effb00cd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:32.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.097+0000 7effa7fff640 1 -- 172.21.15.47:0/3541818805 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7effb01617f0 msgr2=0x7effb00b4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:32.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.097+0000 7effa7fff640 1 --2- 172.21.15.47:0/3541818805 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7effb01617f0 0x7effb00b4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:32.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.097+0000 7effa7fff640 1 -- 172.21.15.47:0/3541818805 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effb00c7cf0 con 0x7effb015d5f0 2024-04-26T23:19:32.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.097+0000 7effa7fff640 1 --2- 172.21.15.47:0/3541818805 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effb015d5f0 0x7effb00cdd60 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7eff9800c820 tx=0x7eff9800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:32.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.098+0000 7effa6ffd640 1 -- 172.21.15.47:0/3541818805 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff98015070 con 0x7effb015d5f0 2024-04-26T23:19:32.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.098+0000 7effa6ffd640 1 -- 172.21.15.47:0/3541818805 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff98007a30 con 0x7effb015d5f0 2024-04-26T23:19:32.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.098+0000 7effb4df0640 1 -- 172.21.15.47:0/3541818805 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effb00c8550 con 0x7effb015d5f0 2024-04-26T23:19:32.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.098+0000 7effa6ffd640 1 -- 172.21.15.47:0/3541818805 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff98007de0 con 0x7effb015d5f0 2024-04-26T23:19:32.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.098+0000 7effb4df0640 1 -- 172.21.15.47:0/3541818805 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effb00c8270 con 0x7effb015d5f0 2024-04-26T23:19:32.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.099+0000 7effb4df0640 1 -- 172.21.15.47:0/3541818805 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effb00694f0 con 0x7effb015d5f0 2024-04-26T23:19:32.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.099+0000 7effa6ffd640 1 -- 172.21.15.47:0/3541818805 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7eff98018070 con 0x7effb015d5f0 2024-04-26T23:19:32.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.099+0000 7effa6ffd640 1 --2- 172.21.15.47:0/3541818805 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7eff9006c8f0 0x7eff9006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:32.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.100+0000 7effa6ffd640 1 -- 172.21.15.47:0/3541818805 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7eff9808bcd0 con 0x7effb015d5f0 2024-04-26T23:19:32.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.100+0000 7effae59c640 1 --2- 172.21.15.47:0/3541818805 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7eff9006c8f0 0x7eff9006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:32.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.100+0000 7effae59c640 1 --2- 172.21.15.47:0/3541818805 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7eff9006c8f0 0x7eff9006edb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7effa0034000 tx=0x7effa002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:32.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.104+0000 7effa6ffd640 1 -- 172.21.15.47:0/3541818805 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7eff98059720 con 0x7effb015d5f0 2024-04-26T23:19:32.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.331+0000 7effb4df0640 1 -- 172.21.15.47:0/3541818805 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7effb0159260 con 0x7effb015d5f0 2024-04-26T23:19:32.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.331+0000 7effa6ffd640 1 -- 172.21.15.47:0/3541818805 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5180 (secure 0 0 0) 0x7eff9805da20 con 0x7effb015d5f0 2024-04-26T23:19:32.333 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:19:32.333 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/1553696401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1553696401},{"type":"v1","addr":"172.21.15.47:6834","nonce":1553696401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/3792767564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3792767564},{"type":"v1","addr":"172.21.15.203:6836","nonce":3792767564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4088100766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":4088100766},{"type":"v1","addr":"172.21.15.47:6835","nonce":4088100766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:18:31.367292+0000","modified":"2024-04-26T23:18:53.430267+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/997115011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":997115011},{"type":"v1","addr":"172.21.15.203:6837","nonce":997115011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4282,"qdb_cluster":[4282]},"id":1}]} 2024-04-26T23:19:32.333 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 9 2024-04-26T23:19:32.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.333+0000 7effb4df0640 1 -- 172.21.15.47:0/3541818805 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7eff9006c8f0 msgr2=0x7eff9006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:32.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.333+0000 7effb4df0640 1 --2- 172.21.15.47:0/3541818805 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7eff9006c8f0 0x7eff9006edb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7effa0034000 tx=0x7effa002e000 comp rx=0 tx=0).stop 2024-04-26T23:19:32.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.333+0000 7effb4df0640 1 -- 172.21.15.47:0/3541818805 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effb015d5f0 msgr2=0x7effb00cdd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:32.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.333+0000 7effb4df0640 1 --2- 172.21.15.47:0/3541818805 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effb015d5f0 0x7effb00cdd60 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7eff9800c820 tx=0x7eff9800cce0 comp rx=0 tx=0).stop 2024-04-26T23:19:32.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.333+0000 7effb4df0640 1 -- 172.21.15.47:0/3541818805 shutdown_connections 2024-04-26T23:19:32.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.333+0000 7effb4df0640 1 --2- 172.21.15.47:0/3541818805 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7eff9006c8f0 0x7eff9006edb0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:32.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.333+0000 7effb4df0640 1 --2- 172.21.15.47:0/3541818805 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7effb01617f0 0x7effb00b4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:32.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.333+0000 7effb4df0640 1 --2- 172.21.15.47:0/3541818805 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effb015d5f0 0x7effb00cdd60 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:32.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.333+0000 7effb4df0640 1 --2- 172.21.15.47:0/3541818805 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7effb015ab60 0x7effb00cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:32.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.333+0000 7effb4df0640 1 -- 172.21.15.47:0/3541818805 >> 172.21.15.47:0/3541818805 conn(0x7effb00d1cf0 msgr2=0x7effb0160210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:32.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.333+0000 7effb4df0640 1 -- 172.21.15.47:0/3541818805 shutdown_connections 2024-04-26T23:19:32.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.333+0000 7effb4df0640 1 -- 172.21.15.47:0/3541818805 wait complete. 2024-04-26T23:19:32.345 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:19:32.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.445+0000 7f6cd2bf0640 1 Processor -- start 2024-04-26T23:19:32.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.445+0000 7f6cd2bf0640 1 -- start start 2024-04-26T23:19:32.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.446+0000 7f6cd2bf0640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6ccc162880 0x7f6ccc0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:32.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.446+0000 7f6cd2bf0640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f6ccc0d3bc0 0x7f6ccc0b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:32.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.446+0000 7f6cd2bf0640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ccc0b7e50 con 0x7f6ccc0b5f20 2024-04-26T23:19:32.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.446+0000 7f6cd2bf0640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f6ccc0d0580 con 0x7f6ccc162880 2024-04-26T23:19:32.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.446+0000 7f6cd2bf0640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f6ccc0d1800 con 0x7f6ccc0d3bc0 2024-04-26T23:19:32.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.446+0000 7f6cd098c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6ccc162880 0x7f6ccc0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:32.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.446+0000 7f6cd098c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6ccc162880 0x7f6ccc0d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33890/0 (socket says 172.21.15.47:33890) 2024-04-26T23:19:32.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.446+0000 7f6cd098c640 1 -- 172.21.15.47:0/3171760533 learned_addr learned my addr 172.21.15.47:0/3171760533 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:32.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.446+0000 7f6ccbfff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f6ccc0b5f20 0x7f6ccc0d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40920/0 (socket says 172.21.15.47:40920) 2024-04-26T23:19:32.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.446+0000 7f6cd118d640 1 --2- 172.21.15.47:0/3171760533 >> v2:172.21.15.203:3300/0 conn(0x7f6ccc0d3bc0 0x7f6ccc0b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:32.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.446+0000 7f6cd098c640 1 -- 172.21.15.47:0/3171760533 >> v2:172.21.15.203:3300/0 conn(0x7f6ccc0d3bc0 msgr2=0x7f6ccc0b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:32.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.446+0000 7f6cd098c640 1 --2- 172.21.15.47:0/3171760533 >> v2:172.21.15.203:3300/0 conn(0x7f6ccc0d3bc0 0x7f6ccc0b8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:32.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.446+0000 7f6cd098c640 1 -- 172.21.15.47:0/3171760533 >> v1:172.21.15.47:6789/0 conn(0x7f6ccc0b5f20 legacy=0x7f6ccc0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:32.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.446+0000 7f6cd098c640 1 -- 172.21.15.47:0/3171760533 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ccc0d07d0 con 0x7f6ccc162880 2024-04-26T23:19:32.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.447+0000 7f6cd098c640 1 --2- 172.21.15.47:0/3171760533 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ccc162880 0x7f6ccc0d2cb0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc0d11d0 tx=0x7f6cc402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5512ce8d375b210 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:32.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.447+0000 7f6ccb7fe640 1 -- 172.21.15.47:0/3171760533 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc4037070 con 0x7f6ccc162880 2024-04-26T23:19:32.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.447+0000 7f6ccb7fe640 1 -- 172.21.15.47:0/3171760533 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cc4032440 con 0x7f6ccc162880 2024-04-26T23:19:32.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.447+0000 7f6ccb7fe640 1 -- 172.21.15.47:0/3171760533 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc4032810 con 0x7f6ccc162880 2024-04-26T23:19:32.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.448+0000 7f6cd2bf0640 1 -- 172.21.15.47:0/3171760533 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ccc162880 msgr2=0x7f6ccc0d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:32.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.448+0000 7f6cd2bf0640 1 --2- 172.21.15.47:0/3171760533 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ccc162880 0x7f6ccc0d2cb0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc0d11d0 tx=0x7f6cc402a270 comp rx=0 tx=0).stop 2024-04-26T23:19:32.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.448+0000 7f6cd2bf0640 1 -- 172.21.15.47:0/3171760533 shutdown_connections 2024-04-26T23:19:32.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.448+0000 7f6cd2bf0640 1 --2- 172.21.15.47:0/3171760533 >> v2:172.21.15.203:3300/0 conn(0x7f6ccc0d3bc0 0x7f6ccc0b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:32.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.448+0000 7f6cd2bf0640 1 --2- 172.21.15.47:0/3171760533 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ccc162880 0x7f6ccc0d2cb0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:32.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.448+0000 7f6cd2bf0640 1 -- 172.21.15.47:0/3171760533 >> 172.21.15.47:0/3171760533 conn(0x7f6ccc0d4f60 msgr2=0x7f6ccc0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:32.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.448+0000 7f6cd2bf0640 1 -- 172.21.15.47:0/3171760533 shutdown_connections 2024-04-26T23:19:32.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.448+0000 7f6cd2bf0640 1 -- 172.21.15.47:0/3171760533 wait complete. 2024-04-26T23:19:32.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.449+0000 7f6cd2bf0640 1 Processor -- start 2024-04-26T23:19:32.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.449+0000 7f6cd2bf0640 1 -- start start 2024-04-26T23:19:32.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.450+0000 7f6cd2bf0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ccc0b5f20 0x7f6ccc0cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:32.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.450+0000 7f6cd2bf0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ccc0d3bc0 0x7f6ccc15b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:32.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.450+0000 7f6cd2bf0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ccc162880 0x7f6ccc0cad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:32.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.450+0000 7f6cd2bf0640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6ccc162c60 con 0x7f6ccc0d3bc0 2024-04-26T23:19:32.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.450+0000 7f6cd2bf0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6ccc0d0d90 con 0x7f6ccc162880 2024-04-26T23:19:32.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.450+0000 7f6cd2bf0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f6ccc162de0 con 0x7f6ccc0b5f20 2024-04-26T23:19:32.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.450+0000 7f6ccbfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ccc0d3bc0 0x7f6ccc15b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:32.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.450+0000 7f6ccbfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ccc0d3bc0 0x7f6ccc15b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33906/0 (socket says 172.21.15.47:33906) 2024-04-26T23:19:32.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.450+0000 7f6ccbfff640 1 -- 172.21.15.47:0/4046891726 learned_addr learned my addr 172.21.15.47:0/4046891726 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:32.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.450+0000 7f6cd098c640 1 --2- 172.21.15.47:0/4046891726 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ccc0b5f20 0x7f6ccc0cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:32.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.450+0000 7f6cd118d640 1 --2- 172.21.15.47:0/4046891726 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ccc162880 0x7f6ccc0cad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:32.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.450+0000 7f6ccbfff640 1 -- 172.21.15.47:0/4046891726 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ccc0b5f20 msgr2=0x7f6ccc0cbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:32.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.450+0000 7f6ccbfff640 1 --2- 172.21.15.47:0/4046891726 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ccc0b5f20 0x7f6ccc0cbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:32.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.450+0000 7f6ccbfff640 1 -- 172.21.15.47:0/4046891726 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ccc162880 msgr2=0x7f6ccc0cad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:32.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.450+0000 7f6ccbfff640 1 --2- 172.21.15.47:0/4046891726 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ccc162880 0x7f6ccc0cad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:32.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.450+0000 7f6ccbfff640 1 -- 172.21.15.47:0/4046891726 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ccc0cc580 con 0x7f6ccc0d3bc0 2024-04-26T23:19:32.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.450+0000 7f6cd118d640 1 --2- 172.21.15.47:0/4046891726 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ccc162880 0x7f6ccc0cad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:19:32.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.450+0000 7f6ccbfff640 1 --2- 172.21.15.47:0/4046891726 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ccc0d3bc0 0x7f6ccc15b230 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc00c730 tx=0x7f6cbc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:32.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.451+0000 7f6cc97fa640 1 -- 172.21.15.47:0/4046891726 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc017070 con 0x7f6ccc0d3bc0 2024-04-26T23:19:32.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.451+0000 7f6cc97fa640 1 -- 172.21.15.47:0/4046891726 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cbc00ce40 con 0x7f6ccc0d3bc0 2024-04-26T23:19:32.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.451+0000 7f6cd2bf0640 1 -- 172.21.15.47:0/4046891726 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ccc0cc860 con 0x7f6ccc0d3bc0 2024-04-26T23:19:32.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.451+0000 7f6cc97fa640 1 -- 172.21.15.47:0/4046891726 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc00a860 con 0x7f6ccc0d3bc0 2024-04-26T23:19:32.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.451+0000 7f6cd2bf0640 1 -- 172.21.15.47:0/4046891726 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ccc0cce20 con 0x7f6ccc0d3bc0 2024-04-26T23:19:32.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.452+0000 7f6cd2bf0640 1 -- 172.21.15.47:0/4046891726 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ccc08cfe0 con 0x7f6ccc0d3bc0 2024-04-26T23:19:32.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.452+0000 7f6cc97fa640 1 -- 172.21.15.47:0/4046891726 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6cbc01a070 con 0x7f6ccc0d3bc0 2024-04-26T23:19:32.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.452+0000 7f6cc97fa640 1 --2- 172.21.15.47:0/4046891726 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f6cac06c8f0 0x7f6cac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:32.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.453+0000 7f6cc97fa640 1 -- 172.21.15.47:0/4046891726 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f6cbc08b4b0 con 0x7f6ccc0d3bc0 2024-04-26T23:19:32.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.453+0000 7f6cd098c640 1 --2- 172.21.15.47:0/4046891726 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f6cac06c8f0 0x7f6cac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:32.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.453+0000 7f6cd098c640 1 --2- 172.21.15.47:0/4046891726 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f6cac06c8f0 0x7f6cac06edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4034000 tx=0x7f6cc402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:32.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.457+0000 7f6cc97fa640 1 -- 172.21.15.47:0/4046891726 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6ccc08cfe0 con 0x7f6ccc0d3bc0 2024-04-26T23:19:32.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.681+0000 7f6cd2bf0640 1 -- 172.21.15.47:0/4046891726 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ccc0694f0 con 0x7f6ccc0d3bc0 2024-04-26T23:19:32.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.681+0000 7f6cc97fa640 1 -- 172.21.15.47:0/4046891726 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5180 (secure 0 0 0) 0x7f6cbc058e70 con 0x7f6ccc0d3bc0 2024-04-26T23:19:32.682 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:19:32.682 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/1553696401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1553696401},{"type":"v1","addr":"172.21.15.47:6834","nonce":1553696401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/3792767564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3792767564},{"type":"v1","addr":"172.21.15.203:6836","nonce":3792767564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4088100766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":4088100766},{"type":"v1","addr":"172.21.15.47:6835","nonce":4088100766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:18:31.367292+0000","modified":"2024-04-26T23:18:53.430267+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/997115011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":997115011},{"type":"v1","addr":"172.21.15.203:6837","nonce":997115011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4282,"qdb_cluster":[4282]},"id":1}]} 2024-04-26T23:19:32.683 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 9 2024-04-26T23:19:32.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.684+0000 7f6cd2bf0640 1 -- 172.21.15.47:0/4046891726 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f6cac06c8f0 msgr2=0x7f6cac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:32.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.684+0000 7f6cd2bf0640 1 --2- 172.21.15.47:0/4046891726 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f6cac06c8f0 0x7f6cac06edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4034000 tx=0x7f6cc402e000 comp rx=0 tx=0).stop 2024-04-26T23:19:32.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.684+0000 7f6cd2bf0640 1 -- 172.21.15.47:0/4046891726 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ccc0d3bc0 msgr2=0x7f6ccc15b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:32.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.684+0000 7f6cd2bf0640 1 --2- 172.21.15.47:0/4046891726 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ccc0d3bc0 0x7f6ccc15b230 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc00c730 tx=0x7f6cbc00cbf0 comp rx=0 tx=0).stop 2024-04-26T23:19:32.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.684+0000 7f6cd2bf0640 1 -- 172.21.15.47:0/4046891726 shutdown_connections 2024-04-26T23:19:32.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.684+0000 7f6cd2bf0640 1 --2- 172.21.15.47:0/4046891726 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f6cac06c8f0 0x7f6cac06edb0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:32.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.684+0000 7f6cd2bf0640 1 --2- 172.21.15.47:0/4046891726 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ccc162880 0x7f6ccc0cad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:32.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.684+0000 7f6cd2bf0640 1 --2- 172.21.15.47:0/4046891726 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ccc0d3bc0 0x7f6ccc15b230 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:32.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.684+0000 7f6cd2bf0640 1 --2- 172.21.15.47:0/4046891726 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ccc0b5f20 0x7f6ccc0cbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:32.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.684+0000 7f6cd2bf0640 1 -- 172.21.15.47:0/4046891726 >> 172.21.15.47:0/4046891726 conn(0x7f6ccc0d4f60 msgr2=0x7f6ccc0b69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:32.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.684+0000 7f6cd2bf0640 1 -- 172.21.15.47:0/4046891726 shutdown_connections 2024-04-26T23:19:32.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.684+0000 7f6cd2bf0640 1 -- 172.21.15.47:0/4046891726 wait complete. 2024-04-26T23:19:32.695 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:19:32.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.795+0000 7f7403322640 1 Processor -- start 2024-04-26T23:19:32.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.795+0000 7f7403322640 1 -- start start 2024-04-26T23:19:32.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.796+0000 7f7403322640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f73fc15f540 0x7f73fc0d3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:32.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.796+0000 7f7403322640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73fc0b3ff0 0x7f73fc0d39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:32.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.796+0000 7f7403322640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f73fc0d1770 0x7f73fc0d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:32.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.796+0000 7f7403322640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f73fc15f920 con 0x7f73fc0d1770 2024-04-26T23:19:32.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.796+0000 7f7403322640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f73fc0d0f30 con 0x7f73fc15f540 2024-04-26T23:19:32.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.796+0000 7f7403322640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f73fc0d2d70 con 0x7f73fc0b3ff0 2024-04-26T23:19:32.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.796+0000 7f74018bf640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f73fc0d1770 0x7f73fc0d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:32.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.796+0000 7f74018bf640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f73fc0d1770 0x7f73fc0d1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33918/0 (socket says 172.21.15.47:33918) 2024-04-26T23:19:32.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.796+0000 7f74018bf640 1 -- 172.21.15.47:0/3213051132 learned_addr learned my addr 172.21.15.47:0/3213051132 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:32.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.797+0000 7f74008bd640 1 --2- 172.21.15.47:0/3213051132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73fc0b3ff0 0x7f73fc0d39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:32.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.797+0000 7f74010be640 1 --2- 172.21.15.47:0/3213051132 >> v2:172.21.15.203:3300/0 conn(0x7f73fc15f540 0x7f73fc0d3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:32.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.797+0000 7f74018bf640 1 -- 172.21.15.47:0/3213051132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73fc0b3ff0 msgr2=0x7f73fc0d39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:32.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.797+0000 7f74018bf640 1 --2- 172.21.15.47:0/3213051132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73fc0b3ff0 0x7f73fc0d39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:32.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.797+0000 7f74018bf640 1 -- 172.21.15.47:0/3213051132 >> v2:172.21.15.203:3300/0 conn(0x7f73fc15f540 msgr2=0x7f73fc0d3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:32.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.797+0000 7f74018bf640 1 --2- 172.21.15.47:0/3213051132 >> v2:172.21.15.203:3300/0 conn(0x7f73fc15f540 0x7f73fc0d3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:32.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.797+0000 7f74018bf640 1 -- 172.21.15.47:0/3213051132 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73fc0d3050 con 0x7f73fc0d1770 2024-04-26T23:19:32.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.797+0000 7f74010be640 1 --2- 172.21.15.47:0/3213051132 >> v2:172.21.15.203:3300/0 conn(0x7f73fc15f540 0x7f73fc0d3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:19:32.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.797+0000 7f74018bf640 1 --2- 172.21.15.47:0/3213051132 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73fc0d1770 0x7f73fc0d1c00 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f73fc08d6d0 tx=0x7f73ec02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28ba1eb3038325d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:32.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.797+0000 7f73fbfff640 1 -- 172.21.15.47:0/3213051132 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73ec036070 con 0x7f73fc0d1770 2024-04-26T23:19:32.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.798+0000 7f73fbfff640 1 -- 172.21.15.47:0/3213051132 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73ec031600 con 0x7f73fc0d1770 2024-04-26T23:19:32.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.798+0000 7f73fbfff640 1 -- 172.21.15.47:0/3213051132 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73ec0319d0 con 0x7f73fc0d1770 2024-04-26T23:19:32.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.798+0000 7f7403322640 1 -- 172.21.15.47:0/3213051132 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73fc0d1770 msgr2=0x7f73fc0d1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:32.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.798+0000 7f7403322640 1 --2- 172.21.15.47:0/3213051132 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73fc0d1770 0x7f73fc0d1c00 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f73fc08d6d0 tx=0x7f73ec02a6a0 comp rx=0 tx=0).stop 2024-04-26T23:19:32.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.798+0000 7f7403322640 1 -- 172.21.15.47:0/3213051132 shutdown_connections 2024-04-26T23:19:32.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.798+0000 7f7403322640 1 --2- 172.21.15.47:0/3213051132 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73fc0d1770 0x7f73fc0d1c00 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:32.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.798+0000 7f7403322640 1 --2- 172.21.15.47:0/3213051132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73fc0b3ff0 0x7f73fc0d39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:32.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.798+0000 7f7403322640 1 --2- 172.21.15.47:0/3213051132 >> v2:172.21.15.203:3300/0 conn(0x7f73fc15f540 0x7f73fc0d3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:32.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.798+0000 7f7403322640 1 -- 172.21.15.47:0/3213051132 >> 172.21.15.47:0/3213051132 conn(0x7f73fc0d4f90 msgr2=0x7f73fc0d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:32.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.798+0000 7f7403322640 1 -- 172.21.15.47:0/3213051132 shutdown_connections 2024-04-26T23:19:32.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.798+0000 7f7403322640 1 -- 172.21.15.47:0/3213051132 wait complete. 2024-04-26T23:19:32.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.799+0000 7f7403322640 1 Processor -- start 2024-04-26T23:19:32.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.799+0000 7f7403322640 1 -- start start 2024-04-26T23:19:32.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.800+0000 7f7403322640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73fc0b3ff0 0x7f73fc1f4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:32.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.800+0000 7f7403322640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73fc0d1770 0x7f73fc1f8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:32.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.800+0000 7f7403322640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73fc15f540 0x7f73fc1f8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:32.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.800+0000 7f7403322640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f73fc068e50 con 0x7f73fc0b3ff0 2024-04-26T23:19:32.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.800+0000 7f7403322640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f73fc0d0bc0 con 0x7f73fc0d1770 2024-04-26T23:19:32.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.800+0000 7f7403322640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f73fc0d2140 con 0x7f73fc15f540 2024-04-26T23:19:32.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.800+0000 7f74010be640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73fc0b3ff0 0x7f73fc1f4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:32.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.800+0000 7f74010be640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73fc0b3ff0 0x7f73fc1f4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33924/0 (socket says 172.21.15.47:33924) 2024-04-26T23:19:32.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.800+0000 7f74010be640 1 -- 172.21.15.47:0/2770993303 learned_addr learned my addr 172.21.15.47:0/2770993303 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:32.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.800+0000 7f74008bd640 1 --2- 172.21.15.47:0/2770993303 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73fc0d1770 0x7f73fc1f8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:32.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.800+0000 7f74018bf640 1 --2- 172.21.15.47:0/2770993303 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73fc15f540 0x7f73fc1f8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:32.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.801+0000 7f74010be640 1 -- 172.21.15.47:0/2770993303 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73fc15f540 msgr2=0x7f73fc1f8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:32.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.801+0000 7f74010be640 1 --2- 172.21.15.47:0/2770993303 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73fc15f540 0x7f73fc1f8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:32.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.801+0000 7f74010be640 1 -- 172.21.15.47:0/2770993303 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73fc0d1770 msgr2=0x7f73fc1f8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:32.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.801+0000 7f74010be640 1 --2- 172.21.15.47:0/2770993303 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73fc0d1770 0x7f73fc1f8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:32.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.801+0000 7f74010be640 1 -- 172.21.15.47:0/2770993303 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73fc15ba40 con 0x7f73fc0b3ff0 2024-04-26T23:19:32.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.801+0000 7f74008bd640 1 --2- 172.21.15.47:0/2770993303 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73fc0d1770 0x7f73fc1f8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:19:32.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.801+0000 7f74010be640 1 --2- 172.21.15.47:0/2770993303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73fc0b3ff0 0x7f73fc1f4b20 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f73f400c820 tx=0x7f73f400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:32.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.801+0000 7f73f9ffb640 1 -- 172.21.15.47:0/2770993303 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f4015070 con 0x7f73fc0b3ff0 2024-04-26T23:19:32.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.801+0000 7f73f9ffb640 1 -- 172.21.15.47:0/2770993303 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73f4007a30 con 0x7f73fc0b3ff0 2024-04-26T23:19:32.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.801+0000 7f7403322640 1 -- 172.21.15.47:0/2770993303 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73fc15bd20 con 0x7f73fc0b3ff0 2024-04-26T23:19:32.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.801+0000 7f73f9ffb640 1 -- 172.21.15.47:0/2770993303 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f4007de0 con 0x7f73fc0b3ff0 2024-04-26T23:19:32.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.801+0000 7f7403322640 1 -- 172.21.15.47:0/2770993303 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73fc0b4d00 con 0x7f73fc0b3ff0 2024-04-26T23:19:32.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.802+0000 7f7403322640 1 -- 172.21.15.47:0/2770993303 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73fc08e6f0 con 0x7f73fc0b3ff0 2024-04-26T23:19:32.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.803+0000 7f73f9ffb640 1 -- 172.21.15.47:0/2770993303 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f73f4018070 con 0x7f73fc0b3ff0 2024-04-26T23:19:32.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.803+0000 7f73f9ffb640 1 --2- 172.21.15.47:0/2770993303 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f73e806c8f0 0x7f73e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:32.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.803+0000 7f73f9ffb640 1 -- 172.21.15.47:0/2770993303 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f73f408bd20 con 0x7f73fc0b3ff0 2024-04-26T23:19:32.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.803+0000 7f74008bd640 1 --2- 172.21.15.47:0/2770993303 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f73e806c8f0 0x7f73e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:32.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.804+0000 7f74008bd640 1 --2- 172.21.15.47:0/2770993303 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f73e806c8f0 0x7f73e806edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f73fc158870 tx=0x7f73f0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:32.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.807+0000 7f73f9ffb640 1 -- 172.21.15.47:0/2770993303 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f73fc08e6f0 con 0x7f73fc0b3ff0 2024-04-26T23:19:32.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.994+0000 7f7403322640 1 -- 172.21.15.47:0/2770993303 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f73fc0694f0 con 0x7f73fc0b3ff0 2024-04-26T23:19:32.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.995+0000 7f73f9ffb640 1 -- 172.21.15.47:0/2770993303 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18857 (secure 0 0 0) 0x7f73f4059770 con 0x7f73fc0b3ff0 2024-04-26T23:19:32.996 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:19:32.996 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":26,"fsid":"399006d3-4db1-43e6-ba81-a79a276eabfd","created":"2024-04-26T23:18:06.731211+0000","modified":"2024-04-26T23:19:12.124051+0000","last_up_change":"2024-04-26T23:18:17.808736+0000","last_in_change":"2024-04-26T23:18:08.945859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:18:16.937492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:18:29.033575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:18:30.318499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:18:34.014536+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ee091567-fea0-4a14-bf4e-0e16c20bb2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6819","nonce":326520779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6821","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6823","nonce":326520779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6829","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6831","nonce":326520779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6825","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6827","nonce":326520779}]},"public_addr":"172.21.15.47:6819/326520779","cluster_addr":"172.21.15.47:6823/326520779","heartbeat_back_addr":"172.21.15.47:6831/326520779","heartbeat_front_addr":"172.21.15.47:6827/326520779","state":["exists","up"]},{"osd":1,"uuid":"c14440a0-bba5-4931-a6db-b736c08e869b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6809","nonce":1912610931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6811","nonce":1912610931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6815","nonce":1912610931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6813","nonce":1912610931}]},"public_addr":"172.21.15.47:6809/1912610931","cluster_addr":"172.21.15.47:6811/1912610931","heartbeat_back_addr":"172.21.15.47:6815/1912610931","heartbeat_front_addr":"172.21.15.47:6813/1912610931","state":["exists","up"]},{"osd":2,"uuid":"3940f8de-c9b2-4cd3-a911-9c301729eb30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6801","nonce":2569869305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6803","nonce":2569869305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6807","nonce":2569869305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6805","nonce":2569869305}]},"public_addr":"172.21.15.47:6801/2569869305","cluster_addr":"172.21.15.47:6803/2569869305","heartbeat_back_addr":"172.21.15.47:6807/2569869305","heartbeat_front_addr":"172.21.15.47:6805/2569869305","state":["exists","up"]},{"osd":3,"uuid":"1286a67e-8958-426a-8c2e-bc85498c619c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6818","nonce":2791411683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6822","nonce":2791411683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6830","nonce":2791411683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6826","nonce":2791411683}]},"public_addr":"172.21.15.47:6818/2791411683","cluster_addr":"172.21.15.47:6822/2791411683","heartbeat_back_addr":"172.21.15.47:6830/2791411683","heartbeat_front_addr":"172.21.15.47:6826/2791411683","state":["exists","up"]},{"osd":4,"uuid":"d8ee2c20-e5b7-46c2-9999-b234070126cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6809","nonce":441313104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6811","nonce":441313104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6815","nonce":441313104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6813","nonce":441313104}]},"public_addr":"172.21.15.203:6809/441313104","cluster_addr":"172.21.15.203:6811/441313104","heartbeat_back_addr":"172.21.15.203:6815/441313104","heartbeat_front_addr":"172.21.15.203:6813/441313104","state":["exists","up"]},{"osd":5,"uuid":"b453a8f0-f324-4840-9ae0-d67bdadb9f8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6801","nonce":56838444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6803","nonce":56838444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6807","nonce":56838444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6805","nonce":56838444}]},"public_addr":"172.21.15.203:6801/56838444","cluster_addr":"172.21.15.203:6803/56838444","heartbeat_back_addr":"172.21.15.203:6807/56838444","heartbeat_front_addr":"172.21.15.203:6805/56838444","state":["exists","up"]},{"osd":6,"uuid":"a6d31951-2d4b-4e8d-a510-6283b01dc158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6825","nonce":2738517080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6827","nonce":2738517080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6831","nonce":2738517080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6829","nonce":2738517080}]},"public_addr":"172.21.15.203:6825/2738517080","cluster_addr":"172.21.15.203:6827/2738517080","heartbeat_back_addr":"172.21.15.203:6831/2738517080","heartbeat_front_addr":"172.21.15.203:6829/2738517080","state":["exists","up"]},{"osd":7,"uuid":"7119b8c0-fe96-402d-9e75-7b509d9d44d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6817","nonce":1315565058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6819","nonce":1315565058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6823","nonce":1315565058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6821","nonce":1315565058}]},"public_addr":"172.21.15.203:6817/1315565058","cluster_addr":"172.21.15.203:6819/1315565058","heartbeat_back_addr":"172.21.15.203:6823/1315565058","heartbeat_front_addr":"172.21.15.203:6821/1315565058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.121921+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.137494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.084814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.111593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.072430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.125066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.097867+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.057061+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":5,"to":1}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:19:32.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.997+0000 7f7403322640 1 -- 172.21.15.47:0/2770993303 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f73e806c8f0 msgr2=0x7f73e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:32.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.997+0000 7f7403322640 1 --2- 172.21.15.47:0/2770993303 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f73e806c8f0 0x7f73e806edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f73fc158870 tx=0x7f73f0008040 comp rx=0 tx=0).stop 2024-04-26T23:19:32.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.997+0000 7f7403322640 1 -- 172.21.15.47:0/2770993303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73fc0b3ff0 msgr2=0x7f73fc1f4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:32.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.997+0000 7f7403322640 1 --2- 172.21.15.47:0/2770993303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73fc0b3ff0 0x7f73fc1f4b20 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f73f400c820 tx=0x7f73f400cce0 comp rx=0 tx=0).stop 2024-04-26T23:19:32.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.997+0000 7f7403322640 1 -- 172.21.15.47:0/2770993303 shutdown_connections 2024-04-26T23:19:32.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.997+0000 7f7403322640 1 --2- 172.21.15.47:0/2770993303 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f73e806c8f0 0x7f73e806edb0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:32.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.997+0000 7f7403322640 1 --2- 172.21.15.47:0/2770993303 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73fc15f540 0x7f73fc1f8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:32.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.997+0000 7f7403322640 1 --2- 172.21.15.47:0/2770993303 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73fc0d1770 0x7f73fc1f8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:32.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.997+0000 7f7403322640 1 --2- 172.21.15.47:0/2770993303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73fc0b3ff0 0x7f73fc1f4b20 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:32.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.997+0000 7f7403322640 1 -- 172.21.15.47:0/2770993303 >> 172.21.15.47:0/2770993303 conn(0x7f73fc0d4f90 msgr2=0x7f73fc06ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:32.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.997+0000 7f7403322640 1 -- 172.21.15.47:0/2770993303 shutdown_connections 2024-04-26T23:19:32.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:32.997+0000 7f7403322640 1 -- 172.21.15.47:0/2770993303 wait complete. 2024-04-26T23:19:33.009 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T23:19:33.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.110+0000 7fa7826cf640 1 Processor -- start 2024-04-26T23:19:33.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.110+0000 7fa7826cf640 1 -- start start 2024-04-26T23:19:33.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.111+0000 7fa7826cf640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa77c0d3bc0 0x7fa77c0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:33.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.111+0000 7fa7826cf640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa77c0b5f20 0x7fa77c0b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:33.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.111+0000 7fa7826cf640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa77c0b7e50 con 0x7fa77c162880 2024-04-26T23:19:33.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.111+0000 7fa7826cf640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fa77c0d0580 con 0x7fa77c0b5f20 2024-04-26T23:19:33.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.111+0000 7fa7826cf640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fa77c0d1800 con 0x7fa77c0d3bc0 2024-04-26T23:19:33.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.111+0000 7fa780c6c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa77c0b5f20 0x7fa77c0b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:33.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.111+0000 7fa780c6c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa77c0b5f20 0x7fa77c0b8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33938/0 (socket says 172.21.15.47:33938) 2024-04-26T23:19:33.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.111+0000 7fa780c6c640 1 -- 172.21.15.47:0/1512780963 learned_addr learned my addr 172.21.15.47:0/1512780963 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:33.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.111+0000 7fa77b7fe640 1 --2- 172.21.15.47:0/1512780963 >> v2:172.21.15.203:3300/0 conn(0x7fa77c0d3bc0 0x7fa77c0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:33.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.111+0000 7fa780c6c640 1 -- 172.21.15.47:0/1512780963 >> v2:172.21.15.203:3300/0 conn(0x7fa77c0d3bc0 msgr2=0x7fa77c0d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:33.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.111+0000 7fa780c6c640 1 --2- 172.21.15.47:0/1512780963 >> v2:172.21.15.203:3300/0 conn(0x7fa77c0d3bc0 0x7fa77c0d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:33.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.111+0000 7fa780c6c640 1 -- 172.21.15.47:0/1512780963 >> v1:172.21.15.203:6789/0 conn(0x7fa77c162880 legacy=0x7fa77c0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:33.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.111+0000 7fa780c6c640 1 -- 172.21.15.47:0/1512780963 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa77c0d07d0 con 0x7fa77c0b5f20 2024-04-26T23:19:33.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.112+0000 7fa780c6c640 1 --2- 172.21.15.47:0/1512780963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa77c0b5f20 0x7fa77c0b8300 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa77c08d3d0 tx=0x7fa77402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cab64dfd629ea433 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:33.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.112+0000 7fa77affd640 1 -- 172.21.15.47:0/1512780963 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa774036070 con 0x7fa77c0b5f20 2024-04-26T23:19:33.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.112+0000 7fa77affd640 1 -- 172.21.15.47:0/1512780963 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa774031600 con 0x7fa77c0b5f20 2024-04-26T23:19:33.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.112+0000 7fa77affd640 1 -- 172.21.15.47:0/1512780963 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7740319d0 con 0x7fa77c0b5f20 2024-04-26T23:19:33.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.113+0000 7fa7826cf640 1 -- 172.21.15.47:0/1512780963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa77c0b5f20 msgr2=0x7fa77c0b8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:33.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.113+0000 7fa7826cf640 1 --2- 172.21.15.47:0/1512780963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa77c0b5f20 0x7fa77c0b8300 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa77c08d3d0 tx=0x7fa77402a730 comp rx=0 tx=0).stop 2024-04-26T23:19:33.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.113+0000 7fa7826cf640 1 -- 172.21.15.47:0/1512780963 shutdown_connections 2024-04-26T23:19:33.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.113+0000 7fa7826cf640 1 --2- 172.21.15.47:0/1512780963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa77c0b5f20 0x7fa77c0b8300 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:33.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.113+0000 7fa7826cf640 1 --2- 172.21.15.47:0/1512780963 >> v2:172.21.15.203:3300/0 conn(0x7fa77c0d3bc0 0x7fa77c0d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:33.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.113+0000 7fa7826cf640 1 -- 172.21.15.47:0/1512780963 >> 172.21.15.47:0/1512780963 conn(0x7fa77c0d4f60 msgr2=0x7fa77c0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:33.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.113+0000 7fa7826cf640 1 -- 172.21.15.47:0/1512780963 shutdown_connections 2024-04-26T23:19:33.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.113+0000 7fa7826cf640 1 -- 172.21.15.47:0/1512780963 wait complete. 2024-04-26T23:19:33.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.114+0000 7fa7826cf640 1 Processor -- start 2024-04-26T23:19:33.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.114+0000 7fa7826cf640 1 -- start start 2024-04-26T23:19:33.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.115+0000 7fa7826cf640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa77c0b5f20 0x7fa77c0b5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:33.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.115+0000 7fa7826cf640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa77c0d3bc0 0x7fa77c1f8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:33.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.115+0000 7fa7826cf640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa77c162880 0x7fa77c15c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:33.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.115+0000 7fa7826cf640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa77c162c60 con 0x7fa77c0d3bc0 2024-04-26T23:19:33.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.115+0000 7fa7826cf640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa77c0d0d90 con 0x7fa77c162880 2024-04-26T23:19:33.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.115+0000 7fa7826cf640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa77c162de0 con 0x7fa77c0b5f20 2024-04-26T23:19:33.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.115+0000 7fa77b7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa77c0d3bc0 0x7fa77c1f8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:33.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.115+0000 7fa77b7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa77c0d3bc0 0x7fa77c1f8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33954/0 (socket says 172.21.15.47:33954) 2024-04-26T23:19:33.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.115+0000 7fa77b7fe640 1 -- 172.21.15.47:0/2437948960 learned_addr learned my addr 172.21.15.47:0/2437948960 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:33.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.115+0000 7fa780c6c640 1 --2- 172.21.15.47:0/2437948960 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa77c162880 0x7fa77c15c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:33.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.115+0000 7fa77bfff640 1 --2- 172.21.15.47:0/2437948960 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa77c0b5f20 0x7fa77c0b5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:33.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.115+0000 7fa77b7fe640 1 -- 172.21.15.47:0/2437948960 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa77c0b5f20 msgr2=0x7fa77c0b5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:33.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.115+0000 7fa77b7fe640 1 --2- 172.21.15.47:0/2437948960 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa77c0b5f20 0x7fa77c0b5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:33.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.115+0000 7fa77b7fe640 1 -- 172.21.15.47:0/2437948960 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa77c162880 msgr2=0x7fa77c15c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:33.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.115+0000 7fa77b7fe640 1 --2- 172.21.15.47:0/2437948960 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa77c162880 0x7fa77c15c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:33.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.115+0000 7fa77b7fe640 1 -- 172.21.15.47:0/2437948960 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa77c0cd9c0 con 0x7fa77c0d3bc0 2024-04-26T23:19:33.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.116+0000 7fa77b7fe640 1 --2- 172.21.15.47:0/2437948960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa77c0d3bc0 0x7fa77c1f8350 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa76c00d800 tx=0x7fa76c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:33.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.116+0000 7fa778ff9640 1 -- 172.21.15.47:0/2437948960 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76c00b6c0 con 0x7fa77c0d3bc0 2024-04-26T23:19:33.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.116+0000 7fa778ff9640 1 -- 172.21.15.47:0/2437948960 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa76c019070 con 0x7fa77c0d3bc0 2024-04-26T23:19:33.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.116+0000 7fa7826cf640 1 -- 172.21.15.47:0/2437948960 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa77c0ccc00 con 0x7fa77c0d3bc0 2024-04-26T23:19:33.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.116+0000 7fa778ff9640 1 -- 172.21.15.47:0/2437948960 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76c00bac0 con 0x7fa77c0d3bc0 2024-04-26T23:19:33.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.116+0000 7fa7826cf640 1 -- 172.21.15.47:0/2437948960 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa77c0cd1c0 con 0x7fa77c0d3bc0 2024-04-26T23:19:33.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.117+0000 7fa7826cf640 1 -- 172.21.15.47:0/2437948960 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa77c0694f0 con 0x7fa77c0d3bc0 2024-04-26T23:19:33.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.117+0000 7fa778ff9640 1 -- 172.21.15.47:0/2437948960 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa76c00bc60 con 0x7fa77c0d3bc0 2024-04-26T23:19:33.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.118+0000 7fa778ff9640 1 --2- 172.21.15.47:0/2437948960 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa76806c8f0 0x7fa76806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:33.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.118+0000 7fa778ff9640 1 -- 172.21.15.47:0/2437948960 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7fa76c08c2b0 con 0x7fa77c0d3bc0 2024-04-26T23:19:33.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.118+0000 7fa77bfff640 1 --2- 172.21.15.47:0/2437948960 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa76806c8f0 0x7fa76806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:33.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.119+0000 7fa77bfff640 1 --2- 172.21.15.47:0/2437948960 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa76806c8f0 0x7fa76806edb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa770001780 tx=0x7fa770009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:33.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.122+0000 7fa778ff9640 1 -- 172.21.15.47:0/2437948960 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa76c059ce0 con 0x7fa77c0d3bc0 2024-04-26T23:19:33.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.346+0000 7fa7826cf640 1 -- 172.21.15.47:0/2437948960 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa77c0d35e0 con 0x7fa77c0d3bc0 2024-04-26T23:19:33.348 INFO:tasks.ceph.mon.a.smithi047.stderr:2024-04-26T23:19:33.347+0000 7fb226ccb640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T23:19:33.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.348+0000 7fa778ff9640 1 -- 172.21.15.47:0/2437948960 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) ==== 161+0+0 (secure 0 0 0) 0x7fa76c05dfe0 con 0x7fa77c0d3bc0 2024-04-26T23:19:33.349 INFO:teuthology.orchestra.run.smithi047.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T23:19:33.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.351+0000 7fa7826cf640 1 -- 172.21.15.47:0/2437948960 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa76806c8f0 msgr2=0x7fa76806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:33.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.351+0000 7fa7826cf640 1 --2- 172.21.15.47:0/2437948960 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa76806c8f0 0x7fa76806edb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa770001780 tx=0x7fa770009040 comp rx=0 tx=0).stop 2024-04-26T23:19:33.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.351+0000 7fa7826cf640 1 -- 172.21.15.47:0/2437948960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa77c0d3bc0 msgr2=0x7fa77c1f8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:33.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.351+0000 7fa7826cf640 1 --2- 172.21.15.47:0/2437948960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa77c0d3bc0 0x7fa77c1f8350 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa76c00d800 tx=0x7fa76c00dcc0 comp rx=0 tx=0).stop 2024-04-26T23:19:33.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.351+0000 7fa7826cf640 1 -- 172.21.15.47:0/2437948960 shutdown_connections 2024-04-26T23:19:33.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.351+0000 7fa7826cf640 1 --2- 172.21.15.47:0/2437948960 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa76806c8f0 0x7fa76806edb0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:33.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.351+0000 7fa7826cf640 1 --2- 172.21.15.47:0/2437948960 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa77c162880 0x7fa77c15c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:33.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.351+0000 7fa7826cf640 1 --2- 172.21.15.47:0/2437948960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa77c0d3bc0 0x7fa77c1f8350 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:33.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.351+0000 7fa7826cf640 1 --2- 172.21.15.47:0/2437948960 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa77c0b5f20 0x7fa77c0b5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:33.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.351+0000 7fa7826cf640 1 -- 172.21.15.47:0/2437948960 >> 172.21.15.47:0/2437948960 conn(0x7fa77c0d4f60 msgr2=0x7fa77c161ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:33.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.351+0000 7fa7826cf640 1 -- 172.21.15.47:0/2437948960 shutdown_connections 2024-04-26T23:19:33.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.351+0000 7fa7826cf640 1 -- 172.21.15.47:0/2437948960 wait complete. 2024-04-26T23:19:33.355 INFO:tasks.ceph.mds.b.smithi203.stderr: -6653> 2024-04-26T23:18:24.523+0000 7f082451f640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-04-26T23:19:33.356 INFO:tasks.ceph.mds.b.smithi203.stderr: -5430> 2024-04-26T23:18:35.522+0000 7f081e513640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T23:19:33.363 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T23:19:33.389 INFO:tasks.ceph.mds.b.smithi203.stdout:starting mds.b at 2024-04-26T23:19:33.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.465+0000 7fd7ae259640 1 Processor -- start 2024-04-26T23:19:33.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.465+0000 7fd7ae259640 1 -- start start 2024-04-26T23:19:33.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.466+0000 7fd7ae259640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd7a8162810 0x7fd7a8070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:33.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.466+0000 7fd7ae259640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd7a80d1b30 0x7fd7a80d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:33.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.466+0000 7fd7ae259640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7a80d10d0 con 0x7fd7a80b2340 2024-04-26T23:19:33.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.466+0000 7fd7ae259640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fd7a80d0750 con 0x7fd7a80d1b30 2024-04-26T23:19:33.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.466+0000 7fd7ae259640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fd7a80d16a0 con 0x7fd7a8162810 2024-04-26T23:19:33.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.466+0000 7fd7a7fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd7a80d1b30 0x7fd7a80d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:33.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.466+0000 7fd7a6ffd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd7a80b2340 0x7fd7a80cd3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40928/0 (socket says 172.21.15.47:40928) 2024-04-26T23:19:33.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.466+0000 7fd7a7fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd7a80d1b30 0x7fd7a80d1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33958/0 (socket says 172.21.15.47:33958) 2024-04-26T23:19:33.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.466+0000 7fd7a6ffd640 1 -- 172.21.15.47:0/305994225 learned_addr learned my addr 172.21.15.47:0/305994225 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:33.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.466+0000 7fd7a77fe640 1 --2- 172.21.15.47:0/305994225 >> v2:172.21.15.203:3300/0 conn(0x7fd7a8162810 0x7fd7a8070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:33.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.466+0000 7fd7a7fff640 1 -- 172.21.15.47:0/305994225 >> v2:172.21.15.203:3300/0 conn(0x7fd7a8162810 msgr2=0x7fd7a8070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:33.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.466+0000 7fd7a7fff640 1 --2- 172.21.15.47:0/305994225 >> v2:172.21.15.203:3300/0 conn(0x7fd7a8162810 0x7fd7a8070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:33.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.466+0000 7fd7a7fff640 1 -- 172.21.15.47:0/305994225 >> v1:172.21.15.47:6789/0 conn(0x7fd7a80b2340 legacy=0x7fd7a80cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:33.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.466+0000 7fd7a7fff640 1 -- 172.21.15.47:0/305994225 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7a80d1370 con 0x7fd7a80d1b30 2024-04-26T23:19:33.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.467+0000 7fd7a77fe640 1 --2- 172.21.15.47:0/305994225 >> v2:172.21.15.203:3300/0 conn(0x7fd7a8162810 0x7fd7a8070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:19:33.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.467+0000 7fd7a7fff640 1 --2- 172.21.15.47:0/305994225 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7a80d1b30 0x7fd7a80d1fc0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8073d90 tx=0x7fd7a002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b536b86ce233dd94 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:33.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.467+0000 7fd7a67fc640 1 -- 172.21.15.47:0/305994225 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a0007740 con 0x7fd7a80d1b30 2024-04-26T23:19:33.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.467+0000 7fd7a67fc640 1 -- 172.21.15.47:0/305994225 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7a00078e0 con 0x7fd7a80d1b30 2024-04-26T23:19:33.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.467+0000 7fd7a67fc640 1 -- 172.21.15.47:0/305994225 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a0007cb0 con 0x7fd7a80d1b30 2024-04-26T23:19:33.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.468+0000 7fd7ae259640 1 -- 172.21.15.47:0/305994225 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7a80d1b30 msgr2=0x7fd7a80d1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:33.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.468+0000 7fd7ae259640 1 --2- 172.21.15.47:0/305994225 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7a80d1b30 0x7fd7a80d1fc0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8073d90 tx=0x7fd7a002a680 comp rx=0 tx=0).stop 2024-04-26T23:19:33.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.468+0000 7fd7ae259640 1 -- 172.21.15.47:0/305994225 shutdown_connections 2024-04-26T23:19:33.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.468+0000 7fd7ae259640 1 --2- 172.21.15.47:0/305994225 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7a80d1b30 0x7fd7a80d1fc0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:33.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.468+0000 7fd7ae259640 1 --2- 172.21.15.47:0/305994225 >> v2:172.21.15.203:3300/0 conn(0x7fd7a8162810 0x7fd7a8070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:33.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.468+0000 7fd7ae259640 1 -- 172.21.15.47:0/305994225 >> 172.21.15.47:0/305994225 conn(0x7fd7a80d2da0 msgr2=0x7fd7a80d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:33.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.468+0000 7fd7ae259640 1 -- 172.21.15.47:0/305994225 shutdown_connections 2024-04-26T23:19:33.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.468+0000 7fd7ae259640 1 -- 172.21.15.47:0/305994225 wait complete. 2024-04-26T23:19:33.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.469+0000 7fd7ae259640 1 Processor -- start 2024-04-26T23:19:33.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.469+0000 7fd7ae259640 1 -- start start 2024-04-26T23:19:33.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.470+0000 7fd7ae259640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7a80b2340 0x7fd7a80ca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:33.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.470+0000 7fd7ae259640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7a80d1b30 0x7fd7a8156f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:33.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.470+0000 7fd7ae259640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7a8162810 0x7fd7a8156420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:33.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.470+0000 7fd7ae259640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a8070890 con 0x7fd7a80b2340 2024-04-26T23:19:33.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.470+0000 7fd7ae259640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a80d0470 con 0x7fd7a8162810 2024-04-26T23:19:33.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.470+0000 7fd7ae259640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd7a80cdb00 con 0x7fd7a80d1b30 2024-04-26T23:19:33.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.470+0000 7fd7a77fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7a80b2340 0x7fd7a80ca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:33.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.470+0000 7fd7a77fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7a80b2340 0x7fd7a80ca0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33966/0 (socket says 172.21.15.47:33966) 2024-04-26T23:19:33.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.470+0000 7fd7a77fe640 1 -- 172.21.15.47:0/1266275240 learned_addr learned my addr 172.21.15.47:0/1266275240 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:33.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.470+0000 7fd7a7fff640 1 --2- 172.21.15.47:0/1266275240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7a8162810 0x7fd7a8156420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:33.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.470+0000 7fd7a6ffd640 1 --2- 172.21.15.47:0/1266275240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7a80d1b30 0x7fd7a8156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:33.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.471+0000 7fd7a77fe640 1 -- 172.21.15.47:0/1266275240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7a80d1b30 msgr2=0x7fd7a8156f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:33.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.471+0000 7fd7a77fe640 1 --2- 172.21.15.47:0/1266275240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7a80d1b30 0x7fd7a8156f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:33.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.471+0000 7fd7a77fe640 1 -- 172.21.15.47:0/1266275240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7a8162810 msgr2=0x7fd7a8156420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:33.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.471+0000 7fd7a77fe640 1 --2- 172.21.15.47:0/1266275240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7a8162810 0x7fd7a8156420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:33.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.471+0000 7fd7a77fe640 1 -- 172.21.15.47:0/1266275240 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7a806d3f0 con 0x7fd7a80b2340 2024-04-26T23:19:33.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.471+0000 7fd7a77fe640 1 --2- 172.21.15.47:0/1266275240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7a80b2340 0x7fd7a80ca0e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd79800c7f0 tx=0x7fd79800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:33.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.471+0000 7fd78ffff640 1 -- 172.21.15.47:0/1266275240 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd798015070 con 0x7fd7a80b2340 2024-04-26T23:19:33.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.471+0000 7fd78ffff640 1 -- 172.21.15.47:0/1266275240 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd798007820 con 0x7fd7a80b2340 2024-04-26T23:19:33.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.471+0000 7fd78ffff640 1 -- 172.21.15.47:0/1266275240 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd798007bd0 con 0x7fd7a80b2340 2024-04-26T23:19:33.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.471+0000 7fd7ae259640 1 -- 172.21.15.47:0/1266275240 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7a806f2c0 con 0x7fd7a80b2340 2024-04-26T23:19:33.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.471+0000 7fd7ae259640 1 -- 172.21.15.47:0/1266275240 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7a806d970 con 0x7fd7a80b2340 2024-04-26T23:19:33.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.472+0000 7fd7ae259640 1 -- 172.21.15.47:0/1266275240 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7a80694f0 con 0x7fd7a80b2340 2024-04-26T23:19:33.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.473+0000 7fd78ffff640 1 -- 172.21.15.47:0/1266275240 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd798018070 con 0x7fd7a80b2340 2024-04-26T23:19:33.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.473+0000 7fd78ffff640 1 --2- 172.21.15.47:0/1266275240 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd79406c8f0 0x7fd79406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:33.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.473+0000 7fd78ffff640 1 -- 172.21.15.47:0/1266275240 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6850+0+0 (secure 0 0 0) 0x7fd79808c030 con 0x7fd7a80b2340 2024-04-26T23:19:33.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.473+0000 7fd7a6ffd640 1 --2- 172.21.15.47:0/1266275240 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd79406c8f0 0x7fd79406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:33.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.474+0000 7fd7a6ffd640 1 --2- 172.21.15.47:0/1266275240 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd79406c8f0 0x7fd79406edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8161d20 tx=0x7fd79c007e80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:33.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.477+0000 7fd78ffff640 1 -- 172.21.15.47:0/1266275240 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd798059990 con 0x7fd7a80b2340 2024-04-26T23:19:33.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:33.701+0000 7fd7ae259640 1 -- 172.21.15.47:0/1266275240 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fd7a80721a0 con 0x7fd7a80b2340 2024-04-26T23:19:34.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.350+0000 7fd78ffff640 1 -- 172.21.15.47:0/1266275240 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7fd79805ee20 con 0x7fd7a80b2340 2024-04-26T23:19:34.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.353+0000 7fd7ae259640 1 -- 172.21.15.47:0/1266275240 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd79406c8f0 msgr2=0x7fd79406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:34.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.353+0000 7fd7ae259640 1 --2- 172.21.15.47:0/1266275240 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd79406c8f0 0x7fd79406edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8161d20 tx=0x7fd79c007e80 comp rx=0 tx=0).stop 2024-04-26T23:19:34.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.353+0000 7fd7ae259640 1 -- 172.21.15.47:0/1266275240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7a80b2340 msgr2=0x7fd7a80ca0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:34.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.353+0000 7fd7ae259640 1 --2- 172.21.15.47:0/1266275240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7a80b2340 0x7fd7a80ca0e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd79800c7f0 tx=0x7fd79800ccb0 comp rx=0 tx=0).stop 2024-04-26T23:19:34.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.353+0000 7fd7ae259640 1 -- 172.21.15.47:0/1266275240 shutdown_connections 2024-04-26T23:19:34.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.353+0000 7fd7ae259640 1 --2- 172.21.15.47:0/1266275240 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd79406c8f0 0x7fd79406edb0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:34.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.353+0000 7fd7ae259640 1 --2- 172.21.15.47:0/1266275240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7a8162810 0x7fd7a8156420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:34.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.353+0000 7fd7ae259640 1 --2- 172.21.15.47:0/1266275240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7a80d1b30 0x7fd7a8156f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:34.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.353+0000 7fd7ae259640 1 --2- 172.21.15.47:0/1266275240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7a80b2340 0x7fd7a80ca0e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:34.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.353+0000 7fd7ae259640 1 -- 172.21.15.47:0/1266275240 >> 172.21.15.47:0/1266275240 conn(0x7fd7a80d2da0 msgr2=0x7fd7a80d1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:34.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.353+0000 7fd7ae259640 1 -- 172.21.15.47:0/1266275240 shutdown_connections 2024-04-26T23:19:34.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.353+0000 7fd7ae259640 1 -- 172.21.15.47:0/1266275240 wait complete. 2024-04-26T23:19:34.367 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T23:19:34.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.478+0000 7fef5dada640 1 Processor -- start 2024-04-26T23:19:34.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.478+0000 7fef5dada640 1 -- start start 2024-04-26T23:19:34.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.479+0000 7fef5dada640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fef580d3c80 0x7fef580d1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:34.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.479+0000 7fef5dada640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fef580d1f80 0x7fef5806ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:34.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.479+0000 7fef5dada640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef5815f840 con 0x7fef5815f460 2024-04-26T23:19:34.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.479+0000 7fef5dada640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fef580d37f0 con 0x7fef580d1f80 2024-04-26T23:19:34.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.479+0000 7fef5dada640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fef580d2ec0 con 0x7fef580d3c80 2024-04-26T23:19:34.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.479+0000 7fef57fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fef580d1f80 0x7fef5806ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:34.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.479+0000 7fef577fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fef5815f460 0x7fef580d0760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40936/0 (socket says 172.21.15.47:40936) 2024-04-26T23:19:34.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.479+0000 7fef57fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fef580d1f80 0x7fef5806ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33976/0 (socket says 172.21.15.47:33976) 2024-04-26T23:19:34.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.479+0000 7fef577fe640 1 -- 172.21.15.47:0/2960879289 learned_addr learned my addr 172.21.15.47:0/2960879289 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:34.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.479+0000 7fef56ffd640 1 --2- 172.21.15.47:0/2960879289 >> v2:172.21.15.203:3300/0 conn(0x7fef580d3c80 0x7fef580d1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:34.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.479+0000 7fef57fff640 1 -- 172.21.15.47:0/2960879289 >> v2:172.21.15.203:3300/0 conn(0x7fef580d3c80 msgr2=0x7fef580d1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:34.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.479+0000 7fef57fff640 1 --2- 172.21.15.47:0/2960879289 >> v2:172.21.15.203:3300/0 conn(0x7fef580d3c80 0x7fef580d1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:34.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.479+0000 7fef57fff640 1 -- 172.21.15.47:0/2960879289 >> v1:172.21.15.47:6789/0 conn(0x7fef5815f460 legacy=0x7fef580d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:34.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.479+0000 7fef57fff640 1 -- 172.21.15.47:0/2960879289 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef580d31a0 con 0x7fef580d1f80 2024-04-26T23:19:34.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.479+0000 7fef567fc640 1 -- 172.21.15.47:0/2960879289 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1275439449 0 0) 0x7fef5815f840 con 0x7fef5815f460 2024-04-26T23:19:34.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.480+0000 7fef57fff640 1 --2- 172.21.15.47:0/2960879289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef580d1f80 0x7fef5806ce20 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fef5808dce0 tx=0x7fef5002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fc8baa3e7894841 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:34.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.480+0000 7fef567fc640 1 -- 172.21.15.47:0/2960879289 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef50007740 con 0x7fef580d1f80 2024-04-26T23:19:34.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.480+0000 7fef567fc640 1 -- 172.21.15.47:0/2960879289 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef500078e0 con 0x7fef580d1f80 2024-04-26T23:19:34.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.480+0000 7fef567fc640 1 -- 172.21.15.47:0/2960879289 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef50007cb0 con 0x7fef580d1f80 2024-04-26T23:19:34.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.481+0000 7fef5dada640 1 -- 172.21.15.47:0/2960879289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef580d1f80 msgr2=0x7fef5806ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:34.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.481+0000 7fef5dada640 1 --2- 172.21.15.47:0/2960879289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef580d1f80 0x7fef5806ce20 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fef5808dce0 tx=0x7fef5002a680 comp rx=0 tx=0).stop 2024-04-26T23:19:34.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.481+0000 7fef5dada640 1 -- 172.21.15.47:0/2960879289 shutdown_connections 2024-04-26T23:19:34.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.481+0000 7fef5dada640 1 --2- 172.21.15.47:0/2960879289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef580d1f80 0x7fef5806ce20 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:34.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.481+0000 7fef5dada640 1 --2- 172.21.15.47:0/2960879289 >> v2:172.21.15.203:3300/0 conn(0x7fef580d3c80 0x7fef580d1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:34.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.481+0000 7fef5dada640 1 -- 172.21.15.47:0/2960879289 >> 172.21.15.47:0/2960879289 conn(0x7fef5809e3f0 msgr2=0x7fef580d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:34.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.481+0000 7fef5dada640 1 -- 172.21.15.47:0/2960879289 shutdown_connections 2024-04-26T23:19:34.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.481+0000 7fef5dada640 1 -- 172.21.15.47:0/2960879289 wait complete. 2024-04-26T23:19:34.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.482+0000 7fef5dada640 1 Processor -- start 2024-04-26T23:19:34.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.482+0000 7fef5dada640 1 -- start start 2024-04-26T23:19:34.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.483+0000 7fef5dada640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fef580d1f80 0x7fef5815f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:34.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.483+0000 7fef5dada640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef580d3c80 0x7fef5815cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:34.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.483+0000 7fef5dada640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef5815f460 0x7fef580cc670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:34.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.483+0000 7fef5dada640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fef580d5f30 con 0x7fef5815f460 2024-04-26T23:19:34.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.483+0000 7fef5dada640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fef580d40e0 con 0x7fef580d3c80 2024-04-26T23:19:34.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.483+0000 7fef5dada640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fef580d0e70 con 0x7fef580d1f80 2024-04-26T23:19:34.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.483+0000 7fef57fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef5815f460 0x7fef580cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:34.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.483+0000 7fef57fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef5815f460 0x7fef580cc670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33990/0 (socket says 172.21.15.47:33990) 2024-04-26T23:19:34.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.483+0000 7fef57fff640 1 -- 172.21.15.47:0/257546093 learned_addr learned my addr 172.21.15.47:0/257546093 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:34.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.483+0000 7fef56ffd640 1 --2- 172.21.15.47:0/257546093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef580d3c80 0x7fef5815cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:34.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.483+0000 7fef577fe640 1 --2- 172.21.15.47:0/257546093 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fef580d1f80 0x7fef5815f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:34.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.483+0000 7fef57fff640 1 -- 172.21.15.47:0/257546093 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fef580d1f80 msgr2=0x7fef5815f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:34.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.483+0000 7fef57fff640 1 --2- 172.21.15.47:0/257546093 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fef580d1f80 0x7fef5815f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:34.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.483+0000 7fef57fff640 1 -- 172.21.15.47:0/257546093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef580d3c80 msgr2=0x7fef5815cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:34.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.483+0000 7fef57fff640 1 --2- 172.21.15.47:0/257546093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef580d3c80 0x7fef5815cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:34.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.483+0000 7fef57fff640 1 -- 172.21.15.47:0/257546093 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef5815d080 con 0x7fef5815f460 2024-04-26T23:19:34.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.484+0000 7fef57fff640 1 --2- 172.21.15.47:0/257546093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef5815f460 0x7fef580cc670 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fef50009e80 tx=0x7fef50004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:34.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.484+0000 7fef3ffff640 1 -- 172.21.15.47:0/257546093 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef50040070 con 0x7fef5815f460 2024-04-26T23:19:34.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.484+0000 7fef3ffff640 1 -- 172.21.15.47:0/257546093 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef50031bc0 con 0x7fef5815f460 2024-04-26T23:19:34.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.484+0000 7fef5dada640 1 -- 172.21.15.47:0/257546093 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef5815ffd0 con 0x7fef5815f460 2024-04-26T23:19:34.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.484+0000 7fef3ffff640 1 -- 172.21.15.47:0/257546093 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef50009450 con 0x7fef5815f460 2024-04-26T23:19:34.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.484+0000 7fef5dada640 1 -- 172.21.15.47:0/257546093 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef58160590 con 0x7fef5815f460 2024-04-26T23:19:34.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.485+0000 7fef5dada640 1 -- 172.21.15.47:0/257546093 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef58069600 con 0x7fef5815f460 2024-04-26T23:19:34.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.485+0000 7fef3ffff640 1 -- 172.21.15.47:0/257546093 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fef50039040 con 0x7fef5815f460 2024-04-26T23:19:34.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.486+0000 7fef3ffff640 1 --2- 172.21.15.47:0/257546093 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fef4406c8f0 0x7fef4406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:34.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.486+0000 7fef3ffff640 1 -- 172.21.15.47:0/257546093 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6850+0+0 (secure 0 0 0) 0x7fef500aced0 con 0x7fef5815f460 2024-04-26T23:19:34.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.486+0000 7fef577fe640 1 --2- 172.21.15.47:0/257546093 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fef4406c8f0 0x7fef4406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:34.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.487+0000 7fef577fe640 1 --2- 172.21.15.47:0/257546093 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fef4406c8f0 0x7fef4406edb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fef48007fd0 tx=0x7fef48007de0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:34.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.490+0000 7fef3ffff640 1 -- 172.21.15.47:0/257546093 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fef5007a820 con 0x7fef5815f460 2024-04-26T23:19:34.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:34.674+0000 7fef5dada640 1 -- 172.21.15.47:0/257546093 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fef580c8450 con 0x7fef5815f460 2024-04-26T23:19:35.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.355+0000 7fef3ffff640 1 -- 172.21.15.47:0/257546093 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v28) ==== 181+0+0 (secure 0 0 0) 0x7fef5007eb20 con 0x7fef5815f460 2024-04-26T23:19:35.357 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' removed 2024-04-26T23:19:35.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.359+0000 7fef5dada640 1 -- 172.21.15.47:0/257546093 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fef4406c8f0 msgr2=0x7fef4406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:35.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.359+0000 7fef5dada640 1 --2- 172.21.15.47:0/257546093 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fef4406c8f0 0x7fef4406edb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fef48007fd0 tx=0x7fef48007de0 comp rx=0 tx=0).stop 2024-04-26T23:19:35.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.359+0000 7fef5dada640 1 -- 172.21.15.47:0/257546093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef5815f460 msgr2=0x7fef580cc670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:35.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.359+0000 7fef5dada640 1 --2- 172.21.15.47:0/257546093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef5815f460 0x7fef580cc670 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fef50009e80 tx=0x7fef50004a40 comp rx=0 tx=0).stop 2024-04-26T23:19:35.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.359+0000 7fef5dada640 1 -- 172.21.15.47:0/257546093 shutdown_connections 2024-04-26T23:19:35.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.359+0000 7fef5dada640 1 --2- 172.21.15.47:0/257546093 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fef4406c8f0 0x7fef4406edb0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:35.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.359+0000 7fef5dada640 1 --2- 172.21.15.47:0/257546093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef5815f460 0x7fef580cc670 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:35.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.359+0000 7fef5dada640 1 --2- 172.21.15.47:0/257546093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef580d3c80 0x7fef5815cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:35.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.359+0000 7fef5dada640 1 --2- 172.21.15.47:0/257546093 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fef580d1f80 0x7fef5815f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:35.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.359+0000 7fef5dada640 1 -- 172.21.15.47:0/257546093 >> 172.21.15.47:0/257546093 conn(0x7fef5809e3f0 msgr2=0x7fef580b69a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:35.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.359+0000 7fef5dada640 1 -- 172.21.15.47:0/257546093 shutdown_connections 2024-04-26T23:19:35.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.359+0000 7fef5dada640 1 -- 172.21.15.47:0/257546093 wait complete. 2024-04-26T23:19:35.376 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T23:19:35.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.484+0000 7f7adae47640 1 Processor -- start 2024-04-26T23:19:35.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.484+0000 7f7adae47640 1 -- start start 2024-04-26T23:19:35.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.485+0000 7f7adae47640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7ad415f340 0x7f7ad40d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:35.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.485+0000 7f7adae47640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7ad415fe30 0x7f7ad40cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:35.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.485+0000 7f7adae47640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ad4163160 con 0x7f7ad41606c0 2024-04-26T23:19:35.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.485+0000 7f7adae47640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f7ad4162f10 con 0x7f7ad415f340 2024-04-26T23:19:35.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.485+0000 7f7adae47640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f7ad4163d70 con 0x7f7ad415fe30 2024-04-26T23:19:35.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.485+0000 7f7ad8be3640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7ad415f340 0x7f7ad40d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:35.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.485+0000 7f7ad8be3640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7ad415f340 0x7f7ad40d2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34004/0 (socket says 172.21.15.47:34004) 2024-04-26T23:19:35.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.485+0000 7f7ad8be3640 1 -- 172.21.15.47:0/2777656597 learned_addr learned my addr 172.21.15.47:0/2777656597 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:35.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.485+0000 7f7ad93e4640 1 --2- 172.21.15.47:0/2777656597 >> v2:172.21.15.203:3300/0 conn(0x7f7ad415fe30 0x7f7ad40cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:35.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.485+0000 7f7ad8be3640 1 -- 172.21.15.47:0/2777656597 >> v2:172.21.15.203:3300/0 conn(0x7f7ad415fe30 msgr2=0x7f7ad40cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:35.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.485+0000 7f7ad8be3640 1 --2- 172.21.15.47:0/2777656597 >> v2:172.21.15.203:3300/0 conn(0x7f7ad415fe30 0x7f7ad40cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:35.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.485+0000 7f7ad8be3640 1 -- 172.21.15.47:0/2777656597 >> v1:172.21.15.203:6789/0 conn(0x7f7ad41606c0 legacy=0x7f7ad415f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:35.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.485+0000 7f7ad8be3640 1 -- 172.21.15.47:0/2777656597 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ad4163440 con 0x7f7ad415f340 2024-04-26T23:19:35.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.485+0000 7f7ad93e4640 1 --2- 172.21.15.47:0/2777656597 >> v2:172.21.15.203:3300/0 conn(0x7f7ad415fe30 0x7f7ad40cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:19:35.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.486+0000 7f7ad8be3640 1 --2- 172.21.15.47:0/2777656597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ad415f340 0x7f7ad40d2550 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4068da0 tx=0x7f7ac802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b973a885cac2981 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:35.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.486+0000 7f7ad37fe640 1 -- 172.21.15.47:0/2777656597 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac8037070 con 0x7f7ad415f340 2024-04-26T23:19:35.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.486+0000 7f7ad37fe640 1 -- 172.21.15.47:0/2777656597 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ac8032440 con 0x7f7ad415f340 2024-04-26T23:19:35.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.486+0000 7f7ad37fe640 1 -- 172.21.15.47:0/2777656597 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac8032810 con 0x7f7ad415f340 2024-04-26T23:19:35.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.487+0000 7f7adae47640 1 -- 172.21.15.47:0/2777656597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ad415f340 msgr2=0x7f7ad40d2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:35.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.487+0000 7f7adae47640 1 --2- 172.21.15.47:0/2777656597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ad415f340 0x7f7ad40d2550 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4068da0 tx=0x7f7ac802a230 comp rx=0 tx=0).stop 2024-04-26T23:19:35.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.487+0000 7f7adae47640 1 -- 172.21.15.47:0/2777656597 shutdown_connections 2024-04-26T23:19:35.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.487+0000 7f7adae47640 1 --2- 172.21.15.47:0/2777656597 >> v2:172.21.15.203:3300/0 conn(0x7f7ad415fe30 0x7f7ad40cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:35.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.487+0000 7f7adae47640 1 --2- 172.21.15.47:0/2777656597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ad415f340 0x7f7ad40d2550 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:35.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.487+0000 7f7adae47640 1 -- 172.21.15.47:0/2777656597 >> 172.21.15.47:0/2777656597 conn(0x7f7ad409ba50 msgr2=0x7f7ad409be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:35.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.487+0000 7f7adae47640 1 -- 172.21.15.47:0/2777656597 shutdown_connections 2024-04-26T23:19:35.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.487+0000 7f7adae47640 1 -- 172.21.15.47:0/2777656597 wait complete. 2024-04-26T23:19:35.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.488+0000 7f7adae47640 1 Processor -- start 2024-04-26T23:19:35.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.488+0000 7f7adae47640 1 -- start start 2024-04-26T23:19:35.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.489+0000 7f7adae47640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ad415f340 0x7f7ad4156de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:35.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.489+0000 7f7adae47640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ad415fe30 0x7f7ad40b69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:35.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.489+0000 7f7adae47640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ad41606c0 0x7f7ad415bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:35.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.489+0000 7f7adae47640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad40d2a90 con 0x7f7ad415f340 2024-04-26T23:19:35.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.489+0000 7f7adae47640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad4163a00 con 0x7f7ad41606c0 2024-04-26T23:19:35.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.489+0000 7f7adae47640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f7ad4160230 con 0x7f7ad415fe30 2024-04-26T23:19:35.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.489+0000 7f7ad8be3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ad415f340 0x7f7ad4156de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:35.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.489+0000 7f7ad8be3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ad415f340 0x7f7ad4156de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34014/0 (socket says 172.21.15.47:34014) 2024-04-26T23:19:35.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.489+0000 7f7ad8be3640 1 -- 172.21.15.47:0/2888086449 learned_addr learned my addr 172.21.15.47:0/2888086449 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:35.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.489+0000 7f7ad93e4640 1 --2- 172.21.15.47:0/2888086449 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ad41606c0 0x7f7ad415bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:35.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.489+0000 7f7ad3fff640 1 --2- 172.21.15.47:0/2888086449 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ad415fe30 0x7f7ad40b69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:35.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.490+0000 7f7ad8be3640 1 -- 172.21.15.47:0/2888086449 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ad415fe30 msgr2=0x7f7ad40b69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:35.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.490+0000 7f7ad8be3640 1 --2- 172.21.15.47:0/2888086449 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ad415fe30 0x7f7ad40b69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:35.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.490+0000 7f7ad8be3640 1 -- 172.21.15.47:0/2888086449 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ad41606c0 msgr2=0x7f7ad415bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:35.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.490+0000 7f7ad8be3640 1 --2- 172.21.15.47:0/2888086449 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ad41606c0 0x7f7ad415bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:35.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.490+0000 7f7ad8be3640 1 -- 172.21.15.47:0/2888086449 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ad40b7240 con 0x7f7ad415f340 2024-04-26T23:19:35.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.490+0000 7f7ad8be3640 1 --2- 172.21.15.47:0/2888086449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ad415f340 0x7f7ad4156de0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8000c00 tx=0x7f7ac80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:35.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.490+0000 7f7ad17fa640 1 -- 172.21.15.47:0/2888086449 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac8041020 con 0x7f7ad415f340 2024-04-26T23:19:35.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.490+0000 7f7ad17fa640 1 -- 172.21.15.47:0/2888086449 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ac8030970 con 0x7f7ad415f340 2024-04-26T23:19:35.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.490+0000 7f7adae47640 1 -- 172.21.15.47:0/2888086449 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ad4157dc0 con 0x7f7ad415f340 2024-04-26T23:19:35.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.490+0000 7f7ad17fa640 1 -- 172.21.15.47:0/2888086449 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac8037040 con 0x7f7ad415f340 2024-04-26T23:19:35.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.490+0000 7f7adae47640 1 -- 172.21.15.47:0/2888086449 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ad4158380 con 0x7f7ad415f340 2024-04-26T23:19:35.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.491+0000 7f7adae47640 1 -- 172.21.15.47:0/2888086449 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ad4069600 con 0x7f7ad415f340 2024-04-26T23:19:35.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.492+0000 7f7ad17fa640 1 -- 172.21.15.47:0/2888086449 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7ac8044020 con 0x7f7ad415f340 2024-04-26T23:19:35.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.492+0000 7f7ad17fa640 1 --2- 172.21.15.47:0/2888086449 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f7ab806c8f0 0x7f7ab806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:35.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.492+0000 7f7ad17fa640 1 -- 172.21.15.47:0/2888086449 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6412+0+0 (secure 0 0 0) 0x7f7ac80ac8e0 con 0x7f7ad415f340 2024-04-26T23:19:35.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.493+0000 7f7ad3fff640 1 --2- 172.21.15.47:0/2888086449 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f7ab806c8f0 0x7f7ab806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:35.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.493+0000 7f7ad3fff640 1 --2- 172.21.15.47:0/2888086449 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f7ab806c8f0 0x7f7ab806edb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0006fd0 tx=0x7f7ac0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:35.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.497+0000 7f7ad17fa640 1 -- 172.21.15.47:0/2888086449 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7ac804b030 con 0x7f7ad415f340 2024-04-26T23:19:35.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:35.687+0000 7f7adae47640 1 -- 172.21.15.47:0/2888086449 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7ad4068e00 con 0x7f7ad415f340 2024-04-26T23:19:36.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.357+0000 7f7ad17fa640 1 -- 172.21.15.47:0/2888086449 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v29) ==== 169+0+0 (secure 0 0 0) 0x7f7ac807a9b0 con 0x7f7ad415f340 2024-04-26T23:19:36.358 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' removed 2024-04-26T23:19:36.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.361+0000 7f7adae47640 1 -- 172.21.15.47:0/2888086449 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f7ab806c8f0 msgr2=0x7f7ab806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:36.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.361+0000 7f7adae47640 1 --2- 172.21.15.47:0/2888086449 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f7ab806c8f0 0x7f7ab806edb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0006fd0 tx=0x7f7ac0008040 comp rx=0 tx=0).stop 2024-04-26T23:19:36.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.361+0000 7f7adae47640 1 -- 172.21.15.47:0/2888086449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ad415f340 msgr2=0x7f7ad4156de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:36.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.361+0000 7f7adae47640 1 --2- 172.21.15.47:0/2888086449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ad415f340 0x7f7ad4156de0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8000c00 tx=0x7f7ac80046c0 comp rx=0 tx=0).stop 2024-04-26T23:19:36.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.361+0000 7f7adae47640 1 -- 172.21.15.47:0/2888086449 shutdown_connections 2024-04-26T23:19:36.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.361+0000 7f7adae47640 1 --2- 172.21.15.47:0/2888086449 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f7ab806c8f0 0x7f7ab806edb0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:36.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.361+0000 7f7adae47640 1 --2- 172.21.15.47:0/2888086449 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ad41606c0 0x7f7ad415bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:36.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.361+0000 7f7adae47640 1 --2- 172.21.15.47:0/2888086449 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ad415fe30 0x7f7ad40b69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:36.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.361+0000 7f7adae47640 1 --2- 172.21.15.47:0/2888086449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ad415f340 0x7f7ad4156de0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:36.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.361+0000 7f7adae47640 1 -- 172.21.15.47:0/2888086449 >> 172.21.15.47:0/2888086449 conn(0x7f7ad409ba50 msgr2=0x7f7ad40cf640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:36.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.362+0000 7f7adae47640 1 -- 172.21.15.47:0/2888086449 shutdown_connections 2024-04-26T23:19:36.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.362+0000 7f7adae47640 1 -- 172.21.15.47:0/2888086449 wait complete. 2024-04-26T23:19:36.387 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-26T23:19:36.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.496+0000 7f1879e2a640 1 Processor -- start 2024-04-26T23:19:36.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.496+0000 7f1879e2a640 1 -- start start 2024-04-26T23:19:36.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.497+0000 7f1879e2a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1874145630 0x7f18740cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:36.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.497+0000 7f1879e2a640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f187415f980 0x7f18740b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:36.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.497+0000 7f1879e2a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18740b33e0 con 0x7f18740b2fc0 2024-04-26T23:19:36.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.497+0000 7f1879e2a640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f18740cfd70 con 0x7f187415f980 2024-04-26T23:19:36.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.497+0000 7f1879e2a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f18740d0050 con 0x7f1874145630 2024-04-26T23:19:36.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.497+0000 7f18737fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1874145630 0x7f18740cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:36.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.497+0000 7f1872ffd640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f187415f980 0x7f18740b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:36.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.497+0000 7f18737fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1874145630 0x7f18740cf500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.47:43344/0 (socket says 172.21.15.47:43344) 2024-04-26T23:19:36.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.497+0000 7f1872ffd640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f187415f980 0x7f18740b2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.47:51584/0 (socket says 172.21.15.47:51584) 2024-04-26T23:19:36.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.497+0000 7f18737fe640 1 -- 172.21.15.47:0/3449061796 learned_addr learned my addr 172.21.15.47:0/3449061796 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:36.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.497+0000 7f1873fff640 1 --1- 172.21.15.47:0/3449061796 >> v1:172.21.15.203:6789/0 conn(0x7f18740b2fc0 0x7f18740730d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.47:35036/0 (socket says 172.21.15.47:35036) 2024-04-26T23:19:36.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.498+0000 7f18737fe640 1 -- 172.21.15.47:0/3449061796 >> v2:172.21.15.203:3300/0 conn(0x7f187415f980 msgr2=0x7f18740b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:36.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.498+0000 7f18737fe640 1 --2- 172.21.15.47:0/3449061796 >> v2:172.21.15.203:3300/0 conn(0x7f187415f980 0x7f18740b2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:36.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.498+0000 7f18737fe640 1 -- 172.21.15.47:0/3449061796 >> v1:172.21.15.203:6789/0 conn(0x7f18740b2fc0 legacy=0x7f18740730d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:36.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.498+0000 7f18737fe640 1 -- 172.21.15.47:0/3449061796 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18740d0330 con 0x7f1874145630 2024-04-26T23:19:36.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.498+0000 7f18737fe640 1 --2- 172.21.15.47:0/3449061796 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1874145630 0x7f18740cf500 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1864009870 tx=0x7f186402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3967a0ce9f6b429 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:36.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.499+0000 7f18727fc640 1 -- 172.21.15.47:0/3449061796 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1864037070 con 0x7f1874145630 2024-04-26T23:19:36.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.499+0000 7f18727fc640 1 -- 172.21.15.47:0/3449061796 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1864032440 con 0x7f1874145630 2024-04-26T23:19:36.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.499+0000 7f18727fc640 1 -- 172.21.15.47:0/3449061796 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1864032810 con 0x7f1874145630 2024-04-26T23:19:36.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.499+0000 7f1879e2a640 1 -- 172.21.15.47:0/3449061796 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1874145630 msgr2=0x7f18740cf500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:36.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.499+0000 7f1879e2a640 1 --2- 172.21.15.47:0/3449061796 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1874145630 0x7f18740cf500 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1864009870 tx=0x7f186402a270 comp rx=0 tx=0).stop 2024-04-26T23:19:36.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.499+0000 7f1879e2a640 1 -- 172.21.15.47:0/3449061796 shutdown_connections 2024-04-26T23:19:36.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.499+0000 7f1879e2a640 1 --2- 172.21.15.47:0/3449061796 >> v2:172.21.15.203:3300/0 conn(0x7f187415f980 0x7f18740b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:36.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.499+0000 7f1879e2a640 1 --2- 172.21.15.47:0/3449061796 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1874145630 0x7f18740cf500 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:36.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.499+0000 7f1879e2a640 1 -- 172.21.15.47:0/3449061796 >> 172.21.15.47:0/3449061796 conn(0x7f187406f310 msgr2=0x7f187406f720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:36.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.499+0000 7f1879e2a640 1 -- 172.21.15.47:0/3449061796 shutdown_connections 2024-04-26T23:19:36.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.499+0000 7f1879e2a640 1 -- 172.21.15.47:0/3449061796 wait complete. 2024-04-26T23:19:36.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.500+0000 7f1879e2a640 1 Processor -- start 2024-04-26T23:19:36.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.501+0000 7f1879e2a640 1 -- start start 2024-04-26T23:19:36.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.501+0000 7f1879e2a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18740b2fc0 0x7f1874144360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:36.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.501+0000 7f1879e2a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1874145630 0x7f18741425b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:36.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.501+0000 7f1879e2a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f187415f980 0x7f187415c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:36.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.501+0000 7f1879e2a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f18740cfa40 con 0x7f18740b2fc0 2024-04-26T23:19:36.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.501+0000 7f1879e2a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f18740d08f0 con 0x7f187415f980 2024-04-26T23:19:36.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.501+0000 7f1879e2a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f1874145a10 con 0x7f1874145630 2024-04-26T23:19:36.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.501+0000 7f18737fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18740b2fc0 0x7f1874144360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:36.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.501+0000 7f18737fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18740b2fc0 0x7f1874144360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34016/0 (socket says 172.21.15.47:34016) 2024-04-26T23:19:36.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.501+0000 7f18737fe640 1 -- 172.21.15.47:0/3467624917 learned_addr learned my addr 172.21.15.47:0/3467624917 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:36.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.502+0000 7f1873fff640 1 --2- 172.21.15.47:0/3467624917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f187415f980 0x7f187415c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:36.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.502+0000 7f1872ffd640 1 --2- 172.21.15.47:0/3467624917 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1874145630 0x7f18741425b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:36.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.502+0000 7f18737fe640 1 -- 172.21.15.47:0/3467624917 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1874145630 msgr2=0x7f18741425b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:36.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.502+0000 7f18737fe640 1 --2- 172.21.15.47:0/3467624917 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1874145630 0x7f18741425b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:36.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.502+0000 7f18737fe640 1 -- 172.21.15.47:0/3467624917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f187415f980 msgr2=0x7f187415c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:36.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.502+0000 7f18737fe640 1 --2- 172.21.15.47:0/3467624917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f187415f980 0x7f187415c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:36.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.502+0000 7f18737fe640 1 -- 172.21.15.47:0/3467624917 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f187414dc50 con 0x7f18740b2fc0 2024-04-26T23:19:36.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.502+0000 7f1873fff640 1 --2- 172.21.15.47:0/3467624917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f187415f980 0x7f187415c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:19:36.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.502+0000 7f1872ffd640 1 --2- 172.21.15.47:0/3467624917 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1874145630 0x7f18741425b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:19:36.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.502+0000 7f18737fe640 1 --2- 172.21.15.47:0/3467624917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18740b2fc0 0x7f1874144360 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f186402a2b0 tx=0x7f18640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:36.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.502+0000 7f185bfff640 1 -- 172.21.15.47:0/3467624917 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1864037040 con 0x7f18740b2fc0 2024-04-26T23:19:36.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.502+0000 7f185bfff640 1 -- 172.21.15.47:0/3467624917 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1864042070 con 0x7f18740b2fc0 2024-04-26T23:19:36.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.502+0000 7f1879e2a640 1 -- 172.21.15.47:0/3467624917 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1874141630 con 0x7f18740b2fc0 2024-04-26T23:19:36.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.502+0000 7f185bfff640 1 -- 172.21.15.47:0/3467624917 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1864030880 con 0x7f18740b2fc0 2024-04-26T23:19:36.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.502+0000 7f1879e2a640 1 -- 172.21.15.47:0/3467624917 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1874142af0 con 0x7f18740b2fc0 2024-04-26T23:19:36.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.503+0000 7f1879e2a640 1 -- 172.21.15.47:0/3467624917 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1874141450 con 0x7f18740b2fc0 2024-04-26T23:19:36.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.504+0000 7f185bfff640 1 -- 172.21.15.47:0/3467624917 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1864030a20 con 0x7f18740b2fc0 2024-04-26T23:19:36.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.504+0000 7f185bfff640 1 --2- 172.21.15.47:0/3467624917 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f186006c8f0 0x7f186006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:36.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.504+0000 7f185bfff640 1 -- 172.21.15.47:0/3467624917 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5853+0+0 (secure 0 0 0) 0x7f18640ac410 con 0x7f18740b2fc0 2024-04-26T23:19:36.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.504+0000 7f1872ffd640 1 --2- 172.21.15.47:0/3467624917 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f186006c8f0 0x7f186006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:36.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.505+0000 7f1872ffd640 1 --2- 172.21.15.47:0/3467624917 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f186006c8f0 0x7f186006edb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f18740b5ff0 tx=0x7f1868008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:36.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.509+0000 7f185bfff640 1 -- 172.21.15.47:0/3467624917 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1874141450 con 0x7f18740b2fc0 2024-04-26T23:19:36.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:36.692+0000 7f1879e2a640 1 -- 172.21.15.47:0/3467624917 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1874069600 con 0x7f18740b2fc0 2024-04-26T23:19:37.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.372+0000 7f185bfff640 1 -- 172.21.15.47:0/3467624917 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v30) ==== 178+0+0 (secure 0 0 0) 0x7f1864045360 con 0x7f18740b2fc0 2024-04-26T23:19:37.374 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data_ec' removed 2024-04-26T23:19:37.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.378+0000 7f1879e2a640 1 -- 172.21.15.47:0/3467624917 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f186006c8f0 msgr2=0x7f186006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:37.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.378+0000 7f1879e2a640 1 --2- 172.21.15.47:0/3467624917 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f186006c8f0 0x7f186006edb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f18740b5ff0 tx=0x7f1868008040 comp rx=0 tx=0).stop 2024-04-26T23:19:37.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.378+0000 7f1879e2a640 1 -- 172.21.15.47:0/3467624917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18740b2fc0 msgr2=0x7f1874144360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:37.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.378+0000 7f1879e2a640 1 --2- 172.21.15.47:0/3467624917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18740b2fc0 0x7f1874144360 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f186402a2b0 tx=0x7f18640046c0 comp rx=0 tx=0).stop 2024-04-26T23:19:37.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.378+0000 7f1879e2a640 1 -- 172.21.15.47:0/3467624917 shutdown_connections 2024-04-26T23:19:37.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.378+0000 7f1879e2a640 1 --2- 172.21.15.47:0/3467624917 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f186006c8f0 0x7f186006edb0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:37.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.378+0000 7f1879e2a640 1 --2- 172.21.15.47:0/3467624917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f187415f980 0x7f187415c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:37.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.378+0000 7f1879e2a640 1 --2- 172.21.15.47:0/3467624917 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1874145630 0x7f18741425b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:37.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.378+0000 7f1879e2a640 1 --2- 172.21.15.47:0/3467624917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18740b2fc0 0x7f1874144360 secure :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f186402a2b0 tx=0x7f18640046c0 comp rx=0 tx=0).stop 2024-04-26T23:19:37.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.378+0000 7f1879e2a640 1 -- 172.21.15.47:0/3467624917 >> 172.21.15.47:0/3467624917 conn(0x7f187406f310 msgr2=0x7f18740b1350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:37.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.381+0000 7f1879e2a640 1 -- 172.21.15.47:0/3467624917 shutdown_connections 2024-04-26T23:19:37.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.381+0000 7f1879e2a640 1 -- 172.21.15.47:0/3467624917 wait complete. 2024-04-26T23:19:37.400 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T23:19:37.400 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-26T23:19:37.400 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T23:19:37.400 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:19:37.400 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T23:19:37.400 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-26T23:19:37.401 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T23:19:37.401 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:19:37.401 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T23:19:37.401 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-26T23:19:37.401 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T23:19:37.401 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:19:37.401 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T23:19:37.401 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-26T23:19:37.402 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T23:19:37.402 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:19:37.411 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T23:19:37.411 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T23:19:37.412 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T23:19:37.412 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T23:19:37.413 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T23:19:37.414 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T23:19:37.414 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T23:19:37.415 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T23:19:37.454 INFO:tasks.ceph.mds.c:Started 2024-04-26T23:19:37.455 DEBUG:teuthology.parallel:result is None 2024-04-26T23:19:37.456 INFO:tasks.ceph.mds.a:Started 2024-04-26T23:19:37.456 DEBUG:teuthology.parallel:result is None 2024-04-26T23:19:37.457 INFO:tasks.ceph.mds.d:Started 2024-04-26T23:19:37.457 DEBUG:teuthology.parallel:result is None 2024-04-26T23:19:37.458 INFO:tasks.ceph.mds.b:Started 2024-04-26T23:19:37.458 DEBUG:teuthology.parallel:result is None 2024-04-26T23:19:37.458 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-26T23:19:37.537 INFO:tasks.ceph.mds.c.smithi047.stdout:starting mds.c at 2024-04-26T23:19:37.538 INFO:tasks.ceph.mds.a.smithi047.stdout:starting mds.a at 2024-04-26T23:19:37.538 INFO:tasks.ceph.mds.d.smithi203.stdout:starting mds.d at 2024-04-26T23:19:37.539 INFO:tasks.ceph.mds.b.smithi203.stdout:starting mds.b at 2024-04-26T23:19:37.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.569+0000 7f42bd6a0640 1 Processor -- start 2024-04-26T23:19:37.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.569+0000 7f42bd6a0640 1 -- start start 2024-04-26T23:19:37.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.570+0000 7f42bd6a0640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f42b8163610 0x7f42b80d5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:37.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.570+0000 7f42bd6a0640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f42b80d2830 0x7f42b80cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:37.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.570+0000 7f42bd6a0640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42b80d1a70 con 0x7f42b815f5a0 2024-04-26T23:19:37.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.570+0000 7f42bd6a0640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f42b8163d30 con 0x7f42b8163610 2024-04-26T23:19:37.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.570+0000 7f42bd6a0640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f42b80d1d50 con 0x7f42b80d2830 2024-04-26T23:19:37.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.570+0000 7f42b67fc640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f42b8163610 0x7f42b80d5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:37.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.570+0000 7f42b67fc640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f42b8163610 0x7f42b80d5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34058/0 (socket says 172.21.15.47:34058) 2024-04-26T23:19:37.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.570+0000 7f42b67fc640 1 -- 172.21.15.47:0/2846143787 learned_addr learned my addr 172.21.15.47:0/2846143787 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:37.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.570+0000 7f42b77fe640 1 --2- 172.21.15.47:0/2846143787 >> v2:172.21.15.203:3300/0 conn(0x7f42b80d2830 0x7f42b80cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:37.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.570+0000 7f42b67fc640 1 -- 172.21.15.47:0/2846143787 >> v2:172.21.15.203:3300/0 conn(0x7f42b80d2830 msgr2=0x7f42b80cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:37.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.570+0000 7f42b67fc640 1 --2- 172.21.15.47:0/2846143787 >> v2:172.21.15.203:3300/0 conn(0x7f42b80d2830 0x7f42b80cfda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:37.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.570+0000 7f42b67fc640 1 -- 172.21.15.47:0/2846143787 >> v1:172.21.15.203:6789/0 conn(0x7f42b815f5a0 legacy=0x7f42b8162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:37.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.570+0000 7f42b67fc640 1 -- 172.21.15.47:0/2846143787 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42b80d2030 con 0x7f42b8163610 2024-04-26T23:19:37.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.571+0000 7f42b67fc640 1 --2- 172.21.15.47:0/2846143787 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42b8163610 0x7f42b80d5cb0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f42b808ea00 tx=0x7f42a002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77017f45aa915e8d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:37.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.571+0000 7f42b5ffb640 1 -- 172.21.15.47:0/2846143787 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a0037070 con 0x7f42b8163610 2024-04-26T23:19:37.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.571+0000 7f42b5ffb640 1 -- 172.21.15.47:0/2846143787 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42a0032440 con 0x7f42b8163610 2024-04-26T23:19:37.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.571+0000 7f42b5ffb640 1 -- 172.21.15.47:0/2846143787 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a0032810 con 0x7f42b8163610 2024-04-26T23:19:37.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.572+0000 7f42bd6a0640 1 -- 172.21.15.47:0/2846143787 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42b8163610 msgr2=0x7f42b80d5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:37.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.572+0000 7f42bd6a0640 1 --2- 172.21.15.47:0/2846143787 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42b8163610 0x7f42b80d5cb0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f42b808ea00 tx=0x7f42a002a1e0 comp rx=0 tx=0).stop 2024-04-26T23:19:37.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.572+0000 7f42bd6a0640 1 -- 172.21.15.47:0/2846143787 shutdown_connections 2024-04-26T23:19:37.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.572+0000 7f42bd6a0640 1 --2- 172.21.15.47:0/2846143787 >> v2:172.21.15.203:3300/0 conn(0x7f42b80d2830 0x7f42b80cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:37.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.572+0000 7f42bd6a0640 1 --2- 172.21.15.47:0/2846143787 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42b8163610 0x7f42b80d5cb0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:37.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.572+0000 7f42bd6a0640 1 -- 172.21.15.47:0/2846143787 >> 172.21.15.47:0/2846143787 conn(0x7f42b80d5190 msgr2=0x7f42b80d55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:37.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.572+0000 7f42bd6a0640 1 -- 172.21.15.47:0/2846143787 shutdown_connections 2024-04-26T23:19:37.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.572+0000 7f42bd6a0640 1 -- 172.21.15.47:0/2846143787 wait complete. 2024-04-26T23:19:37.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.573+0000 7f42bd6a0640 1 Processor -- start 2024-04-26T23:19:37.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.573+0000 7f42bd6a0640 1 -- start start 2024-04-26T23:19:37.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.574+0000 7f42bd6a0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42b80d2830 0x7f42b814aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:37.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.574+0000 7f42bd6a0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f42b8163610 0x7f42b81603a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:37.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.574+0000 7f42bd6a0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f42b814df60 0x7f42b814d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:37.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.574+0000 7f42bd6a0640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f42b80d3c20 con 0x7f42b80d2830 2024-04-26T23:19:37.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.574+0000 7f42bd6a0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f42b80d4190 con 0x7f42b8163610 2024-04-26T23:19:37.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.574+0000 7f42bd6a0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f42b80d3da0 con 0x7f42b814df60 2024-04-26T23:19:37.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.574+0000 7f42b6ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42b80d2830 0x7f42b814aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:37.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.574+0000 7f42b6ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42b80d2830 0x7f42b814aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34060/0 (socket says 172.21.15.47:34060) 2024-04-26T23:19:37.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.574+0000 7f42b6ffd640 1 -- 172.21.15.47:0/1837058864 learned_addr learned my addr 172.21.15.47:0/1837058864 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:37.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.574+0000 7f42b67fc640 1 --2- 172.21.15.47:0/1837058864 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f42b8163610 0x7f42b81603a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:37.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.574+0000 7f42b77fe640 1 --2- 172.21.15.47:0/1837058864 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f42b814df60 0x7f42b814d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:37.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.574+0000 7f42b6ffd640 1 -- 172.21.15.47:0/1837058864 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f42b814df60 msgr2=0x7f42b814d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:37.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.575+0000 7f42b6ffd640 1 --2- 172.21.15.47:0/1837058864 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f42b814df60 0x7f42b814d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:37.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.575+0000 7f42b6ffd640 1 -- 172.21.15.47:0/1837058864 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f42b8163610 msgr2=0x7f42b81603a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:37.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.575+0000 7f42b6ffd640 1 --2- 172.21.15.47:0/1837058864 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f42b8163610 0x7f42b81603a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:37.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.575+0000 7f42b6ffd640 1 -- 172.21.15.47:0/1837058864 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42b814b280 con 0x7f42b80d2830 2024-04-26T23:19:37.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.575+0000 7f42b67fc640 1 --2- 172.21.15.47:0/1837058864 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f42b8163610 0x7f42b81603a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:19:37.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.575+0000 7f42b77fe640 1 --2- 172.21.15.47:0/1837058864 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f42b814df60 0x7f42b814d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:19:37.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.575+0000 7f42b6ffd640 1 --2- 172.21.15.47:0/1837058864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42b80d2830 0x7f42b814aad0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f42a800c400 tx=0x7f42a800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:37.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.575+0000 7f42af7fe640 1 -- 172.21.15.47:0/1837058864 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a80052d0 con 0x7f42b80d2830 2024-04-26T23:19:37.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.575+0000 7f42af7fe640 1 -- 172.21.15.47:0/1837058864 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42a8005490 con 0x7f42b80d2830 2024-04-26T23:19:37.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.575+0000 7f42bd6a0640 1 -- 172.21.15.47:0/1837058864 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42b8160bf0 con 0x7f42b80d2830 2024-04-26T23:19:37.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.575+0000 7f42af7fe640 1 -- 172.21.15.47:0/1837058864 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a8007260 con 0x7f42b80d2830 2024-04-26T23:19:37.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.575+0000 7f42bd6a0640 1 -- 172.21.15.47:0/1837058864 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42b81611b0 con 0x7f42b80d2830 2024-04-26T23:19:37.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.576+0000 7f42bd6a0640 1 -- 172.21.15.47:0/1837058864 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42b808d320 con 0x7f42b80d2830 2024-04-26T23:19:37.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.577+0000 7f42af7fe640 1 -- 172.21.15.47:0/1837058864 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f42a8007400 con 0x7f42b80d2830 2024-04-26T23:19:37.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.577+0000 7f42af7fe640 1 --2- 172.21.15.47:0/1837058864 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f429806c920 0x7f429806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:37.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.577+0000 7f42af7fe640 1 -- 172.21.15.47:0/1837058864 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5335+0+0 (secure 0 0 0) 0x7f42a808ae60 con 0x7f42b80d2830 2024-04-26T23:19:37.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.577+0000 7f42b67fc640 1 --2- 172.21.15.47:0/1837058864 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f429806c920 0x7f429806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:37.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.578+0000 7f42b67fc640 1 --2- 172.21.15.47:0/1837058864 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f429806c920 0x7f429806ede0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f42a0034000 tx=0x7f42a002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:37.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.582+0000 7f42af7fe640 1 -- 172.21.15.47:0/1837058864 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f42b808d320 con 0x7f42b80d2830 2024-04-26T23:19:37.615 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-04-26T23:19:37.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:37.779+0000 7f42bd6a0640 1 -- 172.21.15.47:0/1837058864 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f42b80694f0 con 0x7f42b80d2830 2024-04-26T23:19:38.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.382+0000 7f42af7fe640 1 -- 172.21.15.47:0/1837058864 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v32) ==== 97+0+0 (secure 0 0 0) 0x7f42a8059490 con 0x7f42b80d2830 2024-04-26T23:19:38.383 INFO:teuthology.orchestra.run.smithi047.stderr: removed all blocklist entries 2024-04-26T23:19:38.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.384+0000 7f42bd6a0640 1 -- 172.21.15.47:0/1837058864 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f429806c920 msgr2=0x7f429806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:38.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.384+0000 7f42bd6a0640 1 --2- 172.21.15.47:0/1837058864 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f429806c920 0x7f429806ede0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f42a0034000 tx=0x7f42a002e000 comp rx=0 tx=0).stop 2024-04-26T23:19:38.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.384+0000 7f42bd6a0640 1 -- 172.21.15.47:0/1837058864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42b80d2830 msgr2=0x7f42b814aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:38.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.384+0000 7f42bd6a0640 1 --2- 172.21.15.47:0/1837058864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42b80d2830 0x7f42b814aad0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f42a800c400 tx=0x7f42a800c8c0 comp rx=0 tx=0).stop 2024-04-26T23:19:38.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.384+0000 7f42bd6a0640 1 -- 172.21.15.47:0/1837058864 shutdown_connections 2024-04-26T23:19:38.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.384+0000 7f42bd6a0640 1 --2- 172.21.15.47:0/1837058864 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f429806c920 0x7f429806ede0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:38.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.384+0000 7f42bd6a0640 1 --2- 172.21.15.47:0/1837058864 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f42b814df60 0x7f42b814d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:38.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.384+0000 7f42bd6a0640 1 --2- 172.21.15.47:0/1837058864 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f42b8163610 0x7f42b81603a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:38.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.384+0000 7f42bd6a0640 1 --2- 172.21.15.47:0/1837058864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42b80d2830 0x7f42b814aad0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:38.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.384+0000 7f42bd6a0640 1 -- 172.21.15.47:0/1837058864 >> 172.21.15.47:0/1837058864 conn(0x7f42b80d5190 msgr2=0x7f42b80cfd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:38.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.384+0000 7f42bd6a0640 1 -- 172.21.15.47:0/1837058864 shutdown_connections 2024-04-26T23:19:38.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.384+0000 7f42bd6a0640 1 -- 172.21.15.47:0/1837058864 wait complete. 2024-04-26T23:19:38.398 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-26T23:19:38.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.497+0000 7f6600d46640 1 Processor -- start 2024-04-26T23:19:38.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.497+0000 7f6600d46640 1 -- start start 2024-04-26T23:19:38.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.498+0000 7f6600d46640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f65fc0d0070 0x7f65fc0d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:38.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.498+0000 7f6600d46640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65fc06d560 0x7f65fc06db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:38.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.498+0000 7f6600d46640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65fc0d2420 con 0x7f65fc06e0a0 2024-04-26T23:19:38.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.498+0000 7f6600d46640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f65fc162d00 con 0x7f65fc0d0070 2024-04-26T23:19:38.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.498+0000 7f6600d46640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f65fc0d2d10 con 0x7f65fc06d560 2024-04-26T23:19:38.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.498+0000 7f65fa59c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f65fc0d0070 0x7f65fc0d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:38.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.498+0000 7f65fad9d640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f65fc06e0a0 0x7f65fc162f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40942/0 (socket says 172.21.15.47:40942) 2024-04-26T23:19:38.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.498+0000 7f65fad9d640 1 -- 172.21.15.47:0/4155564842 learned_addr learned my addr 172.21.15.47:0/4155564842 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:38.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.499+0000 7f65fa59c640 1 -- 172.21.15.47:0/4155564842 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65fc06d560 msgr2=0x7f65fc06db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:38.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.499+0000 7f65fa59c640 1 --2- 172.21.15.47:0/4155564842 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65fc06d560 0x7f65fc06db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:38.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.499+0000 7f65fa59c640 1 -- 172.21.15.47:0/4155564842 >> v1:172.21.15.47:6789/0 conn(0x7f65fc06e0a0 legacy=0x7f65fc162f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:38.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.499+0000 7f65fa59c640 1 -- 172.21.15.47:0/4155564842 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65fc0cd6b0 con 0x7f65fc0d0070 2024-04-26T23:19:38.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.499+0000 7f65fa59c640 1 --2- 172.21.15.47:0/4155564842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65fc0d0070 0x7f65fc0d1d40 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f65f40097a0 tx=0x7f65f402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a27d6b07260838e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:38.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.500+0000 7f65f959a640 1 -- 172.21.15.47:0/4155564842 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f4037070 con 0x7f65fc0d0070 2024-04-26T23:19:38.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.500+0000 7f65f959a640 1 -- 172.21.15.47:0/4155564842 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65f4032440 con 0x7f65fc0d0070 2024-04-26T23:19:38.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.500+0000 7f65f959a640 1 -- 172.21.15.47:0/4155564842 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f4032810 con 0x7f65fc0d0070 2024-04-26T23:19:38.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.500+0000 7f6600d46640 1 -- 172.21.15.47:0/4155564842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65fc0d0070 msgr2=0x7f65fc0d1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:38.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.500+0000 7f6600d46640 1 --2- 172.21.15.47:0/4155564842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65fc0d0070 0x7f65fc0d1d40 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f65f40097a0 tx=0x7f65f402a210 comp rx=0 tx=0).stop 2024-04-26T23:19:38.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.500+0000 7f6600d46640 1 -- 172.21.15.47:0/4155564842 shutdown_connections 2024-04-26T23:19:38.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.500+0000 7f6600d46640 1 --2- 172.21.15.47:0/4155564842 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65fc06d560 0x7f65fc06db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:38.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.500+0000 7f6600d46640 1 --2- 172.21.15.47:0/4155564842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65fc0d0070 0x7f65fc0d1d40 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:38.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.500+0000 7f6600d46640 1 -- 172.21.15.47:0/4155564842 >> 172.21.15.47:0/4155564842 conn(0x7f65fc0d2ec0 msgr2=0x7f65fc0d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:38.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.500+0000 7f6600d46640 1 -- 172.21.15.47:0/4155564842 shutdown_connections 2024-04-26T23:19:38.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.500+0000 7f6600d46640 1 -- 172.21.15.47:0/4155564842 wait complete. 2024-04-26T23:19:38.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.502+0000 7f6600d46640 1 Processor -- start 2024-04-26T23:19:38.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.502+0000 7f6600d46640 1 -- start start 2024-04-26T23:19:38.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.502+0000 7f6600d46640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65fc06d560 0x7f65fc15b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:38.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.502+0000 7f6600d46640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65fc06e0a0 0x7f65fc1e98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:38.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.502+0000 7f6600d46640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65fc0d0070 0x7f65fc1f4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:38.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.502+0000 7f6600d46640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f65fc0cf1a0 con 0x7f65fc06e0a0 2024-04-26T23:19:38.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.502+0000 7f6600d46640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f65fc0cdc70 con 0x7f65fc0d0070 2024-04-26T23:19:38.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.502+0000 7f6600d46640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f65fc0cf320 con 0x7f65fc06d560 2024-04-26T23:19:38.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.503+0000 7f65f9d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65fc06e0a0 0x7f65fc1e98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:38.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.503+0000 7f65f9d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65fc06e0a0 0x7f65fc1e98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34080/0 (socket says 172.21.15.47:34080) 2024-04-26T23:19:38.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.503+0000 7f65f9d9b640 1 -- 172.21.15.47:0/637630352 learned_addr learned my addr 172.21.15.47:0/637630352 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:38.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.503+0000 7f65fa59c640 1 --2- 172.21.15.47:0/637630352 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65fc06d560 0x7f65fc15b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:38.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.503+0000 7f65fad9d640 1 --2- 172.21.15.47:0/637630352 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65fc0d0070 0x7f65fc1f4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:38.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.503+0000 7f65f9d9b640 1 -- 172.21.15.47:0/637630352 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65fc06d560 msgr2=0x7f65fc15b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:38.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.503+0000 7f65f9d9b640 1 --2- 172.21.15.47:0/637630352 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65fc06d560 0x7f65fc15b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:38.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.503+0000 7f65f9d9b640 1 -- 172.21.15.47:0/637630352 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65fc0d0070 msgr2=0x7f65fc1f4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:38.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.503+0000 7f65f9d9b640 1 --2- 172.21.15.47:0/637630352 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65fc0d0070 0x7f65fc1f4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:38.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.503+0000 7f65f9d9b640 1 -- 172.21.15.47:0/637630352 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65fc1e9e30 con 0x7f65fc06e0a0 2024-04-26T23:19:38.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.503+0000 7f65f9d9b640 1 --2- 172.21.15.47:0/637630352 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65fc06e0a0 0x7f65fc1e98f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f65e400b440 tx=0x7f65e400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:38.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.503+0000 7f65f2ffd640 1 -- 172.21.15.47:0/637630352 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e4015020 con 0x7f65fc06e0a0 2024-04-26T23:19:38.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.503+0000 7f65f2ffd640 1 -- 172.21.15.47:0/637630352 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65e4011890 con 0x7f65fc06e0a0 2024-04-26T23:19:38.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.503+0000 7f6600d46640 1 -- 172.21.15.47:0/637630352 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65fc145a40 con 0x7f65fc06e0a0 2024-04-26T23:19:38.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.503+0000 7f65f2ffd640 1 -- 172.21.15.47:0/637630352 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e4011c80 con 0x7f65fc06e0a0 2024-04-26T23:19:38.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.503+0000 7f6600d46640 1 -- 172.21.15.47:0/637630352 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65fc15bc40 con 0x7f65fc06e0a0 2024-04-26T23:19:38.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.504+0000 7f6600d46640 1 -- 172.21.15.47:0/637630352 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65fc0694f0 con 0x7f65fc06e0a0 2024-04-26T23:19:38.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.505+0000 7f65f2ffd640 1 -- 172.21.15.47:0/637630352 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f65e4018020 con 0x7f65fc06e0a0 2024-04-26T23:19:38.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.505+0000 7f65f2ffd640 1 --2- 172.21.15.47:0/637630352 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f65dc06c8f0 0x7f65dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:38.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.505+0000 7f65f2ffd640 1 -- 172.21.15.47:0/637630352 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f65e408ba80 con 0x7f65fc06e0a0 2024-04-26T23:19:38.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.505+0000 7f65fa59c640 1 --2- 172.21.15.47:0/637630352 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f65dc06c8f0 0x7f65dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:38.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.506+0000 7f65fa59c640 1 --2- 172.21.15.47:0/637630352 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f65dc06c8f0 0x7f65dc06edb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f65f4034000 tx=0x7f65f402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:38.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.509+0000 7f65f2ffd640 1 -- 172.21.15.47:0/637630352 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f65e401f020 con 0x7f65fc06e0a0 2024-04-26T23:19:38.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.763+0000 7f6600d46640 1 -- 172.21.15.47:0/637630352 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f65fc0ca0c0 con 0x7f65fc06e0a0 2024-04-26T23:19:38.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.764+0000 7f65f2ffd640 1 -- 172.21.15.47:0/637630352 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v3) ==== 80+0+6645 (secure 0 0 0) 0x7f65e400d040 con 0x7f65fc06e0a0 2024-04-26T23:19:38.765 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:19:38.765 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-26T23:19:38.765 INFO:teuthology.orchestra.run.smithi047.stdout: "auth_dump": [ 2024-04-26T23:19:38.765 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-26T23:19:38.765 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.a", 2024-04-26T23:19:38.765 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQAKNixmrb+7BhAA9Tl5qHRAAITpDe0o1RpB8A==", 2024-04-26T23:19:38.765 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-26T23:19:38.766 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-04-26T23:19:38.766 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2024-04-26T23:19:38.766 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2024-04-26T23:19:38.766 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-26T23:19:38.766 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-26T23:19:38.766 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-26T23:19:38.766 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-26T23:19:38.766 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.b", 2024-04-26T23:19:38.766 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQAKNixmZYgrEhAA4XwxVuQT3PVAfsalOSSpRg==", 2024-04-26T23:19:38.766 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-26T23:19:38.766 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-04-26T23:19:38.766 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2024-04-26T23:19:38.766 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2024-04-26T23:19:38.766 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-26T23:19:38.766 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-26T23:19:38.767 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-26T23:19:38.767 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-26T23:19:38.767 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.c", 2024-04-26T23:19:38.767 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQAKNixmy+yoDRAAeSQ/dzhhLyysouyco6YrRw==", 2024-04-26T23:19:38.767 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-26T23:19:38.767 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-04-26T23:19:38.767 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2024-04-26T23:19:38.767 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2024-04-26T23:19:38.767 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-26T23:19:38.767 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-26T23:19:38.767 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-26T23:19:38.767 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-26T23:19:38.767 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.d", 2024-04-26T23:19:38.767 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQAKNixmr1gJGRAA2M7//N7u9lk7edXxKxKdJQ==", 2024-04-26T23:19:38.767 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-26T23:19:38.768 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-04-26T23:19:38.768 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2024-04-26T23:19:38.768 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2024-04-26T23:19:38.768 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-26T23:19:38.768 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-26T23:19:38.768 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-26T23:19:38.768 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-26T23:19:38.768 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.0", 2024-04-26T23:19:38.768 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQALNixm3dBSOxAA0heGY4osjvaNAB25znAAZw==", 2024-04-26T23:19:38.768 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-26T23:19:38.768 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-04-26T23:19:38.768 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-04-26T23:19:38.768 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-26T23:19:38.768 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-26T23:19:38.769 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-26T23:19:38.769 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-26T23:19:38.769 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.1", 2024-04-26T23:19:38.769 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQAONixmH0ZoORAAaNuvYE66+ypC0bIeWxVaPw==", 2024-04-26T23:19:38.769 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-26T23:19:38.769 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-04-26T23:19:38.769 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-04-26T23:19:38.769 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-26T23:19:38.769 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-26T23:19:38.769 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-26T23:19:38.769 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-26T23:19:38.769 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.2", 2024-04-26T23:19:38.769 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQARNixmQCOoNxAAl7vmPd2PBbBHjl7VJobIeg==", 2024-04-26T23:19:38.769 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-26T23:19:38.769 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-04-26T23:19:38.770 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-04-26T23:19:38.770 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-26T23:19:38.770 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-26T23:19:38.770 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-26T23:19:38.770 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-26T23:19:38.770 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.3", 2024-04-26T23:19:38.770 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQAUNixmMtdiNhAA+xeZUbroEshbAQO7BQm9kg==", 2024-04-26T23:19:38.770 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-26T23:19:38.770 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-04-26T23:19:38.770 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-04-26T23:19:38.770 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-26T23:19:38.770 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-26T23:19:38.770 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-26T23:19:38.770 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-26T23:19:38.771 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.4", 2024-04-26T23:19:38.771 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQAaNixmiYETOhAAkz/BfU0a8SnawltF0pBU/A==", 2024-04-26T23:19:38.771 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-26T23:19:38.771 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-04-26T23:19:38.771 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-04-26T23:19:38.771 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-26T23:19:38.771 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-26T23:19:38.771 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-26T23:19:38.771 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-26T23:19:38.771 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.5", 2024-04-26T23:19:38.771 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQAdNixmKCoPORAAp3FfD88sxG9IlXK1HqpiPA==", 2024-04-26T23:19:38.771 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-26T23:19:38.771 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-04-26T23:19:38.771 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-04-26T23:19:38.771 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-26T23:19:38.772 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-26T23:19:38.772 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-26T23:19:38.772 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-26T23:19:38.772 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.6", 2024-04-26T23:19:38.772 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQAgNixm/ipONxAAzgJ2wJAHFbHlv09nqA3soQ==", 2024-04-26T23:19:38.772 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-26T23:19:38.772 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-04-26T23:19:38.772 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-04-26T23:19:38.772 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-26T23:19:38.772 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-26T23:19:38.772 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-26T23:19:38.772 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-26T23:19:38.772 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.7", 2024-04-26T23:19:38.772 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQAjNixmXDWkNRAAJYh2B7nH7ViaGPii7Yk69A==", 2024-04-26T23:19:38.772 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-26T23:19:38.773 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-04-26T23:19:38.773 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-04-26T23:19:38.773 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-26T23:19:38.773 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-26T23:19:38.773 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-26T23:19:38.773 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-26T23:19:38.773 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.0", 2024-04-26T23:19:38.773 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQAKNixmA5QFHRAAT89WPyzDs0acPluyvU4svw==", 2024-04-26T23:19:38.773 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-26T23:19:38.773 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-04-26T23:19:38.773 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow r", 2024-04-26T23:19:38.773 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow rw", 2024-04-26T23:19:38.773 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rwx" 2024-04-26T23:19:38.773 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-26T23:19:38.773 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-26T23:19:38.774 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-26T23:19:38.774 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.1", 2024-04-26T23:19:38.774 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQAKNixmtRW8JRAAqh+Q2ZhqvdhTxty04GtU2g==", 2024-04-26T23:19:38.774 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-26T23:19:38.774 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-04-26T23:19:38.774 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow r", 2024-04-26T23:19:38.774 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow rw", 2024-04-26T23:19:38.774 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rwx" 2024-04-26T23:19:38.774 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-26T23:19:38.774 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-26T23:19:38.774 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-26T23:19:38.774 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.2", 2024-04-26T23:19:38.774 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQAKNixmTb2VIhAAhoBSsrzlPIsljZt1jZVscQ==", 2024-04-26T23:19:38.774 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-26T23:19:38.775 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-04-26T23:19:38.775 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow r", 2024-04-26T23:19:38.775 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow rw", 2024-04-26T23:19:38.775 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rwx" 2024-04-26T23:19:38.775 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-26T23:19:38.775 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-26T23:19:38.775 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-26T23:19:38.775 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.3", 2024-04-26T23:19:38.775 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQAKNixmDE8nKxAA9zbeR9CqdL0p8gcuy3k/kQ==", 2024-04-26T23:19:38.775 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-26T23:19:38.775 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-04-26T23:19:38.775 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow r", 2024-04-26T23:19:38.775 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow rw", 2024-04-26T23:19:38.775 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rwx" 2024-04-26T23:19:38.776 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-26T23:19:38.776 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-26T23:19:38.776 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-26T23:19:38.776 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.admin", 2024-04-26T23:19:38.776 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQAJNixmtExhJhAAOUYKNUWtJEGRp9Shql0u8g==", 2024-04-26T23:19:38.776 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-26T23:19:38.776 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2024-04-26T23:19:38.776 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2024-04-26T23:19:38.776 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2024-04-26T23:19:38.776 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-26T23:19:38.776 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-26T23:19:38.776 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-26T23:19:38.776 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-26T23:19:38.776 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.bootstrap-mds", 2024-04-26T23:19:38.776 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQAuNixmyg6gKxAA5gjxBNnSLhSsV4jrfiBbYw==", 2024-04-26T23:19:38.777 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-26T23:19:38.777 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile bootstrap-mds" 2024-04-26T23:19:38.777 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-26T23:19:38.777 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-26T23:19:38.777 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-26T23:19:38.777 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.bootstrap-mgr", 2024-04-26T23:19:38.777 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQAuNixmZRigKxAAxHOT90ZdALzOs7KcbZJw7Q==", 2024-04-26T23:19:38.777 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-26T23:19:38.777 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-26T23:19:38.777 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-26T23:19:38.777 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-26T23:19:38.777 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-26T23:19:38.777 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.bootstrap-osd", 2024-04-26T23:19:38.777 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQAuNixmpiCgKxAAPPPbAZsIsBuv0PmEtwkOew==", 2024-04-26T23:19:38.778 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-26T23:19:38.778 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile bootstrap-osd" 2024-04-26T23:19:38.778 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-26T23:19:38.778 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-26T23:19:38.778 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-26T23:19:38.778 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.bootstrap-rbd", 2024-04-26T23:19:38.778 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQAuNixmQyigKxAA39GCsCjUTF/c1Xzxdq6i6Q==", 2024-04-26T23:19:38.778 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-26T23:19:38.778 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-26T23:19:38.778 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-26T23:19:38.778 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-26T23:19:38.778 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-26T23:19:38.778 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-26T23:19:38.778 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQAuNixmezCgKxAAGOEGbzR/36gNxXUQIyW14g==", 2024-04-26T23:19:38.779 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-26T23:19:38.779 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-26T23:19:38.779 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-26T23:19:38.779 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-26T23:19:38.779 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-26T23:19:38.779 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.bootstrap-rgw", 2024-04-26T23:19:38.779 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQAuNixmgTigKxAAIdzKV8eKd8Tk6rTRreJPOg==", 2024-04-26T23:19:38.779 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-26T23:19:38.779 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-26T23:19:38.779 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-26T23:19:38.779 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-26T23:19:38.779 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-26T23:19:38.779 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.x", 2024-04-26T23:19:38.779 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQAKNixmf2OaAxAAv3TUVplM33vJKhqsWyV84w==", 2024-04-26T23:19:38.779 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-26T23:19:38.780 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2024-04-26T23:19:38.780 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2024-04-26T23:19:38.780 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-26T23:19:38.780 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-26T23:19:38.780 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-26T23:19:38.780 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-26T23:19:38.780 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.y", 2024-04-26T23:19:38.780 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQAJNixmQ2HSOhAACgAm3LAXwI4PB/P1+oRdyw==", 2024-04-26T23:19:38.780 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-26T23:19:38.780 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2024-04-26T23:19:38.780 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2024-04-26T23:19:38.780 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-26T23:19:38.780 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-26T23:19:38.780 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-26T23:19:38.781 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-26T23:19:38.781 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-26T23:19:38.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.766+0000 7f6600d46640 1 -- 172.21.15.47:0/637630352 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f65dc06c8f0 msgr2=0x7f65dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:38.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.766+0000 7f6600d46640 1 --2- 172.21.15.47:0/637630352 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f65dc06c8f0 0x7f65dc06edb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f65f4034000 tx=0x7f65f402e000 comp rx=0 tx=0).stop 2024-04-26T23:19:38.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.766+0000 7f6600d46640 1 -- 172.21.15.47:0/637630352 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65fc06e0a0 msgr2=0x7f65fc1e98f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:38.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.766+0000 7f6600d46640 1 --2- 172.21.15.47:0/637630352 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65fc06e0a0 0x7f65fc1e98f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f65e400b440 tx=0x7f65e400b900 comp rx=0 tx=0).stop 2024-04-26T23:19:38.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.766+0000 7f6600d46640 1 -- 172.21.15.47:0/637630352 shutdown_connections 2024-04-26T23:19:38.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.766+0000 7f6600d46640 1 --2- 172.21.15.47:0/637630352 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f65dc06c8f0 0x7f65dc06edb0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:38.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.766+0000 7f6600d46640 1 --2- 172.21.15.47:0/637630352 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65fc0d0070 0x7f65fc1f4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:38.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.766+0000 7f6600d46640 1 --2- 172.21.15.47:0/637630352 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65fc06e0a0 0x7f65fc1e98f0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:38.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.766+0000 7f6600d46640 1 --2- 172.21.15.47:0/637630352 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65fc06d560 0x7f65fc15b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:38.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.766+0000 7f6600d46640 1 -- 172.21.15.47:0/637630352 >> 172.21.15.47:0/637630352 conn(0x7f65fc0d2ec0 msgr2=0x7f65fc071400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:38.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.766+0000 7f6600d46640 1 -- 172.21.15.47:0/637630352 shutdown_connections 2024-04-26T23:19:38.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.766+0000 7f6600d46640 1 -- 172.21.15.47:0/637630352 wait complete. 2024-04-26T23:19:38.783 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-26T23:19:38.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.885+0000 7f617bcf2640 1 Processor -- start 2024-04-26T23:19:38.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.885+0000 7f617bcf2640 1 -- start start 2024-04-26T23:19:38.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.886+0000 7f617bcf2640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f617406fea0 0x7f6174166d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:38.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.886+0000 7f617bcf2640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6174160970 con 0x7f617415f2d0 2024-04-26T23:19:38.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.886+0000 7f617bcf2640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6174072f70 con 0x7f61741604e0 2024-04-26T23:19:38.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.886+0000 7f617bcf2640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f6174074010 con 0x7f617406fea0 2024-04-26T23:19:38.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.886+0000 7f617a28f640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f617406fea0 0x7f6174166d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:38.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.886+0000 7f617a28f640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f617406fea0 0x7f6174166d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34082/0 (socket says 172.21.15.47:34082) 2024-04-26T23:19:38.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.886+0000 7f617a28f640 1 -- 172.21.15.47:0/1028258475 learned_addr learned my addr 172.21.15.47:0/1028258475 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:38.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.886+0000 7f6179a8e640 1 --1- 172.21.15.47:0/1028258475 >> v1:172.21.15.47:6789/0 conn(0x7f617415f2d0 0x7f6174072700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40958/0 (socket says 172.21.15.47:40958) 2024-04-26T23:19:38.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.887+0000 7f617a28f640 1 -- 172.21.15.47:0/1028258475 >> v1:172.21.15.203:6789/0 conn(0x7f61741604e0 legacy=0x7f617406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:38.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.887+0000 7f617a28f640 1 -- 172.21.15.47:0/1028258475 >> v1:172.21.15.47:6789/0 conn(0x7f617415f2d0 legacy=0x7f6174072700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:38.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.887+0000 7f6178a8c640 1 -- 172.21.15.47:0/1028258475 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1776692468 0 0) 0x7f6174160970 con 0x7f617415f2d0 2024-04-26T23:19:38.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.887+0000 7f617a28f640 1 -- 172.21.15.47:0/1028258475 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6174073250 con 0x7f617406fea0 2024-04-26T23:19:38.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.887+0000 7f617a28f640 1 --2- 172.21.15.47:0/1028258475 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f617406fea0 0x7f6174166d80 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f616c009e50 tx=0x7f616c02a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6ad83007834de66 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:38.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.887+0000 7f6178a8c640 1 -- 172.21.15.47:0/1028258475 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616c036070 con 0x7f617406fea0 2024-04-26T23:19:38.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.888+0000 7f6178a8c640 1 -- 172.21.15.47:0/1028258475 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f616c031600 con 0x7f617406fea0 2024-04-26T23:19:38.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.888+0000 7f6178a8c640 1 -- 172.21.15.47:0/1028258475 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616c0319d0 con 0x7f617406fea0 2024-04-26T23:19:38.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.888+0000 7f617bcf2640 1 -- 172.21.15.47:0/1028258475 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f617406fea0 msgr2=0x7f6174166d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:38.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.888+0000 7f617bcf2640 1 --2- 172.21.15.47:0/1028258475 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f617406fea0 0x7f6174166d80 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f616c009e50 tx=0x7f616c02a760 comp rx=0 tx=0).stop 2024-04-26T23:19:38.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.888+0000 7f617bcf2640 1 -- 172.21.15.47:0/1028258475 shutdown_connections 2024-04-26T23:19:38.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.888+0000 7f617bcf2640 1 --2- 172.21.15.47:0/1028258475 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f617406fea0 0x7f6174166d80 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:38.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.888+0000 7f617bcf2640 1 -- 172.21.15.47:0/1028258475 >> 172.21.15.47:0/1028258475 conn(0x7f61740d0be0 msgr2=0x7f61740d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:38.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.888+0000 7f617bcf2640 1 -- 172.21.15.47:0/1028258475 shutdown_connections 2024-04-26T23:19:38.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.888+0000 7f617bcf2640 1 -- 172.21.15.47:0/1028258475 wait complete. 2024-04-26T23:19:38.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.889+0000 7f617bcf2640 1 Processor -- start 2024-04-26T23:19:38.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.889+0000 7f617bcf2640 1 -- start start 2024-04-26T23:19:38.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.890+0000 7f617bcf2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f617406fea0 0x7f61741564c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:38.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.890+0000 7f617bcf2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f617415f2d0 0x7f61740b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:38.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.890+0000 7f617bcf2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f61741604e0 0x7f61740c4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:38.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.890+0000 7f617bcf2640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6174073af0 con 0x7f617415f2d0 2024-04-26T23:19:38.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.890+0000 7f617bcf2640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6174073810 con 0x7f617406fea0 2024-04-26T23:19:38.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.890+0000 7f617bcf2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f617415f6b0 con 0x7f61741604e0 2024-04-26T23:19:38.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.890+0000 7f617928d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f617415f2d0 0x7f61740b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:38.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.890+0000 7f617928d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f617415f2d0 0x7f61740b21f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34090/0 (socket says 172.21.15.47:34090) 2024-04-26T23:19:38.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.890+0000 7f617928d640 1 -- 172.21.15.47:0/3273026877 learned_addr learned my addr 172.21.15.47:0/3273026877 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:38.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.890+0000 7f617a28f640 1 --2- 172.21.15.47:0/3273026877 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f61741604e0 0x7f61740c4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:38.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.890+0000 7f6179a8e640 1 --2- 172.21.15.47:0/3273026877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f617406fea0 0x7f61741564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:38.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.890+0000 7f617928d640 1 -- 172.21.15.47:0/3273026877 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f61741604e0 msgr2=0x7f61740c4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:38.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.890+0000 7f617928d640 1 --2- 172.21.15.47:0/3273026877 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f61741604e0 0x7f61740c4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:38.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.891+0000 7f617928d640 1 -- 172.21.15.47:0/3273026877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f617406fea0 msgr2=0x7f61741564c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:38.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.891+0000 7f617928d640 1 --2- 172.21.15.47:0/3273026877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f617406fea0 0x7f61741564c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:38.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.891+0000 7f617928d640 1 -- 172.21.15.47:0/3273026877 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61740ced80 con 0x7f617415f2d0 2024-04-26T23:19:38.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.891+0000 7f617a28f640 1 --2- 172.21.15.47:0/3273026877 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f61741604e0 0x7f61740c4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:19:38.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.891+0000 7f617928d640 1 --2- 172.21.15.47:0/3273026877 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f617415f2d0 0x7f61740b21f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f616400b3f0 tx=0x7f616400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:38.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.891+0000 7f616a7fc640 1 -- 172.21.15.47:0/3273026877 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6164015020 con 0x7f617415f2d0 2024-04-26T23:19:38.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.891+0000 7f616a7fc640 1 -- 172.21.15.47:0/3273026877 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6164011830 con 0x7f617415f2d0 2024-04-26T23:19:38.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.891+0000 7f617bcf2640 1 -- 172.21.15.47:0/3273026877 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6174152470 con 0x7f617415f2d0 2024-04-26T23:19:38.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.891+0000 7f616a7fc640 1 -- 172.21.15.47:0/3273026877 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6164011c20 con 0x7f617415f2d0 2024-04-26T23:19:38.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.891+0000 7f617bcf2640 1 -- 172.21.15.47:0/3273026877 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6174152750 con 0x7f617415f2d0 2024-04-26T23:19:38.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.892+0000 7f617bcf2640 1 -- 172.21.15.47:0/3273026877 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61740694f0 con 0x7f617415f2d0 2024-04-26T23:19:38.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.892+0000 7f616a7fc640 1 -- 172.21.15.47:0/3273026877 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6164018020 con 0x7f617415f2d0 2024-04-26T23:19:38.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.893+0000 7f616a7fc640 1 --2- 172.21.15.47:0/3273026877 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f615806c8f0 0x7f615806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:38.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.893+0000 7f616a7fc640 1 -- 172.21.15.47:0/3273026877 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f616408b980 con 0x7f617415f2d0 2024-04-26T23:19:38.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.893+0000 7f6179a8e640 1 --2- 172.21.15.47:0/3273026877 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f615806c8f0 0x7f615806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:38.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.894+0000 7f6179a8e640 1 --2- 172.21.15.47:0/3273026877 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f615806c8f0 0x7f615806edb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6170000f60 tx=0x7f6170007b50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:38.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:38.897+0000 7f616a7fc640 1 -- 172.21.15.47:0/3273026877 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f616401f020 con 0x7f617415f2d0 2024-04-26T23:19:39.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.152+0000 7f617bcf2640 1 -- 172.21.15.47:0/3273026877 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f617406f000 con 0x7f617415f2d0 2024-04-26T23:19:39.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.154+0000 7f616a7fc640 1 -- 172.21.15.47:0/3273026877 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v4) ==== 90+0+0 (secure 0 0 0) 0x7f61640172c0 con 0x7f617415f2d0 2024-04-26T23:19:39.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.156+0000 7f617bcf2640 1 -- 172.21.15.47:0/3273026877 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f615806c8f0 msgr2=0x7f615806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:39.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.156+0000 7f617bcf2640 1 --2- 172.21.15.47:0/3273026877 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f615806c8f0 0x7f615806edb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6170000f60 tx=0x7f6170007b50 comp rx=0 tx=0).stop 2024-04-26T23:19:39.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.156+0000 7f617bcf2640 1 -- 172.21.15.47:0/3273026877 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f617415f2d0 msgr2=0x7f61740b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:39.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.156+0000 7f617bcf2640 1 --2- 172.21.15.47:0/3273026877 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f617415f2d0 0x7f61740b21f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f616400b3f0 tx=0x7f616400b8b0 comp rx=0 tx=0).stop 2024-04-26T23:19:39.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.156+0000 7f617bcf2640 1 -- 172.21.15.47:0/3273026877 shutdown_connections 2024-04-26T23:19:39.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.156+0000 7f617bcf2640 1 --2- 172.21.15.47:0/3273026877 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f615806c8f0 0x7f615806edb0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:39.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.156+0000 7f617bcf2640 1 --2- 172.21.15.47:0/3273026877 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f61741604e0 0x7f61740c4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:39.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.156+0000 7f617bcf2640 1 --2- 172.21.15.47:0/3273026877 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f617415f2d0 0x7f61740b21f0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:39.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.156+0000 7f617bcf2640 1 --2- 172.21.15.47:0/3273026877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f617406fea0 0x7f61741564c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:39.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.156+0000 7f617bcf2640 1 -- 172.21.15.47:0/3273026877 >> 172.21.15.47:0/3273026877 conn(0x7f61740d0be0 msgr2=0x7f617415f920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:39.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.156+0000 7f617bcf2640 1 -- 172.21.15.47:0/3273026877 shutdown_connections 2024-04-26T23:19:39.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.156+0000 7f617bcf2640 1 -- 172.21.15.47:0/3273026877 wait complete. 2024-04-26T23:19:39.168 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-26T23:19:39.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.269+0000 7f314a207640 1 Processor -- start 2024-04-26T23:19:39.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.269+0000 7f314a207640 1 -- start start 2024-04-26T23:19:39.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.270+0000 7f314a207640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31440d1b30 0x7f31440d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:39.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.270+0000 7f314a207640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31440d10d0 con 0x7f31440b2340 2024-04-26T23:19:39.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.270+0000 7f314a207640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31440d1370 con 0x7f3144162810 2024-04-26T23:19:39.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.270+0000 7f314a207640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f31440d0750 con 0x7f31440d1b30 2024-04-26T23:19:39.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.270+0000 7f3142ffd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f31440b2340 0x7f31440cd3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40966/0 (socket says 172.21.15.47:40966) 2024-04-26T23:19:39.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.270+0000 7f3142ffd640 1 -- 172.21.15.47:0/2602360334 learned_addr learned my addr 172.21.15.47:0/2602360334 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:39.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.270+0000 7f3143fff640 1 --2- 172.21.15.47:0/2602360334 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31440d1b30 0x7f31440d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:39.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.270+0000 7f31427fc640 1 -- 172.21.15.47:0/2602360334 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2396974294 0 0) 0x7f31440d10d0 con 0x7f31440b2340 2024-04-26T23:19:39.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.270+0000 7f31427fc640 1 -- 172.21.15.47:0/2602360334 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f312c003520 con 0x7f31440b2340 2024-04-26T23:19:39.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.270+0000 7f31427fc640 1 -- 172.21.15.47:0/2602360334 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1502960395 0 0) 0x7f312c003520 con 0x7f31440b2340 2024-04-26T23:19:39.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.270+0000 7f31427fc640 1 -- 172.21.15.47:0/2602360334 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31440d1b30 msgr2=0x7f31440d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:39.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.270+0000 7f31427fc640 1 --2- 172.21.15.47:0/2602360334 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31440d1b30 0x7f31440d1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:39.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.270+0000 7f31427fc640 1 -- 172.21.15.47:0/2602360334 >> v1:172.21.15.203:6789/0 conn(0x7f3144162810 legacy=0x7f3144070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:39.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.270+0000 7f31427fc640 1 -- 172.21.15.47:0/2602360334 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31440d01b0 con 0x7f31440b2340 2024-04-26T23:19:39.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.270+0000 7f31427fc640 1 -- 172.21.15.47:0/2602360334 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7f3134001670 con 0x7f31440b2340 2024-04-26T23:19:39.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.270+0000 7f31427fc640 1 -- 172.21.15.47:0/2602360334 >> v1:172.21.15.47:6789/0 conn(0x7f31440b2340 legacy=0x7f31440cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:39.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.270+0000 7f31427fc640 1 --2- 172.21.15.47:0/2602360334 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31440d1b30 0x7f312c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:39.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.270+0000 7f31427fc640 1 --2- 172.21.15.47:0/2602360334 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f312c003f60 0x7f312c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:39.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.270+0000 7f31427fc640 1 --2- 172.21.15.47:0/2602360334 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f312c004d40 0x7f312c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:39.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.270+0000 7f31427fc640 1 -- 172.21.15.47:0/2602360334 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f31440d0750 con 0x7f312c004d40 2024-04-26T23:19:39.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.270+0000 7f31427fc640 1 -- 172.21.15.47:0/2602360334 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f312c00baa0 con 0x7f312c003f60 2024-04-26T23:19:39.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.270+0000 7f31427fc640 1 -- 172.21.15.47:0/2602360334 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f312c00bca0 con 0x7f31440d1b30 2024-04-26T23:19:39.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.270+0000 7f3143fff640 1 --2- 172.21.15.47:0/2602360334 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f312c004d40 0x7f312c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:39.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.270+0000 7f31437fe640 1 --2- 172.21.15.47:0/2602360334 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31440d1b30 0x7f312c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:39.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.271+0000 7f3142ffd640 1 --2- 172.21.15.47:0/2602360334 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f312c003f60 0x7f312c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:39.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.271+0000 7f3143fff640 1 -- 172.21.15.47:0/2602360334 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31440d1b30 msgr2=0x7f312c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:39.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.271+0000 7f3143fff640 1 --2- 172.21.15.47:0/2602360334 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31440d1b30 0x7f312c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:39.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.271+0000 7f3143fff640 1 -- 172.21.15.47:0/2602360334 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f312c003f60 msgr2=0x7f312c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:39.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.271+0000 7f3143fff640 1 --2- 172.21.15.47:0/2602360334 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f312c003f60 0x7f312c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:39.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.271+0000 7f3143fff640 1 -- 172.21.15.47:0/2602360334 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f312c003520 con 0x7f312c004d40 2024-04-26T23:19:39.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.271+0000 7f3143fff640 1 --2- 172.21.15.47:0/2602360334 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f312c004d40 0x7f312c0051c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f313c009fc0 tx=0x7f312c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83f4fdb782a89071 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:39.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.272+0000 7f31427fc640 1 -- 172.21.15.47:0/2602360334 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313c009c10 con 0x7f312c004d40 2024-04-26T23:19:39.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.272+0000 7f31427fc640 1 -- 172.21.15.47:0/2602360334 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f313c036070 con 0x7f312c004d40 2024-04-26T23:19:39.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.272+0000 7f31427fc640 1 -- 172.21.15.47:0/2602360334 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313c031880 con 0x7f312c004d40 2024-04-26T23:19:39.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.272+0000 7f314a207640 1 -- 172.21.15.47:0/2602360334 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f312c004d40 msgr2=0x7f312c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:39.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.272+0000 7f314a207640 1 --2- 172.21.15.47:0/2602360334 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f312c004d40 0x7f312c0051c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f313c009fc0 tx=0x7f312c005830 comp rx=0 tx=0).stop 2024-04-26T23:19:39.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.272+0000 7f3143fff640 1 -- 172.21.15.47:0/2602360334 reap_dead start 2024-04-26T23:19:39.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.272+0000 7f314a207640 1 -- 172.21.15.47:0/2602360334 shutdown_connections 2024-04-26T23:19:39.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.272+0000 7f314a207640 1 -- 172.21.15.47:0/2602360334 >> 172.21.15.47:0/2602360334 conn(0x7f31440d2da0 msgr2=0x7f31440d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:39.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.272+0000 7f314a207640 1 -- 172.21.15.47:0/2602360334 shutdown_connections 2024-04-26T23:19:39.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.272+0000 7f314a207640 1 -- 172.21.15.47:0/2602360334 wait complete. 2024-04-26T23:19:39.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.273+0000 7f314a207640 1 Processor -- start 2024-04-26T23:19:39.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.274+0000 7f314a207640 1 -- start start 2024-04-26T23:19:39.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.274+0000 7f314a207640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f312c004d40 0x7f31441528a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:39.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.274+0000 7f314a207640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31440ccae0 0x7f31440afab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:39.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.274+0000 7f314a207640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31440afff0 0x7f31440cbca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:39.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.274+0000 7f314a207640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f31440cdb00 con 0x7f31440afff0 2024-04-26T23:19:39.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.274+0000 7f314a207640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f31440d16a0 con 0x7f31440ccae0 2024-04-26T23:19:39.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.274+0000 7f314a207640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f31440d1fc0 con 0x7f312c004d40 2024-04-26T23:19:39.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.274+0000 7f3143fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31440afff0 0x7f31440cbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:39.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.274+0000 7f3143fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31440afff0 0x7f31440cbca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34102/0 (socket says 172.21.15.47:34102) 2024-04-26T23:19:39.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.274+0000 7f3143fff640 1 -- 172.21.15.47:0/50891248 learned_addr learned my addr 172.21.15.47:0/50891248 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:39.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.274+0000 7f31437fe640 1 --2- 172.21.15.47:0/50891248 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f312c004d40 0x7f31441528a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:39.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.274+0000 7f3142ffd640 1 --2- 172.21.15.47:0/50891248 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31440ccae0 0x7f31440afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:39.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.275+0000 7f3143fff640 1 -- 172.21.15.47:0/50891248 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f312c004d40 msgr2=0x7f31441528a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:39.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.275+0000 7f3143fff640 1 --2- 172.21.15.47:0/50891248 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f312c004d40 0x7f31441528a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:39.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.275+0000 7f3143fff640 1 -- 172.21.15.47:0/50891248 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31440ccae0 msgr2=0x7f31440afab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:39.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.275+0000 7f3143fff640 1 --2- 172.21.15.47:0/50891248 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31440ccae0 0x7f31440afab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:39.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.275+0000 7f3143fff640 1 -- 172.21.15.47:0/50891248 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31440ca130 con 0x7f31440afff0 2024-04-26T23:19:39.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.275+0000 7f3143fff640 1 --2- 172.21.15.47:0/50891248 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31440afff0 0x7f31440cbca0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f313c033040 tx=0x7f313c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:39.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.275+0000 7f312bfff640 1 -- 172.21.15.47:0/50891248 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313c009c10 con 0x7f31440afff0 2024-04-26T23:19:39.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.275+0000 7f312bfff640 1 -- 172.21.15.47:0/50891248 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f313c031dd0 con 0x7f31440afff0 2024-04-26T23:19:39.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.275+0000 7f314a207640 1 -- 172.21.15.47:0/50891248 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31440ca410 con 0x7f31440afff0 2024-04-26T23:19:39.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.275+0000 7f312bfff640 1 -- 172.21.15.47:0/50891248 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313c003bd0 con 0x7f31440afff0 2024-04-26T23:19:39.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.275+0000 7f314a207640 1 -- 172.21.15.47:0/50891248 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31440cc1e0 con 0x7f31440afff0 2024-04-26T23:19:39.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.276+0000 7f314a207640 1 -- 172.21.15.47:0/50891248 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31440694f0 con 0x7f31440afff0 2024-04-26T23:19:39.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.277+0000 7f312bfff640 1 -- 172.21.15.47:0/50891248 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f313c005d10 con 0x7f31440afff0 2024-04-26T23:19:39.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.277+0000 7f312bfff640 1 --2- 172.21.15.47:0/50891248 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f313006c8f0 0x7f313006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:39.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.277+0000 7f312bfff640 1 -- 172.21.15.47:0/50891248 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f313c0acf60 con 0x7f31440afff0 2024-04-26T23:19:39.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.277+0000 7f31437fe640 1 --2- 172.21.15.47:0/50891248 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f313006c8f0 0x7f313006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:39.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.278+0000 7f31437fe640 1 --2- 172.21.15.47:0/50891248 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f313006c8f0 0x7f313006edb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f31380017e0 tx=0x7f3138009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:39.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.281+0000 7f312bfff640 1 -- 172.21.15.47:0/50891248 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f313c04a060 con 0x7f31440afff0 2024-04-26T23:19:39.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.537+0000 7f314a207640 1 -- 172.21.15.47:0/50891248 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f31440c9be0 con 0x7f31440afff0 2024-04-26T23:19:39.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.539+0000 7f312bfff640 1 -- 172.21.15.47:0/50891248 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7f313c07af00 con 0x7f31440afff0 2024-04-26T23:19:39.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.541+0000 7f314a207640 1 -- 172.21.15.47:0/50891248 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f313006c8f0 msgr2=0x7f313006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:39.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.541+0000 7f314a207640 1 --2- 172.21.15.47:0/50891248 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f313006c8f0 0x7f313006edb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f31380017e0 tx=0x7f3138009040 comp rx=0 tx=0).stop 2024-04-26T23:19:39.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.541+0000 7f314a207640 1 -- 172.21.15.47:0/50891248 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31440afff0 msgr2=0x7f31440cbca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:39.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.541+0000 7f314a207640 1 --2- 172.21.15.47:0/50891248 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31440afff0 0x7f31440cbca0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f313c033040 tx=0x7f313c009750 comp rx=0 tx=0).stop 2024-04-26T23:19:39.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.541+0000 7f314a207640 1 -- 172.21.15.47:0/50891248 shutdown_connections 2024-04-26T23:19:39.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.541+0000 7f314a207640 1 --2- 172.21.15.47:0/50891248 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f313006c8f0 0x7f313006edb0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:39.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.541+0000 7f314a207640 1 --2- 172.21.15.47:0/50891248 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31440afff0 0x7f31440cbca0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:39.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.541+0000 7f314a207640 1 --2- 172.21.15.47:0/50891248 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31440ccae0 0x7f31440afab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:39.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.541+0000 7f314a207640 1 --2- 172.21.15.47:0/50891248 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f312c004d40 0x7f31441528a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:39.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.541+0000 7f314a207640 1 -- 172.21.15.47:0/50891248 >> 172.21.15.47:0/50891248 conn(0x7f31440d2da0 msgr2=0x7f314406d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:39.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.541+0000 7f314a207640 1 -- 172.21.15.47:0/50891248 shutdown_connections 2024-04-26T23:19:39.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.541+0000 7f314a207640 1 -- 172.21.15.47:0/50891248 wait complete. 2024-04-26T23:19:39.554 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-26T23:19:39.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.655+0000 7f50500fc640 1 Processor -- start 2024-04-26T23:19:39.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.655+0000 7f50500fc640 1 -- start start 2024-04-26T23:19:39.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.656+0000 7f50500fc640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f504806f790 0x7f504806fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:39.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.656+0000 7f50500fc640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5048072c50 0x7f5048073060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:39.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.656+0000 7f50500fc640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5048156840 con 0x7f5048156460 2024-04-26T23:19:39.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.656+0000 7f50500fc640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f50480cbf20 con 0x7f504806f790 2024-04-26T23:19:39.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.656+0000 7f50500fc640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f50480cc200 con 0x7f5048072c50 2024-04-26T23:19:39.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.656+0000 7f504d697640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f504806f790 0x7f504806fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:39.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.656+0000 7f504de98640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f5048156460 0x7f50480cb4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40978/0 (socket says 172.21.15.47:40978) 2024-04-26T23:19:39.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.656+0000 7f504de98640 1 -- 172.21.15.47:0/2984772765 learned_addr learned my addr 172.21.15.47:0/2984772765 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:39.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.656+0000 7f504e699640 1 --2- 172.21.15.47:0/2984772765 >> v2:172.21.15.203:3300/0 conn(0x7f5048072c50 0x7f5048073060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:39.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.657+0000 7f504d697640 1 -- 172.21.15.47:0/2984772765 >> v2:172.21.15.203:3300/0 conn(0x7f5048072c50 msgr2=0x7f5048073060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:39.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.657+0000 7f504d697640 1 --2- 172.21.15.47:0/2984772765 >> v2:172.21.15.203:3300/0 conn(0x7f5048072c50 0x7f5048073060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:39.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.657+0000 7f504d697640 1 -- 172.21.15.47:0/2984772765 >> v1:172.21.15.47:6789/0 conn(0x7f5048156460 legacy=0x7f50480cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:39.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.657+0000 7f504d697640 1 -- 172.21.15.47:0/2984772765 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50480cc4e0 con 0x7f504806f790 2024-04-26T23:19:39.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.657+0000 7f504ce96640 1 -- 172.21.15.47:0/2984772765 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3311994697 0 0) 0x7f5048156840 con 0x7f5048156460 2024-04-26T23:19:39.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.657+0000 7f504d697640 1 --2- 172.21.15.47:0/2984772765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f504806f790 0x7f504806fc40 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f50480cb1c0 tx=0x7f503802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f98affc7c31519eb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:39.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.657+0000 7f504ce96640 1 -- 172.21.15.47:0/2984772765 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5038037070 con 0x7f504806f790 2024-04-26T23:19:39.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.658+0000 7f504ce96640 1 -- 172.21.15.47:0/2984772765 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5038032440 con 0x7f504806f790 2024-04-26T23:19:39.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.658+0000 7f504ce96640 1 -- 172.21.15.47:0/2984772765 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5038032810 con 0x7f504806f790 2024-04-26T23:19:39.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.658+0000 7f50500fc640 1 -- 172.21.15.47:0/2984772765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f504806f790 msgr2=0x7f504806fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:39.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.658+0000 7f50500fc640 1 --2- 172.21.15.47:0/2984772765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f504806f790 0x7f504806fc40 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f50480cb1c0 tx=0x7f503802a1a0 comp rx=0 tx=0).stop 2024-04-26T23:19:39.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.658+0000 7f50500fc640 1 -- 172.21.15.47:0/2984772765 shutdown_connections 2024-04-26T23:19:39.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.658+0000 7f50500fc640 1 --2- 172.21.15.47:0/2984772765 >> v2:172.21.15.203:3300/0 conn(0x7f5048072c50 0x7f5048073060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:39.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.658+0000 7f50500fc640 1 --2- 172.21.15.47:0/2984772765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f504806f790 0x7f504806fc40 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:39.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.658+0000 7f50500fc640 1 -- 172.21.15.47:0/2984772765 >> 172.21.15.47:0/2984772765 conn(0x7f50480d0be0 msgr2=0x7f50480d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:39.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.658+0000 7f50500fc640 1 -- 172.21.15.47:0/2984772765 shutdown_connections 2024-04-26T23:19:39.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.658+0000 7f50500fc640 1 -- 172.21.15.47:0/2984772765 wait complete. 2024-04-26T23:19:39.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.659+0000 7f50500fc640 1 Processor -- start 2024-04-26T23:19:39.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.659+0000 7f50500fc640 1 -- start start 2024-04-26T23:19:39.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.660+0000 7f50500fc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f504806f790 0x7f504814f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:39.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.660+0000 7f50500fc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5048072c50 0x7f50480c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:39.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.660+0000 7f50500fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5048156460 0x7f50480c9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:39.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.660+0000 7f50500fc640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f50480cbbe0 con 0x7f5048156460 2024-04-26T23:19:39.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.660+0000 7f50500fc640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5048072700 con 0x7f5048072c50 2024-04-26T23:19:39.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.660+0000 7f50500fc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f504806e260 con 0x7f504806f790 2024-04-26T23:19:39.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.660+0000 7f504e699640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5048156460 0x7f50480c9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:39.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.660+0000 7f504e699640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5048156460 0x7f50480c9240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34108/0 (socket says 172.21.15.47:34108) 2024-04-26T23:19:39.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.660+0000 7f504e699640 1 -- 172.21.15.47:0/998851882 learned_addr learned my addr 172.21.15.47:0/998851882 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:39.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.660+0000 7f504d697640 1 --2- 172.21.15.47:0/998851882 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5048072c50 0x7f50480c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:39.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.660+0000 7f504de98640 1 --2- 172.21.15.47:0/998851882 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f504806f790 0x7f504814f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:39.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.660+0000 7f504e699640 1 -- 172.21.15.47:0/998851882 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f504806f790 msgr2=0x7f504814f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:39.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.660+0000 7f504e699640 1 --2- 172.21.15.47:0/998851882 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f504806f790 0x7f504814f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:39.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.660+0000 7f504e699640 1 -- 172.21.15.47:0/998851882 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5048072c50 msgr2=0x7f50480c8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:39.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.660+0000 7f504e699640 1 --2- 172.21.15.47:0/998851882 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5048072c50 0x7f50480c8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:39.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.660+0000 7f504e699640 1 -- 172.21.15.47:0/998851882 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f504814f580 con 0x7f5048156460 2024-04-26T23:19:39.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.661+0000 7f504e699640 1 --2- 172.21.15.47:0/998851882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5048156460 0x7f50480c9240 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f504000d830 tx=0x7f504000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:39.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.661+0000 7f503e7fc640 1 -- 172.21.15.47:0/998851882 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f504000b6c0 con 0x7f5048156460 2024-04-26T23:19:39.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.661+0000 7f503e7fc640 1 -- 172.21.15.47:0/998851882 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5040019070 con 0x7f5048156460 2024-04-26T23:19:39.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.661+0000 7f50500fc640 1 -- 172.21.15.47:0/998851882 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50480c4d30 con 0x7f5048156460 2024-04-26T23:19:39.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.661+0000 7f503e7fc640 1 -- 172.21.15.47:0/998851882 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f504000bac0 con 0x7f5048156460 2024-04-26T23:19:39.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.661+0000 7f50500fc640 1 -- 172.21.15.47:0/998851882 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50480c52f0 con 0x7f5048156460 2024-04-26T23:19:39.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.662+0000 7f50500fc640 1 -- 172.21.15.47:0/998851882 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50480694f0 con 0x7f5048156460 2024-04-26T23:19:39.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.662+0000 7f503e7fc640 1 -- 172.21.15.47:0/998851882 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f504001e460 con 0x7f5048156460 2024-04-26T23:19:39.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.663+0000 7f503e7fc640 1 --2- 172.21.15.47:0/998851882 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f502c06c8f0 0x7f502c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:39.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.663+0000 7f503e7fc640 1 -- 172.21.15.47:0/998851882 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f504008bfb0 con 0x7f5048156460 2024-04-26T23:19:39.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.663+0000 7f504de98640 1 --2- 172.21.15.47:0/998851882 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f502c06c8f0 0x7f502c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:39.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.664+0000 7f504de98640 1 --2- 172.21.15.47:0/998851882 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f502c06c8f0 0x7f502c06edb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f50440012e0 tx=0x7f5044005eb0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:39.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.667+0000 7f503e7fc640 1 -- 172.21.15.47:0/998851882 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f504005a630 con 0x7f5048156460 2024-04-26T23:19:39.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.924+0000 7f50500fc640 1 -- 172.21.15.47:0/998851882 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f504806de80 con 0x7f5048156460 2024-04-26T23:19:39.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.926+0000 7f503e7fc640 1 -- 172.21.15.47:0/998851882 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7f5040059f30 con 0x7f5048156460 2024-04-26T23:19:39.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.928+0000 7f50500fc640 1 -- 172.21.15.47:0/998851882 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f502c06c8f0 msgr2=0x7f502c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:39.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.928+0000 7f50500fc640 1 --2- 172.21.15.47:0/998851882 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f502c06c8f0 0x7f502c06edb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f50440012e0 tx=0x7f5044005eb0 comp rx=0 tx=0).stop 2024-04-26T23:19:39.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.928+0000 7f50500fc640 1 -- 172.21.15.47:0/998851882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5048156460 msgr2=0x7f50480c9240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:39.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.928+0000 7f50500fc640 1 --2- 172.21.15.47:0/998851882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5048156460 0x7f50480c9240 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f504000d830 tx=0x7f504000dcf0 comp rx=0 tx=0).stop 2024-04-26T23:19:39.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.928+0000 7f50500fc640 1 -- 172.21.15.47:0/998851882 shutdown_connections 2024-04-26T23:19:39.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.928+0000 7f50500fc640 1 --2- 172.21.15.47:0/998851882 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f502c06c8f0 0x7f502c06edb0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:39.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.928+0000 7f50500fc640 1 --2- 172.21.15.47:0/998851882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5048156460 0x7f50480c9240 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:39.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.928+0000 7f50500fc640 1 --2- 172.21.15.47:0/998851882 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5048072c50 0x7f50480c8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:39.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.928+0000 7f50500fc640 1 --2- 172.21.15.47:0/998851882 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f504806f790 0x7f504814f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:39.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.928+0000 7f50500fc640 1 -- 172.21.15.47:0/998851882 >> 172.21.15.47:0/998851882 conn(0x7f50480d0be0 msgr2=0x7f50480cfa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:39.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.928+0000 7f50500fc640 1 -- 172.21.15.47:0/998851882 shutdown_connections 2024-04-26T23:19:39.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:39.928+0000 7f50500fc640 1 -- 172.21.15.47:0/998851882 wait complete. 2024-04-26T23:19:39.940 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-26T23:19:40.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.042+0000 7fb12a064640 1 Processor -- start 2024-04-26T23:19:40.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.042+0000 7fb12a064640 1 -- start start 2024-04-26T23:19:40.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.043+0000 7fb12a064640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1240d3bc0 0x7fb1240d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:40.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.043+0000 7fb12a064640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb1240b5f20 0x7fb1240b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:40.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.043+0000 7fb12a064640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1240b7e50 con 0x7fb124162880 2024-04-26T23:19:40.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.043+0000 7fb12a064640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fb1240d0580 con 0x7fb1240b5f20 2024-04-26T23:19:40.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.043+0000 7fb12a064640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb1240d1800 con 0x7fb1240d3bc0 2024-04-26T23:19:40.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.043+0000 7fb123fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb1240b5f20 0x7fb1240b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:40.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.043+0000 7fb1237fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb124162880 0x7fb1240d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36052/0 (socket says 172.21.15.47:36052) 2024-04-26T23:19:40.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.043+0000 7fb1237fe640 1 -- 172.21.15.47:0/1453761842 learned_addr learned my addr 172.21.15.47:0/1453761842 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:40.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.043+0000 7fb122ffd640 1 --2- 172.21.15.47:0/1453761842 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1240d3bc0 0x7fb1240d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:40.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.043+0000 7fb123fff640 1 -- 172.21.15.47:0/1453761842 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1240d3bc0 msgr2=0x7fb1240d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:40.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.043+0000 7fb123fff640 1 --2- 172.21.15.47:0/1453761842 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1240d3bc0 0x7fb1240d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:40.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.043+0000 7fb123fff640 1 -- 172.21.15.47:0/1453761842 >> v1:172.21.15.47:6789/0 conn(0x7fb124162880 legacy=0x7fb1240d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:40.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.043+0000 7fb123fff640 1 -- 172.21.15.47:0/1453761842 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1240d07d0 con 0x7fb1240b5f20 2024-04-26T23:19:40.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.044+0000 7fb123fff640 1 --2- 172.21.15.47:0/1453761842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1240b5f20 0x7fb1240b8300 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb11c009d90 tx=0x7fb11c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d70c7adc0c7d84ab server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:40.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.044+0000 7fb1227fc640 1 -- 172.21.15.47:0/1453761842 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb11c007740 con 0x7fb1240b5f20 2024-04-26T23:19:40.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.044+0000 7fb1227fc640 1 -- 172.21.15.47:0/1453761842 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb11c0078e0 con 0x7fb1240b5f20 2024-04-26T23:19:40.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.044+0000 7fb1227fc640 1 -- 172.21.15.47:0/1453761842 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb11c007cb0 con 0x7fb1240b5f20 2024-04-26T23:19:40.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.045+0000 7fb12a064640 1 -- 172.21.15.47:0/1453761842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1240b5f20 msgr2=0x7fb1240b8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:40.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.045+0000 7fb12a064640 1 --2- 172.21.15.47:0/1453761842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1240b5f20 0x7fb1240b8300 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb11c009d90 tx=0x7fb11c02a660 comp rx=0 tx=0).stop 2024-04-26T23:19:40.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.045+0000 7fb12a064640 1 -- 172.21.15.47:0/1453761842 shutdown_connections 2024-04-26T23:19:40.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.045+0000 7fb12a064640 1 --2- 172.21.15.47:0/1453761842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1240b5f20 0x7fb1240b8300 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:40.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.045+0000 7fb12a064640 1 --2- 172.21.15.47:0/1453761842 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1240d3bc0 0x7fb1240d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:40.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.045+0000 7fb12a064640 1 -- 172.21.15.47:0/1453761842 >> 172.21.15.47:0/1453761842 conn(0x7fb1240d4f60 msgr2=0x7fb1240d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:40.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.045+0000 7fb12a064640 1 -- 172.21.15.47:0/1453761842 shutdown_connections 2024-04-26T23:19:40.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.045+0000 7fb12a064640 1 -- 172.21.15.47:0/1453761842 wait complete. 2024-04-26T23:19:40.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.046+0000 7fb12a064640 1 Processor -- start 2024-04-26T23:19:40.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.046+0000 7fb12a064640 1 -- start start 2024-04-26T23:19:40.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.047+0000 7fb12a064640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1240b5f20 0x7fb12406e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:40.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.047+0000 7fb12a064640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1240d3bc0 0x7fb12406e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:40.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.047+0000 7fb12a064640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb124162880 0x7fb1240b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:40.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.047+0000 7fb12a064640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb124162c60 con 0x7fb124162880 2024-04-26T23:19:40.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.047+0000 7fb12a064640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb1240d0d90 con 0x7fb1240d3bc0 2024-04-26T23:19:40.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.047+0000 7fb12a064640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb124162de0 con 0x7fb1240b5f20 2024-04-26T23:19:40.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.047+0000 7fb123fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb124162880 0x7fb1240b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:40.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.047+0000 7fb123fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb124162880 0x7fb1240b4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58632/0 (socket says 172.21.15.47:58632) 2024-04-26T23:19:40.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.047+0000 7fb123fff640 1 -- 172.21.15.47:0/994718227 learned_addr learned my addr 172.21.15.47:0/994718227 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:40.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.047+0000 7fb1237fe640 1 --2- 172.21.15.47:0/994718227 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1240b5f20 0x7fb12406e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:40.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.047+0000 7fb122ffd640 1 --2- 172.21.15.47:0/994718227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1240d3bc0 0x7fb12406e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:40.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.047+0000 7fb123fff640 1 -- 172.21.15.47:0/994718227 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1240b5f20 msgr2=0x7fb12406e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:40.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.047+0000 7fb123fff640 1 --2- 172.21.15.47:0/994718227 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1240b5f20 0x7fb12406e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:40.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.047+0000 7fb123fff640 1 -- 172.21.15.47:0/994718227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1240d3bc0 msgr2=0x7fb12406e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:40.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.047+0000 7fb123fff640 1 --2- 172.21.15.47:0/994718227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1240d3bc0 0x7fb12406e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:40.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.047+0000 7fb123fff640 1 -- 172.21.15.47:0/994718227 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1240cdf80 con 0x7fb124162880 2024-04-26T23:19:40.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.047+0000 7fb1237fe640 1 --2- 172.21.15.47:0/994718227 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1240b5f20 0x7fb12406e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:19:40.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.047+0000 7fb122ffd640 1 --2- 172.21.15.47:0/994718227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1240d3bc0 0x7fb12406e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:19:40.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.047+0000 7fb123fff640 1 --2- 172.21.15.47:0/994718227 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb124162880 0x7fb1240b4f20 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb11c02ac50 tx=0x7fb11c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:40.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.048+0000 7fb10bfff640 1 -- 172.21.15.47:0/994718227 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb11c03f070 con 0x7fb124162880 2024-04-26T23:19:40.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.048+0000 7fb10bfff640 1 -- 172.21.15.47:0/994718227 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb11c031bc0 con 0x7fb124162880 2024-04-26T23:19:40.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.048+0000 7fb12a064640 1 -- 172.21.15.47:0/994718227 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1240ce220 con 0x7fb124162880 2024-04-26T23:19:40.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.048+0000 7fb10bfff640 1 -- 172.21.15.47:0/994718227 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb11c044460 con 0x7fb124162880 2024-04-26T23:19:40.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.048+0000 7fb12a064640 1 -- 172.21.15.47:0/994718227 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1240ce7e0 con 0x7fb124162880 2024-04-26T23:19:40.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.049+0000 7fb12a064640 1 -- 172.21.15.47:0/994718227 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1240694f0 con 0x7fb124162880 2024-04-26T23:19:40.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.049+0000 7fb10bfff640 1 -- 172.21.15.47:0/994718227 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb11c007790 con 0x7fb124162880 2024-04-26T23:19:40.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.050+0000 7fb10bfff640 1 --2- 172.21.15.47:0/994718227 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb11006c8f0 0x7fb11006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:40.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.050+0000 7fb10bfff640 1 -- 172.21.15.47:0/994718227 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7fb11c0ad210 con 0x7fb124162880 2024-04-26T23:19:40.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.050+0000 7fb1237fe640 1 --2- 172.21.15.47:0/994718227 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb11006c8f0 0x7fb11006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:40.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.050+0000 7fb1237fe640 1 --2- 172.21.15.47:0/994718227 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb11006c8f0 0x7fb11006edb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb1240d2820 tx=0x7fb114009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:40.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.054+0000 7fb10bfff640 1 -- 172.21.15.47:0/994718227 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb11c07b800 con 0x7fb124162880 2024-04-26T23:19:40.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.314+0000 7fb12a064640 1 -- 172.21.15.47:0/994718227 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7fb124074590 con 0x7fb124162880 2024-04-26T23:19:40.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.316+0000 7fb10bfff640 1 -- 172.21.15.47:0/994718227 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7fb11c07b100 con 0x7fb124162880 2024-04-26T23:19:40.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.318+0000 7fb12a064640 1 -- 172.21.15.47:0/994718227 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb11006c8f0 msgr2=0x7fb11006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:40.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.318+0000 7fb12a064640 1 --2- 172.21.15.47:0/994718227 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb11006c8f0 0x7fb11006edb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb1240d2820 tx=0x7fb114009040 comp rx=0 tx=0).stop 2024-04-26T23:19:40.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.318+0000 7fb12a064640 1 -- 172.21.15.47:0/994718227 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb124162880 msgr2=0x7fb1240b4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:40.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.318+0000 7fb12a064640 1 --2- 172.21.15.47:0/994718227 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb124162880 0x7fb1240b4f20 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb11c02ac50 tx=0x7fb11c004a40 comp rx=0 tx=0).stop 2024-04-26T23:19:40.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.318+0000 7fb12a064640 1 -- 172.21.15.47:0/994718227 shutdown_connections 2024-04-26T23:19:40.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.318+0000 7fb12a064640 1 --2- 172.21.15.47:0/994718227 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb11006c8f0 0x7fb11006edb0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:40.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.318+0000 7fb12a064640 1 --2- 172.21.15.47:0/994718227 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb124162880 0x7fb1240b4f20 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:40.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.318+0000 7fb12a064640 1 --2- 172.21.15.47:0/994718227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1240d3bc0 0x7fb12406e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:40.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.318+0000 7fb12a064640 1 --2- 172.21.15.47:0/994718227 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1240b5f20 0x7fb12406e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:40.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.318+0000 7fb12a064640 1 -- 172.21.15.47:0/994718227 >> 172.21.15.47:0/994718227 conn(0x7fb1240d4f60 msgr2=0x7fb1240ba880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:40.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.319+0000 7fb12a064640 1 -- 172.21.15.47:0/994718227 shutdown_connections 2024-04-26T23:19:40.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.319+0000 7fb12a064640 1 -- 172.21.15.47:0/994718227 wait complete. 2024-04-26T23:19:40.330 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-26T23:19:40.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.432+0000 7f8aa4ab0640 1 Processor -- start 2024-04-26T23:19:40.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.432+0000 7f8aa4ab0640 1 -- start start 2024-04-26T23:19:40.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.433+0000 7f8aa4ab0640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8aa0162810 0x7f8aa0070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:40.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.433+0000 7f8aa4ab0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8aa00b2340 0x7f8aa00cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:40.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.433+0000 7f8aa4ab0640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8aa00d10d0 con 0x7f8aa00cd930 2024-04-26T23:19:40.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.433+0000 7f8aa4ab0640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f8aa00d0750 con 0x7f8aa0162810 2024-04-26T23:19:40.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.433+0000 7f8aa4ab0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f8aa00d16a0 con 0x7f8aa00b2340 2024-04-26T23:19:40.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.433+0000 7f8a9e59c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8aa0162810 0x7f8aa0070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:40.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.433+0000 7f8a9e59c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8aa0162810 0x7f8aa0070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58638/0 (socket says 172.21.15.47:58638) 2024-04-26T23:19:40.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.433+0000 7f8a9e59c640 1 -- 172.21.15.47:0/1071606536 learned_addr learned my addr 172.21.15.47:0/1071606536 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:40.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.434+0000 7f8a9e59c640 1 -- 172.21.15.47:0/1071606536 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8aa00b2340 msgr2=0x7f8aa00cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:40.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.434+0000 7f8a9e59c640 1 --2- 172.21.15.47:0/1071606536 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8aa00b2340 0x7f8aa00cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:40.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.434+0000 7f8a9e59c640 1 -- 172.21.15.47:0/1071606536 >> v1:172.21.15.203:6789/0 conn(0x7f8aa00cd930 legacy=0x7f8aa00d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:40.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.434+0000 7f8a9e59c640 1 -- 172.21.15.47:0/1071606536 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aa00d1370 con 0x7f8aa0162810 2024-04-26T23:19:40.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.434+0000 7f8a9e59c640 1 --2- 172.21.15.47:0/1071606536 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8aa0162810 0x7f8aa0070350 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8a98009870 tx=0x7f8a9802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=840bc3583061215 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:40.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.434+0000 7f8a9d59a640 1 -- 172.21.15.47:0/1071606536 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a98037070 con 0x7f8aa0162810 2024-04-26T23:19:40.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.434+0000 7f8a9d59a640 1 -- 172.21.15.47:0/1071606536 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a98032440 con 0x7f8aa0162810 2024-04-26T23:19:40.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.434+0000 7f8a9d59a640 1 -- 172.21.15.47:0/1071606536 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a98032810 con 0x7f8aa0162810 2024-04-26T23:19:40.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.435+0000 7f8aa4ab0640 1 -- 172.21.15.47:0/1071606536 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8aa0162810 msgr2=0x7f8aa0070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:40.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.435+0000 7f8aa4ab0640 1 --2- 172.21.15.47:0/1071606536 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8aa0162810 0x7f8aa0070350 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8a98009870 tx=0x7f8a9802a210 comp rx=0 tx=0).stop 2024-04-26T23:19:40.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.435+0000 7f8aa4ab0640 1 -- 172.21.15.47:0/1071606536 shutdown_connections 2024-04-26T23:19:40.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.435+0000 7f8aa4ab0640 1 --2- 172.21.15.47:0/1071606536 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8aa00b2340 0x7f8aa00cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:40.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.435+0000 7f8aa4ab0640 1 --2- 172.21.15.47:0/1071606536 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8aa0162810 0x7f8aa0070350 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:40.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.435+0000 7f8aa4ab0640 1 -- 172.21.15.47:0/1071606536 >> 172.21.15.47:0/1071606536 conn(0x7f8aa00d2da0 msgr2=0x7f8aa00d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:40.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.435+0000 7f8aa4ab0640 1 -- 172.21.15.47:0/1071606536 shutdown_connections 2024-04-26T23:19:40.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.435+0000 7f8aa4ab0640 1 -- 172.21.15.47:0/1071606536 wait complete. 2024-04-26T23:19:40.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.436+0000 7f8aa4ab0640 1 Processor -- start 2024-04-26T23:19:40.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.436+0000 7f8aa4ab0640 1 -- start start 2024-04-26T23:19:40.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.437+0000 7f8aa4ab0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8aa00b2340 0x7f8aa00c68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:40.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.437+0000 7f8aa4ab0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8aa00cd930 0x7f8aa014e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:40.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.437+0000 7f8aa4ab0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8aa0162810 0x7f8aa014ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:40.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.437+0000 7f8aa4ab0640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa0070890 con 0x7f8aa00b2340 2024-04-26T23:19:40.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.437+0000 7f8aa4ab0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa00d0470 con 0x7f8aa00cd930 2024-04-26T23:19:40.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.437+0000 7f8aa4ab0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f8aa00d2240 con 0x7f8aa0162810 2024-04-26T23:19:40.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.437+0000 7f8a9e59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8aa00b2340 0x7f8aa00c68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:40.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.437+0000 7f8a9e59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8aa00b2340 0x7f8aa00c68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58650/0 (socket says 172.21.15.47:58650) 2024-04-26T23:19:40.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.437+0000 7f8a9e59c640 1 -- 172.21.15.47:0/2673138317 learned_addr learned my addr 172.21.15.47:0/2673138317 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:40.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.437+0000 7f8a9e59c640 1 -- 172.21.15.47:0/2673138317 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8aa0162810 msgr2=0x7f8aa014ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:40.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.437+0000 7f8a9dd9b640 1 --2- 172.21.15.47:0/2673138317 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8aa00cd930 0x7f8aa014e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:40.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.437+0000 7f8a9ed9d640 1 --2- 172.21.15.47:0/2673138317 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8aa0162810 0x7f8aa014ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:40.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.437+0000 7f8a9e59c640 1 --2- 172.21.15.47:0/2673138317 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8aa0162810 0x7f8aa014ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:40.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.437+0000 7f8a9e59c640 1 -- 172.21.15.47:0/2673138317 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8aa00cd930 msgr2=0x7f8aa014e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:40.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.437+0000 7f8a9e59c640 1 --2- 172.21.15.47:0/2673138317 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8aa00cd930 0x7f8aa014e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:40.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.437+0000 7f8a9e59c640 1 -- 172.21.15.47:0/2673138317 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aa00cb1d0 con 0x7f8aa00b2340 2024-04-26T23:19:40.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.437+0000 7f8a9dd9b640 1 --2- 172.21.15.47:0/2673138317 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8aa00cd930 0x7f8aa014e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:19:40.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.437+0000 7f8a9ed9d640 1 --2- 172.21.15.47:0/2673138317 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8aa0162810 0x7f8aa014ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:19:40.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.438+0000 7f8a9e59c640 1 --2- 172.21.15.47:0/2673138317 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8aa00b2340 0x7f8aa00c68d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f8a98009870 tx=0x7f8a980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:40.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.438+0000 7f8a8effd640 1 -- 172.21.15.47:0/2673138317 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a98041070 con 0x7f8aa00b2340 2024-04-26T23:19:40.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.438+0000 7f8a8effd640 1 -- 172.21.15.47:0/2673138317 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a9803c870 con 0x7f8aa00b2340 2024-04-26T23:19:40.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.438+0000 7f8aa4ab0640 1 -- 172.21.15.47:0/2673138317 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8aa00cb4b0 con 0x7f8aa00b2340 2024-04-26T23:19:40.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.438+0000 7f8a8effd640 1 -- 172.21.15.47:0/2673138317 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a98037040 con 0x7f8aa00b2340 2024-04-26T23:19:40.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.438+0000 7f8aa4ab0640 1 -- 172.21.15.47:0/2673138317 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8aa00cba70 con 0x7f8aa00b2340 2024-04-26T23:19:40.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.439+0000 7f8aa4ab0640 1 -- 172.21.15.47:0/2673138317 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8aa00694f0 con 0x7f8aa00b2340 2024-04-26T23:19:40.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.439+0000 7f8a8effd640 1 -- 172.21.15.47:0/2673138317 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8a98044070 con 0x7f8aa00b2340 2024-04-26T23:19:40.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.440+0000 7f8a8effd640 1 --2- 172.21.15.47:0/2673138317 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f8a8006c8f0 0x7f8a8006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:40.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.440+0000 7f8a8effd640 1 -- 172.21.15.47:0/2673138317 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f8a980ac5e0 con 0x7f8aa00b2340 2024-04-26T23:19:40.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.440+0000 7f8a9dd9b640 1 --2- 172.21.15.47:0/2673138317 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f8a8006c8f0 0x7f8a8006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:40.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.441+0000 7f8a9dd9b640 1 --2- 172.21.15.47:0/2673138317 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f8a8006c8f0 0x7f8a8006edb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8a94006fd0 tx=0x7f8a94008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:40.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.444+0000 7f8a8effd640 1 -- 172.21.15.47:0/2673138317 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8a9807ac50 con 0x7f8aa00b2340 2024-04-26T23:19:40.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.701+0000 7f8aa4ab0640 1 -- 172.21.15.47:0/2673138317 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f8aa00b3300 con 0x7f8aa00b2340 2024-04-26T23:19:40.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.703+0000 7f8a8effd640 1 -- 172.21.15.47:0/2673138317 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v8) ==== 97+0+0 (secure 0 0 0) 0x7f8a9807a550 con 0x7f8aa00b2340 2024-04-26T23:19:40.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.705+0000 7f8aa4ab0640 1 -- 172.21.15.47:0/2673138317 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f8a8006c8f0 msgr2=0x7f8a8006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:40.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.705+0000 7f8aa4ab0640 1 --2- 172.21.15.47:0/2673138317 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f8a8006c8f0 0x7f8a8006edb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8a94006fd0 tx=0x7f8a94008040 comp rx=0 tx=0).stop 2024-04-26T23:19:40.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.705+0000 7f8aa4ab0640 1 -- 172.21.15.47:0/2673138317 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8aa00b2340 msgr2=0x7f8aa00c68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:40.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.705+0000 7f8aa4ab0640 1 --2- 172.21.15.47:0/2673138317 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8aa00b2340 0x7f8aa00c68d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f8a98009870 tx=0x7f8a980046c0 comp rx=0 tx=0).stop 2024-04-26T23:19:40.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.705+0000 7f8aa4ab0640 1 -- 172.21.15.47:0/2673138317 shutdown_connections 2024-04-26T23:19:40.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.705+0000 7f8aa4ab0640 1 --2- 172.21.15.47:0/2673138317 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f8a8006c8f0 0x7f8a8006edb0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:40.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.705+0000 7f8aa4ab0640 1 --2- 172.21.15.47:0/2673138317 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8aa0162810 0x7f8aa014ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:40.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.705+0000 7f8aa4ab0640 1 --2- 172.21.15.47:0/2673138317 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8aa00cd930 0x7f8aa014e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:40.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.705+0000 7f8aa4ab0640 1 --2- 172.21.15.47:0/2673138317 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8aa00b2340 0x7f8aa00c68d0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:40.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.705+0000 7f8aa4ab0640 1 -- 172.21.15.47:0/2673138317 >> 172.21.15.47:0/2673138317 conn(0x7f8aa00d2da0 msgr2=0x7f8aa00cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:40.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.705+0000 7f8aa4ab0640 1 -- 172.21.15.47:0/2673138317 shutdown_connections 2024-04-26T23:19:40.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.705+0000 7f8aa4ab0640 1 -- 172.21.15.47:0/2673138317 wait complete. 2024-04-26T23:19:40.717 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-26T23:19:40.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.818+0000 7fb4a66fc640 1 Processor -- start 2024-04-26T23:19:40.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.818+0000 7fb4a66fc640 1 -- start start 2024-04-26T23:19:40.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.819+0000 7fb4a66fc640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb4a015f3a0 0x7fb4a00d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:40.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.819+0000 7fb4a66fc640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb4a00d1770 0x7fb4a00d1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:40.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.819+0000 7fb4a66fc640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4a00d2590 con 0x7fb4a00d2160 2024-04-26T23:19:40.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.819+0000 7fb4a66fc640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fb4a00d0e20 con 0x7fb4a00d1770 2024-04-26T23:19:40.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.819+0000 7fb4a66fc640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fb4a00b1d70 con 0x7fb4a015f3a0 2024-04-26T23:19:40.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.819+0000 7fb49f7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb4a00d1770 0x7fb4a00d1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:40.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.819+0000 7fb49f7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb4a00d1770 0x7fb4a00d1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58666/0 (socket says 172.21.15.47:58666) 2024-04-26T23:19:40.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.819+0000 7fb49f7fe640 1 -- 172.21.15.47:0/1546626827 learned_addr learned my addr 172.21.15.47:0/1546626827 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:40.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.820+0000 7fb49ffff640 1 --2- 172.21.15.47:0/1546626827 >> v2:172.21.15.203:3300/0 conn(0x7fb4a015f3a0 0x7fb4a00d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:40.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.820+0000 7fb49f7fe640 1 -- 172.21.15.47:0/1546626827 >> v2:172.21.15.203:3300/0 conn(0x7fb4a015f3a0 msgr2=0x7fb4a00d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:40.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.820+0000 7fb49f7fe640 1 --2- 172.21.15.47:0/1546626827 >> v2:172.21.15.203:3300/0 conn(0x7fb4a015f3a0 0x7fb4a00d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:40.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.820+0000 7fb49f7fe640 1 -- 172.21.15.47:0/1546626827 >> v1:172.21.15.203:6789/0 conn(0x7fb4a00d2160 legacy=0x7fb4a00d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:40.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.820+0000 7fb49f7fe640 1 -- 172.21.15.47:0/1546626827 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4a00b2050 con 0x7fb4a00d1770 2024-04-26T23:19:40.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.820+0000 7fb49f7fe640 1 --2- 172.21.15.47:0/1546626827 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4a00d1770 0x7fb4a00d1c20 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0073b50 tx=0x7fb49402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc36f6bb882812f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:40.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.821+0000 7fb49effd640 1 -- 172.21.15.47:0/1546626827 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb494037070 con 0x7fb4a00d1770 2024-04-26T23:19:40.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.821+0000 7fb49effd640 1 -- 172.21.15.47:0/1546626827 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb494032440 con 0x7fb4a00d1770 2024-04-26T23:19:40.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.821+0000 7fb49effd640 1 -- 172.21.15.47:0/1546626827 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb494032810 con 0x7fb4a00d1770 2024-04-26T23:19:40.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.821+0000 7fb4a66fc640 1 -- 172.21.15.47:0/1546626827 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4a00d1770 msgr2=0x7fb4a00d1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:40.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.821+0000 7fb4a66fc640 1 --2- 172.21.15.47:0/1546626827 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4a00d1770 0x7fb4a00d1c20 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0073b50 tx=0x7fb49402a270 comp rx=0 tx=0).stop 2024-04-26T23:19:40.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.821+0000 7fb4a66fc640 1 -- 172.21.15.47:0/1546626827 shutdown_connections 2024-04-26T23:19:40.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.821+0000 7fb4a66fc640 1 --2- 172.21.15.47:0/1546626827 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4a00d1770 0x7fb4a00d1c20 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:40.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.821+0000 7fb4a66fc640 1 --2- 172.21.15.47:0/1546626827 >> v2:172.21.15.203:3300/0 conn(0x7fb4a015f3a0 0x7fb4a00d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:40.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.821+0000 7fb4a66fc640 1 -- 172.21.15.47:0/1546626827 >> 172.21.15.47:0/1546626827 conn(0x7fb4a00d4f90 msgr2=0x7fb4a00d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:40.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.821+0000 7fb4a66fc640 1 -- 172.21.15.47:0/1546626827 shutdown_connections 2024-04-26T23:19:40.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.821+0000 7fb4a66fc640 1 -- 172.21.15.47:0/1546626827 wait complete. 2024-04-26T23:19:40.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.822+0000 7fb4a66fc640 1 Processor -- start 2024-04-26T23:19:40.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.823+0000 7fb4a66fc640 1 -- start start 2024-04-26T23:19:40.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.823+0000 7fb4a66fc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4a00d1770 0x7fb4a0154810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:40.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.823+0000 7fb4a66fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4a00d2160 0x7fb4a00caeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:40.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.823+0000 7fb4a66fc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4a015f3a0 0x7fb4a00cb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:40.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.823+0000 7fb4a66fc640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a015f900 con 0x7fb4a00d2160 2024-04-26T23:19:40.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.823+0000 7fb4a66fc640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a015f780 con 0x7fb4a015f3a0 2024-04-26T23:19:40.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.823+0000 7fb4a66fc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb4a00b3860 con 0x7fb4a00d1770 2024-04-26T23:19:40.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.823+0000 7fb49f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4a00d2160 0x7fb4a00caeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:40.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.823+0000 7fb49f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4a00d2160 0x7fb4a00caeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58678/0 (socket says 172.21.15.47:58678) 2024-04-26T23:19:40.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.823+0000 7fb49f7fe640 1 -- 172.21.15.47:0/901097616 learned_addr learned my addr 172.21.15.47:0/901097616 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:40.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.824+0000 7fb49f7fe640 1 -- 172.21.15.47:0/901097616 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4a00d1770 msgr2=0x7fb4a0154810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:40.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.824+0000 7fb49f7fe640 1 --2- 172.21.15.47:0/901097616 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4a00d1770 0x7fb4a0154810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:40.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.824+0000 7fb49f7fe640 1 -- 172.21.15.47:0/901097616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4a015f3a0 msgr2=0x7fb4a00cb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:40.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.824+0000 7fb49f7fe640 1 --2- 172.21.15.47:0/901097616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4a015f3a0 0x7fb4a00cb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:40.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.824+0000 7fb49f7fe640 1 -- 172.21.15.47:0/901097616 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4a00cbc00 con 0x7fb4a00d2160 2024-04-26T23:19:40.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.824+0000 7fb49f7fe640 1 --2- 172.21.15.47:0/901097616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4a00d2160 0x7fb4a00caeb0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb49402a2b0 tx=0x7fb4940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:40.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.824+0000 7fb49cff9640 1 -- 172.21.15.47:0/901097616 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb494041020 con 0x7fb4a00d2160 2024-04-26T23:19:40.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.824+0000 7fb49cff9640 1 -- 172.21.15.47:0/901097616 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4940306c0 con 0x7fb4a00d2160 2024-04-26T23:19:40.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.824+0000 7fb4a66fc640 1 -- 172.21.15.47:0/901097616 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4a00cb930 con 0x7fb4a00d2160 2024-04-26T23:19:40.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.824+0000 7fb49cff9640 1 -- 172.21.15.47:0/901097616 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb494037040 con 0x7fb4a00d2160 2024-04-26T23:19:40.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.824+0000 7fb4a66fc640 1 -- 172.21.15.47:0/901097616 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4a0152250 con 0x7fb4a00d2160 2024-04-26T23:19:40.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.825+0000 7fb4a66fc640 1 -- 172.21.15.47:0/901097616 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4a00694f0 con 0x7fb4a00d2160 2024-04-26T23:19:40.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.825+0000 7fb49cff9640 1 -- 172.21.15.47:0/901097616 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb494044020 con 0x7fb4a00d2160 2024-04-26T23:19:40.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.826+0000 7fb49cff9640 1 --2- 172.21.15.47:0/901097616 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb48c06c8f0 0x7fb48c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:40.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.826+0000 7fb49cff9640 1 -- 172.21.15.47:0/901097616 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7fb4940ac1e0 con 0x7fb4a00d2160 2024-04-26T23:19:40.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.826+0000 7fb49ffff640 1 --2- 172.21.15.47:0/901097616 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb48c06c8f0 0x7fb48c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:40.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.827+0000 7fb49ffff640 1 --2- 172.21.15.47:0/901097616 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb48c06c8f0 0x7fb48c06edb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb490006fd0 tx=0x7fb490008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:40.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:40.830+0000 7fb49cff9640 1 -- 172.21.15.47:0/901097616 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb49404b030 con 0x7fb4a00d2160 2024-04-26T23:19:41.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.087+0000 7fb4a66fc640 1 -- 172.21.15.47:0/901097616 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7fb4a00d0460 con 0x7fb4a00d2160 2024-04-26T23:19:41.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.089+0000 7fb49cff9640 1 -- 172.21.15.47:0/901097616 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v9) ==== 90+0+0 (secure 0 0 0) 0x7fb49407a7d0 con 0x7fb4a00d2160 2024-04-26T23:19:41.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.091+0000 7fb4a66fc640 1 -- 172.21.15.47:0/901097616 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb48c06c8f0 msgr2=0x7fb48c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:41.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.091+0000 7fb4a66fc640 1 --2- 172.21.15.47:0/901097616 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb48c06c8f0 0x7fb48c06edb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb490006fd0 tx=0x7fb490008040 comp rx=0 tx=0).stop 2024-04-26T23:19:41.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.091+0000 7fb4a66fc640 1 -- 172.21.15.47:0/901097616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4a00d2160 msgr2=0x7fb4a00caeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:41.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.091+0000 7fb4a66fc640 1 --2- 172.21.15.47:0/901097616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4a00d2160 0x7fb4a00caeb0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb49402a2b0 tx=0x7fb4940046c0 comp rx=0 tx=0).stop 2024-04-26T23:19:41.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.091+0000 7fb4a66fc640 1 -- 172.21.15.47:0/901097616 shutdown_connections 2024-04-26T23:19:41.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.091+0000 7fb4a66fc640 1 --2- 172.21.15.47:0/901097616 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb48c06c8f0 0x7fb48c06edb0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:41.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.091+0000 7fb4a66fc640 1 --2- 172.21.15.47:0/901097616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4a015f3a0 0x7fb4a00cb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:41.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.091+0000 7fb4a66fc640 1 --2- 172.21.15.47:0/901097616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4a00d2160 0x7fb4a00caeb0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:41.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.091+0000 7fb4a66fc640 1 --2- 172.21.15.47:0/901097616 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4a00d1770 0x7fb4a0154810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:41.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.091+0000 7fb4a66fc640 1 -- 172.21.15.47:0/901097616 >> 172.21.15.47:0/901097616 conn(0x7fb4a00d4f90 msgr2=0x7fb4a006d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:41.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.091+0000 7fb4a66fc640 1 -- 172.21.15.47:0/901097616 shutdown_connections 2024-04-26T23:19:41.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.091+0000 7fb4a66fc640 1 -- 172.21.15.47:0/901097616 wait complete. 2024-04-26T23:19:41.103 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:19:41.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.204+0000 7f2e5fbbf640 1 Processor -- start 2024-04-26T23:19:41.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.204+0000 7f2e5fbbf640 1 -- start start 2024-04-26T23:19:41.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.205+0000 7f2e5fbbf640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e58162880 0x7f2e580d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:41.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.205+0000 7f2e5fbbf640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e580b7e50 con 0x7f2e580d3bc0 2024-04-26T23:19:41.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.205+0000 7f2e5fbbf640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e580d1800 con 0x7f2e580b5f20 2024-04-26T23:19:41.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.205+0000 7f2e5fbbf640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f2e580d0580 con 0x7f2e58162880 2024-04-26T23:19:41.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.205+0000 7f2e5e15c640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f2e580d3bc0 0x7f2e580b8300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36064/0 (socket says 172.21.15.47:36064) 2024-04-26T23:19:41.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.205+0000 7f2e5e15c640 1 -- 172.21.15.47:0/2087838464 learned_addr learned my addr 172.21.15.47:0/2087838464 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:41.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.205+0000 7f2e5d95b640 1 --2- 172.21.15.47:0/2087838464 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e58162880 0x7f2e580d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:41.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.205+0000 7f2e5c959640 1 -- 172.21.15.47:0/2087838464 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 338056425 0 0) 0x7f2e580b7e50 con 0x7f2e580d3bc0 2024-04-26T23:19:41.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.205+0000 7f2e5c959640 1 -- 172.21.15.47:0/2087838464 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e44003520 con 0x7f2e580d3bc0 2024-04-26T23:19:41.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.206+0000 7f2e5c959640 1 -- 172.21.15.47:0/2087838464 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 304496278 0 0) 0x7f2e44003520 con 0x7f2e580d3bc0 2024-04-26T23:19:41.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.206+0000 7f2e5c959640 1 -- 172.21.15.47:0/2087838464 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e58162880 msgr2=0x7f2e580d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:41.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.206+0000 7f2e5c959640 1 --2- 172.21.15.47:0/2087838464 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e58162880 0x7f2e580d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:41.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.206+0000 7f2e5c959640 1 -- 172.21.15.47:0/2087838464 >> v1:172.21.15.203:6789/0 conn(0x7f2e580b5f20 legacy=0x7f2e580d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:41.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.206+0000 7f2e5c959640 1 -- 172.21.15.47:0/2087838464 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e580d07d0 con 0x7f2e580d3bc0 2024-04-26T23:19:41.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.206+0000 7f2e5c959640 1 -- 172.21.15.47:0/2087838464 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7f2e500019f0 con 0x7f2e580d3bc0 2024-04-26T23:19:41.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.206+0000 7f2e5c959640 1 -- 172.21.15.47:0/2087838464 >> v1:172.21.15.47:6789/0 conn(0x7f2e580d3bc0 legacy=0x7f2e580b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:41.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.206+0000 7f2e5c959640 1 --2- 172.21.15.47:0/2087838464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e440039e0 0x7f2e44003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:41.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.206+0000 7f2e5c959640 1 --2- 172.21.15.47:0/2087838464 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e44004390 0x7f2e44004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:41.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.206+0000 7f2e5c959640 1 --2- 172.21.15.47:0/2087838464 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e58162880 0x7f2e440050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:41.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.206+0000 7f2e5c959640 1 -- 172.21.15.47:0/2087838464 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2e580d0580 con 0x7f2e440039e0 2024-04-26T23:19:41.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.206+0000 7f2e5c959640 1 -- 172.21.15.47:0/2087838464 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4400b9c0 con 0x7f2e44004390 2024-04-26T23:19:41.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.206+0000 7f2e5c959640 1 -- 172.21.15.47:0/2087838464 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f2e4400bbc0 con 0x7f2e58162880 2024-04-26T23:19:41.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.206+0000 7f2e5d95b640 1 --2- 172.21.15.47:0/2087838464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e440039e0 0x7f2e44003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:41.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.206+0000 7f2e5e15c640 1 --2- 172.21.15.47:0/2087838464 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e58162880 0x7f2e440050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:41.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.206+0000 7f2e5d15a640 1 --2- 172.21.15.47:0/2087838464 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e44004390 0x7f2e44004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:41.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.206+0000 7f2e5d95b640 1 -- 172.21.15.47:0/2087838464 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e58162880 msgr2=0x7f2e440050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:41.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.206+0000 7f2e5d95b640 1 --2- 172.21.15.47:0/2087838464 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e58162880 0x7f2e440050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:41.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.206+0000 7f2e5d95b640 1 -- 172.21.15.47:0/2087838464 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e44004390 msgr2=0x7f2e44004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:41.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.206+0000 7f2e5d95b640 1 --2- 172.21.15.47:0/2087838464 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e44004390 0x7f2e44004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:41.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.206+0000 7f2e5d95b640 1 -- 172.21.15.47:0/2087838464 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e44003520 con 0x7f2e440039e0 2024-04-26T23:19:41.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.206+0000 7f2e5e15c640 1 --2- 172.21.15.47:0/2087838464 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e58162880 0x7f2e440050c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:19:41.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.207+0000 7f2e5d95b640 1 --2- 172.21.15.47:0/2087838464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e440039e0 0x7f2e44003dc0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2e54009ac0 tx=0x7f2e44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5c599f028e71704 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:41.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.207+0000 7f2e5c959640 1 -- 172.21.15.47:0/2087838464 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e54036070 con 0x7f2e440039e0 2024-04-26T23:19:41.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.207+0000 7f2e5c959640 1 -- 172.21.15.47:0/2087838464 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e54031400 con 0x7f2e440039e0 2024-04-26T23:19:41.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.207+0000 7f2e5c959640 1 -- 172.21.15.47:0/2087838464 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e54009690 con 0x7f2e440039e0 2024-04-26T23:19:41.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.207+0000 7f2e5fbbf640 1 -- 172.21.15.47:0/2087838464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e440039e0 msgr2=0x7f2e44003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:41.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.207+0000 7f2e5fbbf640 1 --2- 172.21.15.47:0/2087838464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e440039e0 0x7f2e44003dc0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2e54009ac0 tx=0x7f2e44005830 comp rx=0 tx=0).stop 2024-04-26T23:19:41.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.208+0000 7f2e5e15c640 1 -- 172.21.15.47:0/2087838464 reap_dead start 2024-04-26T23:19:41.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.208+0000 7f2e5fbbf640 1 -- 172.21.15.47:0/2087838464 shutdown_connections 2024-04-26T23:19:41.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.208+0000 7f2e5fbbf640 1 -- 172.21.15.47:0/2087838464 >> 172.21.15.47:0/2087838464 conn(0x7f2e580d4f60 msgr2=0x7f2e580d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:41.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.208+0000 7f2e5fbbf640 1 -- 172.21.15.47:0/2087838464 shutdown_connections 2024-04-26T23:19:41.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.208+0000 7f2e5fbbf640 1 -- 172.21.15.47:0/2087838464 wait complete. 2024-04-26T23:19:41.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.209+0000 7f2e5fbbf640 1 Processor -- start 2024-04-26T23:19:41.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.209+0000 7f2e5fbbf640 1 -- start start 2024-04-26T23:19:41.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.210+0000 7f2e5fbbf640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e440039e0 0x7f2e580b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:41.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.210+0000 7f2e5fbbf640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e580b4de0 0x7f2e5814ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:41.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.210+0000 7f2e5fbbf640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e5814f160 0x7f2e5814f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:41.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.210+0000 7f2e5fbbf640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2e58162c60 con 0x7f2e5814f160 2024-04-26T23:19:41.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.210+0000 7f2e5fbbf640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2e580d0d90 con 0x7f2e440039e0 2024-04-26T23:19:41.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.210+0000 7f2e5fbbf640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f2e58162de0 con 0x7f2e580b4de0 2024-04-26T23:19:41.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.210+0000 7f2e5e15c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e5814f160 0x7f2e5814f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:41.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.210+0000 7f2e5e15c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e5814f160 0x7f2e5814f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58696/0 (socket says 172.21.15.47:58696) 2024-04-26T23:19:41.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.210+0000 7f2e5e15c640 1 -- 172.21.15.47:0/2211575317 learned_addr learned my addr 172.21.15.47:0/2211575317 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:41.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.210+0000 7f2e5e15c640 1 -- 172.21.15.47:0/2211575317 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e580b4de0 msgr2=0x7f2e5814ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:41.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.210+0000 7f2e5e15c640 1 --2- 172.21.15.47:0/2211575317 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e580b4de0 0x7f2e5814ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:41.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.210+0000 7f2e5e15c640 1 -- 172.21.15.47:0/2211575317 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e440039e0 msgr2=0x7f2e580b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:41.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.210+0000 7f2e5e15c640 1 --2- 172.21.15.47:0/2211575317 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e440039e0 0x7f2e580b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:41.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.210+0000 7f2e5e15c640 1 -- 172.21.15.47:0/2211575317 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e580b5460 con 0x7f2e5814f160 2024-04-26T23:19:41.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.210+0000 7f2e5e15c640 1 --2- 172.21.15.47:0/2211575317 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e5814f160 0x7f2e5814f5e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f2e580d11f0 tx=0x7f2e5000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:41.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.210+0000 7f2e4e7fc640 1 -- 172.21.15.47:0/2211575317 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5000eb90 con 0x7f2e5814f160 2024-04-26T23:19:41.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.210+0000 7f2e4e7fc640 1 -- 172.21.15.47:0/2211575317 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e5000ed30 con 0x7f2e5814f160 2024-04-26T23:19:41.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.210+0000 7f2e5fbbf640 1 -- 172.21.15.47:0/2211575317 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e580b5740 con 0x7f2e5814f160 2024-04-26T23:19:41.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.210+0000 7f2e4e7fc640 1 -- 172.21.15.47:0/2211575317 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e50014650 con 0x7f2e5814f160 2024-04-26T23:19:41.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.210+0000 7f2e5fbbf640 1 -- 172.21.15.47:0/2211575317 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e580b1bf0 con 0x7f2e5814f160 2024-04-26T23:19:41.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.211+0000 7f2e5fbbf640 1 -- 172.21.15.47:0/2211575317 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e580694f0 con 0x7f2e5814f160 2024-04-26T23:19:41.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.212+0000 7f2e4e7fc640 1 -- 172.21.15.47:0/2211575317 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2e500147f0 con 0x7f2e5814f160 2024-04-26T23:19:41.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.212+0000 7f2e4e7fc640 1 --2- 172.21.15.47:0/2211575317 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f2e3c06c8f0 0x7f2e3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:41.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.212+0000 7f2e4e7fc640 1 -- 172.21.15.47:0/2211575317 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f2e5008b770 con 0x7f2e5814f160 2024-04-26T23:19:41.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.212+0000 7f2e5d95b640 1 --2- 172.21.15.47:0/2211575317 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f2e3c06c8f0 0x7f2e3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:41.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.213+0000 7f2e5d95b640 1 --2- 172.21.15.47:0/2211575317 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f2e3c06c8f0 0x7f2e3c06edb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2e54009ac0 tx=0x7f2e5402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:41.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.216+0000 7f2e4e7fc640 1 -- 172.21.15.47:0/2211575317 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2e50059d60 con 0x7f2e5814f160 2024-04-26T23:19:41.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.404+0000 7f2e5fbbf640 1 -- 172.21.15.47:0/2211575317 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2e5806d500 con 0x7f2e5814f160 2024-04-26T23:19:41.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.405+0000 7f2e4e7fc640 1 -- 172.21.15.47:0/2211575317 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+13128 (secure 0 0 0) 0x7f2e50059660 con 0x7f2e5814f160 2024-04-26T23:19:41.406 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:19:41.406 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":34,"fsid":"399006d3-4db1-43e6-ba81-a79a276eabfd","created":"2024-04-26T23:18:06.731211+0000","modified":"2024-04-26T23:19:38.389380+0000","last_up_change":"2024-04-26T23:18:17.808736+0000","last_in_change":"2024-04-26T23:18:08.945859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:18:16.937492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ee091567-fea0-4a14-bf4e-0e16c20bb2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6819","nonce":326520779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6821","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6823","nonce":326520779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6829","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6831","nonce":326520779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6825","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6827","nonce":326520779}]},"public_addr":"172.21.15.47:6819/326520779","cluster_addr":"172.21.15.47:6823/326520779","heartbeat_back_addr":"172.21.15.47:6831/326520779","heartbeat_front_addr":"172.21.15.47:6827/326520779","state":["exists","up"]},{"osd":1,"uuid":"c14440a0-bba5-4931-a6db-b736c08e869b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6809","nonce":1912610931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6811","nonce":1912610931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6815","nonce":1912610931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6813","nonce":1912610931}]},"public_addr":"172.21.15.47:6809/1912610931","cluster_addr":"172.21.15.47:6811/1912610931","heartbeat_back_addr":"172.21.15.47:6815/1912610931","heartbeat_front_addr":"172.21.15.47:6813/1912610931","state":["exists","up"]},{"osd":2,"uuid":"3940f8de-c9b2-4cd3-a911-9c301729eb30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6801","nonce":2569869305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6803","nonce":2569869305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6807","nonce":2569869305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6805","nonce":2569869305}]},"public_addr":"172.21.15.47:6801/2569869305","cluster_addr":"172.21.15.47:6803/2569869305","heartbeat_back_addr":"172.21.15.47:6807/2569869305","heartbeat_front_addr":"172.21.15.47:6805/2569869305","state":["exists","up"]},{"osd":3,"uuid":"1286a67e-8958-426a-8c2e-bc85498c619c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6818","nonce":2791411683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6822","nonce":2791411683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6830","nonce":2791411683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6826","nonce":2791411683}]},"public_addr":"172.21.15.47:6818/2791411683","cluster_addr":"172.21.15.47:6822/2791411683","heartbeat_back_addr":"172.21.15.47:6830/2791411683","heartbeat_front_addr":"172.21.15.47:6826/2791411683","state":["exists","up"]},{"osd":4,"uuid":"d8ee2c20-e5b7-46c2-9999-b234070126cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6809","nonce":441313104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6811","nonce":441313104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6815","nonce":441313104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6813","nonce":441313104}]},"public_addr":"172.21.15.203:6809/441313104","cluster_addr":"172.21.15.203:6811/441313104","heartbeat_back_addr":"172.21.15.203:6815/441313104","heartbeat_front_addr":"172.21.15.203:6813/441313104","state":["exists","up"]},{"osd":5,"uuid":"b453a8f0-f324-4840-9ae0-d67bdadb9f8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6801","nonce":56838444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6803","nonce":56838444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6807","nonce":56838444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6805","nonce":56838444}]},"public_addr":"172.21.15.203:6801/56838444","cluster_addr":"172.21.15.203:6803/56838444","heartbeat_back_addr":"172.21.15.203:6807/56838444","heartbeat_front_addr":"172.21.15.203:6805/56838444","state":["exists","up"]},{"osd":6,"uuid":"a6d31951-2d4b-4e8d-a510-6283b01dc158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6825","nonce":2738517080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6827","nonce":2738517080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6831","nonce":2738517080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6829","nonce":2738517080}]},"public_addr":"172.21.15.203:6825/2738517080","cluster_addr":"172.21.15.203:6827/2738517080","heartbeat_back_addr":"172.21.15.203:6831/2738517080","heartbeat_front_addr":"172.21.15.203:6829/2738517080","state":["exists","up"]},{"osd":7,"uuid":"7119b8c0-fe96-402d-9e75-7b509d9d44d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6817","nonce":1315565058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6819","nonce":1315565058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6823","nonce":1315565058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6821","nonce":1315565058}]},"public_addr":"172.21.15.203:6817/1315565058","cluster_addr":"172.21.15.203:6819/1315565058","heartbeat_back_addr":"172.21.15.203:6823/1315565058","heartbeat_front_addr":"172.21.15.203:6821/1315565058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.121921+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.137494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.084814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.111593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.072430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.125066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.097867+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.057061+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:19:41.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.407+0000 7f2e5fbbf640 1 -- 172.21.15.47:0/2211575317 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f2e3c06c8f0 msgr2=0x7f2e3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:41.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.407+0000 7f2e5fbbf640 1 --2- 172.21.15.47:0/2211575317 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f2e3c06c8f0 0x7f2e3c06edb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2e54009ac0 tx=0x7f2e5402e000 comp rx=0 tx=0).stop 2024-04-26T23:19:41.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.407+0000 7f2e5fbbf640 1 -- 172.21.15.47:0/2211575317 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e5814f160 msgr2=0x7f2e5814f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:41.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.407+0000 7f2e5fbbf640 1 --2- 172.21.15.47:0/2211575317 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e5814f160 0x7f2e5814f5e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f2e580d11f0 tx=0x7f2e5000e530 comp rx=0 tx=0).stop 2024-04-26T23:19:41.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.407+0000 7f2e5fbbf640 1 -- 172.21.15.47:0/2211575317 shutdown_connections 2024-04-26T23:19:41.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.407+0000 7f2e5fbbf640 1 --2- 172.21.15.47:0/2211575317 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f2e3c06c8f0 0x7f2e3c06edb0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:41.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.407+0000 7f2e5fbbf640 1 --2- 172.21.15.47:0/2211575317 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e5814f160 0x7f2e5814f5e0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:41.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.407+0000 7f2e5fbbf640 1 --2- 172.21.15.47:0/2211575317 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e580b4de0 0x7f2e5814ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:41.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.407+0000 7f2e5fbbf640 1 --2- 172.21.15.47:0/2211575317 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e440039e0 0x7f2e580b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:41.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.407+0000 7f2e5fbbf640 1 -- 172.21.15.47:0/2211575317 >> 172.21.15.47:0/2211575317 conn(0x7f2e580d4f60 msgr2=0x7f2e580d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:41.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.407+0000 7f2e5fbbf640 1 -- 172.21.15.47:0/2211575317 shutdown_connections 2024-04-26T23:19:41.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.407+0000 7f2e5fbbf640 1 -- 172.21.15.47:0/2211575317 wait complete. 2024-04-26T23:19:41.419 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:18:16.937492+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:19:41.419 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:19:41.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.522+0000 7f04793e4640 1 Processor -- start 2024-04-26T23:19:41.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.522+0000 7f04793e4640 1 -- start start 2024-04-26T23:19:41.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.523+0000 7f04793e4640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f047415f170 0x7f04740d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:41.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.523+0000 7f04793e4640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f047415f550 0x7f04740d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:41.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.523+0000 7f04793e4640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04740d1040 con 0x7f04740d34b0 2024-04-26T23:19:41.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.523+0000 7f04793e4640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f04740c9430 con 0x7f047415f550 2024-04-26T23:19:41.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.523+0000 7f04793e4640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f04740c9680 con 0x7f047415f170 2024-04-26T23:19:41.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.523+0000 7f04727fc640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f047415f550 0x7f04740d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:41.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.523+0000 7f04737fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f04740d34b0 0x7f04740cfc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36070/0 (socket says 172.21.15.47:36070) 2024-04-26T23:19:41.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.523+0000 7f04737fe640 1 -- 172.21.15.47:0/3707157857 learned_addr learned my addr 172.21.15.47:0/3707157857 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:41.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.523+0000 7f0472ffd640 1 --2- 172.21.15.47:0/3707157857 >> v2:172.21.15.203:3300/0 conn(0x7f047415f170 0x7f04740d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:41.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.523+0000 7f04727fc640 1 -- 172.21.15.47:0/3707157857 >> v2:172.21.15.203:3300/0 conn(0x7f047415f170 msgr2=0x7f04740d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:41.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.523+0000 7f04727fc640 1 --2- 172.21.15.47:0/3707157857 >> v2:172.21.15.203:3300/0 conn(0x7f047415f170 0x7f04740d5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:41.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.523+0000 7f04727fc640 1 -- 172.21.15.47:0/3707157857 >> v1:172.21.15.47:6789/0 conn(0x7f04740d34b0 legacy=0x7f04740cfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:41.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.523+0000 7f04727fc640 1 -- 172.21.15.47:0/3707157857 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04740c9960 con 0x7f047415f550 2024-04-26T23:19:41.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.523+0000 7f0471ffb640 1 -- 172.21.15.47:0/3707157857 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 751163951 0 0) 0x7f0464001490 con 0x7f04740d34b0 2024-04-26T23:19:41.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.523+0000 7f0472ffd640 1 --2- 172.21.15.47:0/3707157857 >> v2:172.21.15.203:3300/0 conn(0x7f047415f170 0x7f04740d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:19:41.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.524+0000 7f04727fc640 1 --2- 172.21.15.47:0/3707157857 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f047415f550 0x7f04740d2cb0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f04680097a0 tx=0x7f046802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80e5dffae413445 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:41.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.524+0000 7f0471ffb640 1 -- 172.21.15.47:0/3707157857 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0468037070 con 0x7f047415f550 2024-04-26T23:19:41.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.524+0000 7f0471ffb640 1 -- 172.21.15.47:0/3707157857 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0468032440 con 0x7f047415f550 2024-04-26T23:19:41.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.524+0000 7f0471ffb640 1 -- 172.21.15.47:0/3707157857 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0468032810 con 0x7f047415f550 2024-04-26T23:19:41.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.525+0000 7f04793e4640 1 -- 172.21.15.47:0/3707157857 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f047415f550 msgr2=0x7f04740d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:41.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.525+0000 7f04793e4640 1 --2- 172.21.15.47:0/3707157857 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f047415f550 0x7f04740d2cb0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f04680097a0 tx=0x7f046802a1e0 comp rx=0 tx=0).stop 2024-04-26T23:19:41.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.525+0000 7f04793e4640 1 -- 172.21.15.47:0/3707157857 shutdown_connections 2024-04-26T23:19:41.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.525+0000 7f04793e4640 1 --2- 172.21.15.47:0/3707157857 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f047415f550 0x7f04740d2cb0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:41.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.525+0000 7f04793e4640 1 --2- 172.21.15.47:0/3707157857 >> v2:172.21.15.203:3300/0 conn(0x7f047415f170 0x7f04740d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:41.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.525+0000 7f04793e4640 1 -- 172.21.15.47:0/3707157857 >> 172.21.15.47:0/3707157857 conn(0x7f04740d4f60 msgr2=0x7f04740d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:41.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.525+0000 7f04793e4640 1 -- 172.21.15.47:0/3707157857 shutdown_connections 2024-04-26T23:19:41.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.525+0000 7f04793e4640 1 -- 172.21.15.47:0/3707157857 wait complete. 2024-04-26T23:19:41.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.526+0000 7f04793e4640 1 Processor -- start 2024-04-26T23:19:41.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.526+0000 7f04793e4640 1 -- start start 2024-04-26T23:19:41.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.527+0000 7f04793e4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04740d34b0 0x7f0474158a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:41.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.527+0000 7f04793e4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f047415f170 0x7f04740a2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:41.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.527+0000 7f04793e4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f04740a2760 0x7f04740a02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:41.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.527+0000 7f04793e4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f04740cb5d0 con 0x7f047415f170 2024-04-26T23:19:41.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.527+0000 7f04793e4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f04740cb450 con 0x7f04740d34b0 2024-04-26T23:19:41.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.527+0000 7f04793e4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f04740cb750 con 0x7f04740a2760 2024-04-26T23:19:41.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.527+0000 7f04727fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f047415f170 0x7f04740a2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:41.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.527+0000 7f04727fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f047415f170 0x7f04740a2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58718/0 (socket says 172.21.15.47:58718) 2024-04-26T23:19:41.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.527+0000 7f04727fc640 1 -- 172.21.15.47:0/3718369290 learned_addr learned my addr 172.21.15.47:0/3718369290 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:41.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.527+0000 7f04727fc640 1 -- 172.21.15.47:0/3718369290 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f04740a2760 msgr2=0x7f04740a02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:41.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.527+0000 7f04727fc640 1 --2- 172.21.15.47:0/3718369290 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f04740a2760 0x7f04740a02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:41.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.527+0000 7f04727fc640 1 -- 172.21.15.47:0/3718369290 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04740d34b0 msgr2=0x7f0474158a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:41.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.527+0000 7f0472ffd640 1 --2- 172.21.15.47:0/3718369290 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04740d34b0 0x7f0474158a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:41.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.527+0000 7f04727fc640 1 --2- 172.21.15.47:0/3718369290 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04740d34b0 0x7f0474158a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:41.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.527+0000 7f04727fc640 1 -- 172.21.15.47:0/3718369290 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04740a1060 con 0x7f047415f170 2024-04-26T23:19:41.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.528+0000 7f04727fc640 1 --2- 172.21.15.47:0/3718369290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f047415f170 0x7f04740a2220 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f04680098c0 tx=0x7f04680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:41.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.528+0000 7f045b7fe640 1 -- 172.21.15.47:0/3718369290 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0468037040 con 0x7f047415f170 2024-04-26T23:19:41.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.528+0000 7f045b7fe640 1 -- 172.21.15.47:0/3718369290 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0468030770 con 0x7f047415f170 2024-04-26T23:19:41.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.528+0000 7f045b7fe640 1 -- 172.21.15.47:0/3718369290 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0468030b40 con 0x7f047415f170 2024-04-26T23:19:41.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.528+0000 7f04793e4640 1 -- 172.21.15.47:0/3718369290 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04740a1340 con 0x7f047415f170 2024-04-26T23:19:41.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.528+0000 7f04793e4640 1 -- 172.21.15.47:0/3718369290 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04740a1900 con 0x7f047415f170 2024-04-26T23:19:41.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.529+0000 7f04793e4640 1 -- 172.21.15.47:0/3718369290 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04740694f0 con 0x7f047415f170 2024-04-26T23:19:41.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.530+0000 7f045b7fe640 1 -- 172.21.15.47:0/3718369290 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0468044020 con 0x7f047415f170 2024-04-26T23:19:41.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.530+0000 7f045b7fe640 1 --2- 172.21.15.47:0/3718369290 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f046006c8f0 0x7f046006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:41.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.530+0000 7f045b7fe640 1 -- 172.21.15.47:0/3718369290 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f04680ac9c0 con 0x7f047415f170 2024-04-26T23:19:41.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.531+0000 7f0472ffd640 1 --2- 172.21.15.47:0/3718369290 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f046006c8f0 0x7f046006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:41.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.531+0000 7f0472ffd640 1 --2- 172.21.15.47:0/3718369290 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f046006c8f0 0x7f046006edb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f046c008850 tx=0x7f046c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:41.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.534+0000 7f045b7fe640 1 -- 172.21.15.47:0/3718369290 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0468032e20 con 0x7f047415f170 2024-04-26T23:19:41.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.719+0000 7f04793e4640 1 -- 172.21.15.47:0/3718369290 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f04740a0060 con 0x7f047415f170 2024-04-26T23:19:41.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.719+0000 7f045b7fe640 1 -- 172.21.15.47:0/3718369290 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) ==== 93+0+10 (secure 0 0 0) 0x7f046807b090 con 0x7f047415f170 2024-04-26T23:19:41.720 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-26T23:19:41.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.721+0000 7f04793e4640 1 -- 172.21.15.47:0/3718369290 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f046006c8f0 msgr2=0x7f046006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:41.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.721+0000 7f04793e4640 1 --2- 172.21.15.47:0/3718369290 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f046006c8f0 0x7f046006edb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f046c008850 tx=0x7f046c008040 comp rx=0 tx=0).stop 2024-04-26T23:19:41.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.721+0000 7f04793e4640 1 -- 172.21.15.47:0/3718369290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f047415f170 msgr2=0x7f04740a2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:41.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.721+0000 7f04793e4640 1 --2- 172.21.15.47:0/3718369290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f047415f170 0x7f04740a2220 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f04680098c0 tx=0x7f04680046c0 comp rx=0 tx=0).stop 2024-04-26T23:19:41.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.721+0000 7f04793e4640 1 -- 172.21.15.47:0/3718369290 shutdown_connections 2024-04-26T23:19:41.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.721+0000 7f04793e4640 1 --2- 172.21.15.47:0/3718369290 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f046006c8f0 0x7f046006edb0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:41.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.721+0000 7f04793e4640 1 --2- 172.21.15.47:0/3718369290 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f04740a2760 0x7f04740a02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:41.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.721+0000 7f04793e4640 1 --2- 172.21.15.47:0/3718369290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f047415f170 0x7f04740a2220 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:41.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.721+0000 7f04793e4640 1 --2- 172.21.15.47:0/3718369290 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04740d34b0 0x7f0474158a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:41.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.721+0000 7f04793e4640 1 -- 172.21.15.47:0/3718369290 >> 172.21.15.47:0/3718369290 conn(0x7f04740d4f60 msgr2=0x7f04740c85f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:41.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.721+0000 7f04793e4640 1 -- 172.21.15.47:0/3718369290 shutdown_connections 2024-04-26T23:19:41.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.721+0000 7f04793e4640 1 -- 172.21.15.47:0/3718369290 wait complete. 2024-04-26T23:19:41.733 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T23:19:41.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.833+0000 7fc6f72f2640 1 Processor -- start 2024-04-26T23:19:41.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.834+0000 7fc6f72f2640 1 -- start start 2024-04-26T23:19:41.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.834+0000 7fc6f72f2640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc6f015f3a0 0x7fc6f00d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:41.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.834+0000 7fc6f72f2640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc6f00d2330 0x7fc6f00b2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:41.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.834+0000 7fc6f72f2640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6f00d0e20 con 0x7fc6f00d1770 2024-04-26T23:19:41.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.834+0000 7fc6f72f2640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fc6f00b1d70 con 0x7fc6f00d2330 2024-04-26T23:19:41.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.834+0000 7fc6f72f2640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fc6f00b2050 con 0x7fc6f015f3a0 2024-04-26T23:19:41.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.834+0000 7fc6f588f640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc6f00d2330 0x7fc6f00b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:41.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.834+0000 7fc6f588f640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc6f00d2330 0x7fc6f00b2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58720/0 (socket says 172.21.15.47:58720) 2024-04-26T23:19:41.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.834+0000 7fc6f588f640 1 -- 172.21.15.47:0/2619463026 learned_addr learned my addr 172.21.15.47:0/2619463026 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:41.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.835+0000 7fc6f588f640 1 -- 172.21.15.47:0/2619463026 >> v2:172.21.15.203:3300/0 conn(0x7fc6f015f3a0 msgr2=0x7fc6f00d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:41.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.835+0000 7fc6f588f640 1 --2- 172.21.15.47:0/2619463026 >> v2:172.21.15.203:3300/0 conn(0x7fc6f015f3a0 0x7fc6f00d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:41.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.835+0000 7fc6f588f640 1 -- 172.21.15.47:0/2619463026 >> v1:172.21.15.203:6789/0 conn(0x7fc6f00d1770 legacy=0x7fc6f00d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:41.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.835+0000 7fc6f588f640 1 -- 172.21.15.47:0/2619463026 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6f00b2330 con 0x7fc6f00d2330 2024-04-26T23:19:41.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.835+0000 7fc6f588f640 1 --2- 172.21.15.47:0/2619463026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6f00d2330 0x7fc6f00b2850 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc6f00c89f0 tx=0x7fc6ec02a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce441687f7397f61 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:41.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.836+0000 7fc6e7fff640 1 -- 172.21.15.47:0/2619463026 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec036070 con 0x7fc6f00d2330 2024-04-26T23:19:41.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.836+0000 7fc6e7fff640 1 -- 172.21.15.47:0/2619463026 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6ec031600 con 0x7fc6f00d2330 2024-04-26T23:19:41.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.836+0000 7fc6e7fff640 1 -- 172.21.15.47:0/2619463026 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec0319d0 con 0x7fc6f00d2330 2024-04-26T23:19:41.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.836+0000 7fc6f72f2640 1 -- 172.21.15.47:0/2619463026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6f00d2330 msgr2=0x7fc6f00b2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:41.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.836+0000 7fc6f72f2640 1 --2- 172.21.15.47:0/2619463026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6f00d2330 0x7fc6f00b2850 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc6f00c89f0 tx=0x7fc6ec02a6d0 comp rx=0 tx=0).stop 2024-04-26T23:19:41.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.836+0000 7fc6f72f2640 1 -- 172.21.15.47:0/2619463026 shutdown_connections 2024-04-26T23:19:41.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.836+0000 7fc6f72f2640 1 --2- 172.21.15.47:0/2619463026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6f00d2330 0x7fc6f00b2850 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:41.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.836+0000 7fc6f72f2640 1 --2- 172.21.15.47:0/2619463026 >> v2:172.21.15.203:3300/0 conn(0x7fc6f015f3a0 0x7fc6f00d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:41.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.836+0000 7fc6f72f2640 1 -- 172.21.15.47:0/2619463026 >> 172.21.15.47:0/2619463026 conn(0x7fc6f00d4f90 msgr2=0x7fc6f00d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:41.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.836+0000 7fc6f72f2640 1 -- 172.21.15.47:0/2619463026 shutdown_connections 2024-04-26T23:19:41.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.836+0000 7fc6f72f2640 1 -- 172.21.15.47:0/2619463026 wait complete. 2024-04-26T23:19:41.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.837+0000 7fc6f72f2640 1 Processor -- start 2024-04-26T23:19:41.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.837+0000 7fc6f72f2640 1 -- start start 2024-04-26T23:19:41.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.838+0000 7fc6f72f2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6f00d1770 0x7fc6f0150570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:41.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.838+0000 7fc6f72f2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6f00d2330 0x7fc6f00b4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:41.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.838+0000 7fc6f72f2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6f015f3a0 0x7fc6f00b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:41.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.838+0000 7fc6f72f2640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f00b2f10 con 0x7fc6f00d1770 2024-04-26T23:19:41.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.838+0000 7fc6f72f2640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f00b2d90 con 0x7fc6f015f3a0 2024-04-26T23:19:41.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.838+0000 7fc6f72f2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc6f00b3090 con 0x7fc6f00d2330 2024-04-26T23:19:41.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.838+0000 7fc6f508e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6f00d1770 0x7fc6f0150570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:41.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.838+0000 7fc6f508e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6f00d1770 0x7fc6f0150570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58726/0 (socket says 172.21.15.47:58726) 2024-04-26T23:19:41.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.838+0000 7fc6f508e640 1 -- 172.21.15.47:0/895273442 learned_addr learned my addr 172.21.15.47:0/895273442 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:41.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.839+0000 7fc6f588f640 1 --2- 172.21.15.47:0/895273442 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6f015f3a0 0x7fc6f00b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:41.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.839+0000 7fc6f488d640 1 --2- 172.21.15.47:0/895273442 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6f00d2330 0x7fc6f00b4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:41.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.839+0000 7fc6f508e640 1 -- 172.21.15.47:0/895273442 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6f00d2330 msgr2=0x7fc6f00b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:41.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.839+0000 7fc6f508e640 1 --2- 172.21.15.47:0/895273442 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6f00d2330 0x7fc6f00b4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:41.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.839+0000 7fc6f508e640 1 -- 172.21.15.47:0/895273442 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6f015f3a0 msgr2=0x7fc6f00b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:41.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.839+0000 7fc6f508e640 1 --2- 172.21.15.47:0/895273442 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6f015f3a0 0x7fc6f00b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:41.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.839+0000 7fc6f508e640 1 -- 172.21.15.47:0/895273442 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6f0161600 con 0x7fc6f00d1770 2024-04-26T23:19:41.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.839+0000 7fc6f588f640 1 --2- 172.21.15.47:0/895273442 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6f015f3a0 0x7fc6f00b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:19:41.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.839+0000 7fc6f508e640 1 --2- 172.21.15.47:0/895273442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6f00d1770 0x7fc6f0150570 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fc6e800b440 tx=0x7fc6e800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:41.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.839+0000 7fc6e5ffb640 1 -- 172.21.15.47:0/895273442 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e8015020 con 0x7fc6f00d1770 2024-04-26T23:19:41.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.839+0000 7fc6e5ffb640 1 -- 172.21.15.47:0/895273442 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6e8011890 con 0x7fc6f00d1770 2024-04-26T23:19:41.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.839+0000 7fc6f72f2640 1 -- 172.21.15.47:0/895273442 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6f01618e0 con 0x7fc6f00d1770 2024-04-26T23:19:41.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.839+0000 7fc6e5ffb640 1 -- 172.21.15.47:0/895273442 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e8011c80 con 0x7fc6f00d1770 2024-04-26T23:19:41.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.839+0000 7fc6f72f2640 1 -- 172.21.15.47:0/895273442 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6f0161ea0 con 0x7fc6f00d1770 2024-04-26T23:19:41.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.840+0000 7fc6f72f2640 1 -- 172.21.15.47:0/895273442 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6f00694f0 con 0x7fc6f00d1770 2024-04-26T23:19:41.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.840+0000 7fc6e5ffb640 1 -- 172.21.15.47:0/895273442 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc6e8018020 con 0x7fc6f00d1770 2024-04-26T23:19:41.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.841+0000 7fc6e5ffb640 1 --2- 172.21.15.47:0/895273442 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fc6d406c8f0 0x7fc6d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:41.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.841+0000 7fc6e5ffb640 1 -- 172.21.15.47:0/895273442 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7fc6e808bb10 con 0x7fc6f00d1770 2024-04-26T23:19:41.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.841+0000 7fc6f488d640 1 --2- 172.21.15.47:0/895273442 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fc6d406c8f0 0x7fc6d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:41.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.842+0000 7fc6f488d640 1 --2- 172.21.15.47:0/895273442 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fc6d406c8f0 0x7fc6d406edb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0069d50 tx=0x7fc6e0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:41.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:41.846+0000 7fc6e5ffb640 1 -- 172.21.15.47:0/895273442 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc6e801f020 con 0x7fc6f00d1770 2024-04-26T23:19:42.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.032+0000 7fc6f72f2640 1 -- 172.21.15.47:0/895273442 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fc6f00b6860 con 0x7fc6f00d1770 2024-04-26T23:19:42.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.033+0000 7fc6e5ffb640 1 -- 172.21.15.47:0/895273442 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v34) ==== 84+0+65 (secure 0 0 0) 0x7fc6e800d040 con 0x7fc6f00d1770 2024-04-26T23:19:42.033 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:19:42.034 INFO:teuthology.orchestra.run.smithi047.stdout:[ 2024-04-26T23:19:42.034 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-26T23:19:42.034 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 1, 2024-04-26T23:19:42.034 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": ".mgr" 2024-04-26T23:19:42.034 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-26T23:19:42.034 INFO:teuthology.orchestra.run.smithi047.stdout:] 2024-04-26T23:19:42.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.034+0000 7fc6f72f2640 1 -- 172.21.15.47:0/895273442 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fc6d406c8f0 msgr2=0x7fc6d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:42.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.034+0000 7fc6f72f2640 1 --2- 172.21.15.47:0/895273442 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fc6d406c8f0 0x7fc6d406edb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0069d50 tx=0x7fc6e0009040 comp rx=0 tx=0).stop 2024-04-26T23:19:42.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.034+0000 7fc6f72f2640 1 -- 172.21.15.47:0/895273442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6f00d1770 msgr2=0x7fc6f0150570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:42.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.034+0000 7fc6f72f2640 1 --2- 172.21.15.47:0/895273442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6f00d1770 0x7fc6f0150570 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fc6e800b440 tx=0x7fc6e800b900 comp rx=0 tx=0).stop 2024-04-26T23:19:42.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.034+0000 7fc6f72f2640 1 -- 172.21.15.47:0/895273442 shutdown_connections 2024-04-26T23:19:42.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.034+0000 7fc6f72f2640 1 --2- 172.21.15.47:0/895273442 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fc6d406c8f0 0x7fc6d406edb0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:42.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.034+0000 7fc6f72f2640 1 --2- 172.21.15.47:0/895273442 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6f015f3a0 0x7fc6f00b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:42.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.034+0000 7fc6f72f2640 1 --2- 172.21.15.47:0/895273442 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6f00d2330 0x7fc6f00b4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:42.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.034+0000 7fc6f72f2640 1 --2- 172.21.15.47:0/895273442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6f00d1770 0x7fc6f0150570 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:42.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.034+0000 7fc6f72f2640 1 -- 172.21.15.47:0/895273442 >> 172.21.15.47:0/895273442 conn(0x7fc6f00d4f90 msgr2=0x7fc6f00b3490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:42.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.035+0000 7fc6f72f2640 1 -- 172.21.15.47:0/895273442 shutdown_connections 2024-04-26T23:19:42.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.035+0000 7fc6f72f2640 1 -- 172.21.15.47:0/895273442 wait complete. 2024-04-26T23:19:42.047 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T23:19:42.047 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T23:19:42.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.151+0000 7f91d78bd640 1 Processor -- start 2024-04-26T23:19:42.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.151+0000 7f91d78bd640 1 -- start start 2024-04-26T23:19:42.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.152+0000 7f91d78bd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f91d015f160 0x7f91d006d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:42.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.152+0000 7f91d78bd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f91d006ee20 0x7f91d00d2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:42.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.152+0000 7f91d78bd640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91d006dcc0 con 0x7f91d00d32b0 2024-04-26T23:19:42.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.152+0000 7f91d78bd640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f91d00d3700 con 0x7f91d006ee20 2024-04-26T23:19:42.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.152+0000 7f91d78bd640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f91d006dfa0 con 0x7f91d015f160 2024-04-26T23:19:42.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.152+0000 7f91d4e58640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f91d006ee20 0x7f91d00d2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:42.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.152+0000 7f91d4e58640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f91d006ee20 0x7f91d00d2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58728/0 (socket says 172.21.15.47:58728) 2024-04-26T23:19:42.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.152+0000 7f91d4e58640 1 -- 172.21.15.47:0/3038920064 learned_addr learned my addr 172.21.15.47:0/3038920064 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:42.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.153+0000 7f91d5659640 1 --2- 172.21.15.47:0/3038920064 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f91d015f160 0x7f91d006d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:42.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.153+0000 7f91d4e58640 1 -- 172.21.15.47:0/3038920064 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f91d015f160 msgr2=0x7f91d006d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:42.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.153+0000 7f91d4e58640 1 --2- 172.21.15.47:0/3038920064 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f91d015f160 0x7f91d006d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:42.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.153+0000 7f91d4e58640 1 -- 172.21.15.47:0/3038920064 >> v1:172.21.15.203:6789/0 conn(0x7f91d00d32b0 legacy=0x7f91d00cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:42.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.153+0000 7f91d4e58640 1 -- 172.21.15.47:0/3038920064 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91d006e280 con 0x7f91d006ee20 2024-04-26T23:19:42.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.153+0000 7f91d5659640 1 --2- 172.21.15.47:0/3038920064 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f91d015f160 0x7f91d006d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:19:42.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.153+0000 7f91d4e58640 1 --2- 172.21.15.47:0/3038920064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91d006ee20 0x7f91d00d2d70 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f91c0009870 tx=0x7f91c002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69c4672c93934d19 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:42.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.154+0000 7f91c7fff640 1 -- 172.21.15.47:0/3038920064 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c0037070 con 0x7f91d006ee20 2024-04-26T23:19:42.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.154+0000 7f91c7fff640 1 -- 172.21.15.47:0/3038920064 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91c002ae50 con 0x7f91d006ee20 2024-04-26T23:19:42.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.154+0000 7f91c7fff640 1 -- 172.21.15.47:0/3038920064 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c0032860 con 0x7f91d006ee20 2024-04-26T23:19:42.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.154+0000 7f91d78bd640 1 -- 172.21.15.47:0/3038920064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91d006ee20 msgr2=0x7f91d00d2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:42.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.154+0000 7f91d78bd640 1 --2- 172.21.15.47:0/3038920064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91d006ee20 0x7f91d00d2d70 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f91c0009870 tx=0x7f91c002a300 comp rx=0 tx=0).stop 2024-04-26T23:19:42.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.154+0000 7f91d78bd640 1 -- 172.21.15.47:0/3038920064 shutdown_connections 2024-04-26T23:19:42.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.154+0000 7f91d78bd640 1 --2- 172.21.15.47:0/3038920064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91d006ee20 0x7f91d00d2d70 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:42.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.154+0000 7f91d78bd640 1 --2- 172.21.15.47:0/3038920064 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f91d015f160 0x7f91d006d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:42.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.154+0000 7f91d78bd640 1 -- 172.21.15.47:0/3038920064 >> 172.21.15.47:0/3038920064 conn(0x7f91d009ba50 msgr2=0x7f91d009be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:42.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.154+0000 7f91d78bd640 1 -- 172.21.15.47:0/3038920064 shutdown_connections 2024-04-26T23:19:42.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.154+0000 7f91d78bd640 1 -- 172.21.15.47:0/3038920064 wait complete. 2024-04-26T23:19:42.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.155+0000 7f91d78bd640 1 Processor -- start 2024-04-26T23:19:42.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.156+0000 7f91d78bd640 1 -- start start 2024-04-26T23:19:42.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.156+0000 7f91d78bd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91d006ee20 0x7f91d00b7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:42.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.156+0000 7f91d78bd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f91d00d32b0 0x7f91d01f63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:42.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.156+0000 7f91d78bd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91d015f160 0x7f91d009f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:42.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.156+0000 7f91d78bd640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f91d006d890 con 0x7f91d015f160 2024-04-26T23:19:42.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.156+0000 7f91d78bd640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f91d006e840 con 0x7f91d006ee20 2024-04-26T23:19:42.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.156+0000 7f91d78bd640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f91d006da10 con 0x7f91d00d32b0 2024-04-26T23:19:42.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.156+0000 7f91d5e5a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91d015f160 0x7f91d009f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:42.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.156+0000 7f91d5e5a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91d015f160 0x7f91d009f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58730/0 (socket says 172.21.15.47:58730) 2024-04-26T23:19:42.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.156+0000 7f91d5e5a640 1 -- 172.21.15.47:0/2279016397 learned_addr learned my addr 172.21.15.47:0/2279016397 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:42.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.157+0000 7f91d4e58640 1 --2- 172.21.15.47:0/2279016397 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f91d00d32b0 0x7f91d01f63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:42.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.157+0000 7f91d5e5a640 1 -- 172.21.15.47:0/2279016397 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f91d00d32b0 msgr2=0x7f91d01f63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:42.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.157+0000 7f91d5659640 1 --2- 172.21.15.47:0/2279016397 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91d006ee20 0x7f91d00b7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:42.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.157+0000 7f91d5e5a640 1 --2- 172.21.15.47:0/2279016397 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f91d00d32b0 0x7f91d01f63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:42.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.157+0000 7f91d5e5a640 1 -- 172.21.15.47:0/2279016397 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91d006ee20 msgr2=0x7f91d00b7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:42.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.157+0000 7f91d5e5a640 1 --2- 172.21.15.47:0/2279016397 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91d006ee20 0x7f91d00b7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:42.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.157+0000 7f91d5e5a640 1 -- 172.21.15.47:0/2279016397 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91d00b7df0 con 0x7f91d015f160 2024-04-26T23:19:42.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.157+0000 7f91d5e5a640 1 --2- 172.21.15.47:0/2279016397 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91d015f160 0x7f91d009f9e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f91cc00c7d0 tx=0x7f91cc00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:42.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.157+0000 7f91c5ffb640 1 -- 172.21.15.47:0/2279016397 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91cc003b70 con 0x7f91d015f160 2024-04-26T23:19:42.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.157+0000 7f91c5ffb640 1 -- 172.21.15.47:0/2279016397 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91cc003d30 con 0x7f91d015f160 2024-04-26T23:19:42.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.157+0000 7f91d78bd640 1 -- 172.21.15.47:0/2279016397 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91d009ef00 con 0x7f91d015f160 2024-04-26T23:19:42.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.157+0000 7f91c5ffb640 1 -- 172.21.15.47:0/2279016397 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91cc00aa90 con 0x7f91d015f160 2024-04-26T23:19:42.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.157+0000 7f91d78bd640 1 -- 172.21.15.47:0/2279016397 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91d009ebb0 con 0x7f91d015f160 2024-04-26T23:19:42.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.158+0000 7f91d78bd640 1 -- 172.21.15.47:0/2279016397 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91d0069600 con 0x7f91d015f160 2024-04-26T23:19:42.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.158+0000 7f91c5ffb640 1 -- 172.21.15.47:0/2279016397 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f91cc00ac30 con 0x7f91d015f160 2024-04-26T23:19:42.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.159+0000 7f91c5ffb640 1 --2- 172.21.15.47:0/2279016397 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f91b406c8f0 0x7f91b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:42.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.159+0000 7f91c5ffb640 1 -- 172.21.15.47:0/2279016397 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f91cc08b8a0 con 0x7f91d015f160 2024-04-26T23:19:42.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.159+0000 7f91d5659640 1 --2- 172.21.15.47:0/2279016397 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f91b406c8f0 0x7f91b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:42.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.160+0000 7f91d5659640 1 --2- 172.21.15.47:0/2279016397 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f91b406c8f0 0x7f91b406edb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f91c8006fd0 tx=0x7f91c8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:42.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.163+0000 7f91c5ffb640 1 -- 172.21.15.47:0/2279016397 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f91cc059f20 con 0x7f91d015f160 2024-04-26T23:19:42.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.347+0000 7f91d78bd640 1 -- 172.21.15.47:0/2279016397 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f91d009f670 con 0x7f91d015f160 2024-04-26T23:19:42.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.706+0000 7f91c5ffb640 1 -- 172.21.15.47:0/2279016397 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v35) ==== 138+0+0 (secure 0 0 0) 0x7f91cc059820 con 0x7f91d015f160 2024-04-26T23:19:42.707 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' created 2024-04-26T23:19:42.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.709+0000 7f91d78bd640 1 -- 172.21.15.47:0/2279016397 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f91b406c8f0 msgr2=0x7f91b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:42.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.709+0000 7f91d78bd640 1 --2- 172.21.15.47:0/2279016397 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f91b406c8f0 0x7f91b406edb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f91c8006fd0 tx=0x7f91c8008040 comp rx=0 tx=0).stop 2024-04-26T23:19:42.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.709+0000 7f91d78bd640 1 -- 172.21.15.47:0/2279016397 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91d015f160 msgr2=0x7f91d009f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:42.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.709+0000 7f91d78bd640 1 --2- 172.21.15.47:0/2279016397 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91d015f160 0x7f91d009f9e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f91cc00c7d0 tx=0x7f91cc00cc90 comp rx=0 tx=0).stop 2024-04-26T23:19:42.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.709+0000 7f91d78bd640 1 -- 172.21.15.47:0/2279016397 shutdown_connections 2024-04-26T23:19:42.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.709+0000 7f91d78bd640 1 --2- 172.21.15.47:0/2279016397 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f91b406c8f0 0x7f91b406edb0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:42.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.709+0000 7f91d78bd640 1 --2- 172.21.15.47:0/2279016397 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91d015f160 0x7f91d009f9e0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:42.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.709+0000 7f91d78bd640 1 --2- 172.21.15.47:0/2279016397 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f91d00d32b0 0x7f91d01f63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:42.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.709+0000 7f91d78bd640 1 --2- 172.21.15.47:0/2279016397 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91d006ee20 0x7f91d00b7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:42.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.709+0000 7f91d78bd640 1 -- 172.21.15.47:0/2279016397 >> 172.21.15.47:0/2279016397 conn(0x7f91d009ba50 msgr2=0x7f91d00b37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:42.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.709+0000 7f91d78bd640 1 -- 172.21.15.47:0/2279016397 shutdown_connections 2024-04-26T23:19:42.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.709+0000 7f91d78bd640 1 -- 172.21.15.47:0/2279016397 wait complete. 2024-04-26T23:19:42.724 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T23:19:42.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.834+0000 7f2b02a25640 1 Processor -- start 2024-04-26T23:19:42.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.834+0000 7f2b02a25640 1 -- start start 2024-04-26T23:19:42.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.835+0000 7f2b02a25640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2afc070350 0x7f2afc070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:42.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.835+0000 7f2b02a25640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2afc071090 0x7f2afc15e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:42.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.835+0000 7f2b02a25640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2afc0d2390 con 0x7f2afc0cfde0 2024-04-26T23:19:42.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.835+0000 7f2b02a25640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f2afc071500 con 0x7f2afc070350 2024-04-26T23:19:42.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.835+0000 7f2b02a25640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f2afc0afab0 con 0x7f2afc071090 2024-04-26T23:19:42.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.835+0000 7f2afb7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2afc070350 0x7f2afc070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:42.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.835+0000 7f2afb7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2afc070350 0x7f2afc070b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58734/0 (socket says 172.21.15.47:58734) 2024-04-26T23:19:42.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.835+0000 7f2afb7fe640 1 -- 172.21.15.47:0/2229834745 learned_addr learned my addr 172.21.15.47:0/2229834745 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:42.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.835+0000 7f2b00fc2640 1 --2- 172.21.15.47:0/2229834745 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2afc071090 0x7f2afc15e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:42.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.835+0000 7f2afb7fe640 1 -- 172.21.15.47:0/2229834745 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2afc071090 msgr2=0x7f2afc15e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:42.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.835+0000 7f2afb7fe640 1 --2- 172.21.15.47:0/2229834745 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2afc071090 0x7f2afc15e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:42.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.835+0000 7f2afb7fe640 1 -- 172.21.15.47:0/2229834745 >> v1:172.21.15.203:6789/0 conn(0x7f2afc0cfde0 legacy=0x7f2afc0d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:42.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.835+0000 7f2afb7fe640 1 -- 172.21.15.47:0/2229834745 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2afc06d3f0 con 0x7f2afc070350 2024-04-26T23:19:42.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.836+0000 7f2afb7fe640 1 --2- 172.21.15.47:0/2229834745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2afc070350 0x7f2afc070b50 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2af0009870 tx=0x7f2af002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad42bca9bbd80468 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:42.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.836+0000 7f2afaffd640 1 -- 172.21.15.47:0/2229834745 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af0037070 con 0x7f2afc070350 2024-04-26T23:19:42.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.836+0000 7f2afaffd640 1 -- 172.21.15.47:0/2229834745 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2af0032440 con 0x7f2afc070350 2024-04-26T23:19:42.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.836+0000 7f2afaffd640 1 -- 172.21.15.47:0/2229834745 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af0032810 con 0x7f2afc070350 2024-04-26T23:19:42.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.836+0000 7f2b02a25640 1 -- 172.21.15.47:0/2229834745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2afc070350 msgr2=0x7f2afc070b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:42.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.836+0000 7f2b02a25640 1 --2- 172.21.15.47:0/2229834745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2afc070350 0x7f2afc070b50 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2af0009870 tx=0x7f2af002a1c0 comp rx=0 tx=0).stop 2024-04-26T23:19:42.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.837+0000 7f2b02a25640 1 -- 172.21.15.47:0/2229834745 shutdown_connections 2024-04-26T23:19:42.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.837+0000 7f2b02a25640 1 --2- 172.21.15.47:0/2229834745 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2afc071090 0x7f2afc15e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:42.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.837+0000 7f2b02a25640 1 --2- 172.21.15.47:0/2229834745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2afc070350 0x7f2afc070b50 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:42.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.837+0000 7f2b02a25640 1 -- 172.21.15.47:0/2229834745 >> 172.21.15.47:0/2229834745 conn(0x7f2afc0d2d80 msgr2=0x7f2afc0d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:42.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.837+0000 7f2b02a25640 1 -- 172.21.15.47:0/2229834745 shutdown_connections 2024-04-26T23:19:42.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.837+0000 7f2b02a25640 1 -- 172.21.15.47:0/2229834745 wait complete. 2024-04-26T23:19:42.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.838+0000 7f2b02a25640 1 Processor -- start 2024-04-26T23:19:42.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.838+0000 7f2b02a25640 1 -- start start 2024-04-26T23:19:42.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.839+0000 7f2b02a25640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2afc070350 0x7f2afc14c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:42.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.839+0000 7f2b02a25640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2afc071090 0x7f2afc14cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:42.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.839+0000 7f2b02a25640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2afc0cfde0 0x7f2afc1601b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:42.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.839+0000 7f2b02a25640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2afc06d9b0 con 0x7f2afc0cfde0 2024-04-26T23:19:42.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.839+0000 7f2b02a25640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2afc06dd20 con 0x7f2afc071090 2024-04-26T23:19:42.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.839+0000 7f2b02a25640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f2afc15ef30 con 0x7f2afc070350 2024-04-26T23:19:42.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.839+0000 7f2b00fc2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2afc0cfde0 0x7f2afc1601b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:42.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.839+0000 7f2b00fc2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2afc0cfde0 0x7f2afc1601b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58738/0 (socket says 172.21.15.47:58738) 2024-04-26T23:19:42.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.839+0000 7f2b00fc2640 1 -- 172.21.15.47:0/915244887 learned_addr learned my addr 172.21.15.47:0/915244887 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:42.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.839+0000 7f2afbfff640 1 --2- 172.21.15.47:0/915244887 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2afc070350 0x7f2afc14c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:42.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.839+0000 7f2afb7fe640 1 --2- 172.21.15.47:0/915244887 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2afc071090 0x7f2afc14cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:42.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.839+0000 7f2b00fc2640 1 -- 172.21.15.47:0/915244887 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2afc070350 msgr2=0x7f2afc14c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:42.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.839+0000 7f2b00fc2640 1 --2- 172.21.15.47:0/915244887 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2afc070350 0x7f2afc14c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:42.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.839+0000 7f2b00fc2640 1 -- 172.21.15.47:0/915244887 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2afc071090 msgr2=0x7f2afc14cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:42.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.839+0000 7f2b00fc2640 1 --2- 172.21.15.47:0/915244887 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2afc071090 0x7f2afc14cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:42.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.839+0000 7f2b00fc2640 1 -- 172.21.15.47:0/915244887 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2afc1625a0 con 0x7f2afc0cfde0 2024-04-26T23:19:42.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.839+0000 7f2b00fc2640 1 --2- 172.21.15.47:0/915244887 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2afc0cfde0 0x7f2afc1601b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2af4007e40 tx=0x7f2af400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:42.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.839+0000 7f2af8ff9640 1 -- 172.21.15.47:0/915244887 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af4014620 con 0x7f2afc0cfde0 2024-04-26T23:19:42.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.839+0000 7f2af8ff9640 1 -- 172.21.15.47:0/915244887 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2af4019070 con 0x7f2afc0cfde0 2024-04-26T23:19:42.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.839+0000 7f2af8ff9640 1 -- 172.21.15.47:0/915244887 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af4014a20 con 0x7f2afc0cfde0 2024-04-26T23:19:42.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.839+0000 7f2b02a25640 1 -- 172.21.15.47:0/915244887 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2afc0c7f70 con 0x7f2afc0cfde0 2024-04-26T23:19:42.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.840+0000 7f2b02a25640 1 -- 172.21.15.47:0/915244887 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2afc0c8530 con 0x7f2afc0cfde0 2024-04-26T23:19:42.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.840+0000 7f2b02a25640 1 -- 172.21.15.47:0/915244887 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2afc0694f0 con 0x7f2afc0cfde0 2024-04-26T23:19:42.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.841+0000 7f2af8ff9640 1 -- 172.21.15.47:0/915244887 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2af4014bc0 con 0x7f2afc0cfde0 2024-04-26T23:19:42.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.842+0000 7f2af8ff9640 1 --2- 172.21.15.47:0/915244887 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f2ae806c8f0 0x7f2ae806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:42.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.842+0000 7f2af8ff9640 1 -- 172.21.15.47:0/915244887 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5619+0+0 (secure 0 0 0) 0x7f2af408d190 con 0x7f2afc0cfde0 2024-04-26T23:19:42.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.842+0000 7f2afbfff640 1 --2- 172.21.15.47:0/915244887 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f2ae806c8f0 0x7f2ae806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:42.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.842+0000 7f2afbfff640 1 --2- 172.21.15.47:0/915244887 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f2ae806c8f0 0x7f2ae806edb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2aec0016e0 tx=0x7f2aec009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:42.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:42.846+0000 7f2af8ff9640 1 -- 172.21.15.47:0/915244887 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2af405afc0 con 0x7f2afc0cfde0 2024-04-26T23:19:43.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.034+0000 7f2b02a25640 1 -- 172.21.15.47:0/915244887 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f2afc0b8c40 con 0x7f2afc0cfde0 2024-04-26T23:19:43.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.711+0000 7f2af8ff9640 1 -- 172.21.15.47:0/915244887 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v36) ==== 170+0+0 (secure 0 0 0) 0x7f2af405f2c0 con 0x7f2afc0cfde0 2024-04-26T23:19:43.713 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' created 2024-04-26T23:19:43.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.716+0000 7f2b02a25640 1 -- 172.21.15.47:0/915244887 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f2ae806c8f0 msgr2=0x7f2ae806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:43.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.716+0000 7f2b02a25640 1 --2- 172.21.15.47:0/915244887 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f2ae806c8f0 0x7f2ae806edb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2aec0016e0 tx=0x7f2aec009040 comp rx=0 tx=0).stop 2024-04-26T23:19:43.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.716+0000 7f2b02a25640 1 -- 172.21.15.47:0/915244887 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2afc0cfde0 msgr2=0x7f2afc1601b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:43.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.716+0000 7f2b02a25640 1 --2- 172.21.15.47:0/915244887 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2afc0cfde0 0x7f2afc1601b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2af4007e40 tx=0x7f2af400e4b0 comp rx=0 tx=0).stop 2024-04-26T23:19:43.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.716+0000 7f2b02a25640 1 -- 172.21.15.47:0/915244887 shutdown_connections 2024-04-26T23:19:43.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.716+0000 7f2b02a25640 1 --2- 172.21.15.47:0/915244887 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f2ae806c8f0 0x7f2ae806edb0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:43.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.716+0000 7f2b02a25640 1 --2- 172.21.15.47:0/915244887 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2afc0cfde0 0x7f2afc1601b0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:43.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.716+0000 7f2b02a25640 1 --2- 172.21.15.47:0/915244887 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2afc071090 0x7f2afc14cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:43.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.716+0000 7f2b02a25640 1 --2- 172.21.15.47:0/915244887 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2afc070350 0x7f2afc14c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:43.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.716+0000 7f2b02a25640 1 -- 172.21.15.47:0/915244887 >> 172.21.15.47:0/915244887 conn(0x7f2afc0d2d80 msgr2=0x7f2afc0afc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:43.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.717+0000 7f2b02a25640 1 -- 172.21.15.47:0/915244887 shutdown_connections 2024-04-26T23:19:43.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.717+0000 7f2b02a25640 1 -- 172.21.15.47:0/915244887 wait complete. 2024-04-26T23:19:43.746 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T23:19:43.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.855+0000 7f3ef9c45640 1 Processor -- start 2024-04-26T23:19:43.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.855+0000 7f3ef9c45640 1 -- start start 2024-04-26T23:19:43.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.855+0000 7f3ef9c45640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3ef40d1a00 0x7f3ef40d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:43.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.855+0000 7f3ef9c45640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3ef40d2320 0x7f3ef41625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:43.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.855+0000 7f3ef9c45640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ef40b40d0 0x7f3ef4162b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:43.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.855+0000 7f3ef9c45640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f3ef4163070 con 0x7f3ef40d2320 2024-04-26T23:19:43.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.855+0000 7f3ef9c45640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f3ef40d0a10 con 0x7f3ef40d1a00 2024-04-26T23:19:43.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.855+0000 7f3ef9c45640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3ef40d0cf0 con 0x7f3ef40b40d0 2024-04-26T23:19:43.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.856+0000 7f3eeaffd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3ef40d2320 0x7f3ef41625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:43.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.856+0000 7f3eeaffd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3ef40d2320 0x7f3ef41625f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58754/0 (socket says 172.21.15.47:58754) 2024-04-26T23:19:43.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.856+0000 7f3eeaffd640 1 -- 172.21.15.47:0/760465925 learned_addr learned my addr 172.21.15.47:0/760465925 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:43.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.856+0000 7f3ef3fff640 1 --2- 172.21.15.47:0/760465925 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ef40b40d0 0x7f3ef4162b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:43.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.856+0000 7f3ef37fe640 1 --2- 172.21.15.47:0/760465925 >> v2:172.21.15.203:3300/0 conn(0x7f3ef40d1a00 0x7f3ef40d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:43.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.856+0000 7f3eeaffd640 1 -- 172.21.15.47:0/760465925 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ef40b40d0 msgr2=0x7f3ef4162b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:43.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.856+0000 7f3eeaffd640 1 --2- 172.21.15.47:0/760465925 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ef40b40d0 0x7f3ef4162b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:43.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.856+0000 7f3eeaffd640 1 -- 172.21.15.47:0/760465925 >> v2:172.21.15.203:3300/0 conn(0x7f3ef40d1a00 msgr2=0x7f3ef40d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:43.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.856+0000 7f3eeaffd640 1 --2- 172.21.15.47:0/760465925 >> v2:172.21.15.203:3300/0 conn(0x7f3ef40d1a00 0x7f3ef40d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:43.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.856+0000 7f3eeaffd640 1 -- 172.21.15.47:0/760465925 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ef40d0fd0 con 0x7f3ef40d2320 2024-04-26T23:19:43.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.856+0000 7f3ef37fe640 1 --2- 172.21.15.47:0/760465925 >> v2:172.21.15.203:3300/0 conn(0x7f3ef40d1a00 0x7f3ef40d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:19:43.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.857+0000 7f3eeaffd640 1 --2- 172.21.15.47:0/760465925 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ef40d2320 0x7f3ef41625f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f3ef406c650 tx=0x7f3edc02a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd05c3f1d6f3708 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:43.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.857+0000 7f3ef2ffd640 1 -- 172.21.15.47:0/760465925 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3edc037070 con 0x7f3ef40d2320 2024-04-26T23:19:43.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.857+0000 7f3ef2ffd640 1 -- 172.21.15.47:0/760465925 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3edc02ae00 con 0x7f3ef40d2320 2024-04-26T23:19:43.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.857+0000 7f3ef2ffd640 1 -- 172.21.15.47:0/760465925 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3edc032660 con 0x7f3ef40d2320 2024-04-26T23:19:43.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.857+0000 7f3ef9c45640 1 -- 172.21.15.47:0/760465925 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ef40d2320 msgr2=0x7f3ef41625f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:43.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.857+0000 7f3ef9c45640 1 --2- 172.21.15.47:0/760465925 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ef40d2320 0x7f3ef41625f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f3ef406c650 tx=0x7f3edc02a0b0 comp rx=0 tx=0).stop 2024-04-26T23:19:43.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.858+0000 7f3ef9c45640 1 -- 172.21.15.47:0/760465925 shutdown_connections 2024-04-26T23:19:43.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.858+0000 7f3ef9c45640 1 --2- 172.21.15.47:0/760465925 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ef40d2320 0x7f3ef41625f0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:43.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.858+0000 7f3ef9c45640 1 --2- 172.21.15.47:0/760465925 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ef40b40d0 0x7f3ef4162b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:43.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.858+0000 7f3ef9c45640 1 --2- 172.21.15.47:0/760465925 >> v2:172.21.15.203:3300/0 conn(0x7f3ef40d1a00 0x7f3ef40d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:43.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.858+0000 7f3ef9c45640 1 -- 172.21.15.47:0/760465925 >> 172.21.15.47:0/760465925 conn(0x7f3ef40d4fd0 msgr2=0x7f3ef40d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:43.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.858+0000 7f3ef9c45640 1 -- 172.21.15.47:0/760465925 shutdown_connections 2024-04-26T23:19:43.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.858+0000 7f3ef9c45640 1 -- 172.21.15.47:0/760465925 wait complete. 2024-04-26T23:19:43.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.859+0000 7f3ef9c45640 1 Processor -- start 2024-04-26T23:19:43.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.859+0000 7f3ef9c45640 1 -- start start 2024-04-26T23:19:43.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.860+0000 7f3ef9c45640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ef40b40d0 0x7f3ef40cb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:43.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.860+0000 7f3ef9c45640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ef40d1a00 0x7f3ef40cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:43.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.860+0000 7f3ef9c45640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ef40d2320 0x7f3ef41f84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:43.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.860+0000 7f3ef9c45640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef40d3050 con 0x7f3ef40d2320 2024-04-26T23:19:43.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.860+0000 7f3ef9c45640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef40d2ed0 con 0x7f3ef40b40d0 2024-04-26T23:19:43.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.860+0000 7f3ef9c45640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3ef40d31d0 con 0x7f3ef40d1a00 2024-04-26T23:19:43.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.860+0000 7f3ef3fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ef40d2320 0x7f3ef41f84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:43.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.860+0000 7f3ef3fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ef40d2320 0x7f3ef41f84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58768/0 (socket says 172.21.15.47:58768) 2024-04-26T23:19:43.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.860+0000 7f3ef3fff640 1 -- 172.21.15.47:0/1180253901 learned_addr learned my addr 172.21.15.47:0/1180253901 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:43.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.860+0000 7f3ef37fe640 1 --2- 172.21.15.47:0/1180253901 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ef40b40d0 0x7f3ef40cb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:43.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.860+0000 7f3eeaffd640 1 --2- 172.21.15.47:0/1180253901 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ef40d1a00 0x7f3ef40cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:43.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.860+0000 7f3ef3fff640 1 -- 172.21.15.47:0/1180253901 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ef40d1a00 msgr2=0x7f3ef40cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:43.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.860+0000 7f3ef3fff640 1 --2- 172.21.15.47:0/1180253901 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ef40d1a00 0x7f3ef40cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:43.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.860+0000 7f3ef3fff640 1 -- 172.21.15.47:0/1180253901 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ef40b40d0 msgr2=0x7f3ef40cb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:43.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.860+0000 7f3ef3fff640 1 --2- 172.21.15.47:0/1180253901 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ef40b40d0 0x7f3ef40cb9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:43.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.860+0000 7f3ef3fff640 1 -- 172.21.15.47:0/1180253901 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ef415edb0 con 0x7f3ef40d2320 2024-04-26T23:19:43.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.860+0000 7f3ef3fff640 1 --2- 172.21.15.47:0/1180253901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ef40d2320 0x7f3ef41f84d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3eec007e60 tx=0x7f3eec00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:43.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.860+0000 7f3ef0ff9640 1 -- 172.21.15.47:0/1180253901 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eec014620 con 0x7f3ef40d2320 2024-04-26T23:19:43.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.860+0000 7f3ef0ff9640 1 -- 172.21.15.47:0/1180253901 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3eec019070 con 0x7f3ef40d2320 2024-04-26T23:19:43.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.861+0000 7f3ef0ff9640 1 -- 172.21.15.47:0/1180253901 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eec014a20 con 0x7f3ef40d2320 2024-04-26T23:19:43.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.861+0000 7f3ef9c45640 1 -- 172.21.15.47:0/1180253901 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ef41f9300 con 0x7f3ef40d2320 2024-04-26T23:19:43.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.861+0000 7f3ef9c45640 1 -- 172.21.15.47:0/1180253901 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ef40cca70 con 0x7f3ef40d2320 2024-04-26T23:19:43.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.861+0000 7f3ef9c45640 1 -- 172.21.15.47:0/1180253901 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ef40d3630 con 0x7f3ef40d2320 2024-04-26T23:19:43.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.862+0000 7f3ef0ff9640 1 -- 172.21.15.47:0/1180253901 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3eec014bc0 con 0x7f3ef40d2320 2024-04-26T23:19:43.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.862+0000 7f3ef0ff9640 1 --2- 172.21.15.47:0/1180253901 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3ed406c8f0 0x7f3ed406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:43.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.863+0000 7f3ef0ff9640 1 -- 172.21.15.47:0/1180253901 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6001+0+0 (secure 0 0 0) 0x7f3eec08c3d0 con 0x7f3ef40d2320 2024-04-26T23:19:43.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.863+0000 7f3ef37fe640 1 --2- 172.21.15.47:0/1180253901 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3ed406c8f0 0x7f3ed406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:43.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.863+0000 7f3ef37fe640 1 --2- 172.21.15.47:0/1180253901 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3ed406c8f0 0x7f3ed406edb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3ee400ab60 tx=0x7f3ee4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:43.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:43.867+0000 7f3ef0ff9640 1 -- 172.21.15.47:0/1180253901 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3ef40d3630 con 0x7f3ef40d2320 2024-04-26T23:19:44.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.090+0000 7f3ef9c45640 1 -- 172.21.15.47:0/1180253901 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f3ef41624f0 con 0x7f3ef40d2320 2024-04-26T23:19:44.327 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-04-26T23:19:44.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.713+0000 7f3ef0ff9640 1 -- 172.21.15.47:0/1180253901 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) ==== 172+0+0 (secure 0 0 0) 0x7f3eec05a080 con 0x7f3ef40d2320 2024-04-26T23:19:44.714 INFO:teuthology.orchestra.run.smithi047.stderr:new fs with metadata pool 5 and data pool 6 2024-04-26T23:19:44.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.720+0000 7f3ef9c45640 1 -- 172.21.15.47:0/1180253901 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3ed406c8f0 msgr2=0x7f3ed406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:44.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.720+0000 7f3ef9c45640 1 --2- 172.21.15.47:0/1180253901 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3ed406c8f0 0x7f3ed406edb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3ee400ab60 tx=0x7f3ee4008040 comp rx=0 tx=0).stop 2024-04-26T23:19:44.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.720+0000 7f3ef9c45640 1 -- 172.21.15.47:0/1180253901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ef40d2320 msgr2=0x7f3ef41f84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:44.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.720+0000 7f3ef9c45640 1 --2- 172.21.15.47:0/1180253901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ef40d2320 0x7f3ef41f84d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3eec007e60 tx=0x7f3eec00e4b0 comp rx=0 tx=0).stop 2024-04-26T23:19:44.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.721+0000 7f3ef9c45640 1 -- 172.21.15.47:0/1180253901 shutdown_connections 2024-04-26T23:19:44.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.721+0000 7f3ef9c45640 1 --2- 172.21.15.47:0/1180253901 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3ed406c8f0 0x7f3ed406edb0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:44.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.721+0000 7f3ef9c45640 1 --2- 172.21.15.47:0/1180253901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ef40d2320 0x7f3ef41f84d0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:44.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.721+0000 7f3ef9c45640 1 --2- 172.21.15.47:0/1180253901 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ef40d1a00 0x7f3ef40cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:44.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.721+0000 7f3ef9c45640 1 --2- 172.21.15.47:0/1180253901 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ef40b40d0 0x7f3ef40cb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:44.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.721+0000 7f3ef9c45640 1 -- 172.21.15.47:0/1180253901 >> 172.21.15.47:0/1180253901 conn(0x7f3ef40d4fd0 msgr2=0x7f3ef40cfe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:44.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.722+0000 7f3ef9c45640 1 -- 172.21.15.47:0/1180253901 shutdown_connections 2024-04-26T23:19:44.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.722+0000 7f3ef9c45640 1 -- 172.21.15.47:0/1180253901 wait complete. 2024-04-26T23:19:44.753 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:19:44.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.857+0000 7f2724f7f640 1 Processor -- start 2024-04-26T23:19:44.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.857+0000 7f2724f7f640 1 -- start start 2024-04-26T23:19:44.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.858+0000 7f2724f7f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f272015f2d0 0x7f27201604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:44.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.858+0000 7f2724f7f640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f272006fea0 0x7f27200ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:44.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.858+0000 7f2724f7f640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2720072f70 con 0x7f2720072700 2024-04-26T23:19:44.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.858+0000 7f2724f7f640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f2720074010 con 0x7f272006fea0 2024-04-26T23:19:44.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.858+0000 7f2724f7f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f2720073250 con 0x7f272015f2d0 2024-04-26T23:19:44.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.858+0000 7f271ed9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f272006fea0 0x7f27200ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:44.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.858+0000 7f271dd9b640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f2720072700 0x7f272006f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36076/0 (socket says 172.21.15.47:36076) 2024-04-26T23:19:44.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.858+0000 7f271dd9b640 1 -- 172.21.15.47:0/1120561329 learned_addr learned my addr 172.21.15.47:0/1120561329 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:44.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.859+0000 7f271e59c640 1 --2- 172.21.15.47:0/1120561329 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f272015f2d0 0x7f27201604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:44.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.859+0000 7f271ed9d640 1 -- 172.21.15.47:0/1120561329 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f272015f2d0 msgr2=0x7f27201604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:44.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.859+0000 7f271ed9d640 1 --2- 172.21.15.47:0/1120561329 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f272015f2d0 0x7f27201604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:44.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.859+0000 7f271ed9d640 1 -- 172.21.15.47:0/1120561329 >> v1:172.21.15.47:6789/0 conn(0x7f2720072700 legacy=0x7f272006f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:44.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.859+0000 7f271ed9d640 1 -- 172.21.15.47:0/1120561329 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2720073530 con 0x7f272006fea0 2024-04-26T23:19:44.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.859+0000 7f271ed9d640 1 --2- 172.21.15.47:0/1120561329 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f272006fea0 0x7f27200ce7b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2718009d90 tx=0x7f271802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dd9edb505a4b25e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:44.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.859+0000 7f271d59a640 1 -- 172.21.15.47:0/1120561329 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2718007740 con 0x7f272006fea0 2024-04-26T23:19:44.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.859+0000 7f271d59a640 1 -- 172.21.15.47:0/1120561329 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27180078e0 con 0x7f272006fea0 2024-04-26T23:19:44.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.860+0000 7f271d59a640 1 -- 172.21.15.47:0/1120561329 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2718007cb0 con 0x7f272006fea0 2024-04-26T23:19:44.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.860+0000 7f2724f7f640 1 -- 172.21.15.47:0/1120561329 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f272006fea0 msgr2=0x7f27200ce7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:44.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.860+0000 7f2724f7f640 1 --2- 172.21.15.47:0/1120561329 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f272006fea0 0x7f27200ce7b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2718009d90 tx=0x7f271802a660 comp rx=0 tx=0).stop 2024-04-26T23:19:44.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.860+0000 7f2724f7f640 1 -- 172.21.15.47:0/1120561329 shutdown_connections 2024-04-26T23:19:44.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.860+0000 7f2724f7f640 1 --2- 172.21.15.47:0/1120561329 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f272006fea0 0x7f27200ce7b0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:44.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.860+0000 7f2724f7f640 1 --2- 172.21.15.47:0/1120561329 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f272015f2d0 0x7f27201604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:44.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.860+0000 7f2724f7f640 1 -- 172.21.15.47:0/1120561329 >> 172.21.15.47:0/1120561329 conn(0x7f27200d0be0 msgr2=0x7f27200d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:44.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.860+0000 7f2724f7f640 1 -- 172.21.15.47:0/1120561329 shutdown_connections 2024-04-26T23:19:44.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.860+0000 7f2724f7f640 1 -- 172.21.15.47:0/1120561329 wait complete. 2024-04-26T23:19:44.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.861+0000 7f2724f7f640 1 Processor -- start 2024-04-26T23:19:44.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.862+0000 7f2724f7f640 1 -- start start 2024-04-26T23:19:44.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.862+0000 7f2724f7f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f272006fea0 0x7f27201f8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:44.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.862+0000 7f2724f7f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2720072700 0x7f272015e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:44.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.862+0000 7f2724f7f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f272015f2d0 0x7f2720150950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:44.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.862+0000 7f2724f7f640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f272015f6b0 con 0x7f2720072700 2024-04-26T23:19:44.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.862+0000 7f2724f7f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2720073af0 con 0x7f272015f2d0 2024-04-26T23:19:44.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.862+0000 7f2724f7f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f2720072b60 con 0x7f272006fea0 2024-04-26T23:19:44.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.862+0000 7f271dd9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2720072700 0x7f272015e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:44.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.862+0000 7f271dd9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2720072700 0x7f272015e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58790/0 (socket says 172.21.15.47:58790) 2024-04-26T23:19:44.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.862+0000 7f271dd9b640 1 -- 172.21.15.47:0/4162688063 learned_addr learned my addr 172.21.15.47:0/4162688063 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:44.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.862+0000 7f271ed9d640 1 --2- 172.21.15.47:0/4162688063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f272015f2d0 0x7f2720150950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:44.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.862+0000 7f271e59c640 1 --2- 172.21.15.47:0/4162688063 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f272006fea0 0x7f27201f8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:44.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.863+0000 7f271dd9b640 1 -- 172.21.15.47:0/4162688063 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f272006fea0 msgr2=0x7f27201f8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:44.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.863+0000 7f271dd9b640 1 --2- 172.21.15.47:0/4162688063 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f272006fea0 0x7f27201f8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:44.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.863+0000 7f271dd9b640 1 -- 172.21.15.47:0/4162688063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f272015f2d0 msgr2=0x7f2720150950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:44.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.863+0000 7f271dd9b640 1 --2- 172.21.15.47:0/4162688063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f272015f2d0 0x7f2720150950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:44.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.863+0000 7f271dd9b640 1 -- 172.21.15.47:0/4162688063 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27201593e0 con 0x7f2720072700 2024-04-26T23:19:44.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.863+0000 7f271ed9d640 1 --2- 172.21.15.47:0/4162688063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f272015f2d0 0x7f2720150950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:19:44.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.863+0000 7f271dd9b640 1 --2- 172.21.15.47:0/4162688063 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2720072700 0x7f272015e0a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f271400c400 tx=0x7f271400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:44.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.863+0000 7f270effd640 1 -- 172.21.15.47:0/4162688063 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2714017070 con 0x7f2720072700 2024-04-26T23:19:44.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.863+0000 7f270effd640 1 -- 172.21.15.47:0/4162688063 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2714005ce0 con 0x7f2720072700 2024-04-26T23:19:44.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.863+0000 7f2724f7f640 1 -- 172.21.15.47:0/4162688063 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27201f91a0 con 0x7f2720072700 2024-04-26T23:19:44.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.863+0000 7f270effd640 1 -- 172.21.15.47:0/4162688063 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27140072f0 con 0x7f2720072700 2024-04-26T23:19:44.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.863+0000 7f2724f7f640 1 -- 172.21.15.47:0/4162688063 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27201f9760 con 0x7f2720072700 2024-04-26T23:19:44.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.864+0000 7f2724f7f640 1 -- 172.21.15.47:0/4162688063 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f272015e640 con 0x7f2720072700 2024-04-26T23:19:44.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.864+0000 7f270effd640 1 -- 172.21.15.47:0/4162688063 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f271401a070 con 0x7f2720072700 2024-04-26T23:19:44.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.865+0000 7f270effd640 1 --2- 172.21.15.47:0/4162688063 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f270006c8f0 0x7f270006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:44.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.865+0000 7f270effd640 1 -- 172.21.15.47:0/4162688063 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6101+0+0 (secure 0 0 0) 0x7f271408b060 con 0x7f2720072700 2024-04-26T23:19:44.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.865+0000 7f271e59c640 1 --2- 172.21.15.47:0/4162688063 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f270006c8f0 0x7f270006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:44.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.866+0000 7f271e59c640 1 --2- 172.21.15.47:0/4162688063 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f270006c8f0 0x7f270006edb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2710006fd0 tx=0x7f2710008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:44.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:44.869+0000 7f270effd640 1 -- 172.21.15.47:0/4162688063 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2714059390 con 0x7f2720072700 2024-04-26T23:19:45.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.054+0000 7f2724f7f640 1 -- 172.21.15.47:0/4162688063 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f27200694f0 con 0x7f2720072700 2024-04-26T23:19:45.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.055+0000 7f270effd640 1 -- 172.21.15.47:0/4162688063 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+16919 (secure 0 0 0) 0x7f2714058c90 con 0x7f2720072700 2024-04-26T23:19:45.056 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:19:45.056 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":37,"fsid":"399006d3-4db1-43e6-ba81-a79a276eabfd","created":"2024-04-26T23:18:06.731211+0000","modified":"2024-04-26T23:19:44.091392+0000","last_up_change":"2024-04-26T23:18:17.808736+0000","last_in_change":"2024-04-26T23:18:08.945859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:18:16.937492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:19:42.351747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-26T23:19:43.037763+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ee091567-fea0-4a14-bf4e-0e16c20bb2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6819","nonce":326520779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6821","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6823","nonce":326520779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6829","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6831","nonce":326520779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6825","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6827","nonce":326520779}]},"public_addr":"172.21.15.47:6819/326520779","cluster_addr":"172.21.15.47:6823/326520779","heartbeat_back_addr":"172.21.15.47:6831/326520779","heartbeat_front_addr":"172.21.15.47:6827/326520779","state":["exists","up"]},{"osd":1,"uuid":"c14440a0-bba5-4931-a6db-b736c08e869b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6809","nonce":1912610931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6811","nonce":1912610931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6815","nonce":1912610931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6813","nonce":1912610931}]},"public_addr":"172.21.15.47:6809/1912610931","cluster_addr":"172.21.15.47:6811/1912610931","heartbeat_back_addr":"172.21.15.47:6815/1912610931","heartbeat_front_addr":"172.21.15.47:6813/1912610931","state":["exists","up"]},{"osd":2,"uuid":"3940f8de-c9b2-4cd3-a911-9c301729eb30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6801","nonce":2569869305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6803","nonce":2569869305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6807","nonce":2569869305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6805","nonce":2569869305}]},"public_addr":"172.21.15.47:6801/2569869305","cluster_addr":"172.21.15.47:6803/2569869305","heartbeat_back_addr":"172.21.15.47:6807/2569869305","heartbeat_front_addr":"172.21.15.47:6805/2569869305","state":["exists","up"]},{"osd":3,"uuid":"1286a67e-8958-426a-8c2e-bc85498c619c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6818","nonce":2791411683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6822","nonce":2791411683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6830","nonce":2791411683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6826","nonce":2791411683}]},"public_addr":"172.21.15.47:6818/2791411683","cluster_addr":"172.21.15.47:6822/2791411683","heartbeat_back_addr":"172.21.15.47:6830/2791411683","heartbeat_front_addr":"172.21.15.47:6826/2791411683","state":["exists","up"]},{"osd":4,"uuid":"d8ee2c20-e5b7-46c2-9999-b234070126cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6809","nonce":441313104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6811","nonce":441313104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6815","nonce":441313104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6813","nonce":441313104}]},"public_addr":"172.21.15.203:6809/441313104","cluster_addr":"172.21.15.203:6811/441313104","heartbeat_back_addr":"172.21.15.203:6815/441313104","heartbeat_front_addr":"172.21.15.203:6813/441313104","state":["exists","up"]},{"osd":5,"uuid":"b453a8f0-f324-4840-9ae0-d67bdadb9f8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6801","nonce":56838444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6803","nonce":56838444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6807","nonce":56838444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6805","nonce":56838444}]},"public_addr":"172.21.15.203:6801/56838444","cluster_addr":"172.21.15.203:6803/56838444","heartbeat_back_addr":"172.21.15.203:6807/56838444","heartbeat_front_addr":"172.21.15.203:6805/56838444","state":["exists","up"]},{"osd":6,"uuid":"a6d31951-2d4b-4e8d-a510-6283b01dc158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6825","nonce":2738517080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6827","nonce":2738517080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6831","nonce":2738517080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6829","nonce":2738517080}]},"public_addr":"172.21.15.203:6825/2738517080","cluster_addr":"172.21.15.203:6827/2738517080","heartbeat_back_addr":"172.21.15.203:6831/2738517080","heartbeat_front_addr":"172.21.15.203:6829/2738517080","state":["exists","up"]},{"osd":7,"uuid":"7119b8c0-fe96-402d-9e75-7b509d9d44d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6817","nonce":1315565058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6819","nonce":1315565058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6823","nonce":1315565058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6821","nonce":1315565058}]},"public_addr":"172.21.15.203:6817/1315565058","cluster_addr":"172.21.15.203:6819/1315565058","heartbeat_back_addr":"172.21.15.203:6823/1315565058","heartbeat_front_addr":"172.21.15.203:6821/1315565058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.121921+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.137494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.084814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.111593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.072430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.125066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.097867+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.057061+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:19:45.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.057+0000 7f2724f7f640 1 -- 172.21.15.47:0/4162688063 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f270006c8f0 msgr2=0x7f270006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:45.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.057+0000 7f2724f7f640 1 --2- 172.21.15.47:0/4162688063 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f270006c8f0 0x7f270006edb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2710006fd0 tx=0x7f2710008040 comp rx=0 tx=0).stop 2024-04-26T23:19:45.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.057+0000 7f2724f7f640 1 -- 172.21.15.47:0/4162688063 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2720072700 msgr2=0x7f272015e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:45.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.057+0000 7f2724f7f640 1 --2- 172.21.15.47:0/4162688063 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2720072700 0x7f272015e0a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f271400c400 tx=0x7f271400c8c0 comp rx=0 tx=0).stop 2024-04-26T23:19:45.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.057+0000 7f2724f7f640 1 -- 172.21.15.47:0/4162688063 shutdown_connections 2024-04-26T23:19:45.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.057+0000 7f2724f7f640 1 --2- 172.21.15.47:0/4162688063 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f270006c8f0 0x7f270006edb0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:45.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.057+0000 7f2724f7f640 1 --2- 172.21.15.47:0/4162688063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f272015f2d0 0x7f2720150950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:45.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.057+0000 7f2724f7f640 1 --2- 172.21.15.47:0/4162688063 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2720072700 0x7f272015e0a0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:45.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.057+0000 7f2724f7f640 1 --2- 172.21.15.47:0/4162688063 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f272006fea0 0x7f27201f8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:45.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.057+0000 7f2724f7f640 1 -- 172.21.15.47:0/4162688063 >> 172.21.15.47:0/4162688063 conn(0x7f27200d0be0 msgr2=0x7f27200b3920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:45.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.057+0000 7f2724f7f640 1 -- 172.21.15.47:0/4162688063 shutdown_connections 2024-04-26T23:19:45.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.057+0000 7f2724f7f640 1 -- 172.21.15.47:0/4162688063 wait complete. 2024-04-26T23:19:45.070 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:19:45.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.170+0000 7fabc9f51640 1 Processor -- start 2024-04-26T23:19:45.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.170+0000 7fabc9f51640 1 -- start start 2024-04-26T23:19:45.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.170+0000 7fabc9f51640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fabc40d06f0 0x7fabc40d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:45.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.170+0000 7fabc9f51640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fabc40b2850 0x7fabc40b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:45.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.171+0000 7fabc9f51640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabc40d0b70 con 0x7fabc415f3a0 2024-04-26T23:19:45.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.171+0000 7fabc9f51640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fabc40d0e20 con 0x7fabc40b2850 2024-04-26T23:19:45.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.171+0000 7fabc9f51640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fabc40b1d70 con 0x7fabc40d06f0 2024-04-26T23:19:45.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.171+0000 7fabc3fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fabc40b2850 0x7fabc40b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:45.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.171+0000 7fabc37fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fabc415f3a0 0x7fabc40d1770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36092/0 (socket says 172.21.15.47:36092) 2024-04-26T23:19:45.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.171+0000 7fabc37fe640 1 -- 172.21.15.47:0/723615407 learned_addr learned my addr 172.21.15.47:0/723615407 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:45.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.171+0000 7fabc2ffd640 1 --2- 172.21.15.47:0/723615407 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fabc40d06f0 0x7fabc40d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:45.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.171+0000 7fabc3fff640 1 -- 172.21.15.47:0/723615407 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fabc40d06f0 msgr2=0x7fabc40d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:45.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.171+0000 7fabc3fff640 1 --2- 172.21.15.47:0/723615407 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fabc40d06f0 0x7fabc40d1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:45.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.171+0000 7fabc3fff640 1 -- 172.21.15.47:0/723615407 >> v1:172.21.15.47:6789/0 conn(0x7fabc415f3a0 legacy=0x7fabc40d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:45.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.171+0000 7fabc3fff640 1 -- 172.21.15.47:0/723615407 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabc40b2050 con 0x7fabc40b2850 2024-04-26T23:19:45.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.171+0000 7fabc27fc640 1 -- 172.21.15.47:0/723615407 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1673438544 0 0) 0x7fabc40d0b70 con 0x7fabc415f3a0 2024-04-26T23:19:45.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.172+0000 7fabc3fff640 1 --2- 172.21.15.47:0/723615407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabc40b2850 0x7fabc40b2cc0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fabbc009d90 tx=0x7fabbc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da771bf719c12ca0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:45.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.172+0000 7fabc27fc640 1 -- 172.21.15.47:0/723615407 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabbc007740 con 0x7fabc40b2850 2024-04-26T23:19:45.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.172+0000 7fabc27fc640 1 -- 172.21.15.47:0/723615407 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabbc0078e0 con 0x7fabc40b2850 2024-04-26T23:19:45.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.172+0000 7fabc27fc640 1 -- 172.21.15.47:0/723615407 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabbc007cb0 con 0x7fabc40b2850 2024-04-26T23:19:45.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.172+0000 7fabc9f51640 1 -- 172.21.15.47:0/723615407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabc40b2850 msgr2=0x7fabc40b2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:45.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.172+0000 7fabc9f51640 1 --2- 172.21.15.47:0/723615407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabc40b2850 0x7fabc40b2cc0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fabbc009d90 tx=0x7fabbc02a660 comp rx=0 tx=0).stop 2024-04-26T23:19:45.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.172+0000 7fabc9f51640 1 -- 172.21.15.47:0/723615407 shutdown_connections 2024-04-26T23:19:45.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.172+0000 7fabc9f51640 1 --2- 172.21.15.47:0/723615407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabc40b2850 0x7fabc40b2cc0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:45.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.172+0000 7fabc9f51640 1 --2- 172.21.15.47:0/723615407 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fabc40d06f0 0x7fabc40d1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:45.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.172+0000 7fabc9f51640 1 -- 172.21.15.47:0/723615407 >> 172.21.15.47:0/723615407 conn(0x7fabc40d4f90 msgr2=0x7fabc40d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:45.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.172+0000 7fabc9f51640 1 -- 172.21.15.47:0/723615407 shutdown_connections 2024-04-26T23:19:45.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.172+0000 7fabc9f51640 1 -- 172.21.15.47:0/723615407 wait complete. 2024-04-26T23:19:45.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.174+0000 7fabc9f51640 1 Processor -- start 2024-04-26T23:19:45.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.174+0000 7fabc9f51640 1 -- start start 2024-04-26T23:19:45.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.175+0000 7fabc9f51640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fabc40b2850 0x7fabc4151560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:45.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.175+0000 7fabc9f51640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fabc40d06f0 0x7fabc40b8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:45.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.175+0000 7fabc9f51640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabc415f3a0 0x7fabc4151f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:45.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.175+0000 7fabc9f51640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fabc415f900 con 0x7fabc415f3a0 2024-04-26T23:19:45.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.175+0000 7fabc9f51640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fabc415f780 con 0x7fabc40d06f0 2024-04-26T23:19:45.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.175+0000 7fabc9f51640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fabc40d23c0 con 0x7fabc40b2850 2024-04-26T23:19:45.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.175+0000 7fabc3fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabc415f3a0 0x7fabc4151f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:45.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.175+0000 7fabc3fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabc415f3a0 0x7fabc4151f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58816/0 (socket says 172.21.15.47:58816) 2024-04-26T23:19:45.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.175+0000 7fabc3fff640 1 -- 172.21.15.47:0/896456898 learned_addr learned my addr 172.21.15.47:0/896456898 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:45.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.175+0000 7fabc2ffd640 1 --2- 172.21.15.47:0/896456898 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fabc40d06f0 0x7fabc40b8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:45.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.175+0000 7fabc37fe640 1 --2- 172.21.15.47:0/896456898 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fabc40b2850 0x7fabc4151560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:45.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.175+0000 7fabc3fff640 1 -- 172.21.15.47:0/896456898 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fabc40b2850 msgr2=0x7fabc4151560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:45.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.175+0000 7fabc3fff640 1 --2- 172.21.15.47:0/896456898 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fabc40b2850 0x7fabc4151560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:45.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.175+0000 7fabc3fff640 1 -- 172.21.15.47:0/896456898 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fabc40d06f0 msgr2=0x7fabc40b8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:45.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.175+0000 7fabc3fff640 1 --2- 172.21.15.47:0/896456898 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fabc40d06f0 0x7fabc40b8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:45.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.175+0000 7fabc3fff640 1 -- 172.21.15.47:0/896456898 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabc40b7d20 con 0x7fabc415f3a0 2024-04-26T23:19:45.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.175+0000 7fabc2ffd640 1 --2- 172.21.15.47:0/896456898 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fabc40d06f0 0x7fabc40b8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:19:45.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.175+0000 7fabc37fe640 1 --2- 172.21.15.47:0/896456898 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fabc40b2850 0x7fabc4151560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:19:45.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.175+0000 7fabc3fff640 1 --2- 172.21.15.47:0/896456898 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabc415f3a0 0x7fabc4151f60 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fabbc009eb0 tx=0x7fabbc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:45.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.175+0000 7fababfff640 1 -- 172.21.15.47:0/896456898 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabbc03f070 con 0x7fabc415f3a0 2024-04-26T23:19:45.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.175+0000 7fababfff640 1 -- 172.21.15.47:0/896456898 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabbc031bc0 con 0x7fabc415f3a0 2024-04-26T23:19:45.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.175+0000 7fabc9f51640 1 -- 172.21.15.47:0/896456898 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabc40b8000 con 0x7fabc415f3a0 2024-04-26T23:19:45.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.175+0000 7fababfff640 1 -- 172.21.15.47:0/896456898 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabbc009450 con 0x7fabc415f3a0 2024-04-26T23:19:45.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.176+0000 7fabc9f51640 1 -- 172.21.15.47:0/896456898 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabc40b85c0 con 0x7fabc415f3a0 2024-04-26T23:19:45.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.176+0000 7fabc9f51640 1 -- 172.21.15.47:0/896456898 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabc40694f0 con 0x7fabc415f3a0 2024-04-26T23:19:45.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.177+0000 7fababfff640 1 -- 172.21.15.47:0/896456898 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fabbc03b410 con 0x7fabc415f3a0 2024-04-26T23:19:45.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.177+0000 7fababfff640 1 --2- 172.21.15.47:0/896456898 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fabb006c8f0 0x7fabb006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:45.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.177+0000 7fababfff640 1 -- 172.21.15.47:0/896456898 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6101+0+0 (secure 0 0 0) 0x7fabbc0ad520 con 0x7fabc415f3a0 2024-04-26T23:19:45.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.178+0000 7fabc37fe640 1 --2- 172.21.15.47:0/896456898 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fabb006c8f0 0x7fabb006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:45.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.178+0000 7fabc37fe640 1 --2- 172.21.15.47:0/896456898 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fabb006c8f0 0x7fabb006edb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fabb4000f60 tx=0x7fabb4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:45.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.182+0000 7fababfff640 1 -- 172.21.15.47:0/896456898 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fabbc07b0d0 con 0x7fabc415f3a0 2024-04-26T23:19:45.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.378+0000 7fabc9f51640 1 -- 172.21.15.47:0/896456898 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fabc40b6860 con 0x7fabc415f3a0 2024-04-26T23:19:45.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.379+0000 7fababfff640 1 -- 172.21.15.47:0/896456898 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+16919 (secure 0 0 0) 0x7fabbc07f3d0 con 0x7fabc415f3a0 2024-04-26T23:19:45.380 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:19:45.380 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":37,"fsid":"399006d3-4db1-43e6-ba81-a79a276eabfd","created":"2024-04-26T23:18:06.731211+0000","modified":"2024-04-26T23:19:44.091392+0000","last_up_change":"2024-04-26T23:18:17.808736+0000","last_in_change":"2024-04-26T23:18:08.945859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:18:16.937492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:19:42.351747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-26T23:19:43.037763+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ee091567-fea0-4a14-bf4e-0e16c20bb2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6819","nonce":326520779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6821","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6823","nonce":326520779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6829","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6831","nonce":326520779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6825","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6827","nonce":326520779}]},"public_addr":"172.21.15.47:6819/326520779","cluster_addr":"172.21.15.47:6823/326520779","heartbeat_back_addr":"172.21.15.47:6831/326520779","heartbeat_front_addr":"172.21.15.47:6827/326520779","state":["exists","up"]},{"osd":1,"uuid":"c14440a0-bba5-4931-a6db-b736c08e869b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6809","nonce":1912610931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6811","nonce":1912610931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6815","nonce":1912610931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6813","nonce":1912610931}]},"public_addr":"172.21.15.47:6809/1912610931","cluster_addr":"172.21.15.47:6811/1912610931","heartbeat_back_addr":"172.21.15.47:6815/1912610931","heartbeat_front_addr":"172.21.15.47:6813/1912610931","state":["exists","up"]},{"osd":2,"uuid":"3940f8de-c9b2-4cd3-a911-9c301729eb30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6801","nonce":2569869305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6803","nonce":2569869305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6807","nonce":2569869305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6805","nonce":2569869305}]},"public_addr":"172.21.15.47:6801/2569869305","cluster_addr":"172.21.15.47:6803/2569869305","heartbeat_back_addr":"172.21.15.47:6807/2569869305","heartbeat_front_addr":"172.21.15.47:6805/2569869305","state":["exists","up"]},{"osd":3,"uuid":"1286a67e-8958-426a-8c2e-bc85498c619c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6818","nonce":2791411683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6822","nonce":2791411683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6830","nonce":2791411683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6826","nonce":2791411683}]},"public_addr":"172.21.15.47:6818/2791411683","cluster_addr":"172.21.15.47:6822/2791411683","heartbeat_back_addr":"172.21.15.47:6830/2791411683","heartbeat_front_addr":"172.21.15.47:6826/2791411683","state":["exists","up"]},{"osd":4,"uuid":"d8ee2c20-e5b7-46c2-9999-b234070126cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6809","nonce":441313104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6811","nonce":441313104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6815","nonce":441313104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6813","nonce":441313104}]},"public_addr":"172.21.15.203:6809/441313104","cluster_addr":"172.21.15.203:6811/441313104","heartbeat_back_addr":"172.21.15.203:6815/441313104","heartbeat_front_addr":"172.21.15.203:6813/441313104","state":["exists","up"]},{"osd":5,"uuid":"b453a8f0-f324-4840-9ae0-d67bdadb9f8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6801","nonce":56838444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6803","nonce":56838444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6807","nonce":56838444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6805","nonce":56838444}]},"public_addr":"172.21.15.203:6801/56838444","cluster_addr":"172.21.15.203:6803/56838444","heartbeat_back_addr":"172.21.15.203:6807/56838444","heartbeat_front_addr":"172.21.15.203:6805/56838444","state":["exists","up"]},{"osd":6,"uuid":"a6d31951-2d4b-4e8d-a510-6283b01dc158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6825","nonce":2738517080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6827","nonce":2738517080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6831","nonce":2738517080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6829","nonce":2738517080}]},"public_addr":"172.21.15.203:6825/2738517080","cluster_addr":"172.21.15.203:6827/2738517080","heartbeat_back_addr":"172.21.15.203:6831/2738517080","heartbeat_front_addr":"172.21.15.203:6829/2738517080","state":["exists","up"]},{"osd":7,"uuid":"7119b8c0-fe96-402d-9e75-7b509d9d44d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6817","nonce":1315565058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6819","nonce":1315565058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6823","nonce":1315565058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6821","nonce":1315565058}]},"public_addr":"172.21.15.203:6817/1315565058","cluster_addr":"172.21.15.203:6819/1315565058","heartbeat_back_addr":"172.21.15.203:6823/1315565058","heartbeat_front_addr":"172.21.15.203:6821/1315565058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.121921+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.137494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.084814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.111593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.072430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.125066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.097867+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.057061+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:19:45.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.381+0000 7fabc9f51640 1 -- 172.21.15.47:0/896456898 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fabb006c8f0 msgr2=0x7fabb006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:45.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.381+0000 7fabc9f51640 1 --2- 172.21.15.47:0/896456898 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fabb006c8f0 0x7fabb006edb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fabb4000f60 tx=0x7fabb4009040 comp rx=0 tx=0).stop 2024-04-26T23:19:45.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.381+0000 7fabc9f51640 1 -- 172.21.15.47:0/896456898 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabc415f3a0 msgr2=0x7fabc4151f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:45.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.381+0000 7fabc9f51640 1 --2- 172.21.15.47:0/896456898 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabc415f3a0 0x7fabc4151f60 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fabbc009eb0 tx=0x7fabbc004a40 comp rx=0 tx=0).stop 2024-04-26T23:19:45.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.381+0000 7fabc9f51640 1 -- 172.21.15.47:0/896456898 shutdown_connections 2024-04-26T23:19:45.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.381+0000 7fabc9f51640 1 --2- 172.21.15.47:0/896456898 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fabb006c8f0 0x7fabb006edb0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:45.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.381+0000 7fabc9f51640 1 --2- 172.21.15.47:0/896456898 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabc415f3a0 0x7fabc4151f60 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:45.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.381+0000 7fabc9f51640 1 --2- 172.21.15.47:0/896456898 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fabc40d06f0 0x7fabc40b8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:45.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.381+0000 7fabc9f51640 1 --2- 172.21.15.47:0/896456898 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fabc40b2850 0x7fabc4151560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:45.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.381+0000 7fabc9f51640 1 -- 172.21.15.47:0/896456898 >> 172.21.15.47:0/896456898 conn(0x7fabc40d4f90 msgr2=0x7fabc406dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:45.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.381+0000 7fabc9f51640 1 -- 172.21.15.47:0/896456898 shutdown_connections 2024-04-26T23:19:45.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.381+0000 7fabc9f51640 1 -- 172.21.15.47:0/896456898 wait complete. 2024-04-26T23:19:45.393 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T23:19:45.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.496+0000 7f66056cc640 1 Processor -- start 2024-04-26T23:19:45.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.496+0000 7f66056cc640 1 -- start start 2024-04-26T23:19:45.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.497+0000 7f66056cc640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f66000d1240 0x7f66000d1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:45.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.497+0000 7f66056cc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f66000b4360 0x7f66000cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:45.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.497+0000 7f66056cc640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66000d1b60 con 0x7f66000d0d60 2024-04-26T23:19:45.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.497+0000 7f66056cc640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f66000cf210 con 0x7f66000d1240 2024-04-26T23:19:45.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.497+0000 7f66056cc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f66000ce5e0 con 0x7f66000b4360 2024-04-26T23:19:45.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.498+0000 7f65feffd640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f66000d1240 0x7f66000d1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:45.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.498+0000 7f65fe7fc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f66000b4360 0x7f66000cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:45.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.498+0000 7f65feffd640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f66000d1240 0x7f66000d1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.47:59078/0 (socket says 172.21.15.47:59078) 2024-04-26T23:19:45.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.498+0000 7f65ff7fe640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f66000d0d60 0x7f66000cfdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.47:42518/0 (socket says 172.21.15.47:42518) 2024-04-26T23:19:45.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.498+0000 7f65feffd640 1 -- 172.21.15.47:0/2817142856 learned_addr learned my addr 172.21.15.47:0/2817142856 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:45.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.498+0000 7f65feffd640 1 -- 172.21.15.47:0/2817142856 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f66000b4360 msgr2=0x7f66000cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:45.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.498+0000 7f65feffd640 1 --2- 172.21.15.47:0/2817142856 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f66000b4360 0x7f66000cf870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:45.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.498+0000 7f65feffd640 1 -- 172.21.15.47:0/2817142856 >> v1:172.21.15.203:6789/0 conn(0x7f66000d0d60 legacy=0x7f66000cfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:45.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.498+0000 7f65feffd640 1 -- 172.21.15.47:0/2817142856 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66000ce8c0 con 0x7f66000d1240 2024-04-26T23:19:45.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.498+0000 7f65fe7fc640 1 --2- 172.21.15.47:0/2817142856 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f66000b4360 0x7f66000cf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:19:45.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.499+0000 7f65feffd640 1 --2- 172.21.15.47:0/2817142856 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66000d1240 0x7f66000d1620 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f65f0009870 tx=0x7f65f002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81c6ca56bd03112a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:45.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.499+0000 7f65fdffb640 1 -- 172.21.15.47:0/2817142856 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f0037070 con 0x7f66000d1240 2024-04-26T23:19:45.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.499+0000 7f65fdffb640 1 -- 172.21.15.47:0/2817142856 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65f0032440 con 0x7f66000d1240 2024-04-26T23:19:45.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.499+0000 7f65fdffb640 1 -- 172.21.15.47:0/2817142856 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f0032810 con 0x7f66000d1240 2024-04-26T23:19:45.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.499+0000 7f66056cc640 1 -- 172.21.15.47:0/2817142856 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66000d1240 msgr2=0x7f66000d1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:45.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.499+0000 7f66056cc640 1 --2- 172.21.15.47:0/2817142856 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66000d1240 0x7f66000d1620 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f65f0009870 tx=0x7f65f002a270 comp rx=0 tx=0).stop 2024-04-26T23:19:45.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.499+0000 7f66056cc640 1 -- 172.21.15.47:0/2817142856 shutdown_connections 2024-04-26T23:19:45.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.499+0000 7f66056cc640 1 --2- 172.21.15.47:0/2817142856 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f66000b4360 0x7f66000cf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:45.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.499+0000 7f66056cc640 1 --2- 172.21.15.47:0/2817142856 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66000d1240 0x7f66000d1620 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:45.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.499+0000 7f66056cc640 1 -- 172.21.15.47:0/2817142856 >> 172.21.15.47:0/2817142856 conn(0x7f66000d2e60 msgr2=0x7f66000d3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:45.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.500+0000 7f66056cc640 1 -- 172.21.15.47:0/2817142856 shutdown_connections 2024-04-26T23:19:45.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.500+0000 7f66056cc640 1 -- 172.21.15.47:0/2817142856 wait complete. 2024-04-26T23:19:45.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.501+0000 7f66056cc640 1 Processor -- start 2024-04-26T23:19:45.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.501+0000 7f66056cc640 1 -- start start 2024-04-26T23:19:45.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.502+0000 7f66056cc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66000d0d60 0x7f660006d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:45.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.502+0000 7f66056cc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66000d1240 0x7f660015b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:45.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.502+0000 7f66056cc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f66000b2c10 0x7f66000b7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:45.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.502+0000 7f66056cc640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f66000c6610 con 0x7f66000d1240 2024-04-26T23:19:45.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.502+0000 7f66056cc640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f66000cf460 con 0x7f66000d0d60 2024-04-26T23:19:45.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.502+0000 7f66056cc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f66000c6790 con 0x7f66000b2c10 2024-04-26T23:19:45.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.502+0000 7f65fe7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66000d1240 0x7f660015b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:45.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.502+0000 7f65fe7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66000d1240 0x7f660015b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58828/0 (socket says 172.21.15.47:58828) 2024-04-26T23:19:45.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.502+0000 7f65fe7fc640 1 -- 172.21.15.47:0/3911132240 learned_addr learned my addr 172.21.15.47:0/3911132240 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:45.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.502+0000 7f65feffd640 1 --2- 172.21.15.47:0/3911132240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66000d0d60 0x7f660006d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:45.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.502+0000 7f65ff7fe640 1 --2- 172.21.15.47:0/3911132240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f66000b2c10 0x7f66000b7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:45.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.502+0000 7f65fe7fc640 1 -- 172.21.15.47:0/3911132240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f66000b2c10 msgr2=0x7f66000b7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:45.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.502+0000 7f65fe7fc640 1 --2- 172.21.15.47:0/3911132240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f66000b2c10 0x7f66000b7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:45.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.502+0000 7f65fe7fc640 1 -- 172.21.15.47:0/3911132240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66000d0d60 msgr2=0x7f660006d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:45.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.502+0000 7f65fe7fc640 1 --2- 172.21.15.47:0/3911132240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66000d0d60 0x7f660006d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:45.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.502+0000 7f65fe7fc640 1 -- 172.21.15.47:0/3911132240 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66000b86c0 con 0x7f66000d1240 2024-04-26T23:19:45.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.502+0000 7f65fe7fc640 1 --2- 172.21.15.47:0/3911132240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66000d1240 0x7f660015b7a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f65f400ebe0 tx=0x7f65f400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:45.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.502+0000 7f65e77fe640 1 -- 172.21.15.47:0/3911132240 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f4017070 con 0x7f66000d1240 2024-04-26T23:19:45.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.502+0000 7f65e77fe640 1 -- 172.21.15.47:0/3911132240 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65f400cc30 con 0x7f66000d1240 2024-04-26T23:19:45.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.502+0000 7f65e77fe640 1 -- 172.21.15.47:0/3911132240 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f4013470 con 0x7f66000d1240 2024-04-26T23:19:45.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.502+0000 7f66056cc640 1 -- 172.21.15.47:0/3911132240 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66000c7fc0 con 0x7f66000d1240 2024-04-26T23:19:45.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.503+0000 7f66056cc640 1 -- 172.21.15.47:0/3911132240 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66000c8580 con 0x7f66000d1240 2024-04-26T23:19:45.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.504+0000 7f66056cc640 1 -- 172.21.15.47:0/3911132240 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66000694f0 con 0x7f66000d1240 2024-04-26T23:19:45.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.504+0000 7f65e77fe640 1 -- 172.21.15.47:0/3911132240 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f65f401a070 con 0x7f66000d1240 2024-04-26T23:19:45.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.504+0000 7f65e77fe640 1 --2- 172.21.15.47:0/3911132240 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f65ec06c8f0 0x7f65ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:45.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.504+0000 7f65e77fe640 1 -- 172.21.15.47:0/3911132240 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6101+0+0 (secure 0 0 0) 0x7f65f408c010 con 0x7f66000d1240 2024-04-26T23:19:45.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.505+0000 7f65feffd640 1 --2- 172.21.15.47:0/3911132240 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f65ec06c8f0 0x7f65ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:45.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.505+0000 7f65feffd640 1 --2- 172.21.15.47:0/3911132240 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f65ec06c8f0 0x7f65ec06edb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f65f0034000 tx=0x7f65f002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:45.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.509+0000 7f65e77fe640 1 -- 172.21.15.47:0/3911132240 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f65f405a230 con 0x7f66000d1240 2024-04-26T23:19:45.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:45.736+0000 7f66056cc640 1 -- 172.21.15.47:0/3911132240 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f66000d2330 con 0x7f66000d1240 2024-04-26T23:19:46.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.743+0000 7f65e77fe640 1 -- 172.21.15.47:0/3911132240 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f65f4059b30 con 0x7f66000d1240 2024-04-26T23:19:46.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.745+0000 7f66056cc640 1 -- 172.21.15.47:0/3911132240 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f65ec06c8f0 msgr2=0x7f65ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:46.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.746+0000 7f66056cc640 1 --2- 172.21.15.47:0/3911132240 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f65ec06c8f0 0x7f65ec06edb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f65f0034000 tx=0x7f65f002e000 comp rx=0 tx=0).stop 2024-04-26T23:19:46.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.746+0000 7f66056cc640 1 -- 172.21.15.47:0/3911132240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66000d1240 msgr2=0x7f660015b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:46.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.746+0000 7f66056cc640 1 --2- 172.21.15.47:0/3911132240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66000d1240 0x7f660015b7a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f65f400ebe0 tx=0x7f65f400c520 comp rx=0 tx=0).stop 2024-04-26T23:19:46.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.746+0000 7f66056cc640 1 -- 172.21.15.47:0/3911132240 shutdown_connections 2024-04-26T23:19:46.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.746+0000 7f66056cc640 1 --2- 172.21.15.47:0/3911132240 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f65ec06c8f0 0x7f65ec06edb0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:46.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.746+0000 7f66056cc640 1 --2- 172.21.15.47:0/3911132240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f66000b2c10 0x7f66000b7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:46.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.746+0000 7f66056cc640 1 --2- 172.21.15.47:0/3911132240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66000d1240 0x7f660015b7a0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:46.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.746+0000 7f66056cc640 1 --2- 172.21.15.47:0/3911132240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66000d0d60 0x7f660006d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:46.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.746+0000 7f66056cc640 1 -- 172.21.15.47:0/3911132240 >> 172.21.15.47:0/3911132240 conn(0x7f66000d2e60 msgr2=0x7f66000b1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:46.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.746+0000 7f66056cc640 1 -- 172.21.15.47:0/3911132240 shutdown_connections 2024-04-26T23:19:46.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.746+0000 7f66056cc640 1 -- 172.21.15.47:0/3911132240 wait complete. 2024-04-26T23:19:46.761 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-26T23:19:46.761 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T23:19:46.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.869+0000 7f0c10afc640 1 Processor -- start 2024-04-26T23:19:46.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.869+0000 7f0c10afc640 1 -- start start 2024-04-26T23:19:46.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.870+0000 7f0c10afc640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0c0c15f770 0x7f0c0c0b1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:46.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.870+0000 7f0c10afc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c0c0d1920 0x7f0c0c0cdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:46.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.870+0000 7f0c10afc640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c0c06e580 con 0x7f0c0c0d0d60 2024-04-26T23:19:46.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.870+0000 7f0c10afc640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f0c0c06f3e0 con 0x7f0c0c15f770 2024-04-26T23:19:46.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.870+0000 7f0c10afc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f0c0c0cd6e0 con 0x7f0c0c0d1920 2024-04-26T23:19:46.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.871+0000 7f0c0a59c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0c0c15f770 0x7f0c0c0b1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:46.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.871+0000 7f0c09d9b640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f0c0c0d0d60 0x7f0c0c0d1210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.47:42530/0 (socket says 172.21.15.47:42530) 2024-04-26T23:19:46.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.871+0000 7f0c0ad9d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c0c0d1920 0x7f0c0c0cdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:46.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.871+0000 7f0c09d9b640 1 -- 172.21.15.47:0/560718326 learned_addr learned my addr 172.21.15.47:0/560718326 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:46.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.871+0000 7f0c0959a640 1 -- 172.21.15.47:0/560718326 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2304161474 0 0) 0x7f0c0c06e580 con 0x7f0c0c0d0d60 2024-04-26T23:19:46.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.871+0000 7f0c0959a640 1 -- 172.21.15.47:0/560718326 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0bf4003520 con 0x7f0c0c0d0d60 2024-04-26T23:19:46.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.871+0000 7f0c0ad9d640 1 -- 172.21.15.47:0/560718326 >> v2:172.21.15.203:3300/0 conn(0x7f0c0c15f770 msgr2=0x7f0c0c0b1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:46.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.871+0000 7f0c0ad9d640 1 --2- 172.21.15.47:0/560718326 >> v2:172.21.15.203:3300/0 conn(0x7f0c0c15f770 0x7f0c0c0b1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:46.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.871+0000 7f0c0ad9d640 1 -- 172.21.15.47:0/560718326 >> v1:172.21.15.203:6789/0 conn(0x7f0c0c0d0d60 legacy=0x7f0c0c0d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:46.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.871+0000 7f0c0ad9d640 1 -- 172.21.15.47:0/560718326 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c0c06e860 con 0x7f0c0c0d1920 2024-04-26T23:19:46.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.871+0000 7f0c0a59c640 1 --2- 172.21.15.47:0/560718326 >> v2:172.21.15.203:3300/0 conn(0x7f0c0c15f770 0x7f0c0c0b1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:19:46.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.872+0000 7f0c0ad9d640 1 --2- 172.21.15.47:0/560718326 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c0c0d1920 0x7f0c0c0cdb70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc009d90 tx=0x7f0bfc02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc03910eec460d81 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:46.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.872+0000 7f0c0959a640 1 -- 172.21.15.47:0/560718326 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc007740 con 0x7f0c0c0d1920 2024-04-26T23:19:46.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.872+0000 7f0c0959a640 1 -- 172.21.15.47:0/560718326 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bfc0078e0 con 0x7f0c0c0d1920 2024-04-26T23:19:46.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.872+0000 7f0c0959a640 1 -- 172.21.15.47:0/560718326 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc007c90 con 0x7f0c0c0d1920 2024-04-26T23:19:46.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.873+0000 7f0c10afc640 1 -- 172.21.15.47:0/560718326 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c0c0d1920 msgr2=0x7f0c0c0cdb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:46.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.873+0000 7f0c10afc640 1 --2- 172.21.15.47:0/560718326 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c0c0d1920 0x7f0c0c0cdb70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc009d90 tx=0x7f0bfc02a660 comp rx=0 tx=0).stop 2024-04-26T23:19:46.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.873+0000 7f0c10afc640 1 -- 172.21.15.47:0/560718326 shutdown_connections 2024-04-26T23:19:46.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.873+0000 7f0c10afc640 1 --2- 172.21.15.47:0/560718326 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c0c0d1920 0x7f0c0c0cdb70 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:46.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.873+0000 7f0c10afc640 1 --2- 172.21.15.47:0/560718326 >> v2:172.21.15.203:3300/0 conn(0x7f0c0c15f770 0x7f0c0c0b1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:46.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.873+0000 7f0c10afc640 1 -- 172.21.15.47:0/560718326 >> 172.21.15.47:0/560718326 conn(0x7f0c0c0d2e60 msgr2=0x7f0c0c0d3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:46.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.873+0000 7f0c10afc640 1 -- 172.21.15.47:0/560718326 shutdown_connections 2024-04-26T23:19:46.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.873+0000 7f0c10afc640 1 -- 172.21.15.47:0/560718326 wait complete. 2024-04-26T23:19:46.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.874+0000 7f0c10afc640 1 Processor -- start 2024-04-26T23:19:46.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.874+0000 7f0c10afc640 1 -- start start 2024-04-26T23:19:46.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.875+0000 7f0c10afc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c0c0d0d60 0x7f0c0c15a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:46.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.875+0000 7f0c10afc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c0c0d1920 0x7f0c0c0b2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:46.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.875+0000 7f0c10afc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c0c15f770 0x7f0c0c0b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:46.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.875+0000 7f0c10afc640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0c0ce230 con 0x7f0c0c15f770 2024-04-26T23:19:46.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.875+0000 7f0c10afc640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0c0ce0b0 con 0x7f0c0c0d0d60 2024-04-26T23:19:46.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.875+0000 7f0c10afc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f0c0c0ce3b0 con 0x7f0c0c0d1920 2024-04-26T23:19:46.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.875+0000 7f0c0ad9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c0c15f770 0x7f0c0c0b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:46.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.875+0000 7f0c0ad9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c0c15f770 0x7f0c0c0b2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58838/0 (socket says 172.21.15.47:58838) 2024-04-26T23:19:46.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.875+0000 7f0c0ad9d640 1 -- 172.21.15.47:0/108780308 learned_addr learned my addr 172.21.15.47:0/108780308 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:46.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.875+0000 7f0c0a59c640 1 --2- 172.21.15.47:0/108780308 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c0c0d0d60 0x7f0c0c15a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:46.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.875+0000 7f0c09d9b640 1 --2- 172.21.15.47:0/108780308 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c0c0d1920 0x7f0c0c0b2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:46.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.875+0000 7f0c0ad9d640 1 -- 172.21.15.47:0/108780308 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c0c0d1920 msgr2=0x7f0c0c0b2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:46.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.875+0000 7f0c0ad9d640 1 --2- 172.21.15.47:0/108780308 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c0c0d1920 0x7f0c0c0b2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:46.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.875+0000 7f0c0ad9d640 1 -- 172.21.15.47:0/108780308 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c0c0d0d60 msgr2=0x7f0c0c15a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:46.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.875+0000 7f0c0ad9d640 1 --2- 172.21.15.47:0/108780308 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c0c0d0d60 0x7f0c0c15a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:46.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.875+0000 7f0c0ad9d640 1 -- 172.21.15.47:0/108780308 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c0c0c86a0 con 0x7f0c0c15f770 2024-04-26T23:19:46.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.875+0000 7f0c0ad9d640 1 --2- 172.21.15.47:0/108780308 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c0c15f770 0x7f0c0c0b2cc0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc02ac50 tx=0x7f0bfc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:46.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.876+0000 7f0bfaffd640 1 -- 172.21.15.47:0/108780308 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc040070 con 0x7f0c0c15f770 2024-04-26T23:19:46.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.876+0000 7f0bfaffd640 1 -- 172.21.15.47:0/108780308 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bfc031c60 con 0x7f0c0c15f770 2024-04-26T23:19:46.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.876+0000 7f0c10afc640 1 -- 172.21.15.47:0/108780308 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c0c0c8980 con 0x7f0c0c15f770 2024-04-26T23:19:46.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.876+0000 7f0bfaffd640 1 -- 172.21.15.47:0/108780308 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc009450 con 0x7f0c0c15f770 2024-04-26T23:19:46.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.876+0000 7f0c10afc640 1 -- 172.21.15.47:0/108780308 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c0c0c8f40 con 0x7f0c0c15f770 2024-04-26T23:19:46.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.877+0000 7f0c10afc640 1 -- 172.21.15.47:0/108780308 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c0c0694f0 con 0x7f0c0c15f770 2024-04-26T23:19:46.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.877+0000 7f0bfaffd640 1 -- 172.21.15.47:0/108780308 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0bfc039040 con 0x7f0c0c15f770 2024-04-26T23:19:46.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.878+0000 7f0bfaffd640 1 --2- 172.21.15.47:0/108780308 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f0bec06c8f0 0x7f0bec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:46.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.878+0000 7f0bfaffd640 1 -- 172.21.15.47:0/108780308 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f0bfc0acf10 con 0x7f0c0c15f770 2024-04-26T23:19:46.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.878+0000 7f0c0a59c640 1 --2- 172.21.15.47:0/108780308 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f0bec06c8f0 0x7f0bec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:46.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.878+0000 7f0c0a59c640 1 --2- 172.21.15.47:0/108780308 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f0bec06c8f0 0x7f0bec06edb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c0b3370 tx=0x7f0c04009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:46.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:46.882+0000 7f0bfaffd640 1 -- 172.21.15.47:0/108780308 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0bfc07ab60 con 0x7f0c0c15f770 2024-04-26T23:19:47.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.107+0000 7f0c10afc640 1 -- 172.21.15.47:0/108780308 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f0c0c06fe50 con 0x7f0c0c15f770 2024-04-26T23:19:47.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.747+0000 7f0bfaffd640 1 -- 172.21.15.47:0/108780308 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7f0bfc07ee60 con 0x7f0c0c15f770 2024-04-26T23:19:47.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.751+0000 7f0c10afc640 1 -- 172.21.15.47:0/108780308 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f0bec06c8f0 msgr2=0x7f0bec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:47.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.751+0000 7f0c10afc640 1 --2- 172.21.15.47:0/108780308 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f0bec06c8f0 0x7f0bec06edb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c0b3370 tx=0x7f0c04009190 comp rx=0 tx=0).stop 2024-04-26T23:19:47.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.751+0000 7f0c10afc640 1 -- 172.21.15.47:0/108780308 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c0c15f770 msgr2=0x7f0c0c0b2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:47.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.751+0000 7f0c10afc640 1 --2- 172.21.15.47:0/108780308 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c0c15f770 0x7f0c0c0b2cc0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc02ac50 tx=0x7f0bfc004a40 comp rx=0 tx=0).stop 2024-04-26T23:19:47.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.751+0000 7f0c10afc640 1 -- 172.21.15.47:0/108780308 shutdown_connections 2024-04-26T23:19:47.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.751+0000 7f0c10afc640 1 --2- 172.21.15.47:0/108780308 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f0bec06c8f0 0x7f0bec06edb0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:47.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.751+0000 7f0c10afc640 1 --2- 172.21.15.47:0/108780308 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c0c15f770 0x7f0c0c0b2cc0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:47.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.751+0000 7f0c10afc640 1 --2- 172.21.15.47:0/108780308 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c0c0d1920 0x7f0c0c0b2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:47.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.751+0000 7f0c10afc640 1 --2- 172.21.15.47:0/108780308 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c0c0d0d60 0x7f0c0c15a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:47.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.751+0000 7f0c10afc640 1 -- 172.21.15.47:0/108780308 >> 172.21.15.47:0/108780308 conn(0x7f0c0c0d2e60 msgr2=0x7f0c0c0b3da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:47.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.751+0000 7f0c10afc640 1 -- 172.21.15.47:0/108780308 shutdown_connections 2024-04-26T23:19:47.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.751+0000 7f0c10afc640 1 -- 172.21.15.47:0/108780308 wait complete. 2024-04-26T23:19:47.766 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T23:19:47.766 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:19:47.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.870+0000 7f1091a1d640 1 Processor -- start 2024-04-26T23:19:47.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.870+0000 7f1091a1d640 1 -- start start 2024-04-26T23:19:47.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.871+0000 7f1091a1d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f108c0d0070 0x7f108c0d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:47.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.871+0000 7f1091a1d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f108c06e270 0x7f108c0ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:47.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.871+0000 7f1091a1d640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f108c0d2420 con 0x7f108c06d560 2024-04-26T23:19:47.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.871+0000 7f1091a1d640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f108c162d00 con 0x7f108c0d0070 2024-04-26T23:19:47.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.871+0000 7f1091a1d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f108c0d2d10 con 0x7f108c06e270 2024-04-26T23:19:47.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.871+0000 7f108affd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f108c0d0070 0x7f108c0d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:47.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.871+0000 7f108a7fc640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f108c06d560 0x7f108c06db60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36100/0 (socket says 172.21.15.47:36100) 2024-04-26T23:19:47.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.871+0000 7f108a7fc640 1 -- 172.21.15.47:0/2042292461 learned_addr learned my addr 172.21.15.47:0/2042292461 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:47.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.872+0000 7f108b7fe640 1 --2- 172.21.15.47:0/2042292461 >> v2:172.21.15.203:3300/0 conn(0x7f108c06e270 0x7f108c0ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:47.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.872+0000 7f108affd640 1 -- 172.21.15.47:0/2042292461 >> v2:172.21.15.203:3300/0 conn(0x7f108c06e270 msgr2=0x7f108c0ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:47.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.872+0000 7f108affd640 1 --2- 172.21.15.47:0/2042292461 >> v2:172.21.15.203:3300/0 conn(0x7f108c06e270 0x7f108c0ce190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:47.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.872+0000 7f108affd640 1 -- 172.21.15.47:0/2042292461 >> v1:172.21.15.47:6789/0 conn(0x7f108c06d560 legacy=0x7f108c06db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:47.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.872+0000 7f1089ffb640 1 -- 172.21.15.47:0/2042292461 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3196489959 0 0) 0x7f108c0d2420 con 0x7f108c06d560 2024-04-26T23:19:47.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.872+0000 7f108affd640 1 -- 172.21.15.47:0/2042292461 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f108c0cd6b0 con 0x7f108c0d0070 2024-04-26T23:19:47.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.872+0000 7f108b7fe640 1 --2- 172.21.15.47:0/2042292461 >> v2:172.21.15.203:3300/0 conn(0x7f108c06e270 0x7f108c0ce190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:19:47.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.872+0000 7f108affd640 1 --2- 172.21.15.47:0/2042292461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f108c0d0070 0x7f108c0d1d40 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f108c073530 tx=0x7f108402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30b04e954b2decf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:47.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.873+0000 7f1089ffb640 1 -- 172.21.15.47:0/2042292461 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1084037070 con 0x7f108c0d0070 2024-04-26T23:19:47.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.873+0000 7f1089ffb640 1 -- 172.21.15.47:0/2042292461 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1084032440 con 0x7f108c0d0070 2024-04-26T23:19:47.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.873+0000 7f1089ffb640 1 -- 172.21.15.47:0/2042292461 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1084032810 con 0x7f108c0d0070 2024-04-26T23:19:47.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.873+0000 7f1091a1d640 1 -- 172.21.15.47:0/2042292461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f108c0d0070 msgr2=0x7f108c0d1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:47.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.873+0000 7f1091a1d640 1 --2- 172.21.15.47:0/2042292461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f108c0d0070 0x7f108c0d1d40 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f108c073530 tx=0x7f108402a1a0 comp rx=0 tx=0).stop 2024-04-26T23:19:47.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.873+0000 7f1091a1d640 1 -- 172.21.15.47:0/2042292461 shutdown_connections 2024-04-26T23:19:47.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.873+0000 7f1091a1d640 1 --2- 172.21.15.47:0/2042292461 >> v2:172.21.15.203:3300/0 conn(0x7f108c06e270 0x7f108c0ce190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:47.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.873+0000 7f1091a1d640 1 --2- 172.21.15.47:0/2042292461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f108c0d0070 0x7f108c0d1d40 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:47.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.873+0000 7f1091a1d640 1 -- 172.21.15.47:0/2042292461 >> 172.21.15.47:0/2042292461 conn(0x7f108c0d2ec0 msgr2=0x7f108c0d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:47.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.873+0000 7f1091a1d640 1 -- 172.21.15.47:0/2042292461 shutdown_connections 2024-04-26T23:19:47.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.873+0000 7f1091a1d640 1 -- 172.21.15.47:0/2042292461 wait complete. 2024-04-26T23:19:47.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.874+0000 7f1091a1d640 1 Processor -- start 2024-04-26T23:19:47.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.875+0000 7f1091a1d640 1 -- start start 2024-04-26T23:19:47.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.875+0000 7f1091a1d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f108c06d560 0x7f108c0c9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:47.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.875+0000 7f1091a1d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f108c06e270 0x7f108c0cb660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:47.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.875+0000 7f1091a1d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f108c0d0070 0x7f108c15d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:47.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.875+0000 7f1091a1d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f108c0ce6d0 con 0x7f108c06e270 2024-04-26T23:19:47.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.875+0000 7f1091a1d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f108c0cdc70 con 0x7f108c0d0070 2024-04-26T23:19:47.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.875+0000 7f1091a1d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f108c0ce850 con 0x7f108c06d560 2024-04-26T23:19:47.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.875+0000 7f108a7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f108c06e270 0x7f108c0cb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:47.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.875+0000 7f108a7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f108c06e270 0x7f108c0cb660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58860/0 (socket says 172.21.15.47:58860) 2024-04-26T23:19:47.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.875+0000 7f108a7fc640 1 -- 172.21.15.47:0/229776407 learned_addr learned my addr 172.21.15.47:0/229776407 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:47.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.876+0000 7f108b7fe640 1 --2- 172.21.15.47:0/229776407 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f108c0d0070 0x7f108c15d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:47.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.876+0000 7f108affd640 1 --2- 172.21.15.47:0/229776407 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f108c06d560 0x7f108c0c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:47.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.876+0000 7f108a7fc640 1 -- 172.21.15.47:0/229776407 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f108c06d560 msgr2=0x7f108c0c9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:47.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.876+0000 7f108a7fc640 1 --2- 172.21.15.47:0/229776407 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f108c06d560 0x7f108c0c9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:47.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.876+0000 7f108a7fc640 1 -- 172.21.15.47:0/229776407 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f108c0d0070 msgr2=0x7f108c15d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:47.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.876+0000 7f108a7fc640 1 --2- 172.21.15.47:0/229776407 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f108c0d0070 0x7f108c15d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:47.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.876+0000 7f108a7fc640 1 -- 172.21.15.47:0/229776407 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f108c0c7f90 con 0x7f108c06e270 2024-04-26T23:19:47.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.876+0000 7f108a7fc640 1 --2- 172.21.15.47:0/229776407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f108c06e270 0x7f108c0cb660 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f107400c4d0 tx=0x7f107400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:47.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.876+0000 7f1083fff640 1 -- 172.21.15.47:0/229776407 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1074018070 con 0x7f108c06e270 2024-04-26T23:19:47.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.876+0000 7f1083fff640 1 -- 172.21.15.47:0/229776407 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1074007500 con 0x7f108c06e270 2024-04-26T23:19:47.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.876+0000 7f1091a1d640 1 -- 172.21.15.47:0/229776407 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f108c0c8270 con 0x7f108c06e270 2024-04-26T23:19:47.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.876+0000 7f1083fff640 1 -- 172.21.15.47:0/229776407 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1074004a20 con 0x7f108c06e270 2024-04-26T23:19:47.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.876+0000 7f1091a1d640 1 -- 172.21.15.47:0/229776407 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f108c0c8830 con 0x7f108c06e270 2024-04-26T23:19:47.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.877+0000 7f1091a1d640 1 -- 172.21.15.47:0/229776407 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f108c0694f0 con 0x7f108c06e270 2024-04-26T23:19:47.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.878+0000 7f1083fff640 1 -- 172.21.15.47:0/229776407 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f107401b070 con 0x7f108c06e270 2024-04-26T23:19:47.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.878+0000 7f1083fff640 1 --2- 172.21.15.47:0/229776407 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f106c06c8f0 0x7f106c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:47.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.878+0000 7f1083fff640 1 -- 172.21.15.47:0/229776407 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f107408aff0 con 0x7f108c06e270 2024-04-26T23:19:47.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.878+0000 7f108affd640 1 --2- 172.21.15.47:0/229776407 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f106c06c8f0 0x7f106c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:47.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.879+0000 7f108affd640 1 --2- 172.21.15.47:0/229776407 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f106c06c8f0 0x7f106c06edb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1084034000 tx=0x7f108402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:47.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:47.882+0000 7f1083fff640 1 -- 172.21.15.47:0/229776407 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1074059290 con 0x7f108c06e270 2024-04-26T23:19:48.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.107+0000 7f1091a1d640 1 -- 172.21.15.47:0/229776407 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f108c0d0790 con 0x7f108c06e270 2024-04-26T23:19:48.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.108+0000 7f1083fff640 1 -- 172.21.15.47:0/229776407 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5189 (secure 0 0 0) 0x7f1074058b90 con 0x7f108c06e270 2024-04-26T23:19:48.110 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:19:48.110 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5080,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/1970124054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1970124054},{"type":"v1","addr":"172.21.15.203:6836","nonce":1970124054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5310,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/889573201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":889573201},{"type":"v1","addr":"172.21.15.47:6834","nonce":889573201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5316,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3898612760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":3898612760},{"type":"v1","addr":"172.21.15.47:6835","nonce":3898612760}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:19:44.091353+0000","modified":"2024-04-26T23:19:47.109087+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":14738},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14738":{"gid":14738,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/2526477051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":2526477051},{"type":"v1","addr":"172.21.15.203:6837","nonce":2526477051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":14738,"qdb_cluster":[14738]},"id":2}]} 2024-04-26T23:19:48.110 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 18 2024-04-26T23:19:48.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.110+0000 7f1091a1d640 1 -- 172.21.15.47:0/229776407 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f106c06c8f0 msgr2=0x7f106c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:48.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.110+0000 7f1091a1d640 1 --2- 172.21.15.47:0/229776407 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f106c06c8f0 0x7f106c06edb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1084034000 tx=0x7f108402e000 comp rx=0 tx=0).stop 2024-04-26T23:19:48.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.110+0000 7f1091a1d640 1 -- 172.21.15.47:0/229776407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f108c06e270 msgr2=0x7f108c0cb660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:48.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.110+0000 7f1091a1d640 1 --2- 172.21.15.47:0/229776407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f108c06e270 0x7f108c0cb660 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f107400c4d0 tx=0x7f107400c990 comp rx=0 tx=0).stop 2024-04-26T23:19:48.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.110+0000 7f1091a1d640 1 -- 172.21.15.47:0/229776407 shutdown_connections 2024-04-26T23:19:48.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.110+0000 7f1091a1d640 1 --2- 172.21.15.47:0/229776407 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f106c06c8f0 0x7f106c06edb0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:48.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.110+0000 7f1091a1d640 1 --2- 172.21.15.47:0/229776407 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f108c0d0070 0x7f108c15d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:48.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.110+0000 7f1091a1d640 1 --2- 172.21.15.47:0/229776407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f108c06e270 0x7f108c0cb660 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:48.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.110+0000 7f1091a1d640 1 --2- 172.21.15.47:0/229776407 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f108c06d560 0x7f108c0c9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:48.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.110+0000 7f1091a1d640 1 -- 172.21.15.47:0/229776407 >> 172.21.15.47:0/229776407 conn(0x7f108c0d2ec0 msgr2=0x7f108c0d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:48.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.110+0000 7f1091a1d640 1 -- 172.21.15.47:0/229776407 shutdown_connections 2024-04-26T23:19:48.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.110+0000 7f1091a1d640 1 -- 172.21.15.47:0/229776407 wait complete. 2024-04-26T23:19:48.123 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:19:48.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.221+0000 7f25fbaf1640 1 Processor -- start 2024-04-26T23:19:48.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.221+0000 7f25fbaf1640 1 -- start start 2024-04-26T23:19:48.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.222+0000 7f25fbaf1640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f25f40d06f0 0x7f25f40d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:48.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.222+0000 7f25fbaf1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f25f40b2850 0x7f25f40b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:48.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.222+0000 7f25fbaf1640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25f40d0ba0 con 0x7f25f415f3a0 2024-04-26T23:19:48.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.222+0000 7f25fbaf1640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f25f40d0e20 con 0x7f25f40d06f0 2024-04-26T23:19:48.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.222+0000 7f25fbaf1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f25f40b1d70 con 0x7f25f40b2850 2024-04-26T23:19:48.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.222+0000 7f25f988d640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f25f415f3a0 0x7f25f40d1770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36106/0 (socket says 172.21.15.47:36106) 2024-04-26T23:19:48.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.222+0000 7f25f988d640 1 -- 172.21.15.47:0/1182568114 learned_addr learned my addr 172.21.15.47:0/1182568114 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:48.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.222+0000 7f25f908c640 1 --2- 172.21.15.47:0/1182568114 >> v2:172.21.15.203:3300/0 conn(0x7f25f40d06f0 0x7f25f40d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:48.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.223+0000 7f25fa08e640 1 --2- 172.21.15.47:0/1182568114 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f25f40b2850 0x7f25f40b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:48.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.223+0000 7f25f888b640 1 -- 172.21.15.47:0/1182568114 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2320427325 0 0) 0x7f25e4001490 con 0x7f25f415f3a0 2024-04-26T23:19:48.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.223+0000 7f25f888b640 1 -- 172.21.15.47:0/1182568114 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f25dc003520 con 0x7f25f415f3a0 2024-04-26T23:19:48.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.223+0000 7f25f908c640 1 -- 172.21.15.47:0/1182568114 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f25f40b2850 msgr2=0x7f25f40b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:48.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.223+0000 7f25f908c640 1 --2- 172.21.15.47:0/1182568114 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f25f40b2850 0x7f25f40b2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:48.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.223+0000 7f25f908c640 1 -- 172.21.15.47:0/1182568114 >> v1:172.21.15.47:6789/0 conn(0x7f25f415f3a0 legacy=0x7f25f40d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:48.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.223+0000 7f25f888b640 1 -- 172.21.15.47:0/1182568114 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2245547579 0 0) 0x7f25dc003520 con 0x7f25f415f3a0 2024-04-26T23:19:48.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.223+0000 7f25f908c640 1 -- 172.21.15.47:0/1182568114 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25f40b2050 con 0x7f25f40d06f0 2024-04-26T23:19:48.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.224+0000 7f25f908c640 1 --2- 172.21.15.47:0/1182568114 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25f40d06f0 0x7f25f40d1e80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f25e80097a0 tx=0x7f25e802a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2938a3c8662f9f0a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:48.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.224+0000 7f25f888b640 1 -- 172.21.15.47:0/1182568114 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e8037070 con 0x7f25f40d06f0 2024-04-26T23:19:48.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.224+0000 7f25f888b640 1 -- 172.21.15.47:0/1182568114 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25e802aba0 con 0x7f25f40d06f0 2024-04-26T23:19:48.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.224+0000 7f25f888b640 1 -- 172.21.15.47:0/1182568114 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e4001bc0 con 0x7f25f40d06f0 2024-04-26T23:19:48.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.225+0000 7f25fbaf1640 1 -- 172.21.15.47:0/1182568114 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25f40d06f0 msgr2=0x7f25f40d1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:48.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.225+0000 7f25fbaf1640 1 --2- 172.21.15.47:0/1182568114 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25f40d06f0 0x7f25f40d1e80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f25e80097a0 tx=0x7f25e802a110 comp rx=0 tx=0).stop 2024-04-26T23:19:48.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.225+0000 7f25fbaf1640 1 -- 172.21.15.47:0/1182568114 shutdown_connections 2024-04-26T23:19:48.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.225+0000 7f25fbaf1640 1 --2- 172.21.15.47:0/1182568114 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f25f40b2850 0x7f25f40b2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:48.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.225+0000 7f25fbaf1640 1 --2- 172.21.15.47:0/1182568114 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25f40d06f0 0x7f25f40d1e80 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:48.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.225+0000 7f25fbaf1640 1 -- 172.21.15.47:0/1182568114 >> 172.21.15.47:0/1182568114 conn(0x7f25f40d4f90 msgr2=0x7f25f40d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:48.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.225+0000 7f25fbaf1640 1 -- 172.21.15.47:0/1182568114 shutdown_connections 2024-04-26T23:19:48.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.225+0000 7f25fbaf1640 1 -- 172.21.15.47:0/1182568114 wait complete. 2024-04-26T23:19:48.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.226+0000 7f25fbaf1640 1 Processor -- start 2024-04-26T23:19:48.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.226+0000 7f25fbaf1640 1 -- start start 2024-04-26T23:19:48.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.227+0000 7f25fbaf1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f25f40b2850 0x7f25f4151b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:48.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.227+0000 7f25fbaf1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25f40d06f0 0x7f25f40ce8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:48.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.227+0000 7f25fbaf1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25f415f3a0 0x7f25f40cee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:48.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.227+0000 7f25fbaf1640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f25f415f900 con 0x7f25f40d06f0 2024-04-26T23:19:48.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.227+0000 7f25fbaf1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f25f415f780 con 0x7f25f415f3a0 2024-04-26T23:19:48.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.227+0000 7f25fbaf1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f25f40d23c0 con 0x7f25f40b2850 2024-04-26T23:19:48.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.227+0000 7f25f908c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25f40d06f0 0x7f25f40ce8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:48.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.227+0000 7f25f908c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25f40d06f0 0x7f25f40ce8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58868/0 (socket says 172.21.15.47:58868) 2024-04-26T23:19:48.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.227+0000 7f25f908c640 1 -- 172.21.15.47:0/516447406 learned_addr learned my addr 172.21.15.47:0/516447406 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:48.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.227+0000 7f25f988d640 1 --2- 172.21.15.47:0/516447406 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f25f40b2850 0x7f25f4151b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:48.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.227+0000 7f25f908c640 1 -- 172.21.15.47:0/516447406 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f25f40b2850 msgr2=0x7f25f4151b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:48.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.227+0000 7f25fa08e640 1 --2- 172.21.15.47:0/516447406 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25f415f3a0 0x7f25f40cee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:48.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.227+0000 7f25f908c640 1 --2- 172.21.15.47:0/516447406 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f25f40b2850 0x7f25f4151b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:48.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.227+0000 7f25f908c640 1 -- 172.21.15.47:0/516447406 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25f415f3a0 msgr2=0x7f25f40cee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:48.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.227+0000 7f25f908c640 1 --2- 172.21.15.47:0/516447406 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25f415f3a0 0x7f25f40cee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:48.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.227+0000 7f25f908c640 1 -- 172.21.15.47:0/516447406 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25f40cb630 con 0x7f25f40d06f0 2024-04-26T23:19:48.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.227+0000 7f25f908c640 1 --2- 172.21.15.47:0/516447406 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25f40d06f0 0x7f25f40ce8e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f25e80098c0 tx=0x7f25e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:48.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.227+0000 7f25f27fc640 1 -- 172.21.15.47:0/516447406 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e8041020 con 0x7f25f40d06f0 2024-04-26T23:19:48.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.227+0000 7f25f27fc640 1 -- 172.21.15.47:0/516447406 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25e803d620 con 0x7f25f40d06f0 2024-04-26T23:19:48.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.227+0000 7f25fbaf1640 1 -- 172.21.15.47:0/516447406 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25f41524a0 con 0x7f25f40d06f0 2024-04-26T23:19:48.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.227+0000 7f25f27fc640 1 -- 172.21.15.47:0/516447406 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e8037040 con 0x7f25f40d06f0 2024-04-26T23:19:48.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.227+0000 7f25fbaf1640 1 -- 172.21.15.47:0/516447406 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25f4152a60 con 0x7f25f40d06f0 2024-04-26T23:19:48.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.228+0000 7f25fbaf1640 1 -- 172.21.15.47:0/516447406 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25f40694f0 con 0x7f25f40d06f0 2024-04-26T23:19:48.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.229+0000 7f25f27fc640 1 -- 172.21.15.47:0/516447406 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f25e8044020 con 0x7f25f40d06f0 2024-04-26T23:19:48.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.229+0000 7f25f27fc640 1 --2- 172.21.15.47:0/516447406 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f25e006c8f0 0x7f25e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:48.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.229+0000 7f25f27fc640 1 -- 172.21.15.47:0/516447406 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f25e80ac310 con 0x7f25f40d06f0 2024-04-26T23:19:48.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.230+0000 7f25f988d640 1 --2- 172.21.15.47:0/516447406 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f25e006c8f0 0x7f25e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:48.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.230+0000 7f25f988d640 1 --2- 172.21.15.47:0/516447406 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f25e006c8f0 0x7f25e006edb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f25e4002230 tx=0x7f25e4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:48.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.233+0000 7f25f27fc640 1 -- 172.21.15.47:0/516447406 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f25e804b030 con 0x7f25f40d06f0 2024-04-26T23:19:48.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.420+0000 7f25fbaf1640 1 -- 172.21.15.47:0/516447406 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f25f40b6840 con 0x7f25f40d06f0 2024-04-26T23:19:48.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.421+0000 7f25f27fc640 1 -- 172.21.15.47:0/516447406 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16906 (secure 0 0 0) 0x7f25e807a510 con 0x7f25f40d06f0 2024-04-26T23:19:48.424 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:19:48.424 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":38,"fsid":"399006d3-4db1-43e6-ba81-a79a276eabfd","created":"2024-04-26T23:18:06.731211+0000","modified":"2024-04-26T23:19:45.729640+0000","last_up_change":"2024-04-26T23:18:17.808736+0000","last_in_change":"2024-04-26T23:18:08.945859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:18:16.937492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:19:42.351747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-26T23:19:43.037763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ee091567-fea0-4a14-bf4e-0e16c20bb2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6819","nonce":326520779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6821","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6823","nonce":326520779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6829","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6831","nonce":326520779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6825","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6827","nonce":326520779}]},"public_addr":"172.21.15.47:6819/326520779","cluster_addr":"172.21.15.47:6823/326520779","heartbeat_back_addr":"172.21.15.47:6831/326520779","heartbeat_front_addr":"172.21.15.47:6827/326520779","state":["exists","up"]},{"osd":1,"uuid":"c14440a0-bba5-4931-a6db-b736c08e869b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6809","nonce":1912610931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6811","nonce":1912610931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6815","nonce":1912610931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6813","nonce":1912610931}]},"public_addr":"172.21.15.47:6809/1912610931","cluster_addr":"172.21.15.47:6811/1912610931","heartbeat_back_addr":"172.21.15.47:6815/1912610931","heartbeat_front_addr":"172.21.15.47:6813/1912610931","state":["exists","up"]},{"osd":2,"uuid":"3940f8de-c9b2-4cd3-a911-9c301729eb30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6801","nonce":2569869305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6803","nonce":2569869305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6807","nonce":2569869305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6805","nonce":2569869305}]},"public_addr":"172.21.15.47:6801/2569869305","cluster_addr":"172.21.15.47:6803/2569869305","heartbeat_back_addr":"172.21.15.47:6807/2569869305","heartbeat_front_addr":"172.21.15.47:6805/2569869305","state":["exists","up"]},{"osd":3,"uuid":"1286a67e-8958-426a-8c2e-bc85498c619c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6818","nonce":2791411683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6822","nonce":2791411683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6830","nonce":2791411683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6826","nonce":2791411683}]},"public_addr":"172.21.15.47:6818/2791411683","cluster_addr":"172.21.15.47:6822/2791411683","heartbeat_back_addr":"172.21.15.47:6830/2791411683","heartbeat_front_addr":"172.21.15.47:6826/2791411683","state":["exists","up"]},{"osd":4,"uuid":"d8ee2c20-e5b7-46c2-9999-b234070126cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6809","nonce":441313104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6811","nonce":441313104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6815","nonce":441313104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6813","nonce":441313104}]},"public_addr":"172.21.15.203:6809/441313104","cluster_addr":"172.21.15.203:6811/441313104","heartbeat_back_addr":"172.21.15.203:6815/441313104","heartbeat_front_addr":"172.21.15.203:6813/441313104","state":["exists","up"]},{"osd":5,"uuid":"b453a8f0-f324-4840-9ae0-d67bdadb9f8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6801","nonce":56838444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6803","nonce":56838444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6807","nonce":56838444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6805","nonce":56838444}]},"public_addr":"172.21.15.203:6801/56838444","cluster_addr":"172.21.15.203:6803/56838444","heartbeat_back_addr":"172.21.15.203:6807/56838444","heartbeat_front_addr":"172.21.15.203:6805/56838444","state":["exists","up"]},{"osd":6,"uuid":"a6d31951-2d4b-4e8d-a510-6283b01dc158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6825","nonce":2738517080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6827","nonce":2738517080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6831","nonce":2738517080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6829","nonce":2738517080}]},"public_addr":"172.21.15.203:6825/2738517080","cluster_addr":"172.21.15.203:6827/2738517080","heartbeat_back_addr":"172.21.15.203:6831/2738517080","heartbeat_front_addr":"172.21.15.203:6829/2738517080","state":["exists","up"]},{"osd":7,"uuid":"7119b8c0-fe96-402d-9e75-7b509d9d44d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6817","nonce":1315565058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6819","nonce":1315565058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6823","nonce":1315565058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6821","nonce":1315565058}]},"public_addr":"172.21.15.203:6817/1315565058","cluster_addr":"172.21.15.203:6819/1315565058","heartbeat_back_addr":"172.21.15.203:6823/1315565058","heartbeat_front_addr":"172.21.15.203:6821/1315565058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.121921+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.137494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.084814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.111593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.072430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.125066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.097867+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.057061+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:19:48.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.423+0000 7f25fbaf1640 1 -- 172.21.15.47:0/516447406 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f25e006c8f0 msgr2=0x7f25e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:48.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.423+0000 7f25fbaf1640 1 --2- 172.21.15.47:0/516447406 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f25e006c8f0 0x7f25e006edb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f25e4002230 tx=0x7f25e4009040 comp rx=0 tx=0).stop 2024-04-26T23:19:48.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.423+0000 7f25fbaf1640 1 -- 172.21.15.47:0/516447406 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25f40d06f0 msgr2=0x7f25f40ce8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:48.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.423+0000 7f25fbaf1640 1 --2- 172.21.15.47:0/516447406 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25f40d06f0 0x7f25f40ce8e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f25e80098c0 tx=0x7f25e80046c0 comp rx=0 tx=0).stop 2024-04-26T23:19:48.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.423+0000 7f25fbaf1640 1 -- 172.21.15.47:0/516447406 shutdown_connections 2024-04-26T23:19:48.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.423+0000 7f25fbaf1640 1 --2- 172.21.15.47:0/516447406 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f25e006c8f0 0x7f25e006edb0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:48.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.423+0000 7f25fbaf1640 1 --2- 172.21.15.47:0/516447406 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25f415f3a0 0x7f25f40cee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:48.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.423+0000 7f25fbaf1640 1 --2- 172.21.15.47:0/516447406 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25f40d06f0 0x7f25f40ce8e0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:48.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.423+0000 7f25fbaf1640 1 --2- 172.21.15.47:0/516447406 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f25f40b2850 0x7f25f4151b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:48.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.423+0000 7f25fbaf1640 1 -- 172.21.15.47:0/516447406 >> 172.21.15.47:0/516447406 conn(0x7f25f40d4f90 msgr2=0x7f25f406e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:48.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.424+0000 7f25fbaf1640 1 -- 172.21.15.47:0/516447406 shutdown_connections 2024-04-26T23:19:48.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.424+0000 7f25fbaf1640 1 -- 172.21.15.47:0/516447406 wait complete. 2024-04-26T23:19:48.436 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T23:19:48.437 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T23:19:48.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.535+0000 7f05e7b40640 1 Processor -- start 2024-04-26T23:19:48.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.535+0000 7f05e7b40640 1 -- start start 2024-04-26T23:19:48.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.536+0000 7f05e7b40640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05e0162810 0x7f05e0070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:48.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.536+0000 7f05e7b40640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f05e00b2340 0x7f05e00cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:48.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.536+0000 7f05e7b40640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f05e00cd930 0x7f05e00d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:48.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.536+0000 7f05e7b40640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f05e00d0750 con 0x7f05e00b2340 2024-04-26T23:19:48.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.536+0000 7f05e7b40640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f05e00d16a0 con 0x7f05e00cd930 2024-04-26T23:19:48.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.536+0000 7f05e7b40640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f05e00d10d0 con 0x7f05e0162810 2024-04-26T23:19:48.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.536+0000 7f05e50db640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f05e00b2340 0x7f05e00cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:48.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.536+0000 7f05e50db640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f05e00b2340 0x7f05e00cd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58882/0 (socket says 172.21.15.47:58882) 2024-04-26T23:19:48.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.536+0000 7f05e50db640 1 -- 172.21.15.47:0/2070126461 learned_addr learned my addr 172.21.15.47:0/2070126461 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:48.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.537+0000 7f05e60dd640 1 --2- 172.21.15.47:0/2070126461 >> v2:172.21.15.203:3300/0 conn(0x7f05e00cd930 0x7f05e00d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:48.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.537+0000 7f05e58dc640 1 --2- 172.21.15.47:0/2070126461 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05e0162810 0x7f05e0070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:48.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.537+0000 7f05e50db640 1 -- 172.21.15.47:0/2070126461 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05e0162810 msgr2=0x7f05e0070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:48.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.537+0000 7f05e50db640 1 --2- 172.21.15.47:0/2070126461 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05e0162810 0x7f05e0070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:48.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.537+0000 7f05e50db640 1 -- 172.21.15.47:0/2070126461 >> v2:172.21.15.203:3300/0 conn(0x7f05e00cd930 msgr2=0x7f05e00d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:48.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.537+0000 7f05e50db640 1 --2- 172.21.15.47:0/2070126461 >> v2:172.21.15.203:3300/0 conn(0x7f05e00cd930 0x7f05e00d1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:48.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.537+0000 7f05e50db640 1 -- 172.21.15.47:0/2070126461 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05e00d1370 con 0x7f05e00b2340 2024-04-26T23:19:48.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.537+0000 7f05e58dc640 1 --2- 172.21.15.47:0/2070126461 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05e0162810 0x7f05e0070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:19:48.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.537+0000 7f05e50db640 1 --2- 172.21.15.47:0/2070126461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05e00b2340 0x7f05e00cd3f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f05e00b2250 tx=0x7f05d002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a76dcbfd9525be3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:48.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.537+0000 7f05e48da640 1 -- 172.21.15.47:0/2070126461 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d0037070 con 0x7f05e00b2340 2024-04-26T23:19:48.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.538+0000 7f05e48da640 1 -- 172.21.15.47:0/2070126461 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05d0032440 con 0x7f05e00b2340 2024-04-26T23:19:48.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.538+0000 7f05e48da640 1 -- 172.21.15.47:0/2070126461 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d0032810 con 0x7f05e00b2340 2024-04-26T23:19:48.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.538+0000 7f05e7b40640 1 -- 172.21.15.47:0/2070126461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05e00b2340 msgr2=0x7f05e00cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:48.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.538+0000 7f05e7b40640 1 --2- 172.21.15.47:0/2070126461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05e00b2340 0x7f05e00cd3f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f05e00b2250 tx=0x7f05d002a1e0 comp rx=0 tx=0).stop 2024-04-26T23:19:48.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.538+0000 7f05e7b40640 1 -- 172.21.15.47:0/2070126461 shutdown_connections 2024-04-26T23:19:48.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.538+0000 7f05e7b40640 1 --2- 172.21.15.47:0/2070126461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05e00b2340 0x7f05e00cd3f0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:48.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.538+0000 7f05e7b40640 1 --2- 172.21.15.47:0/2070126461 >> v2:172.21.15.203:3300/0 conn(0x7f05e00cd930 0x7f05e00d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:48.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.538+0000 7f05e7b40640 1 --2- 172.21.15.47:0/2070126461 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05e0162810 0x7f05e0070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:48.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.538+0000 7f05e7b40640 1 -- 172.21.15.47:0/2070126461 >> 172.21.15.47:0/2070126461 conn(0x7f05e00d2da0 msgr2=0x7f05e00d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:48.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.538+0000 7f05e7b40640 1 -- 172.21.15.47:0/2070126461 shutdown_connections 2024-04-26T23:19:48.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.538+0000 7f05e7b40640 1 -- 172.21.15.47:0/2070126461 wait complete. 2024-04-26T23:19:48.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.539+0000 7f05e7b40640 1 Processor -- start 2024-04-26T23:19:48.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.539+0000 7f05e7b40640 1 -- start start 2024-04-26T23:19:48.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.540+0000 7f05e7b40640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05e00b2340 0x7f05e00cc520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:48.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.540+0000 7f05e7b40640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05e00cd930 0x7f05e0149b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:48.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.540+0000 7f05e7b40640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05e0162810 0x7f05e015d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:48.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.540+0000 7f05e7b40640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f05e0070890 con 0x7f05e00b2340 2024-04-26T23:19:48.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.540+0000 7f05e7b40640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f05e00d0470 con 0x7f05e00cd930 2024-04-26T23:19:48.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.540+0000 7f05e7b40640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f05e00d2070 con 0x7f05e0162810 2024-04-26T23:19:48.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.540+0000 7f05e58dc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05e00b2340 0x7f05e00cc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:48.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.540+0000 7f05e58dc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05e00b2340 0x7f05e00cc520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58892/0 (socket says 172.21.15.47:58892) 2024-04-26T23:19:48.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.540+0000 7f05e58dc640 1 -- 172.21.15.47:0/556480945 learned_addr learned my addr 172.21.15.47:0/556480945 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:48.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.540+0000 7f05e50db640 1 --2- 172.21.15.47:0/556480945 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05e00cd930 0x7f05e0149b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:48.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.540+0000 7f05e60dd640 1 --2- 172.21.15.47:0/556480945 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05e0162810 0x7f05e015d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:48.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.541+0000 7f05e58dc640 1 -- 172.21.15.47:0/556480945 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05e0162810 msgr2=0x7f05e015d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:48.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.541+0000 7f05e58dc640 1 --2- 172.21.15.47:0/556480945 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05e0162810 0x7f05e015d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:48.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.541+0000 7f05e58dc640 1 -- 172.21.15.47:0/556480945 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05e00cd930 msgr2=0x7f05e0149b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:48.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.541+0000 7f05e58dc640 1 --2- 172.21.15.47:0/556480945 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05e00cd930 0x7f05e0149b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:48.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.541+0000 7f05e58dc640 1 -- 172.21.15.47:0/556480945 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05e00cb480 con 0x7f05e00b2340 2024-04-26T23:19:48.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.541+0000 7f05e58dc640 1 --2- 172.21.15.47:0/556480945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05e00b2340 0x7f05e00cc520 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f05dc00c7f0 tx=0x7f05dc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:48.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.541+0000 7f05d67fc640 1 -- 172.21.15.47:0/556480945 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05dc015070 con 0x7f05e00b2340 2024-04-26T23:19:48.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.541+0000 7f05d67fc640 1 -- 172.21.15.47:0/556480945 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05dc007820 con 0x7f05e00b2340 2024-04-26T23:19:48.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.541+0000 7f05e7b40640 1 -- 172.21.15.47:0/556480945 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05e00cb760 con 0x7f05e00b2340 2024-04-26T23:19:48.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.541+0000 7f05d67fc640 1 -- 172.21.15.47:0/556480945 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05dc007bd0 con 0x7f05e00b2340 2024-04-26T23:19:48.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.541+0000 7f05e7b40640 1 -- 172.21.15.47:0/556480945 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05e00cbd20 con 0x7f05e00b2340 2024-04-26T23:19:48.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.542+0000 7f05e7b40640 1 -- 172.21.15.47:0/556480945 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05e00694f0 con 0x7f05e00b2340 2024-04-26T23:19:48.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.542+0000 7f05d67fc640 1 -- 172.21.15.47:0/556480945 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f05dc018070 con 0x7f05e00b2340 2024-04-26T23:19:48.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.543+0000 7f05d67fc640 1 --2- 172.21.15.47:0/556480945 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f05c406c8f0 0x7f05c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:48.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.543+0000 7f05d67fc640 1 -- 172.21.15.47:0/556480945 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f05dc08be50 con 0x7f05e00b2340 2024-04-26T23:19:48.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.543+0000 7f05e50db640 1 --2- 172.21.15.47:0/556480945 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f05c406c8f0 0x7f05c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:48.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.544+0000 7f05e50db640 1 --2- 172.21.15.47:0/556480945 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f05c406c8f0 0x7f05c406edb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f05d0034000 tx=0x7f05d002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:48.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.547+0000 7f05d67fc640 1 -- 172.21.15.47:0/556480945 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f05dc05a0f0 con 0x7f05e00b2340 2024-04-26T23:19:48.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.726+0000 7f05e7b40640 1 -- 172.21.15.47:0/556480945 --> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f05e006de70 con 0x7f05c406c8f0 2024-04-26T23:19:48.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.733+0000 7f05d67fc640 1 -- 172.21.15.47:0/556480945 <== mgr.4099 v2:172.21.15.203:6832/992162732 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+243401 (secure 0 0 0) 0x7f05e006de70 con 0x7f05c406c8f0 2024-04-26T23:19:48.739 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:19:48.740 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-26T23:19:48.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.736+0000 7f05e7b40640 1 -- 172.21.15.47:0/556480945 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f05c406c8f0 msgr2=0x7f05c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:48.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.736+0000 7f05e7b40640 1 --2- 172.21.15.47:0/556480945 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f05c406c8f0 0x7f05c406edb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f05d0034000 tx=0x7f05d002e000 comp rx=0 tx=0).stop 2024-04-26T23:19:48.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.736+0000 7f05e7b40640 1 -- 172.21.15.47:0/556480945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05e00b2340 msgr2=0x7f05e00cc520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:48.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.736+0000 7f05e7b40640 1 --2- 172.21.15.47:0/556480945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05e00b2340 0x7f05e00cc520 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f05dc00c7f0 tx=0x7f05dc00ccb0 comp rx=0 tx=0).stop 2024-04-26T23:19:48.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.736+0000 7f05e7b40640 1 -- 172.21.15.47:0/556480945 shutdown_connections 2024-04-26T23:19:48.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.736+0000 7f05e7b40640 1 --2- 172.21.15.47:0/556480945 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f05c406c8f0 0x7f05c406edb0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:48.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.736+0000 7f05e7b40640 1 --2- 172.21.15.47:0/556480945 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05e0162810 0x7f05e015d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:48.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.736+0000 7f05e7b40640 1 --2- 172.21.15.47:0/556480945 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05e00cd930 0x7f05e0149b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:48.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.736+0000 7f05e7b40640 1 --2- 172.21.15.47:0/556480945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05e00b2340 0x7f05e00cc520 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:48.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.736+0000 7f05e7b40640 1 -- 172.21.15.47:0/556480945 >> 172.21.15.47:0/556480945 conn(0x7f05e00d2da0 msgr2=0x7f05e00b1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:48.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.736+0000 7f05e7b40640 1 -- 172.21.15.47:0/556480945 shutdown_connections 2024-04-26T23:19:48.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.736+0000 7f05e7b40640 1 -- 172.21.15.47:0/556480945 wait complete. 2024-04-26T23:19:48.749 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":76,"stamp":"2024-04-26T23:19:46.946608+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":134,"acting":134,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237072,"kb_used_data":4216,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737648,"statfs":{"total":773094113280,"available":772851351552,"internally_reserved":0,"allocated":4317184,"data_stored":2504316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.234265"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:19:43.707477+0000","last_change":"2024-04-26T23:19:43.707477+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.707477+0000","last_undegraded":"2024-04-26T23:19:43.707477+0000","last_fullsized":"2024-04-26T23:19:43.707477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:19:43.707477+0000","last_change":"2024-04-26T23:19:43.707477+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.707477+0000","last_undegraded":"2024-04-26T23:19:43.707477+0000","last_fullsized":"2024-04-26T23:19:43.707477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:19:43.707477+0000","last_change":"2024-04-26T23:19:43.707477+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.707477+0000","last_undegraded":"2024-04-26T23:19:43.707477+0000","last_fullsized":"2024-04-26T23:19:43.707477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:19:43.707477+0000","last_change":"2024-04-26T23:19:43.707477+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.707477+0000","last_undegraded":"2024-04-26T23:19:43.707477+0000","last_fullsized":"2024-04-26T23:19:43.707477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:19:43.707477+0000","last_change":"2024-04-26T23:19:43.707477+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.707477+0000","last_undegraded":"2024-04-26T23:19:43.707477+0000","last_fullsized":"2024-04-26T23:19:43.707477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:19:43.707477+0000","last_change":"2024-04-26T23:19:43.707477+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.707477+0000","last_undegraded":"2024-04-26T23:19:43.707477+0000","last_fullsized":"2024-04-26T23:19:43.707477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T23:19:43.720630+0000","last_change":"2024-04-26T23:19:43.720630+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.720630+0000","last_undegraded":"2024-04-26T23:19:43.720630+0000","last_fullsized":"2024-04-26T23:19:43.720630+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:19:43.707477+0000","last_change":"2024-04-26T23:19:43.707477+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.707477+0000","last_undegraded":"2024-04-26T23:19:43.707477+0000","last_fullsized":"2024-04-26T23:19:43.707477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:19:43.707477+0000","last_change":"2024-04-26T23:19:43.707477+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.707477+0000","last_undegraded":"2024-04-26T23:19:43.707477+0000","last_fullsized":"2024-04-26T23:19:43.707477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T23:19:43.723155+0000","last_change":"2024-04-26T23:19:43.723155+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.723155+0000","last_undegraded":"2024-04-26T23:19:43.723155+0000","last_fullsized":"2024-04-26T23:19:43.723155+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T23:19:43.721767+0000","last_change":"2024-04-26T23:19:43.721767+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.721767+0000","last_undegraded":"2024-04-26T23:19:43.721767+0000","last_fullsized":"2024-04-26T23:19:43.721767+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:19:43.707477+0000","last_change":"2024-04-26T23:19:43.707477+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.707477+0000","last_undegraded":"2024-04-26T23:19:43.707477+0000","last_fullsized":"2024-04-26T23:19:43.707477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T23:19:43.721168+0000","last_change":"2024-04-26T23:19:43.721168+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.721168+0000","last_undegraded":"2024-04-26T23:19:43.721168+0000","last_fullsized":"2024-04-26T23:19:43.721168+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T23:19:43.725498+0000","last_change":"2024-04-26T23:19:43.725498+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.725498+0000","last_undegraded":"2024-04-26T23:19:43.725498+0000","last_fullsized":"2024-04-26T23:19:43.725498+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T23:19:43.719932+0000","last_change":"2024-04-26T23:19:43.719932+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.719932+0000","last_undegraded":"2024-04-26T23:19:43.719932+0000","last_fullsized":"2024-04-26T23:19:43.719932+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T23:19:43.726758+0000","last_change":"2024-04-26T23:19:43.726758+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.726758+0000","last_undegraded":"2024-04-26T23:19:43.726758+0000","last_fullsized":"2024-04-26T23:19:43.726758+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T23:19:43.728736+0000","last_change":"2024-04-26T23:19:43.728736+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.728736+0000","last_undegraded":"2024-04-26T23:19:43.728736+0000","last_fullsized":"2024-04-26T23:19:43.728736+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T23:19:43.719936+0000","last_change":"2024-04-26T23:19:43.719936+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.719936+0000","last_undegraded":"2024-04-26T23:19:43.719936+0000","last_fullsized":"2024-04-26T23:19:43.719936+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:19:43.707477+0000","last_change":"2024-04-26T23:19:43.707477+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.707477+0000","last_undegraded":"2024-04-26T23:19:43.707477+0000","last_fullsized":"2024-04-26T23:19:43.707477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T23:19:43.718689+0000","last_change":"2024-04-26T23:19:43.718689+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.718689+0000","last_undegraded":"2024-04-26T23:19:43.718689+0000","last_fullsized":"2024-04-26T23:19:43.718689+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T23:19:43.720744+0000","last_change":"2024-04-26T23:19:43.720744+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.720744+0000","last_undegraded":"2024-04-26T23:19:43.720744+0000","last_fullsized":"2024-04-26T23:19:43.720744+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T23:19:43.722714+0000","last_change":"2024-04-26T23:19:43.722714+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.722714+0000","last_undegraded":"2024-04-26T23:19:43.722714+0000","last_fullsized":"2024-04-26T23:19:43.722714+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T23:19:43.726847+0000","last_change":"2024-04-26T23:19:43.726847+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.726847+0000","last_undegraded":"2024-04-26T23:19:43.726847+0000","last_fullsized":"2024-04-26T23:19:43.726847+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:19:43.707477+0000","last_change":"2024-04-26T23:19:43.707477+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.707477+0000","last_undegraded":"2024-04-26T23:19:43.707477+0000","last_fullsized":"2024-04-26T23:19:43.707477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T23:19:43.719370+0000","last_change":"2024-04-26T23:19:43.719370+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.719370+0000","last_undegraded":"2024-04-26T23:19:43.719370+0000","last_fullsized":"2024-04-26T23:19:43.719370+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:19:43.707477+0000","last_change":"2024-04-26T23:19:43.707477+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.707477+0000","last_undegraded":"2024-04-26T23:19:43.707477+0000","last_fullsized":"2024-04-26T23:19:43.707477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T23:19:43.719843+0000","last_change":"2024-04-26T23:19:43.719843+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.719843+0000","last_undegraded":"2024-04-26T23:19:43.719843+0000","last_fullsized":"2024-04-26T23:19:43.719843+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T23:19:43.721219+0000","last_change":"2024-04-26T23:19:43.721219+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.721219+0000","last_undegraded":"2024-04-26T23:19:43.721219+0000","last_fullsized":"2024-04-26T23:19:43.721219+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T23:19:43.718209+0000","last_change":"2024-04-26T23:19:43.718209+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.718209+0000","last_undegraded":"2024-04-26T23:19:43.718209+0000","last_fullsized":"2024-04-26T23:19:43.718209+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T23:19:43.718663+0000","last_change":"2024-04-26T23:19:43.718663+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.718663+0000","last_undegraded":"2024-04-26T23:19:43.718663+0000","last_fullsized":"2024-04-26T23:19:43.718663+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T23:19:43.726304+0000","last_change":"2024-04-26T23:19:43.726304+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.726304+0000","last_undegraded":"2024-04-26T23:19:43.726304+0000","last_fullsized":"2024-04-26T23:19:43.726304+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:19:43.707477+0000","last_change":"2024-04-26T23:19:43.707477+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.707477+0000","last_undegraded":"2024-04-26T23:19:43.707477+0000","last_fullsized":"2024-04-26T23:19:43.707477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T23:19:43.727375+0000","last_change":"2024-04-26T23:19:43.727375+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.727375+0000","last_undegraded":"2024-04-26T23:19:43.727375+0000","last_fullsized":"2024-04-26T23:19:43.727375+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T23:19:43.730724+0000","last_change":"2024-04-26T23:19:43.728273+0000","last_active":"2024-04-26T23:19:43.730724+0000","last_peered":"2024-04-26T23:19:43.730724+0000","last_clean":"2024-04-26T23:19:43.730724+0000","last_became_active":"2024-04-26T23:19:43.728009+0000","last_became_peered":"2024-04-26T23:19:43.728009+0000","last_unstale":"2024-04-26T23:19:43.730724+0000","last_undegraded":"2024-04-26T23:19:43.730724+0000","last_fullsized":"2024-04-26T23:19:43.730724+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:39:46.731984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:19:43.707477+0000","last_change":"2024-04-26T23:19:43.707477+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.707477+0000","last_undegraded":"2024-04-26T23:19:43.707477+0000","last_fullsized":"2024-04-26T23:19:43.707477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T23:19:42.710052+0000","last_change":"2024-04-26T23:19:42.710052+0000","last_active":"2024-04-26T23:19:42.704965+0000","last_peered":"2024-04-26T23:19:42.704965+0000","last_clean":"2024-04-26T23:19:42.704965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:42.710052+0000","last_undegraded":"2024-04-26T23:19:42.710052+0000","last_fullsized":"2024-04-26T23:19:42.710052+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T23:19:43.723317+0000","last_change":"2024-04-26T23:19:43.723317+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.723317+0000","last_undegraded":"2024-04-26T23:19:43.723317+0000","last_fullsized":"2024-04-26T23:19:43.723317+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T23:19:42.713244+0000","last_change":"2024-04-26T23:19:42.713244+0000","last_active":"2024-04-26T23:19:42.704965+0000","last_peered":"2024-04-26T23:19:42.704965+0000","last_clean":"2024-04-26T23:19:42.704965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:42.713244+0000","last_undegraded":"2024-04-26T23:19:42.713244+0000","last_fullsized":"2024-04-26T23:19:42.713244+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:19:43.707477+0000","last_change":"2024-04-26T23:19:43.707477+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.707477+0000","last_undegraded":"2024-04-26T23:19:43.707477+0000","last_fullsized":"2024-04-26T23:19:43.707477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T23:19:43.733762+0000","last_change":"2024-04-26T23:19:43.727630+0000","last_active":"2024-04-26T23:19:43.733762+0000","last_peered":"2024-04-26T23:19:43.733762+0000","last_clean":"2024-04-26T23:19:43.733762+0000","last_became_active":"2024-04-26T23:19:43.726916+0000","last_became_peered":"2024-04-26T23:19:43.726916+0000","last_unstale":"2024-04-26T23:19:43.733762+0000","last_undegraded":"2024-04-26T23:19:43.733762+0000","last_fullsized":"2024-04-26T23:19:43.733762+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:55:00.849518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:19:43.707477+0000","last_change":"2024-04-26T23:19:43.707477+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.707477+0000","last_undegraded":"2024-04-26T23:19:43.707477+0000","last_fullsized":"2024-04-26T23:19:43.707477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T23:19:43.737128+0000","last_change":"2024-04-26T23:19:43.730835+0000","last_active":"2024-04-26T23:19:43.737128+0000","last_peered":"2024-04-26T23:19:43.737128+0000","last_clean":"2024-04-26T23:19:43.737128+0000","last_became_active":"2024-04-26T23:19:43.730566+0000","last_became_peered":"2024-04-26T23:19:43.730566+0000","last_unstale":"2024-04-26T23:19:43.737128+0000","last_undegraded":"2024-04-26T23:19:43.737128+0000","last_fullsized":"2024-04-26T23:19:43.737128+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:34:07.760043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T23:19:43.719430+0000","last_change":"2024-04-26T23:19:43.719430+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.719430+0000","last_undegraded":"2024-04-26T23:19:43.719430+0000","last_fullsized":"2024-04-26T23:19:43.719430+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T23:19:43.740076+0000","last_change":"2024-04-26T23:19:43.737887+0000","last_active":"2024-04-26T23:19:43.740076+0000","last_peered":"2024-04-26T23:19:43.740076+0000","last_clean":"2024-04-26T23:19:43.740076+0000","last_became_active":"2024-04-26T23:19:43.737494+0000","last_became_peered":"2024-04-26T23:19:43.737494+0000","last_unstale":"2024-04-26T23:19:43.740076+0000","last_undegraded":"2024-04-26T23:19:43.740076+0000","last_fullsized":"2024-04-26T23:19:43.740076+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:37.017797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T23:19:43.724661+0000","last_change":"2024-04-26T23:19:43.724661+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.724661+0000","last_undegraded":"2024-04-26T23:19:43.724661+0000","last_fullsized":"2024-04-26T23:19:43.724661+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T23:19:42.713255+0000","last_change":"2024-04-26T23:19:42.713255+0000","last_active":"2024-04-26T23:19:42.704965+0000","last_peered":"2024-04-26T23:19:42.704965+0000","last_clean":"2024-04-26T23:19:42.704965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:42.713255+0000","last_undegraded":"2024-04-26T23:19:42.713255+0000","last_fullsized":"2024-04-26T23:19:42.713255+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:19:43.707477+0000","last_change":"2024-04-26T23:19:43.707477+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.707477+0000","last_undegraded":"2024-04-26T23:19:43.707477+0000","last_fullsized":"2024-04-26T23:19:43.707477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T23:19:43.737062+0000","last_change":"2024-04-26T23:19:43.731843+0000","last_active":"2024-04-26T23:19:43.737062+0000","last_peered":"2024-04-26T23:19:43.737062+0000","last_clean":"2024-04-26T23:19:43.737062+0000","last_became_active":"2024-04-26T23:19:43.731575+0000","last_became_peered":"2024-04-26T23:19:43.731575+0000","last_unstale":"2024-04-26T23:19:43.737062+0000","last_undegraded":"2024-04-26T23:19:43.737062+0000","last_fullsized":"2024-04-26T23:19:43.737062+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:17.697208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:19:43.707477+0000","last_change":"2024-04-26T23:19:43.707477+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.707477+0000","last_undegraded":"2024-04-26T23:19:43.707477+0000","last_fullsized":"2024-04-26T23:19:43.707477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T23:19:42.718393+0000","last_change":"2024-04-26T23:19:42.718393+0000","last_active":"2024-04-26T23:19:42.704965+0000","last_peered":"2024-04-26T23:19:42.704965+0000","last_clean":"2024-04-26T23:19:42.704965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:42.718393+0000","last_undegraded":"2024-04-26T23:19:42.718393+0000","last_fullsized":"2024-04-26T23:19:42.718393+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T23:19:43.728051+0000","last_change":"2024-04-26T23:19:43.728051+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.728051+0000","last_undegraded":"2024-04-26T23:19:43.728051+0000","last_fullsized":"2024-04-26T23:19:43.728051+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T23:19:43.739712+0000","last_change":"2024-04-26T23:19:43.734757+0000","last_active":"2024-04-26T23:19:43.739712+0000","last_peered":"2024-04-26T23:19:43.739712+0000","last_clean":"2024-04-26T23:19:43.739712+0000","last_became_active":"2024-04-26T23:19:43.734295+0000","last_became_peered":"2024-04-26T23:19:43.734295+0000","last_unstale":"2024-04-26T23:19:43.739712+0000","last_undegraded":"2024-04-26T23:19:43.739712+0000","last_fullsized":"2024-04-26T23:19:43.739712+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:37.017797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:19:43.707477+0000","last_change":"2024-04-26T23:19:43.707477+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.707477+0000","last_undegraded":"2024-04-26T23:19:43.707477+0000","last_fullsized":"2024-04-26T23:19:43.707477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T23:19:43.739695+0000","last_change":"2024-04-26T23:19:43.734517+0000","last_active":"2024-04-26T23:19:43.739695+0000","last_peered":"2024-04-26T23:19:43.739695+0000","last_clean":"2024-04-26T23:19:43.739695+0000","last_became_active":"2024-04-26T23:19:43.734234+0000","last_became_peered":"2024-04-26T23:19:43.734234+0000","last_unstale":"2024-04-26T23:19:43.739695+0000","last_undegraded":"2024-04-26T23:19:43.739695+0000","last_fullsized":"2024-04-26T23:19:43.739695+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:44:49.809340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T23:19:43.725988+0000","last_change":"2024-04-26T23:19:43.725988+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.725988+0000","last_undegraded":"2024-04-26T23:19:43.725988+0000","last_fullsized":"2024-04-26T23:19:43.725988+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T23:19:42.718472+0000","last_change":"2024-04-26T23:19:42.718472+0000","last_active":"2024-04-26T23:19:42.704965+0000","last_peered":"2024-04-26T23:19:42.704965+0000","last_clean":"2024-04-26T23:19:42.704965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:42.718472+0000","last_undegraded":"2024-04-26T23:19:42.718472+0000","last_fullsized":"2024-04-26T23:19:42.718472+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:19:43.707477+0000","last_change":"2024-04-26T23:19:43.707477+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.707477+0000","last_undegraded":"2024-04-26T23:19:43.707477+0000","last_fullsized":"2024-04-26T23:19:43.707477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T23:19:42.718295+0000","last_change":"2024-04-26T23:19:42.718295+0000","last_active":"2024-04-26T23:19:42.704965+0000","last_peered":"2024-04-26T23:19:42.704965+0000","last_clean":"2024-04-26T23:19:42.704965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:42.718295+0000","last_undegraded":"2024-04-26T23:19:42.718295+0000","last_fullsized":"2024-04-26T23:19:42.718295+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:19:43.707477+0000","last_change":"2024-04-26T23:19:43.707477+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.707477+0000","last_undegraded":"2024-04-26T23:19:43.707477+0000","last_fullsized":"2024-04-26T23:19:43.707477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T23:19:42.713163+0000","last_change":"2024-04-26T23:19:42.713163+0000","last_active":"2024-04-26T23:19:42.704965+0000","last_peered":"2024-04-26T23:19:42.704965+0000","last_clean":"2024-04-26T23:19:42.704965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:42.713163+0000","last_undegraded":"2024-04-26T23:19:42.713163+0000","last_fullsized":"2024-04-26T23:19:42.713163+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T23:19:43.726915+0000","last_change":"2024-04-26T23:19:43.726915+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.726915+0000","last_undegraded":"2024-04-26T23:19:43.726915+0000","last_fullsized":"2024-04-26T23:19:43.726915+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T23:19:43.730698+0000","last_change":"2024-04-26T23:19:43.729238+0000","last_active":"2024-04-26T23:19:43.730698+0000","last_peered":"2024-04-26T23:19:43.730698+0000","last_clean":"2024-04-26T23:19:43.730698+0000","last_became_active":"2024-04-26T23:19:43.728884+0000","last_became_peered":"2024-04-26T23:19:43.728884+0000","last_unstale":"2024-04-26T23:19:43.730698+0000","last_undegraded":"2024-04-26T23:19:43.730698+0000","last_fullsized":"2024-04-26T23:19:43.730698+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:57:49.822573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:19:43.707477+0000","last_change":"2024-04-26T23:19:43.707477+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.707477+0000","last_undegraded":"2024-04-26T23:19:43.707477+0000","last_fullsized":"2024-04-26T23:19:43.707477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T23:19:43.740059+0000","last_change":"2024-04-26T23:19:43.737740+0000","last_active":"2024-04-26T23:19:43.740059+0000","last_peered":"2024-04-26T23:19:43.740059+0000","last_clean":"2024-04-26T23:19:43.740059+0000","last_became_active":"2024-04-26T23:19:43.737443+0000","last_became_peered":"2024-04-26T23:19:43.737443+0000","last_unstale":"2024-04-26T23:19:43.740059+0000","last_undegraded":"2024-04-26T23:19:43.740059+0000","last_fullsized":"2024-04-26T23:19:43.740059+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:44:49.809340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:19:43.707477+0000","last_change":"2024-04-26T23:19:43.707477+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.707477+0000","last_undegraded":"2024-04-26T23:19:43.707477+0000","last_fullsized":"2024-04-26T23:19:43.707477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T23:19:43.739765+0000","last_change":"2024-04-26T23:19:43.735048+0000","last_active":"2024-04-26T23:19:43.739765+0000","last_peered":"2024-04-26T23:19:43.739765+0000","last_clean":"2024-04-26T23:19:43.739765+0000","last_became_active":"2024-04-26T23:19:43.734332+0000","last_became_peered":"2024-04-26T23:19:43.734332+0000","last_unstale":"2024-04-26T23:19:43.739765+0000","last_undegraded":"2024-04-26T23:19:43.739765+0000","last_fullsized":"2024-04-26T23:19:43.739765+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:56.075572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T23:19:43.721351+0000","last_change":"2024-04-26T23:19:43.721351+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.721351+0000","last_undegraded":"2024-04-26T23:19:43.721351+0000","last_fullsized":"2024-04-26T23:19:43.721351+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T23:19:43.739680+0000","last_change":"2024-04-26T23:19:43.735311+0000","last_active":"2024-04-26T23:19:43.739680+0000","last_peered":"2024-04-26T23:19:43.739680+0000","last_clean":"2024-04-26T23:19:43.739680+0000","last_became_active":"2024-04-26T23:19:43.735022+0000","last_became_peered":"2024-04-26T23:19:43.735022+0000","last_unstale":"2024-04-26T23:19:43.739680+0000","last_undegraded":"2024-04-26T23:19:43.739680+0000","last_fullsized":"2024-04-26T23:19:43.739680+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:25:12.211704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T23:19:43.724449+0000","last_change":"2024-04-26T23:19:43.724449+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.724449+0000","last_undegraded":"2024-04-26T23:19:43.724449+0000","last_fullsized":"2024-04-26T23:19:43.724449+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T23:19:42.717508+0000","last_change":"2024-04-26T23:19:42.717508+0000","last_active":"2024-04-26T23:19:42.704965+0000","last_peered":"2024-04-26T23:19:42.704965+0000","last_clean":"2024-04-26T23:19:42.704965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:42.717508+0000","last_undegraded":"2024-04-26T23:19:42.717508+0000","last_fullsized":"2024-04-26T23:19:42.717508+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:19:43.707477+0000","last_change":"2024-04-26T23:19:43.707477+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.707477+0000","last_undegraded":"2024-04-26T23:19:43.707477+0000","last_fullsized":"2024-04-26T23:19:43.707477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T23:19:42.713234+0000","last_change":"2024-04-26T23:19:42.713234+0000","last_active":"2024-04-26T23:19:42.704965+0000","last_peered":"2024-04-26T23:19:42.704965+0000","last_clean":"2024-04-26T23:19:42.704965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:42.713234+0000","last_undegraded":"2024-04-26T23:19:42.713234+0000","last_fullsized":"2024-04-26T23:19:42.713234+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T23:19:43.726204+0000","last_change":"2024-04-26T23:19:43.726204+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.726204+0000","last_undegraded":"2024-04-26T23:19:43.726204+0000","last_fullsized":"2024-04-26T23:19:43.726204+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T23:19:42.717340+0000","last_change":"2024-04-26T23:19:42.717340+0000","last_active":"2024-04-26T23:19:42.704965+0000","last_peered":"2024-04-26T23:19:42.704965+0000","last_clean":"2024-04-26T23:19:42.704965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:42.717340+0000","last_undegraded":"2024-04-26T23:19:42.717340+0000","last_fullsized":"2024-04-26T23:19:42.717340+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:19:43.707477+0000","last_change":"2024-04-26T23:19:43.707477+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.707477+0000","last_undegraded":"2024-04-26T23:19:43.707477+0000","last_fullsized":"2024-04-26T23:19:43.707477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T23:19:42.717354+0000","last_change":"2024-04-26T23:19:42.717354+0000","last_active":"2024-04-26T23:19:42.704965+0000","last_peered":"2024-04-26T23:19:42.704965+0000","last_clean":"2024-04-26T23:19:42.704965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:42.717354+0000","last_undegraded":"2024-04-26T23:19:42.717354+0000","last_fullsized":"2024-04-26T23:19:42.717354+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T23:19:43.719440+0000","last_change":"2024-04-26T23:19:43.719440+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.719440+0000","last_undegraded":"2024-04-26T23:19:43.719440+0000","last_fullsized":"2024-04-26T23:19:43.719440+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T23:19:43.740047+0000","last_change":"2024-04-26T23:19:43.737669+0000","last_active":"2024-04-26T23:19:43.740047+0000","last_peered":"2024-04-26T23:19:43.740047+0000","last_clean":"2024-04-26T23:19:43.740047+0000","last_became_active":"2024-04-26T23:19:43.737369+0000","last_became_peered":"2024-04-26T23:19:43.737369+0000","last_unstale":"2024-04-26T23:19:43.740047+0000","last_undegraded":"2024-04-26T23:19:43.740047+0000","last_fullsized":"2024-04-26T23:19:43.740047+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:34:07.760043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:19:43.707477+0000","last_change":"2024-04-26T23:19:43.707477+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.707477+0000","last_undegraded":"2024-04-26T23:19:43.707477+0000","last_fullsized":"2024-04-26T23:19:43.707477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T23:19:42.717312+0000","last_change":"2024-04-26T23:19:42.717312+0000","last_active":"2024-04-26T23:19:42.704965+0000","last_peered":"2024-04-26T23:19:42.704965+0000","last_clean":"2024-04-26T23:19:42.704965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:42.717312+0000","last_undegraded":"2024-04-26T23:19:42.717312+0000","last_fullsized":"2024-04-26T23:19:42.717312+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:19:43.707477+0000","last_change":"2024-04-26T23:19:43.707477+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.707477+0000","last_undegraded":"2024-04-26T23:19:43.707477+0000","last_fullsized":"2024-04-26T23:19:43.707477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T23:19:43.736929+0000","last_change":"2024-04-26T23:19:43.730239+0000","last_active":"2024-04-26T23:19:43.736929+0000","last_peered":"2024-04-26T23:19:43.736929+0000","last_clean":"2024-04-26T23:19:43.736929+0000","last_became_active":"2024-04-26T23:19:43.729976+0000","last_became_peered":"2024-04-26T23:19:43.729976+0000","last_unstale":"2024-04-26T23:19:43.736929+0000","last_undegraded":"2024-04-26T23:19:43.736929+0000","last_fullsized":"2024-04-26T23:19:43.736929+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:17:20.585223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T23:19:43.727545+0000","last_change":"2024-04-26T23:19:43.727545+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.727545+0000","last_undegraded":"2024-04-26T23:19:43.727545+0000","last_fullsized":"2024-04-26T23:19:43.727545+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T23:19:42.717493+0000","last_change":"2024-04-26T23:19:42.717493+0000","last_active":"2024-04-26T23:19:42.704965+0000","last_peered":"2024-04-26T23:19:42.704965+0000","last_clean":"2024-04-26T23:19:42.704965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:42.717493+0000","last_undegraded":"2024-04-26T23:19:42.717493+0000","last_fullsized":"2024-04-26T23:19:42.717493+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:19:43.707477+0000","last_change":"2024-04-26T23:19:43.707477+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.707477+0000","last_undegraded":"2024-04-26T23:19:43.707477+0000","last_fullsized":"2024-04-26T23:19:43.707477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T23:19:42.710055+0000","last_change":"2024-04-26T23:19:42.710055+0000","last_active":"2024-04-26T23:19:42.704965+0000","last_peered":"2024-04-26T23:19:42.704965+0000","last_clean":"2024-04-26T23:19:42.704965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:42.710055+0000","last_undegraded":"2024-04-26T23:19:42.710055+0000","last_fullsized":"2024-04-26T23:19:42.710055+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:19:43.707477+0000","last_change":"2024-04-26T23:19:43.707477+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.707477+0000","last_undegraded":"2024-04-26T23:19:43.707477+0000","last_fullsized":"2024-04-26T23:19:43.707477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T23:19:42.709913+0000","last_change":"2024-04-26T23:19:42.709913+0000","last_active":"2024-04-26T23:19:42.704965+0000","last_peered":"2024-04-26T23:19:42.704965+0000","last_clean":"2024-04-26T23:19:42.704965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:42.709913+0000","last_undegraded":"2024-04-26T23:19:42.709913+0000","last_fullsized":"2024-04-26T23:19:42.709913+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T23:19:43.707477+0000","last_change":"2024-04-26T23:19:43.707477+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.707477+0000","last_undegraded":"2024-04-26T23:19:43.707477+0000","last_fullsized":"2024-04-26T23:19:43.707477+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T23:19:43.727029+0000","last_change":"2024-04-26T23:19:43.726318+0000","last_active":"2024-04-26T23:19:43.727029+0000","last_peered":"2024-04-26T23:19:43.727029+0000","last_clean":"2024-04-26T23:19:43.727029+0000","last_became_active":"2024-04-26T23:19:43.725993+0000","last_became_peered":"2024-04-26T23:19:43.725993+0000","last_unstale":"2024-04-26T23:19:43.727029+0000","last_undegraded":"2024-04-26T23:19:43.727029+0000","last_fullsized":"2024-04-26T23:19:43.727029+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:24:39.956920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T23:19:43.725082+0000","last_change":"2024-04-26T23:19:43.725082+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.725082+0000","last_undegraded":"2024-04-26T23:19:43.725082+0000","last_fullsized":"2024-04-26T23:19:43.725082+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T23:19:43.737142+0000","last_change":"2024-04-26T23:19:43.733216+0000","last_active":"2024-04-26T23:19:43.737142+0000","last_peered":"2024-04-26T23:19:43.737142+0000","last_clean":"2024-04-26T23:19:43.737142+0000","last_became_active":"2024-04-26T23:19:43.732950+0000","last_became_peered":"2024-04-26T23:19:43.732950+0000","last_unstale":"2024-04-26T23:19:43.737142+0000","last_undegraded":"2024-04-26T23:19:43.737142+0000","last_fullsized":"2024-04-26T23:19:43.737142+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:41.199643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T23:19:43.726264+0000","last_change":"2024-04-26T23:19:43.726264+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.726264+0000","last_undegraded":"2024-04-26T23:19:43.726264+0000","last_fullsized":"2024-04-26T23:19:43.726264+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-26T23:19:42.718788+0000","last_change":"2024-04-26T23:19:42.718788+0000","last_active":"2024-04-26T23:19:42.704965+0000","last_peered":"2024-04-26T23:19:42.704965+0000","last_clean":"2024-04-26T23:19:42.704965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:42.718788+0000","last_undegraded":"2024-04-26T23:19:42.718788+0000","last_fullsized":"2024-04-26T23:19:42.718788+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T23:19:43.725165+0000","last_change":"2024-04-26T23:19:43.725165+0000","last_active":"2024-04-26T23:19:43.707477+0000","last_peered":"2024-04-26T23:19:43.707477+0000","last_clean":"2024-04-26T23:19:43.707477+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T23:19:43.725165+0000","last_undegraded":"2024-04-26T23:19:43.725165+0000","last_fullsized":"2024-04-26T23:19:43.725165+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T23:19:43.739582+0000","last_change":"2024-04-26T23:19:43.736319+0000","last_active":"2024-04-26T23:19:43.739582+0000","last_peered":"2024-04-26T23:19:43.739582+0000","last_clean":"2024-04-26T23:19:43.739582+0000","last_became_active":"2024-04-26T23:19:43.736032+0000","last_became_peered":"2024-04-26T23:19:43.736032+0000","last_unstale":"2024-04-26T23:19:43.739582+0000","last_undegraded":"2024-04-26T23:19:43.739582+0000","last_fullsized":"2024-04-26T23:19:43.739582+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:17:20.585223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":199,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T23:19:43.729323+0000","last_change":"2024-04-26T23:18:18.862631+0000","last_active":"2024-04-26T23:19:43.729323+0000","last_peered":"2024-04-26T23:19:43.729323+0000","last_clean":"2024-04-26T23:19:43.729323+0000","last_became_active":"2024-04-26T23:18:18.861536+0000","last_became_peered":"2024-04-26T23:18:18.861536+0000","last_unstale":"2024-04-26T23:19:43.729323+0000","last_undegraded":"2024-04-26T23:19:43.729323+0000","last_fullsized":"2024-04-26T23:19:43.729323+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:17.808736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:17.808736+0000","last_clean_scrub_stamp":"2024-04-26T23:18:17.808736+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:59:26.359282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":68,"acting":68,"num_store_stats":4},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542164,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343852,"statfs":{"total":96636764160,"available":96608104448,"internally_reserved":0,"allocated":991232,"data_stored":758920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542166,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":397312,"data_stored":168552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542165,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":397312,"data_stored":168552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542165,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":397312,"data_stored":168552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31576,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340264,"statfs":{"total":96636764160,"available":96604430336,"internally_reserved":0,"allocated":385024,"data_stored":162343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31572,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340268,"statfs":{"total":96636764160,"available":96604434432,"internally_reserved":0,"allocated":385024,"data_stored":162343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31576,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340264,"statfs":{"total":96636764160,"available":96604430336,"internally_reserved":0,"allocated":385024,"data_stored":162343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32136,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339704,"statfs":{"total":96636764160,"available":96603856896,"internally_reserved":0,"allocated":978944,"data_stored":752711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:19:48.753 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T23:19:48.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.887+0000 7f4f3f8c8640 1 Processor -- start 2024-04-26T23:19:48.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.887+0000 7f4f3f8c8640 1 -- start start 2024-04-26T23:19:48.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.888+0000 7f4f3f8c8640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4f38162880 0x7f4f380d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:48.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.888+0000 7f4f3f8c8640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4f380b5f20 0x7f4f380d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:48.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.888+0000 7f4f3f8c8640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f380b7e50 con 0x7f4f380d39f0 2024-04-26T23:19:48.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.888+0000 7f4f3f8c8640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f4f380d3e40 con 0x7f4f38162880 2024-04-26T23:19:48.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.888+0000 7f4f3f8c8640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f4f380d0580 con 0x7f4f380b5f20 2024-04-26T23:19:48.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.888+0000 7f4f3d664640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4f38162880 0x7f4f380d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:48.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.888+0000 7f4f3d664640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4f38162880 0x7f4f380d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58896/0 (socket says 172.21.15.47:58896) 2024-04-26T23:19:48.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.888+0000 7f4f3d664640 1 -- 172.21.15.47:0/1632414369 learned_addr learned my addr 172.21.15.47:0/1632414369 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:48.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.889+0000 7f4f3d664640 1 -- 172.21.15.47:0/1632414369 >> v2:172.21.15.203:3300/0 conn(0x7f4f380b5f20 msgr2=0x7f4f380d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:48.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.889+0000 7f4f3d664640 1 --2- 172.21.15.47:0/1632414369 >> v2:172.21.15.203:3300/0 conn(0x7f4f380b5f20 0x7f4f380d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:48.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.889+0000 7f4f3d664640 1 -- 172.21.15.47:0/1632414369 >> v1:172.21.15.203:6789/0 conn(0x7f4f380d39f0 legacy=0x7f4f380b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:48.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.889+0000 7f4f3d664640 1 -- 172.21.15.47:0/1632414369 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f380d1800 con 0x7f4f38162880 2024-04-26T23:19:48.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.889+0000 7f4f3d664640 1 --2- 172.21.15.47:0/1632414369 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f38162880 0x7f4f380d2cb0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4f3808d3d0 tx=0x7f4f3002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce67386e7bba3fa6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:48.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.889+0000 7f4f37fff640 1 -- 172.21.15.47:0/1632414369 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f30037070 con 0x7f4f38162880 2024-04-26T23:19:48.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.889+0000 7f4f37fff640 1 -- 172.21.15.47:0/1632414369 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f30032440 con 0x7f4f38162880 2024-04-26T23:19:48.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.889+0000 7f4f37fff640 1 -- 172.21.15.47:0/1632414369 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f30032810 con 0x7f4f38162880 2024-04-26T23:19:48.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.890+0000 7f4f3f8c8640 1 -- 172.21.15.47:0/1632414369 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f38162880 msgr2=0x7f4f380d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:48.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.890+0000 7f4f3f8c8640 1 --2- 172.21.15.47:0/1632414369 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f38162880 0x7f4f380d2cb0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4f3808d3d0 tx=0x7f4f3002a230 comp rx=0 tx=0).stop 2024-04-26T23:19:48.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.890+0000 7f4f3f8c8640 1 -- 172.21.15.47:0/1632414369 shutdown_connections 2024-04-26T23:19:48.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.890+0000 7f4f3f8c8640 1 --2- 172.21.15.47:0/1632414369 >> v2:172.21.15.203:3300/0 conn(0x7f4f380b5f20 0x7f4f380d34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:48.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.890+0000 7f4f3f8c8640 1 --2- 172.21.15.47:0/1632414369 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f38162880 0x7f4f380d2cb0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:48.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.890+0000 7f4f3f8c8640 1 -- 172.21.15.47:0/1632414369 >> 172.21.15.47:0/1632414369 conn(0x7f4f380d4f60 msgr2=0x7f4f380d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:48.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.890+0000 7f4f3f8c8640 1 -- 172.21.15.47:0/1632414369 shutdown_connections 2024-04-26T23:19:48.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.890+0000 7f4f3f8c8640 1 -- 172.21.15.47:0/1632414369 wait complete. 2024-04-26T23:19:48.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.891+0000 7f4f3f8c8640 1 Processor -- start 2024-04-26T23:19:48.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.891+0000 7f4f3f8c8640 1 -- start start 2024-04-26T23:19:48.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.892+0000 7f4f3f8c8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f380b5f20 0x7f4f380b27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:48.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.892+0000 7f4f3f8c8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f380d39f0 0x7f4f380cd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:48.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.892+0000 7f4f3f8c8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f38162880 0x7f4f3814f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:48.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.892+0000 7f4f3f8c8640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4f380d0d90 con 0x7f4f38162880 2024-04-26T23:19:48.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.892+0000 7f4f3f8c8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4f380d0ab0 con 0x7f4f380d39f0 2024-04-26T23:19:48.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.892+0000 7f4f3f8c8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f4f38162c60 con 0x7f4f380b5f20 2024-04-26T23:19:48.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.892+0000 7f4f3de65640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f38162880 0x7f4f3814f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:48.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.892+0000 7f4f3de65640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f38162880 0x7f4f3814f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58900/0 (socket says 172.21.15.47:58900) 2024-04-26T23:19:48.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.892+0000 7f4f3de65640 1 -- 172.21.15.47:0/952621529 learned_addr learned my addr 172.21.15.47:0/952621529 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:48.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.892+0000 7f4f3ce63640 1 --2- 172.21.15.47:0/952621529 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f380d39f0 0x7f4f380cd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:48.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.892+0000 7f4f3d664640 1 --2- 172.21.15.47:0/952621529 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f380b5f20 0x7f4f380b27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:48.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.893+0000 7f4f3de65640 1 -- 172.21.15.47:0/952621529 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f380b5f20 msgr2=0x7f4f380b27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:48.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.893+0000 7f4f3de65640 1 --2- 172.21.15.47:0/952621529 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f380b5f20 0x7f4f380b27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:48.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.893+0000 7f4f3de65640 1 -- 172.21.15.47:0/952621529 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f380d39f0 msgr2=0x7f4f380cd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:48.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.893+0000 7f4f3de65640 1 --2- 172.21.15.47:0/952621529 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f380d39f0 0x7f4f380cd9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:48.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.893+0000 7f4f3de65640 1 -- 172.21.15.47:0/952621529 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f3814f8e0 con 0x7f4f38162880 2024-04-26T23:19:48.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.893+0000 7f4f3ce63640 1 --2- 172.21.15.47:0/952621529 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f380d39f0 0x7f4f380cd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:19:48.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.893+0000 7f4f3de65640 1 --2- 172.21.15.47:0/952621529 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f38162880 0x7f4f3814f3a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4f2800b9c0 tx=0x7f4f2800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:48.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.893+0000 7f4f35ffb640 1 -- 172.21.15.47:0/952621529 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f28019070 con 0x7f4f38162880 2024-04-26T23:19:48.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.893+0000 7f4f35ffb640 1 -- 172.21.15.47:0/952621529 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f2800cba0 con 0x7f4f38162880 2024-04-26T23:19:48.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.893+0000 7f4f35ffb640 1 -- 172.21.15.47:0/952621529 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f28014440 con 0x7f4f38162880 2024-04-26T23:19:48.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.893+0000 7f4f3f8c8640 1 -- 172.21.15.47:0/952621529 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f380ce150 con 0x7f4f38162880 2024-04-26T23:19:48.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.893+0000 7f4f3f8c8640 1 -- 172.21.15.47:0/952621529 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f380ce710 con 0x7f4f38162880 2024-04-26T23:19:48.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.894+0000 7f4f3f8c8640 1 -- 172.21.15.47:0/952621529 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f380694f0 con 0x7f4f38162880 2024-04-26T23:19:48.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.894+0000 7f4f35ffb640 1 -- 172.21.15.47:0/952621529 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4f2801c070 con 0x7f4f38162880 2024-04-26T23:19:48.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.895+0000 7f4f35ffb640 1 --2- 172.21.15.47:0/952621529 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4f2406c8f0 0x7f4f2406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:48.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.895+0000 7f4f35ffb640 1 -- 172.21.15.47:0/952621529 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f4f2808b9d0 con 0x7f4f38162880 2024-04-26T23:19:48.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.895+0000 7f4f3d664640 1 --2- 172.21.15.47:0/952621529 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4f2406c8f0 0x7f4f2406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:48.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.896+0000 7f4f3d664640 1 --2- 172.21.15.47:0/952621529 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4f2406c8f0 0x7f4f2406edb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4f30034000 tx=0x7f4f3002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:48.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:48.899+0000 7f4f35ffb640 1 -- 172.21.15.47:0/952621529 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4f3814f8e0 con 0x7f4f38162880 2024-04-26T23:19:49.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.081+0000 7f4f3f8c8640 1 -- 172.21.15.47:0/952621529 --> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4f380cdef0 con 0x7f4f2406c8f0 2024-04-26T23:19:49.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.087+0000 7f4f35ffb640 1 -- 172.21.15.47:0/952621529 <== mgr.4099 v2:172.21.15.203:6832/992162732 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251959 (secure 0 0 0) 0x7f4f3003c460 con 0x7f4f2406c8f0 2024-04-26T23:19:49.091 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:19:49.093 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-26T23:19:49.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.090+0000 7f4f3f8c8640 1 -- 172.21.15.47:0/952621529 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4f2406c8f0 msgr2=0x7f4f2406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:49.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.090+0000 7f4f3f8c8640 1 --2- 172.21.15.47:0/952621529 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4f2406c8f0 0x7f4f2406edb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4f30034000 tx=0x7f4f3002e000 comp rx=0 tx=0).stop 2024-04-26T23:19:49.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.090+0000 7f4f3f8c8640 1 -- 172.21.15.47:0/952621529 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f38162880 msgr2=0x7f4f3814f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:49.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.090+0000 7f4f3f8c8640 1 --2- 172.21.15.47:0/952621529 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f38162880 0x7f4f3814f3a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4f2800b9c0 tx=0x7f4f2800be80 comp rx=0 tx=0).stop 2024-04-26T23:19:49.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.090+0000 7f4f3f8c8640 1 -- 172.21.15.47:0/952621529 shutdown_connections 2024-04-26T23:19:49.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.090+0000 7f4f3f8c8640 1 --2- 172.21.15.47:0/952621529 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4f2406c8f0 0x7f4f2406edb0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:49.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.090+0000 7f4f3f8c8640 1 --2- 172.21.15.47:0/952621529 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f38162880 0x7f4f3814f3a0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:49.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.090+0000 7f4f3f8c8640 1 --2- 172.21.15.47:0/952621529 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f380d39f0 0x7f4f380cd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:49.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.090+0000 7f4f3f8c8640 1 --2- 172.21.15.47:0/952621529 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f380b5f20 0x7f4f380b27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:49.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.090+0000 7f4f3f8c8640 1 -- 172.21.15.47:0/952621529 >> 172.21.15.47:0/952621529 conn(0x7f4f380d4f60 msgr2=0x7f4f38160ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:49.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.090+0000 7f4f3f8c8640 1 -- 172.21.15.47:0/952621529 shutdown_connections 2024-04-26T23:19:49.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.090+0000 7f4f3f8c8640 1 -- 172.21.15.47:0/952621529 wait complete. 2024-04-26T23:19:49.103 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2024-04-26T23:19:48.947036+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":177,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237448,"kb_used_data":4616,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737272,"statfs":{"total":773094113280,"available":772850966528,"internally_reserved":0,"allocated":4726784,"data_stored":2698180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001199"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.170476+0000","last_change":"2024-04-26T23:19:44.734389+0000","last_active":"2024-04-26T23:19:46.170476+0000","last_peered":"2024-04-26T23:19:46.170476+0000","last_clean":"2024-04-26T23:19:46.170476+0000","last_became_active":"2024-04-26T23:19:44.734098+0000","last_became_peered":"2024-04-26T23:19:44.734098+0000","last_unstale":"2024-04-26T23:19:46.170476+0000","last_undegraded":"2024-04-26T23:19:46.170476+0000","last_fullsized":"2024-04-26T23:19:46.170476+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:25:26.534137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.170292+0000","last_change":"2024-04-26T23:19:44.725004+0000","last_active":"2024-04-26T23:19:46.170292+0000","last_peered":"2024-04-26T23:19:46.170292+0000","last_clean":"2024-04-26T23:19:46.170292+0000","last_became_active":"2024-04-26T23:19:44.724657+0000","last_became_peered":"2024-04-26T23:19:44.724657+0000","last_unstale":"2024-04-26T23:19:46.170292+0000","last_undegraded":"2024-04-26T23:19:46.170292+0000","last_fullsized":"2024-04-26T23:19:46.170292+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:14:15.379049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.051646+0000","last_change":"2024-04-26T23:19:44.747842+0000","last_active":"2024-04-26T23:19:46.051646+0000","last_peered":"2024-04-26T23:19:46.051646+0000","last_clean":"2024-04-26T23:19:46.051646+0000","last_became_active":"2024-04-26T23:19:44.747538+0000","last_became_peered":"2024-04-26T23:19:44.747538+0000","last_unstale":"2024-04-26T23:19:46.051646+0000","last_undegraded":"2024-04-26T23:19:46.051646+0000","last_fullsized":"2024-04-26T23:19:46.051646+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:24:21.554815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:45.736229+0000","last_change":"2024-04-26T23:19:44.735547+0000","last_active":"2024-04-26T23:19:45.736229+0000","last_peered":"2024-04-26T23:19:45.736229+0000","last_clean":"2024-04-26T23:19:45.736229+0000","last_became_active":"2024-04-26T23:19:44.733793+0000","last_became_peered":"2024-04-26T23:19:44.733793+0000","last_unstale":"2024-04-26T23:19:45.736229+0000","last_undegraded":"2024-04-26T23:19:45.736229+0000","last_fullsized":"2024-04-26T23:19:45.736229+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:25:13.214216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:45.736004+0000","last_change":"2024-04-26T23:19:44.738239+0000","last_active":"2024-04-26T23:19:45.736004+0000","last_peered":"2024-04-26T23:19:45.736004+0000","last_clean":"2024-04-26T23:19:45.736004+0000","last_became_active":"2024-04-26T23:19:44.737936+0000","last_became_peered":"2024-04-26T23:19:44.737936+0000","last_unstale":"2024-04-26T23:19:45.736004+0000","last_undegraded":"2024-04-26T23:19:45.736004+0000","last_fullsized":"2024-04-26T23:19:45.736004+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:55:01.852031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:45.741403+0000","last_change":"2024-04-26T23:19:44.726426+0000","last_active":"2024-04-26T23:19:45.741403+0000","last_peered":"2024-04-26T23:19:45.741403+0000","last_clean":"2024-04-26T23:19:45.741403+0000","last_became_active":"2024-04-26T23:19:44.726155+0000","last_became_peered":"2024-04-26T23:19:44.726155+0000","last_unstale":"2024-04-26T23:19:45.741403+0000","last_undegraded":"2024-04-26T23:19:45.741403+0000","last_fullsized":"2024-04-26T23:19:45.741403+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:24:21.554815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.170893+0000","last_change":"2024-04-26T23:19:44.744930+0000","last_active":"2024-04-26T23:19:46.170893+0000","last_peered":"2024-04-26T23:19:46.170893+0000","last_clean":"2024-04-26T23:19:46.170893+0000","last_became_active":"2024-04-26T23:19:44.744532+0000","last_became_peered":"2024-04-26T23:19:44.744532+0000","last_unstale":"2024-04-26T23:19:46.170893+0000","last_undegraded":"2024-04-26T23:19:46.170893+0000","last_fullsized":"2024-04-26T23:19:46.170893+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:31:09.766783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.170017+0000","last_change":"2024-04-26T23:19:44.742771+0000","last_active":"2024-04-26T23:19:46.170017+0000","last_peered":"2024-04-26T23:19:46.170017+0000","last_clean":"2024-04-26T23:19:46.170017+0000","last_became_active":"2024-04-26T23:19:44.742454+0000","last_became_peered":"2024-04-26T23:19:44.742454+0000","last_unstale":"2024-04-26T23:19:46.170017+0000","last_undegraded":"2024-04-26T23:19:46.170017+0000","last_fullsized":"2024-04-26T23:19:46.170017+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:30:13.827745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.051847+0000","last_change":"2024-04-26T23:19:44.747533+0000","last_active":"2024-04-26T23:19:46.051847+0000","last_peered":"2024-04-26T23:19:46.051847+0000","last_clean":"2024-04-26T23:19:46.051847+0000","last_became_active":"2024-04-26T23:19:44.747235+0000","last_became_peered":"2024-04-26T23:19:44.747235+0000","last_unstale":"2024-04-26T23:19:46.051847+0000","last_undegraded":"2024-04-26T23:19:46.051847+0000","last_fullsized":"2024-04-26T23:19:46.051847+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:25:16.575810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.171359+0000","last_change":"2024-04-26T23:19:44.747024+0000","last_active":"2024-04-26T23:19:46.171359+0000","last_peered":"2024-04-26T23:19:46.171359+0000","last_clean":"2024-04-26T23:19:46.171359+0000","last_became_active":"2024-04-26T23:19:44.745936+0000","last_became_peered":"2024-04-26T23:19:44.745936+0000","last_unstale":"2024-04-26T23:19:46.171359+0000","last_undegraded":"2024-04-26T23:19:46.171359+0000","last_fullsized":"2024-04-26T23:19:46.171359+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:05:31.972319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.171787+0000","last_change":"2024-04-26T23:19:44.748978+0000","last_active":"2024-04-26T23:19:46.171787+0000","last_peered":"2024-04-26T23:19:46.171787+0000","last_clean":"2024-04-26T23:19:46.171787+0000","last_became_active":"2024-04-26T23:19:44.747177+0000","last_became_peered":"2024-04-26T23:19:44.747177+0000","last_unstale":"2024-04-26T23:19:46.171787+0000","last_undegraded":"2024-04-26T23:19:46.171787+0000","last_fullsized":"2024-04-26T23:19:46.171787+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:25:49.056874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.051548+0000","last_change":"2024-04-26T23:19:44.737537+0000","last_active":"2024-04-26T23:19:46.051548+0000","last_peered":"2024-04-26T23:19:46.051548+0000","last_clean":"2024-04-26T23:19:46.051548+0000","last_became_active":"2024-04-26T23:19:44.737241+0000","last_became_peered":"2024-04-26T23:19:44.737241+0000","last_unstale":"2024-04-26T23:19:46.051548+0000","last_undegraded":"2024-04-26T23:19:46.051548+0000","last_fullsized":"2024-04-26T23:19:46.051548+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:05:06.952605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.171253+0000","last_change":"2024-04-26T23:19:44.745921+0000","last_active":"2024-04-26T23:19:46.171253+0000","last_peered":"2024-04-26T23:19:46.171253+0000","last_clean":"2024-04-26T23:19:46.171253+0000","last_became_active":"2024-04-26T23:19:44.745624+0000","last_became_peered":"2024-04-26T23:19:44.745624+0000","last_unstale":"2024-04-26T23:19:46.171253+0000","last_undegraded":"2024-04-26T23:19:46.171253+0000","last_fullsized":"2024-04-26T23:19:46.171253+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:41:04.761590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.170979+0000","last_change":"2024-04-26T23:19:44.744246+0000","last_active":"2024-04-26T23:19:46.170979+0000","last_peered":"2024-04-26T23:19:46.170979+0000","last_clean":"2024-04-26T23:19:46.170979+0000","last_became_active":"2024-04-26T23:19:44.743602+0000","last_became_peered":"2024-04-26T23:19:44.743602+0000","last_unstale":"2024-04-26T23:19:46.170979+0000","last_undegraded":"2024-04-26T23:19:46.170979+0000","last_fullsized":"2024-04-26T23:19:46.170979+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:03:02.445698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.171185+0000","last_change":"2024-04-26T23:19:44.747581+0000","last_active":"2024-04-26T23:19:46.171185+0000","last_peered":"2024-04-26T23:19:46.171185+0000","last_clean":"2024-04-26T23:19:46.171185+0000","last_became_active":"2024-04-26T23:19:44.747279+0000","last_became_peered":"2024-04-26T23:19:44.747279+0000","last_unstale":"2024-04-26T23:19:46.171185+0000","last_undegraded":"2024-04-26T23:19:46.171185+0000","last_fullsized":"2024-04-26T23:19:46.171185+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:30:13.827745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.171748+0000","last_change":"2024-04-26T23:19:44.745673+0000","last_active":"2024-04-26T23:19:46.171748+0000","last_peered":"2024-04-26T23:19:46.171748+0000","last_clean":"2024-04-26T23:19:46.171748+0000","last_became_active":"2024-04-26T23:19:44.744971+0000","last_became_peered":"2024-04-26T23:19:44.744971+0000","last_unstale":"2024-04-26T23:19:46.171748+0000","last_undegraded":"2024-04-26T23:19:46.171748+0000","last_fullsized":"2024-04-26T23:19:46.171748+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:31:09.766783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.172302+0000","last_change":"2024-04-26T23:19:44.746739+0000","last_active":"2024-04-26T23:19:46.172302+0000","last_peered":"2024-04-26T23:19:46.172302+0000","last_clean":"2024-04-26T23:19:46.172302+0000","last_became_active":"2024-04-26T23:19:44.746445+0000","last_became_peered":"2024-04-26T23:19:44.746445+0000","last_unstale":"2024-04-26T23:19:46.172302+0000","last_undegraded":"2024-04-26T23:19:46.172302+0000","last_fullsized":"2024-04-26T23:19:46.172302+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:05:31.972319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:45.741657+0000","last_change":"2024-04-26T23:19:44.738341+0000","last_active":"2024-04-26T23:19:45.741657+0000","last_peered":"2024-04-26T23:19:45.741657+0000","last_clean":"2024-04-26T23:19:45.741657+0000","last_became_active":"2024-04-26T23:19:44.737189+0000","last_became_peered":"2024-04-26T23:19:44.737189+0000","last_unstale":"2024-04-26T23:19:45.741657+0000","last_undegraded":"2024-04-26T23:19:45.741657+0000","last_fullsized":"2024-04-26T23:19:45.741657+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:10:04.483872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:45.736033+0000","last_change":"2024-04-26T23:19:44.730700+0000","last_active":"2024-04-26T23:19:45.736033+0000","last_peered":"2024-04-26T23:19:45.736033+0000","last_clean":"2024-04-26T23:19:45.736033+0000","last_became_active":"2024-04-26T23:19:44.730406+0000","last_became_peered":"2024-04-26T23:19:44.730406+0000","last_unstale":"2024-04-26T23:19:45.736033+0000","last_undegraded":"2024-04-26T23:19:45.736033+0000","last_fullsized":"2024-04-26T23:19:45.736033+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:18.699721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:45.741560+0000","last_change":"2024-04-26T23:19:44.737341+0000","last_active":"2024-04-26T23:19:45.741560+0000","last_peered":"2024-04-26T23:19:45.741560+0000","last_clean":"2024-04-26T23:19:45.741560+0000","last_became_active":"2024-04-26T23:19:44.737071+0000","last_became_peered":"2024-04-26T23:19:44.737071+0000","last_unstale":"2024-04-26T23:19:45.741560+0000","last_undegraded":"2024-04-26T23:19:45.741560+0000","last_fullsized":"2024-04-26T23:19:45.741560+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:00.258608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.171452+0000","last_change":"2024-04-26T23:19:44.744750+0000","last_active":"2024-04-26T23:19:46.171452+0000","last_peered":"2024-04-26T23:19:46.171452+0000","last_clean":"2024-04-26T23:19:46.171452+0000","last_became_active":"2024-04-26T23:19:44.740490+0000","last_became_peered":"2024-04-26T23:19:44.740490+0000","last_unstale":"2024-04-26T23:19:46.171452+0000","last_undegraded":"2024-04-26T23:19:46.171452+0000","last_fullsized":"2024-04-26T23:19:46.171452+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:21:39.753813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.172498+0000","last_change":"2024-04-26T23:19:44.737318+0000","last_active":"2024-04-26T23:19:46.172498+0000","last_peered":"2024-04-26T23:19:46.172498+0000","last_clean":"2024-04-26T23:19:46.172498+0000","last_became_active":"2024-04-26T23:19:44.737053+0000","last_became_peered":"2024-04-26T23:19:44.737053+0000","last_unstale":"2024-04-26T23:19:46.172498+0000","last_undegraded":"2024-04-26T23:19:46.172498+0000","last_fullsized":"2024-04-26T23:19:46.172498+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:04:50.465540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.171169+0000","last_change":"2024-04-26T23:19:44.743410+0000","last_active":"2024-04-26T23:19:46.171169+0000","last_peered":"2024-04-26T23:19:46.171169+0000","last_clean":"2024-04-26T23:19:46.171169+0000","last_became_active":"2024-04-26T23:19:44.743146+0000","last_became_peered":"2024-04-26T23:19:44.743146+0000","last_unstale":"2024-04-26T23:19:46.171169+0000","last_undegraded":"2024-04-26T23:19:46.171169+0000","last_fullsized":"2024-04-26T23:19:46.171169+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:25:16.575810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:45.735861+0000","last_change":"2024-04-26T23:19:44.741921+0000","last_active":"2024-04-26T23:19:45.735861+0000","last_peered":"2024-04-26T23:19:45.735861+0000","last_clean":"2024-04-26T23:19:45.735861+0000","last_became_active":"2024-04-26T23:19:44.741622+0000","last_became_peered":"2024-04-26T23:19:44.741622+0000","last_unstale":"2024-04-26T23:19:45.735861+0000","last_undegraded":"2024-04-26T23:19:45.735861+0000","last_fullsized":"2024-04-26T23:19:45.735861+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:57:50.825085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:45.741762+0000","last_change":"2024-04-26T23:19:44.731737+0000","last_active":"2024-04-26T23:19:45.741762+0000","last_peered":"2024-04-26T23:19:45.741762+0000","last_clean":"2024-04-26T23:19:45.741762+0000","last_became_active":"2024-04-26T23:19:44.731419+0000","last_became_peered":"2024-04-26T23:19:44.731419+0000","last_unstale":"2024-04-26T23:19:45.741762+0000","last_undegraded":"2024-04-26T23:19:45.741762+0000","last_fullsized":"2024-04-26T23:19:45.741762+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:03:02.445698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:45.741373+0000","last_change":"2024-04-26T23:19:44.747854+0000","last_active":"2024-04-26T23:19:45.741373+0000","last_peered":"2024-04-26T23:19:45.741373+0000","last_clean":"2024-04-26T23:19:45.741373+0000","last_became_active":"2024-04-26T23:19:44.747549+0000","last_became_peered":"2024-04-26T23:19:44.747549+0000","last_unstale":"2024-04-26T23:19:45.741373+0000","last_undegraded":"2024-04-26T23:19:45.741373+0000","last_fullsized":"2024-04-26T23:19:45.741373+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:51:35.553507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:45.741682+0000","last_change":"2024-04-26T23:19:44.737981+0000","last_active":"2024-04-26T23:19:45.741682+0000","last_peered":"2024-04-26T23:19:45.741682+0000","last_clean":"2024-04-26T23:19:45.741682+0000","last_became_active":"2024-04-26T23:19:44.737726+0000","last_became_peered":"2024-04-26T23:19:44.737726+0000","last_unstale":"2024-04-26T23:19:45.741682+0000","last_undegraded":"2024-04-26T23:19:45.741682+0000","last_fullsized":"2024-04-26T23:19:45.741682+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:23:54.684004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.171672+0000","last_change":"2024-04-26T23:19:44.740659+0000","last_active":"2024-04-26T23:19:46.171672+0000","last_peered":"2024-04-26T23:19:46.171672+0000","last_clean":"2024-04-26T23:19:46.171672+0000","last_became_active":"2024-04-26T23:19:44.740392+0000","last_became_peered":"2024-04-26T23:19:44.740392+0000","last_unstale":"2024-04-26T23:19:46.171672+0000","last_undegraded":"2024-04-26T23:19:46.171672+0000","last_fullsized":"2024-04-26T23:19:46.171672+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:01.479058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:45.741469+0000","last_change":"2024-04-26T23:19:44.740193+0000","last_active":"2024-04-26T23:19:45.741469+0000","last_peered":"2024-04-26T23:19:45.741469+0000","last_clean":"2024-04-26T23:19:45.741469+0000","last_became_active":"2024-04-26T23:19:44.739875+0000","last_became_peered":"2024-04-26T23:19:44.739875+0000","last_unstale":"2024-04-26T23:19:45.741469+0000","last_undegraded":"2024-04-26T23:19:45.741469+0000","last_fullsized":"2024-04-26T23:19:45.741469+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:13:27.220211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:45.741259+0000","last_change":"2024-04-26T23:19:44.743937+0000","last_active":"2024-04-26T23:19:45.741259+0000","last_peered":"2024-04-26T23:19:45.741259+0000","last_clean":"2024-04-26T23:19:45.741259+0000","last_became_active":"2024-04-26T23:19:44.743682+0000","last_became_peered":"2024-04-26T23:19:44.743682+0000","last_unstale":"2024-04-26T23:19:45.741259+0000","last_undegraded":"2024-04-26T23:19:45.741259+0000","last_fullsized":"2024-04-26T23:19:45.741259+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:42:32.347144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.171335+0000","last_change":"2024-04-26T23:19:44.742617+0000","last_active":"2024-04-26T23:19:46.171335+0000","last_peered":"2024-04-26T23:19:46.171335+0000","last_clean":"2024-04-26T23:19:46.171335+0000","last_became_active":"2024-04-26T23:19:44.742206+0000","last_became_peered":"2024-04-26T23:19:44.742206+0000","last_unstale":"2024-04-26T23:19:46.171335+0000","last_undegraded":"2024-04-26T23:19:46.171335+0000","last_fullsized":"2024-04-26T23:19:46.171335+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:21:00.734571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:45.741179+0000","last_change":"2024-04-26T23:19:44.724223+0000","last_active":"2024-04-26T23:19:45.741179+0000","last_peered":"2024-04-26T23:19:45.741179+0000","last_clean":"2024-04-26T23:19:45.741179+0000","last_became_active":"2024-04-26T23:19:44.723914+0000","last_became_peered":"2024-04-26T23:19:44.723914+0000","last_unstale":"2024-04-26T23:19:45.741179+0000","last_undegraded":"2024-04-26T23:19:45.741179+0000","last_fullsized":"2024-04-26T23:19:45.741179+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:13.113216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.171072+0000","last_change":"2024-04-26T23:19:44.743706+0000","last_active":"2024-04-26T23:19:46.171072+0000","last_peered":"2024-04-26T23:19:46.171072+0000","last_clean":"2024-04-26T23:19:46.171072+0000","last_became_active":"2024-04-26T23:19:44.743441+0000","last_became_peered":"2024-04-26T23:19:44.743441+0000","last_unstale":"2024-04-26T23:19:46.171072+0000","last_undegraded":"2024-04-26T23:19:46.171072+0000","last_fullsized":"2024-04-26T23:19:46.171072+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:38:45.475720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:45.741709+0000","last_change":"2024-04-26T23:19:43.728273+0000","last_active":"2024-04-26T23:19:45.741709+0000","last_peered":"2024-04-26T23:19:45.741709+0000","last_clean":"2024-04-26T23:19:45.741709+0000","last_became_active":"2024-04-26T23:19:43.728009+0000","last_became_peered":"2024-04-26T23:19:43.728009+0000","last_unstale":"2024-04-26T23:19:45.741709+0000","last_undegraded":"2024-04-26T23:19:45.741709+0000","last_fullsized":"2024-04-26T23:19:45.741709+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:17.697208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:45.741403+0000","last_change":"2024-04-26T23:19:44.747892+0000","last_active":"2024-04-26T23:19:45.741403+0000","last_peered":"2024-04-26T23:19:45.741403+0000","last_clean":"2024-04-26T23:19:45.741403+0000","last_became_active":"2024-04-26T23:19:44.747447+0000","last_became_peered":"2024-04-26T23:19:44.747447+0000","last_unstale":"2024-04-26T23:19:45.741403+0000","last_undegraded":"2024-04-26T23:19:45.741403+0000","last_fullsized":"2024-04-26T23:19:45.741403+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:38:45.475720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.051675+0000","last_change":"2024-04-26T23:19:43.734459+0000","last_active":"2024-04-26T23:19:46.051675+0000","last_peered":"2024-04-26T23:19:46.051675+0000","last_clean":"2024-04-26T23:19:46.051675+0000","last_became_active":"2024-04-26T23:19:43.733945+0000","last_became_peered":"2024-04-26T23:19:43.733945+0000","last_unstale":"2024-04-26T23:19:46.051675+0000","last_undegraded":"2024-04-26T23:19:46.051675+0000","last_fullsized":"2024-04-26T23:19:46.051675+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:37.017797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.172596+0000","last_change":"2024-04-26T23:19:44.744198+0000","last_active":"2024-04-26T23:19:46.172596+0000","last_peered":"2024-04-26T23:19:46.172596+0000","last_clean":"2024-04-26T23:19:46.172596+0000","last_became_active":"2024-04-26T23:19:44.743700+0000","last_became_peered":"2024-04-26T23:19:44.743700+0000","last_unstale":"2024-04-26T23:19:46.172596+0000","last_undegraded":"2024-04-26T23:19:46.172596+0000","last_fullsized":"2024-04-26T23:19:46.172596+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:05:12.028008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:45.741388+0000","last_change":"2024-04-26T23:19:43.728369+0000","last_active":"2024-04-26T23:19:45.741388+0000","last_peered":"2024-04-26T23:19:45.741388+0000","last_clean":"2024-04-26T23:19:45.741388+0000","last_became_active":"2024-04-26T23:19:43.728090+0000","last_became_peered":"2024-04-26T23:19:43.728090+0000","last_unstale":"2024-04-26T23:19:45.741388+0000","last_undegraded":"2024-04-26T23:19:45.741388+0000","last_fullsized":"2024-04-26T23:19:45.741388+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:37.017797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.170474+0000","last_change":"2024-04-26T23:19:44.737276+0000","last_active":"2024-04-26T23:19:46.170474+0000","last_peered":"2024-04-26T23:19:46.170474+0000","last_clean":"2024-04-26T23:19:46.170474+0000","last_became_active":"2024-04-26T23:19:44.736873+0000","last_became_peered":"2024-04-26T23:19:44.736873+0000","last_unstale":"2024-04-26T23:19:46.170474+0000","last_undegraded":"2024-04-26T23:19:46.170474+0000","last_fullsized":"2024-04-26T23:19:46.170474+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:11:53.259288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.172735+0000","last_change":"2024-04-26T23:19:43.727630+0000","last_active":"2024-04-26T23:19:46.172735+0000","last_peered":"2024-04-26T23:19:46.172735+0000","last_clean":"2024-04-26T23:19:46.172735+0000","last_became_active":"2024-04-26T23:19:43.726916+0000","last_became_peered":"2024-04-26T23:19:43.726916+0000","last_unstale":"2024-04-26T23:19:46.172735+0000","last_undegraded":"2024-04-26T23:19:46.172735+0000","last_fullsized":"2024-04-26T23:19:46.172735+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:34:07.760043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.051397+0000","last_change":"2024-04-26T23:19:44.736174+0000","last_active":"2024-04-26T23:19:46.051397+0000","last_peered":"2024-04-26T23:19:46.051397+0000","last_clean":"2024-04-26T23:19:46.051397+0000","last_became_active":"2024-04-26T23:19:44.735861+0000","last_became_peered":"2024-04-26T23:19:44.735861+0000","last_unstale":"2024-04-26T23:19:46.051397+0000","last_undegraded":"2024-04-26T23:19:46.051397+0000","last_fullsized":"2024-04-26T23:19:46.051397+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:04:50.465540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.171446+0000","last_change":"2024-04-26T23:19:43.730835+0000","last_active":"2024-04-26T23:19:46.171446+0000","last_peered":"2024-04-26T23:19:46.171446+0000","last_clean":"2024-04-26T23:19:46.171446+0000","last_became_active":"2024-04-26T23:19:43.730566+0000","last_became_peered":"2024-04-26T23:19:43.730566+0000","last_unstale":"2024-04-26T23:19:46.171446+0000","last_undegraded":"2024-04-26T23:19:46.171446+0000","last_fullsized":"2024-04-26T23:19:46.171446+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:42:31.344631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.171477+0000","last_change":"2024-04-26T23:19:44.732932+0000","last_active":"2024-04-26T23:19:46.171477+0000","last_peered":"2024-04-26T23:19:46.171477+0000","last_clean":"2024-04-26T23:19:46.171477+0000","last_became_active":"2024-04-26T23:19:44.732585+0000","last_became_peered":"2024-04-26T23:19:44.732585+0000","last_unstale":"2024-04-26T23:19:46.171477+0000","last_undegraded":"2024-04-26T23:19:46.171477+0000","last_fullsized":"2024-04-26T23:19:46.171477+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:14:15.379049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.171154+0000","last_change":"2024-04-26T23:19:43.737887+0000","last_active":"2024-04-26T23:19:46.171154+0000","last_peered":"2024-04-26T23:19:46.171154+0000","last_clean":"2024-04-26T23:19:46.171154+0000","last_became_active":"2024-04-26T23:19:43.737494+0000","last_became_peered":"2024-04-26T23:19:43.737494+0000","last_unstale":"2024-04-26T23:19:46.171154+0000","last_undegraded":"2024-04-26T23:19:46.171154+0000","last_fullsized":"2024-04-26T23:19:46.171154+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:56.075572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.172171+0000","last_change":"2024-04-26T23:19:44.748773+0000","last_active":"2024-04-26T23:19:46.172171+0000","last_peered":"2024-04-26T23:19:46.172171+0000","last_clean":"2024-04-26T23:19:46.172171+0000","last_became_active":"2024-04-26T23:19:44.748469+0000","last_became_peered":"2024-04-26T23:19:44.748469+0000","last_unstale":"2024-04-26T23:19:46.172171+0000","last_undegraded":"2024-04-26T23:19:46.172171+0000","last_fullsized":"2024-04-26T23:19:46.172171+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:24:23.961628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:45.741116+0000","last_change":"2024-04-26T23:19:43.726706+0000","last_active":"2024-04-26T23:19:45.741116+0000","last_peered":"2024-04-26T23:19:45.741116+0000","last_clean":"2024-04-26T23:19:45.741116+0000","last_became_active":"2024-04-26T23:19:43.726399+0000","last_became_peered":"2024-04-26T23:19:43.726399+0000","last_unstale":"2024-04-26T23:19:45.741116+0000","last_undegraded":"2024-04-26T23:19:45.741116+0000","last_fullsized":"2024-04-26T23:19:45.741116+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:42:31.344631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.170509+0000","last_change":"2024-04-26T23:19:44.737772+0000","last_active":"2024-04-26T23:19:46.170509+0000","last_peered":"2024-04-26T23:19:46.170509+0000","last_clean":"2024-04-26T23:19:46.170509+0000","last_became_active":"2024-04-26T23:19:44.737243+0000","last_became_peered":"2024-04-26T23:19:44.737243+0000","last_unstale":"2024-04-26T23:19:46.170509+0000","last_undegraded":"2024-04-26T23:19:46.170509+0000","last_fullsized":"2024-04-26T23:19:46.170509+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:05:31.972319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.170990+0000","last_change":"2024-04-26T23:19:43.731843+0000","last_active":"2024-04-26T23:19:46.170990+0000","last_peered":"2024-04-26T23:19:46.170990+0000","last_clean":"2024-04-26T23:19:46.170990+0000","last_became_active":"2024-04-26T23:19:43.731575+0000","last_became_peered":"2024-04-26T23:19:43.731575+0000","last_unstale":"2024-04-26T23:19:46.170990+0000","last_undegraded":"2024-04-26T23:19:46.170990+0000","last_fullsized":"2024-04-26T23:19:46.170990+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:23:52.008045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:45.736253+0000","last_change":"2024-04-26T23:19:44.734983+0000","last_active":"2024-04-26T23:19:45.736253+0000","last_peered":"2024-04-26T23:19:45.736253+0000","last_clean":"2024-04-26T23:19:45.736253+0000","last_became_active":"2024-04-26T23:19:44.733325+0000","last_became_peered":"2024-04-26T23:19:44.733325+0000","last_unstale":"2024-04-26T23:19:45.736253+0000","last_undegraded":"2024-04-26T23:19:45.736253+0000","last_fullsized":"2024-04-26T23:19:45.736253+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:23:53.010557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.170507+0000","last_change":"2024-04-26T23:19:43.730610+0000","last_active":"2024-04-26T23:19:46.170507+0000","last_peered":"2024-04-26T23:19:46.170507+0000","last_clean":"2024-04-26T23:19:46.170507+0000","last_became_active":"2024-04-26T23:19:43.730288+0000","last_became_peered":"2024-04-26T23:19:43.730288+0000","last_unstale":"2024-04-26T23:19:46.170507+0000","last_undegraded":"2024-04-26T23:19:46.170507+0000","last_fullsized":"2024-04-26T23:19:46.170507+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:44:44.154186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.172144+0000","last_change":"2024-04-26T23:19:44.737797+0000","last_active":"2024-04-26T23:19:46.172144+0000","last_peered":"2024-04-26T23:19:46.172144+0000","last_clean":"2024-04-26T23:19:46.172144+0000","last_became_active":"2024-04-26T23:19:44.737193+0000","last_became_peered":"2024-04-26T23:19:44.737193+0000","last_unstale":"2024-04-26T23:19:46.172144+0000","last_undegraded":"2024-04-26T23:19:46.172144+0000","last_fullsized":"2024-04-26T23:19:46.172144+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:09:16.884371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.171789+0000","last_change":"2024-04-26T23:19:43.734757+0000","last_active":"2024-04-26T23:19:46.171789+0000","last_peered":"2024-04-26T23:19:46.171789+0000","last_clean":"2024-04-26T23:19:46.171789+0000","last_became_active":"2024-04-26T23:19:43.734295+0000","last_became_peered":"2024-04-26T23:19:43.734295+0000","last_unstale":"2024-04-26T23:19:46.171789+0000","last_undegraded":"2024-04-26T23:19:46.171789+0000","last_fullsized":"2024-04-26T23:19:46.171789+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:13:26.217699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.051747+0000","last_change":"2024-04-26T23:19:44.745014+0000","last_active":"2024-04-26T23:19:46.051747+0000","last_peered":"2024-04-26T23:19:46.051747+0000","last_clean":"2024-04-26T23:19:46.051747+0000","last_became_active":"2024-04-26T23:19:44.744659+0000","last_became_peered":"2024-04-26T23:19:44.744659+0000","last_unstale":"2024-04-26T23:19:46.051747+0000","last_undegraded":"2024-04-26T23:19:46.051747+0000","last_fullsized":"2024-04-26T23:19:46.051747+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:38:45.475720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":17,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.172011+0000","last_change":"2024-04-26T23:19:43.734517+0000","last_active":"2024-04-26T23:19:46.172011+0000","last_peered":"2024-04-26T23:19:46.172011+0000","last_clean":"2024-04-26T23:19:46.172011+0000","last_became_active":"2024-04-26T23:19:43.734234+0000","last_became_peered":"2024-04-26T23:19:43.734234+0000","last_unstale":"2024-04-26T23:19:46.172011+0000","last_undegraded":"2024-04-26T23:19:46.172011+0000","last_fullsized":"2024-04-26T23:19:46.172011+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:03:01.443185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.172506+0000","last_change":"2024-04-26T23:19:44.738505+0000","last_active":"2024-04-26T23:19:46.172506+0000","last_peered":"2024-04-26T23:19:46.172506+0000","last_clean":"2024-04-26T23:19:46.172506+0000","last_became_active":"2024-04-26T23:19:44.738013+0000","last_became_peered":"2024-04-26T23:19:44.738013+0000","last_unstale":"2024-04-26T23:19:46.172506+0000","last_undegraded":"2024-04-26T23:19:46.172506+0000","last_fullsized":"2024-04-26T23:19:46.172506+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:13.113216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'6","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.170200+0000","last_change":"2024-04-26T23:19:43.731960+0000","last_active":"2024-04-26T23:19:46.170200+0000","last_peered":"2024-04-26T23:19:46.170200+0000","last_clean":"2024-04-26T23:19:46.170200+0000","last_became_active":"2024-04-26T23:19:43.731659+0000","last_became_peered":"2024-04-26T23:19:43.731659+0000","last_unstale":"2024-04-26T23:19:46.170200+0000","last_undegraded":"2024-04-26T23:19:46.170200+0000","last_fullsized":"2024-04-26T23:19:46.170200+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:24:22.959116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.051419+0000","last_change":"2024-04-26T23:19:44.749061+0000","last_active":"2024-04-26T23:19:46.051419+0000","last_peered":"2024-04-26T23:19:46.051419+0000","last_clean":"2024-04-26T23:19:46.051419+0000","last_became_active":"2024-04-26T23:19:44.748755+0000","last_became_peered":"2024-04-26T23:19:44.748755+0000","last_unstale":"2024-04-26T23:19:46.051419+0000","last_undegraded":"2024-04-26T23:19:46.051419+0000","last_fullsized":"2024-04-26T23:19:46.051419+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:13.113216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.170460+0000","last_change":"2024-04-26T23:19:43.732888+0000","last_active":"2024-04-26T23:19:46.170460+0000","last_peered":"2024-04-26T23:19:46.170460+0000","last_clean":"2024-04-26T23:19:46.170460+0000","last_became_active":"2024-04-26T23:19:43.732606+0000","last_became_peered":"2024-04-26T23:19:43.732606+0000","last_unstale":"2024-04-26T23:19:46.170460+0000","last_undegraded":"2024-04-26T23:19:46.170460+0000","last_fullsized":"2024-04-26T23:19:46.170460+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:03:01.443185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.170168+0000","last_change":"2024-04-26T23:19:44.733999+0000","last_active":"2024-04-26T23:19:46.170168+0000","last_peered":"2024-04-26T23:19:46.170168+0000","last_clean":"2024-04-26T23:19:46.170168+0000","last_became_active":"2024-04-26T23:19:44.733702+0000","last_became_peered":"2024-04-26T23:19:44.733702+0000","last_unstale":"2024-04-26T23:19:46.170168+0000","last_undegraded":"2024-04-26T23:19:46.170168+0000","last_fullsized":"2024-04-26T23:19:46.170168+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:41:04.761590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:45.741154+0000","last_change":"2024-04-26T23:19:43.732756+0000","last_active":"2024-04-26T23:19:45.741154+0000","last_peered":"2024-04-26T23:19:45.741154+0000","last_clean":"2024-04-26T23:19:45.741154+0000","last_became_active":"2024-04-26T23:19:43.732452+0000","last_became_peered":"2024-04-26T23:19:43.732452+0000","last_unstale":"2024-04-26T23:19:45.741154+0000","last_undegraded":"2024-04-26T23:19:45.741154+0000","last_fullsized":"2024-04-26T23:19:45.741154+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:56.075572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.171036+0000","last_change":"2024-04-26T23:19:44.741851+0000","last_active":"2024-04-26T23:19:46.171036+0000","last_peered":"2024-04-26T23:19:46.171036+0000","last_clean":"2024-04-26T23:19:46.171036+0000","last_became_active":"2024-04-26T23:19:44.740080+0000","last_became_peered":"2024-04-26T23:19:44.740080+0000","last_unstale":"2024-04-26T23:19:46.171036+0000","last_undegraded":"2024-04-26T23:19:46.171036+0000","last_fullsized":"2024-04-26T23:19:46.171036+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:24:21.554815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:45.741146+0000","last_change":"2024-04-26T23:19:43.729238+0000","last_active":"2024-04-26T23:19:45.741146+0000","last_peered":"2024-04-26T23:19:45.741146+0000","last_clean":"2024-04-26T23:19:45.741146+0000","last_became_active":"2024-04-26T23:19:43.728884+0000","last_became_peered":"2024-04-26T23:19:43.728884+0000","last_unstale":"2024-04-26T23:19:45.741146+0000","last_undegraded":"2024-04-26T23:19:45.741146+0000","last_fullsized":"2024-04-26T23:19:45.741146+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:41.199643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.170228+0000","last_change":"2024-04-26T23:19:44.734360+0000","last_active":"2024-04-26T23:19:46.170228+0000","last_peered":"2024-04-26T23:19:46.170228+0000","last_clean":"2024-04-26T23:19:46.170228+0000","last_became_active":"2024-04-26T23:19:44.733497+0000","last_became_peered":"2024-04-26T23:19:44.733497+0000","last_unstale":"2024-04-26T23:19:46.170228+0000","last_undegraded":"2024-04-26T23:19:46.170228+0000","last_fullsized":"2024-04-26T23:19:46.170228+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:09:16.884371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.171803+0000","last_change":"2024-04-26T23:19:43.737740+0000","last_active":"2024-04-26T23:19:46.171803+0000","last_peered":"2024-04-26T23:19:46.171803+0000","last_clean":"2024-04-26T23:19:46.171803+0000","last_became_active":"2024-04-26T23:19:43.737443+0000","last_became_peered":"2024-04-26T23:19:43.737443+0000","last_unstale":"2024-04-26T23:19:46.171803+0000","last_undegraded":"2024-04-26T23:19:46.171803+0000","last_fullsized":"2024-04-26T23:19:46.171803+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:42:31.344631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.170275+0000","last_change":"2024-04-26T23:19:44.735472+0000","last_active":"2024-04-26T23:19:46.170275+0000","last_peered":"2024-04-26T23:19:46.170275+0000","last_clean":"2024-04-26T23:19:46.170275+0000","last_became_active":"2024-04-26T23:19:44.735154+0000","last_became_peered":"2024-04-26T23:19:44.735154+0000","last_unstale":"2024-04-26T23:19:46.170275+0000","last_undegraded":"2024-04-26T23:19:46.170275+0000","last_fullsized":"2024-04-26T23:19:46.170275+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:21:39.753813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.171623+0000","last_change":"2024-04-26T23:19:43.735048+0000","last_active":"2024-04-26T23:19:46.171623+0000","last_peered":"2024-04-26T23:19:46.171623+0000","last_clean":"2024-04-26T23:19:46.171623+0000","last_became_active":"2024-04-26T23:19:43.734332+0000","last_became_peered":"2024-04-26T23:19:43.734332+0000","last_unstale":"2024-04-26T23:19:46.171623+0000","last_undegraded":"2024-04-26T23:19:46.171623+0000","last_fullsized":"2024-04-26T23:19:46.171623+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:35:59.256095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.171978+0000","last_change":"2024-04-26T23:19:44.744348+0000","last_active":"2024-04-26T23:19:46.171978+0000","last_peered":"2024-04-26T23:19:46.171978+0000","last_clean":"2024-04-26T23:19:46.171978+0000","last_became_active":"2024-04-26T23:19:44.743957+0000","last_became_peered":"2024-04-26T23:19:44.743957+0000","last_unstale":"2024-04-26T23:19:46.171978+0000","last_undegraded":"2024-04-26T23:19:46.171978+0000","last_fullsized":"2024-04-26T23:19:46.171978+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:11:53.259288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.172684+0000","last_change":"2024-04-26T23:19:43.735311+0000","last_active":"2024-04-26T23:19:46.172684+0000","last_peered":"2024-04-26T23:19:46.172684+0000","last_clean":"2024-04-26T23:19:46.172684+0000","last_became_active":"2024-04-26T23:19:43.735022+0000","last_became_peered":"2024-04-26T23:19:43.735022+0000","last_unstale":"2024-04-26T23:19:46.172684+0000","last_undegraded":"2024-04-26T23:19:46.172684+0000","last_fullsized":"2024-04-26T23:19:46.172684+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:44:49.809340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.172555+0000","last_change":"2024-04-26T23:19:44.739631+0000","last_active":"2024-04-26T23:19:46.172555+0000","last_peered":"2024-04-26T23:19:46.172555+0000","last_clean":"2024-04-26T23:19:46.172555+0000","last_became_active":"2024-04-26T23:19:44.739343+0000","last_became_peered":"2024-04-26T23:19:44.739343+0000","last_unstale":"2024-04-26T23:19:46.172555+0000","last_undegraded":"2024-04-26T23:19:46.172555+0000","last_fullsized":"2024-04-26T23:19:46.172555+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:05:06.952605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:45.735966+0000","last_change":"2024-04-26T23:19:43.732071+0000","last_active":"2024-04-26T23:19:45.735966+0000","last_peered":"2024-04-26T23:19:45.735966+0000","last_clean":"2024-04-26T23:19:45.735966+0000","last_became_active":"2024-04-26T23:19:43.731719+0000","last_became_peered":"2024-04-26T23:19:43.731719+0000","last_unstale":"2024-04-26T23:19:45.735966+0000","last_undegraded":"2024-04-26T23:19:45.735966+0000","last_fullsized":"2024-04-26T23:19:45.735966+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:38:57.307175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:45.736167+0000","last_change":"2024-04-26T23:19:44.728005+0000","last_active":"2024-04-26T23:19:45.736167+0000","last_peered":"2024-04-26T23:19:45.736167+0000","last_clean":"2024-04-26T23:19:45.736167+0000","last_became_active":"2024-04-26T23:19:44.727721+0000","last_became_peered":"2024-04-26T23:19:44.727721+0000","last_unstale":"2024-04-26T23:19:45.736167+0000","last_undegraded":"2024-04-26T23:19:45.736167+0000","last_fullsized":"2024-04-26T23:19:45.736167+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:42.202155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:45.741175+0000","last_change":"2024-04-26T23:19:43.728356+0000","last_active":"2024-04-26T23:19:45.741175+0000","last_peered":"2024-04-26T23:19:45.741175+0000","last_clean":"2024-04-26T23:19:45.741175+0000","last_became_active":"2024-04-26T23:19:43.728037+0000","last_became_peered":"2024-04-26T23:19:43.728037+0000","last_unstale":"2024-04-26T23:19:45.741175+0000","last_undegraded":"2024-04-26T23:19:45.741175+0000","last_fullsized":"2024-04-26T23:19:45.741175+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:44:49.809340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.171931+0000","last_change":"2024-04-26T23:19:44.742843+0000","last_active":"2024-04-26T23:19:46.171931+0000","last_peered":"2024-04-26T23:19:46.171931+0000","last_clean":"2024-04-26T23:19:46.171931+0000","last_became_active":"2024-04-26T23:19:44.742597+0000","last_became_peered":"2024-04-26T23:19:44.742597+0000","last_unstale":"2024-04-26T23:19:46.171931+0000","last_undegraded":"2024-04-26T23:19:46.171931+0000","last_fullsized":"2024-04-26T23:19:46.171931+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:30:13.827745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:45.735949+0000","last_change":"2024-04-26T23:19:43.729782+0000","last_active":"2024-04-26T23:19:45.735949+0000","last_peered":"2024-04-26T23:19:45.735949+0000","last_clean":"2024-04-26T23:19:45.735949+0000","last_became_active":"2024-04-26T23:19:43.729470+0000","last_became_peered":"2024-04-26T23:19:43.729470+0000","last_unstale":"2024-04-26T23:19:45.735949+0000","last_undegraded":"2024-04-26T23:19:45.735949+0000","last_fullsized":"2024-04-26T23:19:45.735949+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:35:10.661502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.051569+0000","last_change":"2024-04-26T23:19:44.745025+0000","last_active":"2024-04-26T23:19:46.051569+0000","last_peered":"2024-04-26T23:19:46.051569+0000","last_clean":"2024-04-26T23:19:46.051569+0000","last_became_active":"2024-04-26T23:19:44.744650+0000","last_became_peered":"2024-04-26T23:19:44.744650+0000","last_unstale":"2024-04-26T23:19:46.051569+0000","last_undegraded":"2024-04-26T23:19:46.051569+0000","last_fullsized":"2024-04-26T23:19:46.051569+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:05:12.028008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:45.735876+0000","last_change":"2024-04-26T23:19:43.726984+0000","last_active":"2024-04-26T23:19:45.735876+0000","last_peered":"2024-04-26T23:19:45.735876+0000","last_clean":"2024-04-26T23:19:45.735876+0000","last_became_active":"2024-04-26T23:19:43.726697+0000","last_became_peered":"2024-04-26T23:19:43.726697+0000","last_unstale":"2024-04-26T23:19:45.735876+0000","last_undegraded":"2024-04-26T23:19:45.735876+0000","last_fullsized":"2024-04-26T23:19:45.735876+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:00:25.695941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.171082+0000","last_change":"2024-04-26T23:19:44.745078+0000","last_active":"2024-04-26T23:19:46.171082+0000","last_peered":"2024-04-26T23:19:46.171082+0000","last_clean":"2024-04-26T23:19:46.171082+0000","last_became_active":"2024-04-26T23:19:44.741475+0000","last_became_peered":"2024-04-26T23:19:44.741475+0000","last_unstale":"2024-04-26T23:19:46.171082+0000","last_undegraded":"2024-04-26T23:19:46.171082+0000","last_fullsized":"2024-04-26T23:19:46.171082+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:09:16.884371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.171429+0000","last_change":"2024-04-26T23:19:43.737669+0000","last_active":"2024-04-26T23:19:46.171429+0000","last_peered":"2024-04-26T23:19:46.171429+0000","last_clean":"2024-04-26T23:19:46.171429+0000","last_became_active":"2024-04-26T23:19:43.737369+0000","last_became_peered":"2024-04-26T23:19:43.737369+0000","last_unstale":"2024-04-26T23:19:46.171429+0000","last_undegraded":"2024-04-26T23:19:46.171429+0000","last_fullsized":"2024-04-26T23:19:46.171429+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:35:59.256095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.170394+0000","last_change":"2024-04-26T23:19:44.737428+0000","last_active":"2024-04-26T23:19:46.170394+0000","last_peered":"2024-04-26T23:19:46.170394+0000","last_clean":"2024-04-26T23:19:46.170394+0000","last_became_active":"2024-04-26T23:19:44.736850+0000","last_became_peered":"2024-04-26T23:19:44.736850+0000","last_unstale":"2024-04-26T23:19:46.170394+0000","last_undegraded":"2024-04-26T23:19:46.170394+0000","last_fullsized":"2024-04-26T23:19:46.170394+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:25:49.056874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:45.735743+0000","last_change":"2024-04-26T23:19:43.726759+0000","last_active":"2024-04-26T23:19:45.735743+0000","last_peered":"2024-04-26T23:19:45.735743+0000","last_clean":"2024-04-26T23:19:45.735743+0000","last_became_active":"2024-04-26T23:19:43.726410+0000","last_became_peered":"2024-04-26T23:19:43.726410+0000","last_unstale":"2024-04-26T23:19:45.735743+0000","last_undegraded":"2024-04-26T23:19:45.735743+0000","last_fullsized":"2024-04-26T23:19:45.735743+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:20:04.267432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:45.741207+0000","last_change":"2024-04-26T23:19:44.734344+0000","last_active":"2024-04-26T23:19:45.741207+0000","last_peered":"2024-04-26T23:19:45.741207+0000","last_clean":"2024-04-26T23:19:45.741207+0000","last_became_active":"2024-04-26T23:19:44.734045+0000","last_became_peered":"2024-04-26T23:19:44.734045+0000","last_unstale":"2024-04-26T23:19:45.741207+0000","last_undegraded":"2024-04-26T23:19:45.741207+0000","last_fullsized":"2024-04-26T23:19:45.741207+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:05:06.952605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.171832+0000","last_change":"2024-04-26T23:19:43.730239+0000","last_active":"2024-04-26T23:19:46.171832+0000","last_peered":"2024-04-26T23:19:46.171832+0000","last_clean":"2024-04-26T23:19:46.171832+0000","last_became_active":"2024-04-26T23:19:43.729976+0000","last_became_peered":"2024-04-26T23:19:43.729976+0000","last_unstale":"2024-04-26T23:19:46.171832+0000","last_undegraded":"2024-04-26T23:19:46.171832+0000","last_fullsized":"2024-04-26T23:19:46.171832+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:10:03.481359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.172311+0000","last_change":"2024-04-26T23:19:44.738761+0000","last_active":"2024-04-26T23:19:46.172311+0000","last_peered":"2024-04-26T23:19:46.172311+0000","last_clean":"2024-04-26T23:19:46.172311+0000","last_became_active":"2024-04-26T23:19:44.737545+0000","last_became_peered":"2024-04-26T23:19:44.737545+0000","last_unstale":"2024-04-26T23:19:46.172311+0000","last_undegraded":"2024-04-26T23:19:46.172311+0000","last_fullsized":"2024-04-26T23:19:46.172311+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:21:39.753813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"38'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:45.736121+0000","last_change":"2024-04-26T23:19:43.735727+0000","last_active":"2024-04-26T23:19:45.736121+0000","last_peered":"2024-04-26T23:19:45.736121+0000","last_clean":"2024-04-26T23:19:45.736121+0000","last_became_active":"2024-04-26T23:19:43.735412+0000","last_became_peered":"2024-04-26T23:19:43.735412+0000","last_unstale":"2024-04-26T23:19:45.736121+0000","last_undegraded":"2024-04-26T23:19:45.736121+0000","last_fullsized":"2024-04-26T23:19:45.736121+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:11:05.145256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.170393+0000","last_change":"2024-04-26T23:19:44.736242+0000","last_active":"2024-04-26T23:19:46.170393+0000","last_peered":"2024-04-26T23:19:46.170393+0000","last_clean":"2024-04-26T23:19:46.170393+0000","last_became_active":"2024-04-26T23:19:44.735960+0000","last_became_peered":"2024-04-26T23:19:44.735960+0000","last_unstale":"2024-04-26T23:19:46.170393+0000","last_undegraded":"2024-04-26T23:19:46.170393+0000","last_fullsized":"2024-04-26T23:19:46.170393+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:01.479058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.051579+0000","last_change":"2024-04-26T23:19:43.729872+0000","last_active":"2024-04-26T23:19:46.051579+0000","last_peered":"2024-04-26T23:19:46.051579+0000","last_clean":"2024-04-26T23:19:46.051579+0000","last_became_active":"2024-04-26T23:19:43.729499+0000","last_became_peered":"2024-04-26T23:19:43.729499+0000","last_unstale":"2024-04-26T23:19:46.051579+0000","last_undegraded":"2024-04-26T23:19:46.051579+0000","last_fullsized":"2024-04-26T23:19:46.051579+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:34:07.760043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.051603+0000","last_change":"2024-04-26T23:19:44.739851+0000","last_active":"2024-04-26T23:19:46.051603+0000","last_peered":"2024-04-26T23:19:46.051603+0000","last_clean":"2024-04-26T23:19:46.051603+0000","last_became_active":"2024-04-26T23:19:44.739505+0000","last_became_peered":"2024-04-26T23:19:44.739505+0000","last_unstale":"2024-04-26T23:19:46.051603+0000","last_undegraded":"2024-04-26T23:19:46.051603+0000","last_fullsized":"2024-04-26T23:19:46.051603+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:51:35.553507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.051530+0000","last_change":"2024-04-26T23:19:43.727609+0000","last_active":"2024-04-26T23:19:46.051530+0000","last_peered":"2024-04-26T23:19:46.051530+0000","last_clean":"2024-04-26T23:19:46.051530+0000","last_became_active":"2024-04-26T23:19:43.727298+0000","last_became_peered":"2024-04-26T23:19:43.727298+0000","last_unstale":"2024-04-26T23:19:46.051530+0000","last_undegraded":"2024-04-26T23:19:46.051530+0000","last_fullsized":"2024-04-26T23:19:46.051530+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:44:49.809340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:45.741330+0000","last_change":"2024-04-26T23:19:44.737992+0000","last_active":"2024-04-26T23:19:45.741330+0000","last_peered":"2024-04-26T23:19:45.741330+0000","last_clean":"2024-04-26T23:19:45.741330+0000","last_became_active":"2024-04-26T23:19:44.734600+0000","last_became_peered":"2024-04-26T23:19:44.734600+0000","last_unstale":"2024-04-26T23:19:45.741330+0000","last_undegraded":"2024-04-26T23:19:45.741330+0000","last_fullsized":"2024-04-26T23:19:45.741330+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:05:12.028008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:45.741213+0000","last_change":"2024-04-26T23:19:43.726318+0000","last_active":"2024-04-26T23:19:45.741213+0000","last_peered":"2024-04-26T23:19:45.741213+0000","last_clean":"2024-04-26T23:19:45.741213+0000","last_became_active":"2024-04-26T23:19:43.725993+0000","last_became_peered":"2024-04-26T23:19:43.725993+0000","last_unstale":"2024-04-26T23:19:45.741213+0000","last_undegraded":"2024-04-26T23:19:45.741213+0000","last_fullsized":"2024-04-26T23:19:45.741213+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:25:12.211704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.171146+0000","last_change":"2024-04-26T23:19:44.732350+0000","last_active":"2024-04-26T23:19:46.171146+0000","last_peered":"2024-04-26T23:19:46.171146+0000","last_clean":"2024-04-26T23:19:46.171146+0000","last_became_active":"2024-04-26T23:19:44.732091+0000","last_became_peered":"2024-04-26T23:19:44.732091+0000","last_unstale":"2024-04-26T23:19:46.171146+0000","last_undegraded":"2024-04-26T23:19:46.171146+0000","last_fullsized":"2024-04-26T23:19:46.171146+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:44:45.156698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.171182+0000","last_change":"2024-04-26T23:19:43.733216+0000","last_active":"2024-04-26T23:19:46.171182+0000","last_peered":"2024-04-26T23:19:46.171182+0000","last_clean":"2024-04-26T23:19:46.171182+0000","last_became_active":"2024-04-26T23:19:43.732950+0000","last_became_peered":"2024-04-26T23:19:43.732950+0000","last_unstale":"2024-04-26T23:19:46.171182+0000","last_undegraded":"2024-04-26T23:19:46.171182+0000","last_fullsized":"2024-04-26T23:19:46.171182+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:17:20.585223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.171977+0000","last_change":"2024-04-26T23:19:44.737642+0000","last_active":"2024-04-26T23:19:46.171977+0000","last_peered":"2024-04-26T23:19:46.171977+0000","last_clean":"2024-04-26T23:19:46.171977+0000","last_became_active":"2024-04-26T23:19:44.737347+0000","last_became_peered":"2024-04-26T23:19:44.737347+0000","last_unstale":"2024-04-26T23:19:46.171977+0000","last_undegraded":"2024-04-26T23:19:46.171977+0000","last_fullsized":"2024-04-26T23:19:46.171977+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:41:04.761590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.169979+0000","last_change":"2024-04-26T23:19:43.730629+0000","last_active":"2024-04-26T23:19:46.169979+0000","last_peered":"2024-04-26T23:19:46.169979+0000","last_clean":"2024-04-26T23:19:46.169979+0000","last_became_active":"2024-04-26T23:19:43.730300+0000","last_became_peered":"2024-04-26T23:19:43.730300+0000","last_unstale":"2024-04-26T23:19:46.169979+0000","last_undegraded":"2024-04-26T23:19:46.169979+0000","last_fullsized":"2024-04-26T23:19:46.169979+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:23:53.681492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.170969+0000","last_change":"2024-04-26T23:19:44.745514+0000","last_active":"2024-04-26T23:19:46.170969+0000","last_peered":"2024-04-26T23:19:46.170969+0000","last_clean":"2024-04-26T23:19:46.170969+0000","last_became_active":"2024-04-26T23:19:44.744710+0000","last_became_peered":"2024-04-26T23:19:44.744710+0000","last_unstale":"2024-04-26T23:19:46.170969+0000","last_undegraded":"2024-04-26T23:19:46.170969+0000","last_fullsized":"2024-04-26T23:19:46.170969+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:43.707477+0000","last_clean_scrub_stamp":"2024-04-26T23:19:43.707477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:21:00.734571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.172051+0000","last_change":"2024-04-26T23:19:43.736319+0000","last_active":"2024-04-26T23:19:46.172051+0000","last_peered":"2024-04-26T23:19:46.172051+0000","last_clean":"2024-04-26T23:19:46.172051+0000","last_became_active":"2024-04-26T23:19:43.736032+0000","last_became_peered":"2024-04-26T23:19:43.736032+0000","last_unstale":"2024-04-26T23:19:46.172051+0000","last_undegraded":"2024-04-26T23:19:46.172051+0000","last_fullsized":"2024-04-26T23:19:46.172051+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:19:42.704965+0000","last_clean_scrub_stamp":"2024-04-26T23:19:42.704965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:13:26.217699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":203,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T23:19:46.171926+0000","last_change":"2024-04-26T23:18:18.862631+0000","last_active":"2024-04-26T23:19:46.171926+0000","last_peered":"2024-04-26T23:19:46.171926+0000","last_clean":"2024-04-26T23:19:46.171926+0000","last_became_active":"2024-04-26T23:18:18.861536+0000","last_became_peered":"2024-04-26T23:18:18.861536+0000","last_unstale":"2024-04-26T23:19:46.171926+0000","last_undegraded":"2024-04-26T23:19:46.171926+0000","last_fullsized":"2024-04-26T23:19:46.171926+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:17.808736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:17.808736+0000","last_clean_scrub_stamp":"2024-04-26T23:18:17.808736+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:59:26.359282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542165,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343796,"statfs":{"total":96636764160,"available":96608047104,"internally_reserved":0,"allocated":1044480,"data_stored":781128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542167,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":438272,"data_stored":189660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542166,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":442368,"data_stored":189190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542166,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":442368,"data_stored":189682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31620,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340220,"statfs":{"total":96636764160,"available":96604385280,"internally_reserved":0,"allocated":450560,"data_stored":190724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31608,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340232,"statfs":{"total":96636764160,"available":96604397568,"internally_reserved":0,"allocated":438272,"data_stored":189148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31608,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340232,"statfs":{"total":96636764160,"available":96604397568,"internally_reserved":0,"allocated":438272,"data_stored":189168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32192,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339648,"statfs":{"total":96636764160,"available":96603799552,"internally_reserved":0,"allocated":1032192,"data_stored":779480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:19:49.108 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T23:19:49.108 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-26T23:19:49.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.249+0000 7f5a1fe9b640 1 Processor -- start 2024-04-26T23:19:49.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.249+0000 7f5a1fe9b640 1 -- start start 2024-04-26T23:19:49.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.250+0000 7f5a1fe9b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a18162a20 0x7f5a180d1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:49.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.250+0000 7f5a1fe9b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5a180d1c40 0x7f5a180d20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:49.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.250+0000 7f5a1fe9b640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a180c8bc0 con 0x7f5a180d2ed0 2024-04-26T23:19:49.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.250+0000 7f5a1fe9b640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f5a180d07a0 con 0x7f5a180d1c40 2024-04-26T23:19:49.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.250+0000 7f5a1fe9b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f5a180d09f0 con 0x7f5a18162a20 2024-04-26T23:19:49.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.250+0000 7f5a1e438640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f5a180d2ed0 0x7f5a180d3360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36114/0 (socket says 172.21.15.47:36114) 2024-04-26T23:19:49.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.250+0000 7f5a1e438640 1 -- 172.21.15.47:0/3008206864 learned_addr learned my addr 172.21.15.47:0/3008206864 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:49.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.250+0000 7f5a1d436640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5a180d1c40 0x7f5a180d20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:49.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.251+0000 7f5a1cc35640 1 -- 172.21.15.47:0/3008206864 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3234299423 0 0) 0x7f5a180c8bc0 con 0x7f5a180d2ed0 2024-04-26T23:19:49.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.251+0000 7f5a1dc37640 1 --2- 172.21.15.47:0/3008206864 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a18162a20 0x7f5a180d1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:49.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.251+0000 7f5a1cc35640 1 -- 172.21.15.47:0/3008206864 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a00003520 con 0x7f5a180d2ed0 2024-04-26T23:19:49.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.251+0000 7f5a1d436640 1 -- 172.21.15.47:0/3008206864 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a18162a20 msgr2=0x7f5a180d1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:49.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.251+0000 7f5a1d436640 1 --2- 172.21.15.47:0/3008206864 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a18162a20 0x7f5a180d1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:49.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.251+0000 7f5a1d436640 1 -- 172.21.15.47:0/3008206864 >> v1:172.21.15.47:6789/0 conn(0x7f5a180d2ed0 legacy=0x7f5a180d3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:49.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.251+0000 7f5a1d436640 1 -- 172.21.15.47:0/3008206864 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a180d0cd0 con 0x7f5a180d1c40 2024-04-26T23:19:49.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.251+0000 7f5a1dc37640 1 --2- 172.21.15.47:0/3008206864 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a18162a20 0x7f5a180d1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:19:49.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.251+0000 7f5a1d436640 1 --2- 172.21.15.47:0/3008206864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a180d1c40 0x7f5a180d20a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c0097a0 tx=0x7f5a0c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ddef1e8583c990f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:49.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.252+0000 7f5a1cc35640 1 -- 172.21.15.47:0/3008206864 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c037070 con 0x7f5a180d1c40 2024-04-26T23:19:49.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.252+0000 7f5a1cc35640 1 -- 172.21.15.47:0/3008206864 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a0c02ae50 con 0x7f5a180d1c40 2024-04-26T23:19:49.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.252+0000 7f5a1cc35640 1 -- 172.21.15.47:0/3008206864 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c032660 con 0x7f5a180d1c40 2024-04-26T23:19:49.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.252+0000 7f5a1fe9b640 1 -- 172.21.15.47:0/3008206864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a180d1c40 msgr2=0x7f5a180d20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:49.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.252+0000 7f5a1fe9b640 1 --2- 172.21.15.47:0/3008206864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a180d1c40 0x7f5a180d20a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c0097a0 tx=0x7f5a0c02a100 comp rx=0 tx=0).stop 2024-04-26T23:19:49.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.252+0000 7f5a1fe9b640 1 -- 172.21.15.47:0/3008206864 shutdown_connections 2024-04-26T23:19:49.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.252+0000 7f5a1fe9b640 1 --2- 172.21.15.47:0/3008206864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a180d1c40 0x7f5a180d20a0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:49.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.252+0000 7f5a1fe9b640 1 --2- 172.21.15.47:0/3008206864 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a18162a20 0x7f5a180d1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:49.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.252+0000 7f5a1fe9b640 1 -- 172.21.15.47:0/3008206864 >> 172.21.15.47:0/3008206864 conn(0x7f5a180d4fd0 msgr2=0x7f5a180d53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:49.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.252+0000 7f5a1fe9b640 1 -- 172.21.15.47:0/3008206864 shutdown_connections 2024-04-26T23:19:49.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.252+0000 7f5a1fe9b640 1 -- 172.21.15.47:0/3008206864 wait complete. 2024-04-26T23:19:49.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.253+0000 7f5a1fe9b640 1 Processor -- start 2024-04-26T23:19:49.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.254+0000 7f5a1fe9b640 1 -- start start 2024-04-26T23:19:49.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.254+0000 7f5a1fe9b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a180d1c40 0x7f5a1815d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:49.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.254+0000 7f5a1fe9b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a180d2ed0 0x7f5a180a02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:49.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.254+0000 7f5a1fe9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a18162a20 0x7f5a180b9f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:49.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.254+0000 7f5a1fe9b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5a180cbd90 con 0x7f5a18162a20 2024-04-26T23:19:49.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.254+0000 7f5a1fe9b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5a180d0fb0 con 0x7f5a180d2ed0 2024-04-26T23:19:49.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.254+0000 7f5a1fe9b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f5a18162e00 con 0x7f5a180d1c40 2024-04-26T23:19:49.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.254+0000 7f5a1e438640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a18162a20 0x7f5a180b9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:49.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.254+0000 7f5a1e438640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a18162a20 0x7f5a180b9f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58932/0 (socket says 172.21.15.47:58932) 2024-04-26T23:19:49.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.254+0000 7f5a1e438640 1 -- 172.21.15.47:0/3033950394 learned_addr learned my addr 172.21.15.47:0/3033950394 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:49.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.254+0000 7f5a1dc37640 1 --2- 172.21.15.47:0/3033950394 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a180d1c40 0x7f5a1815d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:49.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.254+0000 7f5a1d436640 1 --2- 172.21.15.47:0/3033950394 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a180d2ed0 0x7f5a180a02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:49.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.255+0000 7f5a1e438640 1 -- 172.21.15.47:0/3033950394 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a180d1c40 msgr2=0x7f5a1815d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:49.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.255+0000 7f5a1e438640 1 --2- 172.21.15.47:0/3033950394 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a180d1c40 0x7f5a1815d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:49.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.255+0000 7f5a1e438640 1 -- 172.21.15.47:0/3033950394 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a180d2ed0 msgr2=0x7f5a180a02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:49.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.255+0000 7f5a1e438640 1 --2- 172.21.15.47:0/3033950394 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a180d2ed0 0x7f5a180a02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:49.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.255+0000 7f5a1e438640 1 -- 172.21.15.47:0/3033950394 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a180b7460 con 0x7f5a18162a20 2024-04-26T23:19:49.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.255+0000 7f5a1d436640 1 --2- 172.21.15.47:0/3033950394 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a180d2ed0 0x7f5a180a02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:19:49.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.255+0000 7f5a1e438640 1 --2- 172.21.15.47:0/3033950394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a18162a20 0x7f5a180b9f90 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5a0800cb40 tx=0x7f5a0800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:49.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.255+0000 7f5a167fc640 1 -- 172.21.15.47:0/3033950394 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a08017070 con 0x7f5a18162a20 2024-04-26T23:19:49.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.255+0000 7f5a167fc640 1 -- 172.21.15.47:0/3033950394 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a0800aca0 con 0x7f5a18162a20 2024-04-26T23:19:49.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.255+0000 7f5a1fe9b640 1 -- 172.21.15.47:0/3033950394 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a180b7700 con 0x7f5a18162a20 2024-04-26T23:19:49.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.255+0000 7f5a167fc640 1 -- 172.21.15.47:0/3033950394 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a08012620 con 0x7f5a18162a20 2024-04-26T23:19:49.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.255+0000 7f5a1fe9b640 1 -- 172.21.15.47:0/3033950394 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a180b7cc0 con 0x7f5a18162a20 2024-04-26T23:19:49.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.256+0000 7f5a1fe9b640 1 -- 172.21.15.47:0/3033950394 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a180694f0 con 0x7f5a18162a20 2024-04-26T23:19:49.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.257+0000 7f5a167fc640 1 -- 172.21.15.47:0/3033950394 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5a0801a070 con 0x7f5a18162a20 2024-04-26T23:19:49.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.257+0000 7f5a167fc640 1 --2- 172.21.15.47:0/3033950394 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f5a0406c920 0x7f5a0406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:49.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.257+0000 7f5a167fc640 1 -- 172.21.15.47:0/3033950394 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f5a0808b960 con 0x7f5a18162a20 2024-04-26T23:19:49.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.257+0000 7f5a1dc37640 1 --2- 172.21.15.47:0/3033950394 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f5a0406c920 0x7f5a0406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:49.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.258+0000 7f5a1dc37640 1 --2- 172.21.15.47:0/3033950394 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f5a0406c920 0x7f5a0406ede0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5a1000ab60 tx=0x7f5a10008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:49.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.262+0000 7f5a167fc640 1 -- 172.21.15.47:0/3033950394 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5a08059640 con 0x7f5a18162a20 2024-04-26T23:19:49.388 INFO:tasks.ceph.mds.b.smithi203.stderr:2024-04-26T23:19:49.386+0000 7fd67e004640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T23:19:49.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.526+0000 7f5a1fe9b640 1 -- 172.21.15.47:0/3033950394 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f5a180cae60 con 0x7f5a18162a20 2024-04-26T23:19:49.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.529+0000 7f5a167fc640 1 -- 172.21.15.47:0/3033950394 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v10) ==== 190+0+147 (secure 0 0 0) 0x7f5a0805d940 con 0x7f5a18162a20 2024-04-26T23:19:49.531 INFO:teuthology.orchestra.run.smithi047.stdout:[client.0] 2024-04-26T23:19:49.531 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQAKNixmA5QFHRAAT89WPyzDs0acPluyvU4svw== 2024-04-26T23:19:49.531 INFO:teuthology.orchestra.run.smithi047.stdout: caps mds = "allow" 2024-04-26T23:19:49.531 INFO:teuthology.orchestra.run.smithi047.stdout: caps mon = "allow r" 2024-04-26T23:19:49.531 INFO:teuthology.orchestra.run.smithi047.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-26T23:19:49.531 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.0 2024-04-26T23:19:49.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.531+0000 7f5a1fe9b640 1 -- 172.21.15.47:0/3033950394 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f5a0406c920 msgr2=0x7f5a0406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:49.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.531+0000 7f5a1fe9b640 1 --2- 172.21.15.47:0/3033950394 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f5a0406c920 0x7f5a0406ede0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5a1000ab60 tx=0x7f5a10008040 comp rx=0 tx=0).stop 2024-04-26T23:19:49.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.531+0000 7f5a1fe9b640 1 -- 172.21.15.47:0/3033950394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a18162a20 msgr2=0x7f5a180b9f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:49.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.531+0000 7f5a1fe9b640 1 --2- 172.21.15.47:0/3033950394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a18162a20 0x7f5a180b9f90 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5a0800cb40 tx=0x7f5a0800a060 comp rx=0 tx=0).stop 2024-04-26T23:19:49.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.531+0000 7f5a1fe9b640 1 -- 172.21.15.47:0/3033950394 shutdown_connections 2024-04-26T23:19:49.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.531+0000 7f5a1fe9b640 1 --2- 172.21.15.47:0/3033950394 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f5a0406c920 0x7f5a0406ede0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:49.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.531+0000 7f5a1fe9b640 1 --2- 172.21.15.47:0/3033950394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a18162a20 0x7f5a180b9f90 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:49.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.531+0000 7f5a1fe9b640 1 --2- 172.21.15.47:0/3033950394 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a180d2ed0 0x7f5a180a02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:49.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.531+0000 7f5a1fe9b640 1 --2- 172.21.15.47:0/3033950394 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a180d1c40 0x7f5a1815d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:49.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.531+0000 7f5a1fe9b640 1 -- 172.21.15.47:0/3033950394 >> 172.21.15.47:0/3033950394 conn(0x7f5a180d4fd0 msgr2=0x7f5a180cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:49.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.531+0000 7f5a1fe9b640 1 -- 172.21.15.47:0/3033950394 shutdown_connections 2024-04-26T23:19:49.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.531+0000 7f5a1fe9b640 1 -- 172.21.15.47:0/3033950394 wait complete. 2024-04-26T23:19:49.544 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:19:49.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.643+0000 7f8196b3f640 1 Processor -- start 2024-04-26T23:19:49.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.643+0000 7f8196b3f640 1 -- start start 2024-04-26T23:19:49.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.644+0000 7f8196b3f640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8190162880 0x7f81900d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:49.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.644+0000 7f8196b3f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f81900b5f20 0x7f81900d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:49.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.644+0000 7f8196b3f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f81900d39f0 0x7f81900b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:49.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.644+0000 7f8196b3f640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f81900d0580 con 0x7f8190162880 2024-04-26T23:19:49.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.644+0000 7f8196b3f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f81900b7e50 con 0x7f81900b5f20 2024-04-26T23:19:49.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.644+0000 7f8196b3f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f81900d1800 con 0x7f81900d39f0 2024-04-26T23:19:49.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.644+0000 7f81948db640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8190162880 0x7f81900d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:49.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.644+0000 7f81948db640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8190162880 0x7f81900d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58940/0 (socket says 172.21.15.47:58940) 2024-04-26T23:19:49.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.644+0000 7f81948db640 1 -- 172.21.15.47:0/2090614782 learned_addr learned my addr 172.21.15.47:0/2090614782 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:49.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.644+0000 7f81948db640 1 -- 172.21.15.47:0/2090614782 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f81900d39f0 msgr2=0x7f81900b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:49.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.644+0000 7f81948db640 1 --2- 172.21.15.47:0/2090614782 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f81900d39f0 0x7f81900b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:49.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.644+0000 7f81948db640 1 -- 172.21.15.47:0/2090614782 >> v2:172.21.15.203:3300/0 conn(0x7f81900b5f20 msgr2=0x7f81900d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:49.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.644+0000 7f81948db640 1 --2- 172.21.15.47:0/2090614782 >> v2:172.21.15.203:3300/0 conn(0x7f81900b5f20 0x7f81900d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:49.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.644+0000 7f81948db640 1 -- 172.21.15.47:0/2090614782 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81900d07d0 con 0x7f8190162880 2024-04-26T23:19:49.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.645+0000 7f81948db640 1 --2- 172.21.15.47:0/2090614782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8190162880 0x7f81900d2cb0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f819008d3d0 tx=0x7f818802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d782a01aa3b3865b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:49.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.645+0000 7f818f7fe640 1 -- 172.21.15.47:0/2090614782 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8188037070 con 0x7f8190162880 2024-04-26T23:19:49.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.645+0000 7f818f7fe640 1 -- 172.21.15.47:0/2090614782 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8188032440 con 0x7f8190162880 2024-04-26T23:19:49.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.645+0000 7f818f7fe640 1 -- 172.21.15.47:0/2090614782 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8188032810 con 0x7f8190162880 2024-04-26T23:19:49.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.646+0000 7f8196b3f640 1 -- 172.21.15.47:0/2090614782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8190162880 msgr2=0x7f81900d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:49.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.646+0000 7f8196b3f640 1 --2- 172.21.15.47:0/2090614782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8190162880 0x7f81900d2cb0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f819008d3d0 tx=0x7f818802a1e0 comp rx=0 tx=0).stop 2024-04-26T23:19:49.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.646+0000 7f8196b3f640 1 -- 172.21.15.47:0/2090614782 shutdown_connections 2024-04-26T23:19:49.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.646+0000 7f8196b3f640 1 --2- 172.21.15.47:0/2090614782 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f81900d39f0 0x7f81900b69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:49.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.646+0000 7f8196b3f640 1 --2- 172.21.15.47:0/2090614782 >> v2:172.21.15.203:3300/0 conn(0x7f81900b5f20 0x7f81900d34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:49.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.646+0000 7f8196b3f640 1 --2- 172.21.15.47:0/2090614782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8190162880 0x7f81900d2cb0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:49.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.646+0000 7f8196b3f640 1 -- 172.21.15.47:0/2090614782 >> 172.21.15.47:0/2090614782 conn(0x7f81900d4f60 msgr2=0x7f81900d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:49.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.646+0000 7f8196b3f640 1 -- 172.21.15.47:0/2090614782 shutdown_connections 2024-04-26T23:19:49.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.646+0000 7f8196b3f640 1 -- 172.21.15.47:0/2090614782 wait complete. 2024-04-26T23:19:49.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.647+0000 7f8196b3f640 1 Processor -- start 2024-04-26T23:19:49.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.647+0000 7f8196b3f640 1 -- start start 2024-04-26T23:19:49.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.648+0000 7f8196b3f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81900b5f20 0x7f81900b5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:49.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.648+0000 7f8196b3f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81900d39f0 0x7f819015c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:49.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.648+0000 7f8196b3f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8190162880 0x7f81900b2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:49.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.648+0000 7f8196b3f640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8190162c60 con 0x7f81900b5f20 2024-04-26T23:19:49.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.648+0000 7f8196b3f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f81900d0d90 con 0x7f81900d39f0 2024-04-26T23:19:49.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.648+0000 7f8196b3f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f8190162de0 con 0x7f8190162880 2024-04-26T23:19:49.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.648+0000 7f81948db640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81900b5f20 0x7f81900b5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:49.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.648+0000 7f81948db640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81900b5f20 0x7f81900b5490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58950/0 (socket says 172.21.15.47:58950) 2024-04-26T23:19:49.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.648+0000 7f81948db640 1 -- 172.21.15.47:0/1298646333 learned_addr learned my addr 172.21.15.47:0/1298646333 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:49.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.648+0000 7f818ffff640 1 --2- 172.21.15.47:0/1298646333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81900d39f0 0x7f819015c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:49.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.648+0000 7f81950dc640 1 --2- 172.21.15.47:0/1298646333 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8190162880 0x7f81900b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:49.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.648+0000 7f81948db640 1 -- 172.21.15.47:0/1298646333 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8190162880 msgr2=0x7f81900b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:49.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.648+0000 7f81948db640 1 --2- 172.21.15.47:0/1298646333 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8190162880 0x7f81900b2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:49.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.648+0000 7f81948db640 1 -- 172.21.15.47:0/1298646333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81900d39f0 msgr2=0x7f819015c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:49.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.648+0000 7f81948db640 1 --2- 172.21.15.47:0/1298646333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81900d39f0 0x7f819015c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:49.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.648+0000 7f81948db640 1 -- 172.21.15.47:0/1298646333 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81901f4d90 con 0x7f81900b5f20 2024-04-26T23:19:49.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.648+0000 7f818ffff640 1 --2- 172.21.15.47:0/1298646333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81900d39f0 0x7f819015c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:19:49.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.648+0000 7f81948db640 1 --2- 172.21.15.47:0/1298646333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81900b5f20 0x7f81900b5490 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8188000c00 tx=0x7f81880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:49.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.649+0000 7f818d7fa640 1 -- 172.21.15.47:0/1298646333 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8188041070 con 0x7f81900b5f20 2024-04-26T23:19:49.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.649+0000 7f818d7fa640 1 -- 172.21.15.47:0/1298646333 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f818803c870 con 0x7f81900b5f20 2024-04-26T23:19:49.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.649+0000 7f8196b3f640 1 -- 172.21.15.47:0/1298646333 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81900b2010 con 0x7f81900b5f20 2024-04-26T23:19:49.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.649+0000 7f818d7fa640 1 -- 172.21.15.47:0/1298646333 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8188037040 con 0x7f81900b5f20 2024-04-26T23:19:49.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.649+0000 7f8196b3f640 1 -- 172.21.15.47:0/1298646333 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81900b25d0 con 0x7f81900b5f20 2024-04-26T23:19:49.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.650+0000 7f8196b3f640 1 -- 172.21.15.47:0/1298646333 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81900694f0 con 0x7f81900b5f20 2024-04-26T23:19:49.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.650+0000 7f818d7fa640 1 -- 172.21.15.47:0/1298646333 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8188044070 con 0x7f81900b5f20 2024-04-26T23:19:49.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.651+0000 7f818d7fa640 1 --2- 172.21.15.47:0/1298646333 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f817006c8f0 0x7f817006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:49.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.651+0000 7f818d7fa640 1 -- 172.21.15.47:0/1298646333 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f81880ac6f0 con 0x7f81900b5f20 2024-04-26T23:19:49.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.651+0000 7f818ffff640 1 --2- 172.21.15.47:0/1298646333 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f817006c8f0 0x7f817006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:49.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.652+0000 7f818ffff640 1 --2- 172.21.15.47:0/1298646333 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f817006c8f0 0x7f817006edb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f81901612c0 tx=0x7f81800073c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:49.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.655+0000 7f818d7fa640 1 -- 172.21.15.47:0/1298646333 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f818807aa20 con 0x7f81900b5f20 2024-04-26T23:19:49.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.883+0000 7f8196b3f640 1 -- 172.21.15.47:0/1298646333 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8190075320 con 0x7f81900b5f20 2024-04-26T23:19:49.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.884+0000 7f818d7fa640 1 -- 172.21.15.47:0/1298646333 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5189 (secure 0 0 0) 0x7f818807a320 con 0x7f81900b5f20 2024-04-26T23:19:49.886 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:19:49.886 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5080,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/1970124054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1970124054},{"type":"v1","addr":"172.21.15.203:6836","nonce":1970124054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5310,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/889573201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":889573201},{"type":"v1","addr":"172.21.15.47:6834","nonce":889573201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5316,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3898612760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":3898612760},{"type":"v1","addr":"172.21.15.47:6835","nonce":3898612760}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:19:44.091353+0000","modified":"2024-04-26T23:19:47.109087+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":14738},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14738":{"gid":14738,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/2526477051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":2526477051},{"type":"v1","addr":"172.21.15.203:6837","nonce":2526477051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":14738,"qdb_cluster":[14738]},"id":2}]} 2024-04-26T23:19:49.886 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 18 2024-04-26T23:19:49.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.886+0000 7f8196b3f640 1 -- 172.21.15.47:0/1298646333 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f817006c8f0 msgr2=0x7f817006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:49.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.886+0000 7f8196b3f640 1 --2- 172.21.15.47:0/1298646333 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f817006c8f0 0x7f817006edb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f81901612c0 tx=0x7f81800073c0 comp rx=0 tx=0).stop 2024-04-26T23:19:49.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.886+0000 7f8196b3f640 1 -- 172.21.15.47:0/1298646333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81900b5f20 msgr2=0x7f81900b5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:49.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.886+0000 7f8196b3f640 1 --2- 172.21.15.47:0/1298646333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81900b5f20 0x7f81900b5490 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8188000c00 tx=0x7f81880046c0 comp rx=0 tx=0).stop 2024-04-26T23:19:49.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.886+0000 7f8196b3f640 1 -- 172.21.15.47:0/1298646333 shutdown_connections 2024-04-26T23:19:49.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.886+0000 7f8196b3f640 1 --2- 172.21.15.47:0/1298646333 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f817006c8f0 0x7f817006edb0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:49.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.886+0000 7f8196b3f640 1 --2- 172.21.15.47:0/1298646333 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8190162880 0x7f81900b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:49.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.886+0000 7f8196b3f640 1 --2- 172.21.15.47:0/1298646333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81900d39f0 0x7f819015c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:49.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.886+0000 7f8196b3f640 1 --2- 172.21.15.47:0/1298646333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81900b5f20 0x7f81900b5490 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:49.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.886+0000 7f8196b3f640 1 -- 172.21.15.47:0/1298646333 >> 172.21.15.47:0/1298646333 conn(0x7f81900d4f60 msgr2=0x7f819015f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:49.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.886+0000 7f8196b3f640 1 -- 172.21.15.47:0/1298646333 shutdown_connections 2024-04-26T23:19:49.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:49.886+0000 7f8196b3f640 1 -- 172.21.15.47:0/1298646333 wait complete. 2024-04-26T23:19:50.302 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T23:19:44.091353+0000', 'modified': '2024-04-26T23:19:47.109087+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 14738}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14738': {'gid': 14738, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.203:6837/2526477051', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6835', 'nonce': 2526477051}, {'type': 'v1', 'addr': '172.21.15.203:6837', 'nonce': 2526477051}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 14738, 'qdb_cluster': [14738]} 2024-04-26T23:19:50.302 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T23:19:50.302 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-26T23:19:50.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.410+0000 7f598096a640 1 Processor -- start 2024-04-26T23:19:50.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.410+0000 7f598096a640 1 -- start start 2024-04-26T23:19:50.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.411+0000 7f598096a640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f597c162880 0x7f597c0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:50.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.411+0000 7f598096a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f597c0d3bc0 0x7f597c0b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:50.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.411+0000 7f598096a640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f597c0b7e50 con 0x7f597c0b5f20 2024-04-26T23:19:50.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.411+0000 7f598096a640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f597c0d0580 con 0x7f597c162880 2024-04-26T23:19:50.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.411+0000 7f598096a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f597c0d1800 con 0x7f597c0d3bc0 2024-04-26T23:19:50.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.411+0000 7f597a59c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f597c162880 0x7f597c0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:50.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.411+0000 7f5979d9b640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f597c0b5f20 0x7f597c0d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44708/0 (socket says 172.21.15.47:44708) 2024-04-26T23:19:50.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.411+0000 7f5979d9b640 1 -- 172.21.15.47:0/2040193044 learned_addr learned my addr 172.21.15.47:0/2040193044 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:50.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.411+0000 7f597ad9d640 1 --2- 172.21.15.47:0/2040193044 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f597c0d3bc0 0x7f597c0b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:50.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.411+0000 7f597a59c640 1 -- 172.21.15.47:0/2040193044 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f597c0d3bc0 msgr2=0x7f597c0b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:50.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.411+0000 7f597a59c640 1 --2- 172.21.15.47:0/2040193044 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f597c0d3bc0 0x7f597c0b8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:50.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.411+0000 7f597a59c640 1 -- 172.21.15.47:0/2040193044 >> v1:172.21.15.47:6789/0 conn(0x7f597c0b5f20 legacy=0x7f597c0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:50.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.411+0000 7f597a59c640 1 -- 172.21.15.47:0/2040193044 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f597c0d07d0 con 0x7f597c162880 2024-04-26T23:19:50.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.411+0000 7f597959a640 1 -- 172.21.15.47:0/2040193044 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1461799632 0 0) 0x7f597c0b7e50 con 0x7f597c0b5f20 2024-04-26T23:19:50.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.412+0000 7f597a59c640 1 --2- 172.21.15.47:0/2040193044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f597c162880 0x7f597c0d2cb0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f59740097a0 tx=0x7f597402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b88e98345128ae1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:50.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.412+0000 7f597959a640 1 -- 172.21.15.47:0/2040193044 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5974037070 con 0x7f597c162880 2024-04-26T23:19:50.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.412+0000 7f597959a640 1 -- 172.21.15.47:0/2040193044 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5974032440 con 0x7f597c162880 2024-04-26T23:19:50.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.412+0000 7f597959a640 1 -- 172.21.15.47:0/2040193044 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5974032810 con 0x7f597c162880 2024-04-26T23:19:50.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.412+0000 7f598096a640 1 -- 172.21.15.47:0/2040193044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f597c162880 msgr2=0x7f597c0d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:50.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.412+0000 7f598096a640 1 --2- 172.21.15.47:0/2040193044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f597c162880 0x7f597c0d2cb0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f59740097a0 tx=0x7f597402a180 comp rx=0 tx=0).stop 2024-04-26T23:19:50.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.413+0000 7f598096a640 1 -- 172.21.15.47:0/2040193044 shutdown_connections 2024-04-26T23:19:50.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.413+0000 7f598096a640 1 --2- 172.21.15.47:0/2040193044 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f597c0d3bc0 0x7f597c0b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:50.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.413+0000 7f598096a640 1 --2- 172.21.15.47:0/2040193044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f597c162880 0x7f597c0d2cb0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:50.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.413+0000 7f598096a640 1 -- 172.21.15.47:0/2040193044 >> 172.21.15.47:0/2040193044 conn(0x7f597c0d4f60 msgr2=0x7f597c0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:50.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.413+0000 7f598096a640 1 -- 172.21.15.47:0/2040193044 shutdown_connections 2024-04-26T23:19:50.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.413+0000 7f598096a640 1 -- 172.21.15.47:0/2040193044 wait complete. 2024-04-26T23:19:50.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.414+0000 7f598096a640 1 Processor -- start 2024-04-26T23:19:50.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.414+0000 7f598096a640 1 -- start start 2024-04-26T23:19:50.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.415+0000 7f598096a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f597c0b5f20 0x7f597c0ce290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:50.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.415+0000 7f598096a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f597c0d3bc0 0x7f597c0b1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:50.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.415+0000 7f598096a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f597c162880 0x7f597c14fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:50.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.415+0000 7f598096a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f597c162c60 con 0x7f597c162880 2024-04-26T23:19:50.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.415+0000 7f598096a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f597c0d0d90 con 0x7f597c0b5f20 2024-04-26T23:19:50.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.415+0000 7f598096a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f597c162de0 con 0x7f597c0d3bc0 2024-04-26T23:19:50.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.415+0000 7f597ad9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f597c162880 0x7f597c14fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:50.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.415+0000 7f597ad9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f597c162880 0x7f597c14fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37896/0 (socket says 172.21.15.47:37896) 2024-04-26T23:19:50.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.415+0000 7f597ad9d640 1 -- 172.21.15.47:0/1660458175 learned_addr learned my addr 172.21.15.47:0/1660458175 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:50.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.415+0000 7f5979d9b640 1 --2- 172.21.15.47:0/1660458175 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f597c0d3bc0 0x7f597c0b1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:50.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.415+0000 7f597a59c640 1 --2- 172.21.15.47:0/1660458175 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f597c0b5f20 0x7f597c0ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:50.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.415+0000 7f597ad9d640 1 -- 172.21.15.47:0/1660458175 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f597c0d3bc0 msgr2=0x7f597c0b1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:50.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.415+0000 7f597ad9d640 1 --2- 172.21.15.47:0/1660458175 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f597c0d3bc0 0x7f597c0b1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:50.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.415+0000 7f597ad9d640 1 -- 172.21.15.47:0/1660458175 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f597c0b5f20 msgr2=0x7f597c0ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:50.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.415+0000 7f597ad9d640 1 --2- 172.21.15.47:0/1660458175 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f597c0b5f20 0x7f597c0ce290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:50.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.415+0000 7f597ad9d640 1 -- 172.21.15.47:0/1660458175 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f597c14ebd0 con 0x7f597c162880 2024-04-26T23:19:50.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.415+0000 7f5979d9b640 1 --2- 172.21.15.47:0/1660458175 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f597c0d3bc0 0x7f597c0b1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:19:50.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.416+0000 7f597ad9d640 1 --2- 172.21.15.47:0/1660458175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f597c162880 0x7f597c14fc70 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f596c007d70 tx=0x7f596c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:50.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.416+0000 7f5972ffd640 1 -- 172.21.15.47:0/1660458175 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596c014420 con 0x7f597c162880 2024-04-26T23:19:50.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.416+0000 7f5972ffd640 1 -- 172.21.15.47:0/1660458175 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f596c019070 con 0x7f597c162880 2024-04-26T23:19:50.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.416+0000 7f598096a640 1 -- 172.21.15.47:0/1660458175 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f597c14eeb0 con 0x7f597c162880 2024-04-26T23:19:50.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.416+0000 7f5972ffd640 1 -- 172.21.15.47:0/1660458175 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596c014820 con 0x7f597c162880 2024-04-26T23:19:50.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.416+0000 7f598096a640 1 -- 172.21.15.47:0/1660458175 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f597c14f470 con 0x7f597c162880 2024-04-26T23:19:50.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.417+0000 7f5972ffd640 1 -- 172.21.15.47:0/1660458175 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f596c0149c0 con 0x7f597c162880 2024-04-26T23:19:50.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.418+0000 7f5972ffd640 1 --2- 172.21.15.47:0/1660458175 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f595c06c8f0 0x7f595c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:50.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.418+0000 7f5972ffd640 1 -- 172.21.15.47:0/1660458175 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f596c08c510 con 0x7f597c162880 2024-04-26T23:19:50.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.418+0000 7f597a59c640 1 --2- 172.21.15.47:0/1660458175 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f595c06c8f0 0x7f595c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:50.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.418+0000 7f597a59c640 1 --2- 172.21.15.47:0/1660458175 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f595c06c8f0 0x7f595c06edb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5974034000 tx=0x7f597402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:50.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.419+0000 7f5983322740 1 Processor -- start 2024-04-26T23:19:50.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.419+0000 7f5983322740 1 -- start start 2024-04-26T23:19:50.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.419+0000 7f5983322740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ddff302440 0x55ddff302820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:50.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.419+0000 7f5983322740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ddff31fce0 0x55ddff320230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:50.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.419+0000 7f5983322740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ddff2d8120 0x55ddff31e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:50.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.419+0000 7f5983322740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55ddff23c740 con 0x55ddff302440 2024-04-26T23:19:50.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.419+0000 7f5983322740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ddff1d83a0 con 0x55ddff31fce0 2024-04-26T23:19:50.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.419+0000 7f5983322740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x55ddff238700 con 0x55ddff2d8120 2024-04-26T23:19:50.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.420+0000 7f597a59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ddff302440 0x55ddff302820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:50.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.420+0000 7f597a59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ddff302440 0x55ddff302820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37898/0 (socket says 172.21.15.47:37898) 2024-04-26T23:19:50.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.420+0000 7f597a59c640 1 -- 172.21.15.47:0/2844735974 learned_addr learned my addr 172.21.15.47:0/2844735974 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:50.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.420+0000 7f5979d9b640 1 --2- 172.21.15.47:0/2844735974 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ddff31fce0 0x55ddff320230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:50.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.420+0000 7f597ad9d640 1 --2- 172.21.15.47:0/2844735974 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ddff2d8120 0x55ddff31e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:50.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.420+0000 7f597a59c640 1 -- 172.21.15.47:0/2844735974 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ddff2d8120 msgr2=0x55ddff31e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:50.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.420+0000 7f597a59c640 1 --2- 172.21.15.47:0/2844735974 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ddff2d8120 0x55ddff31e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:50.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.420+0000 7f597a59c640 1 -- 172.21.15.47:0/2844735974 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ddff31fce0 msgr2=0x55ddff320230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:50.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.420+0000 7f597a59c640 1 --2- 172.21.15.47:0/2844735974 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ddff31fce0 0x55ddff320230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:50.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.420+0000 7f597a59c640 1 -- 172.21.15.47:0/2844735974 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddff1f4750 con 0x55ddff302440 2024-04-26T23:19:50.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.420+0000 7f597a59c640 1 --2- 172.21.15.47:0/2844735974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ddff302440 0x55ddff302820 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f597402a1c0 tx=0x7f597402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9df567c13d0a7001 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:50.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.420+0000 7f595bfff640 1 -- 172.21.15.47:0/2844735974 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f597404e070 con 0x55ddff302440 2024-04-26T23:19:50.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.420+0000 7f595bfff640 1 -- 172.21.15.47:0/2844735974 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5974032d90 con 0x55ddff302440 2024-04-26T23:19:50.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.420+0000 7f595bfff640 1 -- 172.21.15.47:0/2844735974 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5974037040 con 0x55ddff302440 2024-04-26T23:19:50.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.421+0000 7f5983322740 1 -- 172.21.15.47:0/2844735974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ddff302440 msgr2=0x55ddff302820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:50.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.421+0000 7f5983322740 1 --2- 172.21.15.47:0/2844735974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ddff302440 0x55ddff302820 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f597402a1c0 tx=0x7f597402c000 comp rx=0 tx=0).stop 2024-04-26T23:19:50.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.421+0000 7f5983322740 1 -- 172.21.15.47:0/2844735974 shutdown_connections 2024-04-26T23:19:50.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.421+0000 7f5983322740 1 --2- 172.21.15.47:0/2844735974 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ddff2d8120 0x55ddff31e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:50.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.421+0000 7f5983322740 1 --2- 172.21.15.47:0/2844735974 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ddff31fce0 0x55ddff320230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:50.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.421+0000 7f5983322740 1 --2- 172.21.15.47:0/2844735974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ddff302440 0x55ddff302820 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:50.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.421+0000 7f5983322740 1 -- 172.21.15.47:0/2844735974 >> 172.21.15.47:0/2844735974 conn(0x55ddff2d4a50 msgr2=0x55ddff31d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:50.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.421+0000 7f5983322740 1 -- 172.21.15.47:0/2844735974 shutdown_connections 2024-04-26T23:19:50.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.421+0000 7f5983322740 1 -- 172.21.15.47:0/2844735974 wait complete. 2024-04-26T23:19:50.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.422+0000 7f5983322740 1 -- start start 2024-04-26T23:19:50.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.422+0000 7f5983322740 1 Processor -- start 2024-04-26T23:19:50.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.423+0000 7f5983322740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ddff302440 0x55ddff3001f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:50.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.423+0000 7f5983322740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ddff31fce0 0x55ddff315bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:50.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.423+0000 7f5983322740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ddff31a410 0x55ddff31a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:50.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.423+0000 7f5983322740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55ddff2d8ec0 con 0x55ddff31fce0 2024-04-26T23:19:50.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.423+0000 7f5983322740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ddff1c3fe0 con 0x55ddff31a410 2024-04-26T23:19:50.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.423+0000 7f5983322740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x55ddff2d2980 con 0x55ddff302440 2024-04-26T23:19:50.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.423+0000 7f5983322740 10 client.0 _ll_register_callbacks cb 0x55ddff1da150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:19:50.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.423+0000 7f5979d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ddff31fce0 0x55ddff315bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:50.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.423+0000 7f5979d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ddff31fce0 0x55ddff315bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37912/0 (socket says 172.21.15.47:37912) 2024-04-26T23:19:50.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.423+0000 7f5979d9b640 1 -- 172.21.15.47:0/3565767179 learned_addr learned my addr 172.21.15.47:0/3565767179 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:50.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.423+0000 7f597a59c640 1 --2- 172.21.15.47:0/3565767179 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ddff302440 0x55ddff3001f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:50.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.423+0000 7f597ad9d640 1 --2- 172.21.15.47:0/3565767179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ddff31a410 0x55ddff31a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:50.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.423+0000 7f5979d9b640 1 -- 172.21.15.47:0/3565767179 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ddff302440 msgr2=0x55ddff3001f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:50.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.423+0000 7f5979d9b640 1 --2- 172.21.15.47:0/3565767179 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ddff302440 0x55ddff3001f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:50.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.423+0000 7f5979d9b640 1 -- 172.21.15.47:0/3565767179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ddff31a410 msgr2=0x55ddff31a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:50.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.423+0000 7f5979d9b640 1 --2- 172.21.15.47:0/3565767179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ddff31a410 0x55ddff31a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:50.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.423+0000 7f5979d9b640 1 -- 172.21.15.47:0/3565767179 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddfef4e2d0 con 0x55ddff31fce0 2024-04-26T23:19:50.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.423+0000 7f5979d9b640 1 --2- 172.21.15.47:0/3565767179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ddff31fce0 0x55ddff315bf0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5964005fb0 tx=0x7f59640015e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb96a21d2e83772c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:50.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.423+0000 7f59597fa640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-26T23:19:50.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.423+0000 7f59597fa640 1 -- 172.21.15.47:0/3565767179 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59640145e0 con 0x55ddff31fce0 2024-04-26T23:19:50.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.423+0000 7f59597fa640 1 -- 172.21.15.47:0/3565767179 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f5964014780 con 0x55ddff31fce0 2024-04-26T23:19:50.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.423+0000 7f5983322740 1 -- 172.21.15.47:0/3565767179 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5974032d90 con 0x55ddff31fce0 2024-04-26T23:19:50.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.423+0000 7f59597fa640 1 -- 172.21.15.47:0/3565767179 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f5964022050 con 0x55ddff31fce0 2024-04-26T23:19:50.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.423+0000 7f5983322740 10 client.5499 fetch_fsmap learned FSMap version 18 2024-04-26T23:19:50.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.424+0000 7f5983322740 1 -- 172.21.15.47:0/3565767179 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x55ddff0186b0 con 0x55ddff31fce0 2024-04-26T23:19:50.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.424+0000 7f59597fa640 1 -- 172.21.15.47:0/3565767179 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5974032d90 con 0x55ddff31fce0 2024-04-26T23:19:50.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.424+0000 7f59597fa640 1 -- 172.21.15.47:0/3565767179 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f5964025020 con 0x55ddff31fce0 2024-04-26T23:19:50.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.424+0000 7f5983322740 10 client.5499 fetch_fsmap finished waiting for FSMap version 18 2024-04-26T23:19:50.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.424+0000 7f5983322740 10 client.5499 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T23:19:50.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.424+0000 7f5983322740 20 client.5499 populate_metadata read hostname 'smithi047' 2024-04-26T23:19:50.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.424+0000 7f5983322740 1 --2- 172.21.15.47:0/3565767179 >> [v2:172.21.15.203:6835/2526477051,v1:172.21.15.203:6837/2526477051] conn(0x55ddff330da0 0x55ddff331180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:50.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.424+0000 7f5983322740 4 client.5499 mds_command: new command op to 14738 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:19:50.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.424+0000 7f5983322740 1 -- 172.21.15.47:0/3565767179 --> [v2:172.21.15.203:6835/2526477051,v1:172.21.15.203:6837/2526477051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ddff24a350 con 0x55ddff330da0 2024-04-26T23:19:50.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.424+0000 7f597a59c640 1 --2- 172.21.15.47:0/3565767179 >> [v2:172.21.15.203:6835/2526477051,v1:172.21.15.203:6837/2526477051] conn(0x55ddff330da0 0x55ddff331180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:50.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.425+0000 7f597a59c640 1 --2- 172.21.15.47:0/3565767179 >> [v2:172.21.15.203:6835/2526477051,v1:172.21.15.203:6837/2526477051] conn(0x55ddff330da0 0x55ddff331180 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff4af81128a08190 server_cookie=8eb026fc62e0d865 in_seq=0 out_seq=0 2024-04-26T23:19:50.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.425+0000 7f59597fa640 10 client.5499 ms_handle_connect on v2:172.21.15.203:6835/2526477051 2024-04-26T23:19:50.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.425+0000 7f59597fa640 1 -- 172.21.15.47:0/3565767179 <== mds.0 v2:172.21.15.203:6835/2526477051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ddff24a350 con 0x55ddff330da0 2024-04-26T23:19:50.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.425+0000 7f59597fa640 10 client.5499 handle_command_reply: tid=0 2024-04-26T23:19:50.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.425+0000 7f5983322740 1 client.5499 shutdown 2024-04-26T23:19:50.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.426+0000 7f5983322740 1 -- 172.21.15.47:0/3565767179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ddff31fce0 msgr2=0x55ddff315bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:50.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.426+0000 7f5983322740 1 --2- 172.21.15.47:0/3565767179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ddff31fce0 0x55ddff315bf0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5964005fb0 tx=0x7f59640015e0 comp rx=0 tx=0).stop 2024-04-26T23:19:50.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.426+0000 7f5983322740 1 -- 172.21.15.47:0/3565767179 shutdown_connections 2024-04-26T23:19:50.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.426+0000 7f5983322740 1 --2- 172.21.15.47:0/3565767179 >> [v2:172.21.15.203:6835/2526477051,v1:172.21.15.203:6837/2526477051] conn(0x55ddff330da0 0x55ddff331180 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:50.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.426+0000 7f5983322740 1 --2- 172.21.15.47:0/3565767179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ddff31a410 0x55ddff31a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:50.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.426+0000 7f5983322740 1 --2- 172.21.15.47:0/3565767179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ddff31fce0 0x55ddff315bf0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:50.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.426+0000 7f59597fa640 0 client.5499 ms_handle_reset on v2:172.21.15.203:6835/2526477051 2024-04-26T23:19:50.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.426+0000 7f5983322740 1 --2- 172.21.15.47:0/3565767179 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ddff302440 0x55ddff3001f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:50.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.426+0000 7f5983322740 1 -- 172.21.15.47:0/3565767179 >> 172.21.15.47:0/3565767179 conn(0x55ddff2d8120 msgr2=0x55ddff321060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:50.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.426+0000 7f5983322740 1 -- 172.21.15.47:0/3565767179 shutdown_connections 2024-04-26T23:19:50.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.426+0000 7f5983322740 1 -- 172.21.15.47:0/3565767179 wait complete. 2024-04-26T23:19:50.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.426+0000 7f5983322740 20 client.5499 trim_cache size 0 max 16384 2024-04-26T23:19:50.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.444+0000 7f5983322740 1 Processor -- start 2024-04-26T23:19:50.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.444+0000 7f5983322740 1 -- start start 2024-04-26T23:19:50.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.445+0000 7f5983322740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ddff302440 0x55ddff3413b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:50.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.445+0000 7f5983322740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ddff31a410 0x55ddff2d8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:50.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.445+0000 7f5983322740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ddff31fce0 0x55ddff2d8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:50.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.445+0000 7f5983322740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f597c200c90 con 0x55ddff31fce0 2024-04-26T23:19:50.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.445+0000 7f5983322740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f597c200020 con 0x55ddff31a410 2024-04-26T23:19:50.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.445+0000 7f5983322740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f597c200640 con 0x55ddff302440 2024-04-26T23:19:50.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.445+0000 7f597ad9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ddff31fce0 0x55ddff2d8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:50.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.445+0000 7f597ad9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ddff31fce0 0x55ddff2d8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37920/0 (socket says 172.21.15.47:37920) 2024-04-26T23:19:50.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.445+0000 7f597ad9d640 1 -- 172.21.15.47:0/251155814 learned_addr learned my addr 172.21.15.47:0/251155814 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:50.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.445+0000 7f597ad9d640 1 -- 172.21.15.47:0/251155814 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ddff302440 msgr2=0x55ddff3413b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:50.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.445+0000 7f597ad9d640 1 --2- 172.21.15.47:0/251155814 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ddff302440 0x55ddff3413b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:50.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.445+0000 7f597ad9d640 1 -- 172.21.15.47:0/251155814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ddff31a410 msgr2=0x55ddff2d8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:50.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.445+0000 7f597ad9d640 1 --2- 172.21.15.47:0/251155814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ddff31a410 0x55ddff2d8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:50.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.445+0000 7f597ad9d640 1 -- 172.21.15.47:0/251155814 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddff315c00 con 0x55ddff31fce0 2024-04-26T23:19:50.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.445+0000 7f597ad9d640 1 --2- 172.21.15.47:0/251155814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ddff31fce0 0x55ddff2d8a40 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f596c059db0 tx=0x7f596c08c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac9ef9e73fdac09 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:50.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.446+0000 7f595b7fe640 1 -- 172.21.15.47:0/251155814 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596c05e460 con 0x55ddff31fce0 2024-04-26T23:19:50.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.446+0000 7f595b7fe640 1 -- 172.21.15.47:0/251155814 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f596c062c90 con 0x55ddff31fce0 2024-04-26T23:19:50.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.446+0000 7f595b7fe640 1 -- 172.21.15.47:0/251155814 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596c05e0e0 con 0x55ddff31fce0 2024-04-26T23:19:50.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.446+0000 7f5983322740 1 -- 172.21.15.47:0/251155814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ddff31fce0 msgr2=0x55ddff2d8a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:50.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.446+0000 7f5983322740 1 --2- 172.21.15.47:0/251155814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ddff31fce0 0x55ddff2d8a40 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f596c059db0 tx=0x7f596c08c040 comp rx=0 tx=0).stop 2024-04-26T23:19:50.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.446+0000 7f5983322740 1 -- 172.21.15.47:0/251155814 shutdown_connections 2024-04-26T23:19:50.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.446+0000 7f5983322740 1 --2- 172.21.15.47:0/251155814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ddff31fce0 0x55ddff2d8a40 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:50.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.446+0000 7f5983322740 1 --2- 172.21.15.47:0/251155814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ddff31a410 0x55ddff2d8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:50.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.446+0000 7f5983322740 1 --2- 172.21.15.47:0/251155814 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ddff302440 0x55ddff3413b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:50.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.446+0000 7f5983322740 1 -- 172.21.15.47:0/251155814 >> 172.21.15.47:0/251155814 conn(0x55ddff2d8120 msgr2=0x55ddff3197b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:50.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.446+0000 7f5983322740 1 -- 172.21.15.47:0/251155814 shutdown_connections 2024-04-26T23:19:50.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.446+0000 7f5983322740 1 -- 172.21.15.47:0/251155814 wait complete. 2024-04-26T23:19:50.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.447+0000 7f5983322740 1 -- start start 2024-04-26T23:19:50.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.447+0000 7f5983322740 1 Processor -- start 2024-04-26T23:19:50.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.448+0000 7f5983322740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ddff302440 0x55ddff314050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:50.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.448+0000 7f5983322740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ddff31a410 0x55ddff314590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:50.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.448+0000 7f5983322740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ddff31fce0 0x55ddff340290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:50.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.448+0000 7f5983322740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55ddff31c4e0 con 0x55ddff302440 2024-04-26T23:19:50.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.448+0000 7f5983322740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ddff316130 con 0x55ddff31a410 2024-04-26T23:19:50.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.448+0000 7f5983322740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x55ddff318dd0 con 0x55ddff31fce0 2024-04-26T23:19:50.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.448+0000 7f5983322740 10 client.0 _ll_register_callbacks cb 0x7f5964007280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:19:50.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.448+0000 7f597a59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ddff302440 0x55ddff314050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:50.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.448+0000 7f597a59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ddff302440 0x55ddff314050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37936/0 (socket says 172.21.15.47:37936) 2024-04-26T23:19:50.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.448+0000 7f597a59c640 1 -- 172.21.15.47:0/531320086 learned_addr learned my addr 172.21.15.47:0/531320086 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:50.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.448+0000 7f597a59c640 1 -- 172.21.15.47:0/531320086 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ddff31fce0 msgr2=0x55ddff340290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:50.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.448+0000 7f597a59c640 1 --2- 172.21.15.47:0/531320086 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ddff31fce0 0x55ddff340290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:50.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.448+0000 7f597a59c640 1 -- 172.21.15.47:0/531320086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ddff31a410 msgr2=0x55ddff314590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:50.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.448+0000 7f5979d9b640 1 --2- 172.21.15.47:0/531320086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ddff31a410 0x55ddff314590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:50.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.448+0000 7f597a59c640 1 --2- 172.21.15.47:0/531320086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ddff31a410 0x55ddff314590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:50.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.448+0000 7f597a59c640 1 -- 172.21.15.47:0/531320086 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddff2ff9b0 con 0x55ddff302440 2024-04-26T23:19:50.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.448+0000 7f597a59c640 1 --2- 172.21.15.47:0/531320086 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ddff302440 0x55ddff314050 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f5974006f80 tx=0x7f59740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96457a16fd917555 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:50.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.448+0000 7f59597fa640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-26T23:19:50.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.448+0000 7f59597fa640 1 -- 172.21.15.47:0/531320086 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f597404e040 con 0x55ddff302440 2024-04-26T23:19:50.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.448+0000 7f59597fa640 1 -- 172.21.15.47:0/531320086 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f5974032d90 con 0x55ddff302440 2024-04-26T23:19:50.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.449+0000 7f5983322740 1 -- 172.21.15.47:0/531320086 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f596c062c90 con 0x55ddff302440 2024-04-26T23:19:50.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.449+0000 7f59597fa640 1 -- 172.21.15.47:0/531320086 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f597405d050 con 0x55ddff302440 2024-04-26T23:19:50.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.449+0000 7f5983322740 10 client.5505 fetch_fsmap learned FSMap version 18 2024-04-26T23:19:50.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.449+0000 7f5983322740 1 -- 172.21.15.47:0/531320086 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x55ddff3323b0 con 0x55ddff302440 2024-04-26T23:19:50.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.449+0000 7f59597fa640 1 -- 172.21.15.47:0/531320086 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f596c062c90 con 0x55ddff302440 2024-04-26T23:19:50.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.449+0000 7f59597fa640 1 -- 172.21.15.47:0/531320086 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f5974060020 con 0x55ddff302440 2024-04-26T23:19:50.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.449+0000 7f5983322740 10 client.5505 fetch_fsmap finished waiting for FSMap version 18 2024-04-26T23:19:50.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.449+0000 7f5983322740 10 client.5505 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T23:19:50.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.449+0000 7f5983322740 20 client.5505 populate_metadata read hostname 'smithi047' 2024-04-26T23:19:50.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.449+0000 7f5983322740 1 --2- 172.21.15.47:0/531320086 >> [v2:172.21.15.203:6835/2526477051,v1:172.21.15.203:6837/2526477051] conn(0x55ddff340c50 0x55ddff331330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:50.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.449+0000 7f5983322740 4 client.5505 mds_command: new command op to 14738 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T23:19:50.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.449+0000 7f5983322740 1 -- 172.21.15.47:0/531320086 --> [v2:172.21.15.203:6835/2526477051,v1:172.21.15.203:6837/2526477051] -- command(tid 0: {"prefix": "status"}) -- 0x55ddff2441c0 con 0x55ddff340c50 2024-04-26T23:19:50.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.449+0000 7f5979d9b640 1 --2- 172.21.15.47:0/531320086 >> [v2:172.21.15.203:6835/2526477051,v1:172.21.15.203:6837/2526477051] conn(0x55ddff340c50 0x55ddff331330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:50.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.450+0000 7f5979d9b640 1 --2- 172.21.15.47:0/531320086 >> [v2:172.21.15.203:6835/2526477051,v1:172.21.15.203:6837/2526477051] conn(0x55ddff340c50 0x55ddff331330 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40830690b23d200a server_cookie=da55ecc095006197 in_seq=0 out_seq=0 2024-04-26T23:19:50.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.450+0000 7f59597fa640 10 client.5505 ms_handle_connect on v2:172.21.15.203:6835/2526477051 2024-04-26T23:19:50.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.450+0000 7f59597fa640 1 -- 172.21.15.47:0/531320086 <== mds.0 v2:172.21.15.203:6835/2526477051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55ddff2441c0 con 0x55ddff340c50 2024-04-26T23:19:50.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.450+0000 7f59597fa640 10 client.5505 handle_command_reply: tid=0 2024-04-26T23:19:50.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.450+0000 7f5983322740 1 client.5505 shutdown 2024-04-26T23:19:50.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.450+0000 7f5983322740 1 -- 172.21.15.47:0/531320086 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ddff302440 msgr2=0x55ddff314050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:50.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.450+0000 7f5983322740 1 --2- 172.21.15.47:0/531320086 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ddff302440 0x55ddff314050 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f5974006f80 tx=0x7f59740046c0 comp rx=0 tx=0).stop 2024-04-26T23:19:50.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.450+0000 7f5983322740 1 -- 172.21.15.47:0/531320086 shutdown_connections 2024-04-26T23:19:50.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.450+0000 7f5983322740 1 --2- 172.21.15.47:0/531320086 >> [v2:172.21.15.203:6835/2526477051,v1:172.21.15.203:6837/2526477051] conn(0x55ddff340c50 0x55ddff331330 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:50.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.450+0000 7f5983322740 1 --2- 172.21.15.47:0/531320086 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ddff31fce0 0x55ddff340290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:50.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.450+0000 7f5983322740 1 --2- 172.21.15.47:0/531320086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ddff31a410 0x55ddff314590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:50.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.450+0000 7f59597fa640 0 client.5505 ms_handle_reset on v2:172.21.15.203:6835/2526477051 2024-04-26T23:19:50.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.450+0000 7f5983322740 1 --2- 172.21.15.47:0/531320086 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ddff302440 0x55ddff314050 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:50.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.450+0000 7f5983322740 1 -- 172.21.15.47:0/531320086 >> 172.21.15.47:0/531320086 conn(0x55ddff2d8120 msgr2=0x55ddff2d8500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:50.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.450+0000 7f5983322740 1 -- 172.21.15.47:0/531320086 shutdown_connections 2024-04-26T23:19:50.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.450+0000 7f5983322740 1 -- 172.21.15.47:0/531320086 wait complete. 2024-04-26T23:19:50.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.450+0000 7f5983322740 20 client.5505 trim_cache size 0 max 16384 2024-04-26T23:19:50.458 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-26T23:19:50.458 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "399006d3-4db1-43e6-ba81-a79a276eabfd", 2024-04-26T23:19:50.458 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 0, 2024-04-26T23:19:50.458 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 14738, 2024-04-26T23:19:50.458 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-04-26T23:19:50.458 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-04-26T23:19:50.458 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-04-26T23:19:50.458 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 5.7365580779999998, 2024-04-26T23:19:50.458 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 18, 2024-04-26T23:19:50.458 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 38, 2024-04-26T23:19:50.459 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 38, 2024-04-26T23:19:50.459 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 12.911619923 2024-04-26T23:19:50.459 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-26T23:19:50.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.451+0000 7f598096a640 1 -- 172.21.15.47:0/1660458175 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f595c06c8f0 msgr2=0x7f595c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:50.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.451+0000 7f598096a640 1 --2- 172.21.15.47:0/1660458175 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f595c06c8f0 0x7f595c06edb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5974034000 tx=0x7f597402e000 comp rx=0 tx=0).stop 2024-04-26T23:19:50.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.451+0000 7f598096a640 1 -- 172.21.15.47:0/1660458175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f597c162880 msgr2=0x7f597c14fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:50.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.451+0000 7f598096a640 1 --2- 172.21.15.47:0/1660458175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f597c162880 0x7f597c14fc70 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f596c007d70 tx=0x7f596c00e4b0 comp rx=0 tx=0).stop 2024-04-26T23:19:50.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.451+0000 7f598096a640 1 -- 172.21.15.47:0/1660458175 shutdown_connections 2024-04-26T23:19:50.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.451+0000 7f598096a640 1 --2- 172.21.15.47:0/1660458175 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f595c06c8f0 0x7f595c06edb0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:50.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.451+0000 7f598096a640 1 --2- 172.21.15.47:0/1660458175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f597c162880 0x7f597c14fc70 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:50.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.451+0000 7f598096a640 1 --2- 172.21.15.47:0/1660458175 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f597c0d3bc0 0x7f597c0b1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:50.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.451+0000 7f598096a640 1 --2- 172.21.15.47:0/1660458175 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f597c0b5f20 0x7f597c0ce290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:50.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.451+0000 7f598096a640 1 -- 172.21.15.47:0/1660458175 >> 172.21.15.47:0/1660458175 conn(0x7f597c0d4f60 msgr2=0x7f597c1614d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:50.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.451+0000 7f598096a640 1 -- 172.21.15.47:0/1660458175 shutdown_connections 2024-04-26T23:19:50.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.451+0000 7f598096a640 1 -- 172.21.15.47:0/1660458175 wait complete. 2024-04-26T23:19:50.463 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T23:19:50.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.562+0000 7f94e67cf640 1 Processor -- start 2024-04-26T23:19:50.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.562+0000 7f94e67cf640 1 -- start start 2024-04-26T23:19:50.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.563+0000 7f94e67cf640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f94e0162810 0x7f94e0070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:50.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.563+0000 7f94e67cf640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f94e00d1b30 0x7f94e00d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:50.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.563+0000 7f94e67cf640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94e00d10d0 con 0x7f94e00b2340 2024-04-26T23:19:50.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.563+0000 7f94e67cf640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f94e00d0750 con 0x7f94e0162810 2024-04-26T23:19:50.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.563+0000 7f94e67cf640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f94e00d16a0 con 0x7f94e00d1b30 2024-04-26T23:19:50.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.563+0000 7f94dffff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f94e0162810 0x7f94e0070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:50.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.563+0000 7f94dffff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f94e0162810 0x7f94e0070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37952/0 (socket says 172.21.15.47:37952) 2024-04-26T23:19:50.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.563+0000 7f94dffff640 1 -- 172.21.15.47:0/2982032390 learned_addr learned my addr 172.21.15.47:0/2982032390 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:50.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.563+0000 7f94e4d6c640 1 --2- 172.21.15.47:0/2982032390 >> v2:172.21.15.203:3300/0 conn(0x7f94e00d1b30 0x7f94e00d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:50.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.563+0000 7f94dffff640 1 -- 172.21.15.47:0/2982032390 >> v2:172.21.15.203:3300/0 conn(0x7f94e00d1b30 msgr2=0x7f94e00d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:50.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.563+0000 7f94dffff640 1 --2- 172.21.15.47:0/2982032390 >> v2:172.21.15.203:3300/0 conn(0x7f94e00d1b30 0x7f94e00d1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:50.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.563+0000 7f94dffff640 1 -- 172.21.15.47:0/2982032390 >> v1:172.21.15.203:6789/0 conn(0x7f94e00b2340 legacy=0x7f94e00cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:50.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.563+0000 7f94dffff640 1 -- 172.21.15.47:0/2982032390 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94e00d1370 con 0x7f94e0162810 2024-04-26T23:19:50.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.564+0000 7f94dffff640 1 --2- 172.21.15.47:0/2982032390 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94e0162810 0x7f94e0070350 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f94e0073330 tx=0x7f94c802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9742b0b8d9f25d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:50.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.564+0000 7f94deffd640 1 -- 172.21.15.47:0/2982032390 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c8037070 con 0x7f94e0162810 2024-04-26T23:19:50.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.564+0000 7f94deffd640 1 -- 172.21.15.47:0/2982032390 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94c8032440 con 0x7f94e0162810 2024-04-26T23:19:50.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.564+0000 7f94deffd640 1 -- 172.21.15.47:0/2982032390 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c8032810 con 0x7f94e0162810 2024-04-26T23:19:50.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.565+0000 7f94e67cf640 1 -- 172.21.15.47:0/2982032390 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94e0162810 msgr2=0x7f94e0070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:50.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.565+0000 7f94e67cf640 1 --2- 172.21.15.47:0/2982032390 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94e0162810 0x7f94e0070350 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f94e0073330 tx=0x7f94c802a270 comp rx=0 tx=0).stop 2024-04-26T23:19:50.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.565+0000 7f94e67cf640 1 -- 172.21.15.47:0/2982032390 shutdown_connections 2024-04-26T23:19:50.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.565+0000 7f94e67cf640 1 --2- 172.21.15.47:0/2982032390 >> v2:172.21.15.203:3300/0 conn(0x7f94e00d1b30 0x7f94e00d1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:50.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.565+0000 7f94e67cf640 1 --2- 172.21.15.47:0/2982032390 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94e0162810 0x7f94e0070350 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:50.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.565+0000 7f94e67cf640 1 -- 172.21.15.47:0/2982032390 >> 172.21.15.47:0/2982032390 conn(0x7f94e00d2da0 msgr2=0x7f94e00d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:50.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.565+0000 7f94e67cf640 1 -- 172.21.15.47:0/2982032390 shutdown_connections 2024-04-26T23:19:50.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.565+0000 7f94e67cf640 1 -- 172.21.15.47:0/2982032390 wait complete. 2024-04-26T23:19:50.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.566+0000 7f94e67cf640 1 Processor -- start 2024-04-26T23:19:50.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.566+0000 7f94e67cf640 1 -- start start 2024-04-26T23:19:50.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.567+0000 7f94e67cf640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94e00b2340 0x7f94e014bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:50.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.567+0000 7f94e67cf640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94e00d1b30 0x7f94e006d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:50.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.567+0000 7f94e67cf640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94e0162810 0x7f94e006d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:50.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.567+0000 7f94e67cf640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f94e0070890 con 0x7f94e0162810 2024-04-26T23:19:50.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.567+0000 7f94e67cf640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f94e00d0470 con 0x7f94e00d1b30 2024-04-26T23:19:50.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.567+0000 7f94e67cf640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f94e00cdb00 con 0x7f94e00b2340 2024-04-26T23:19:50.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.567+0000 7f94e4d6c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94e0162810 0x7f94e006d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:50.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.567+0000 7f94e4d6c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94e0162810 0x7f94e006d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37962/0 (socket says 172.21.15.47:37962) 2024-04-26T23:19:50.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.567+0000 7f94e4d6c640 1 -- 172.21.15.47:0/565862951 learned_addr learned my addr 172.21.15.47:0/565862951 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:50.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.567+0000 7f94df7fe640 1 --2- 172.21.15.47:0/565862951 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94e00d1b30 0x7f94e006d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:50.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.567+0000 7f94dffff640 1 --2- 172.21.15.47:0/565862951 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94e00b2340 0x7f94e014bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:50.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.567+0000 7f94e4d6c640 1 -- 172.21.15.47:0/565862951 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94e00b2340 msgr2=0x7f94e014bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:50.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.567+0000 7f94e4d6c640 1 --2- 172.21.15.47:0/565862951 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94e00b2340 0x7f94e014bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:50.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.567+0000 7f94e4d6c640 1 -- 172.21.15.47:0/565862951 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94e00d1b30 msgr2=0x7f94e006d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:50.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.567+0000 7f94e4d6c640 1 --2- 172.21.15.47:0/565862951 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94e00d1b30 0x7f94e006d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:50.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.567+0000 7f94e4d6c640 1 -- 172.21.15.47:0/565862951 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94e01f4dd0 con 0x7f94e0162810 2024-04-26T23:19:50.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.567+0000 7f94df7fe640 1 --2- 172.21.15.47:0/565862951 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94e00d1b30 0x7f94e006d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:19:50.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.568+0000 7f94e4d6c640 1 --2- 172.21.15.47:0/565862951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94e0162810 0x7f94e006d890 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f94d800efc0 tx=0x7f94d800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:50.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.568+0000 7f94dcff9640 1 -- 172.21.15.47:0/565862951 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d8018020 con 0x7f94e0162810 2024-04-26T23:19:50.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.568+0000 7f94dcff9640 1 -- 172.21.15.47:0/565862951 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94d8014480 con 0x7f94e0162810 2024-04-26T23:19:50.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.568+0000 7f94dcff9640 1 -- 172.21.15.47:0/565862951 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d8014870 con 0x7f94e0162810 2024-04-26T23:19:50.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.568+0000 7f94e67cf640 1 -- 172.21.15.47:0/565862951 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94e014f320 con 0x7f94e0162810 2024-04-26T23:19:50.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.568+0000 7f94e67cf640 1 -- 172.21.15.47:0/565862951 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94e01f5370 con 0x7f94e0162810 2024-04-26T23:19:50.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.569+0000 7f94e67cf640 1 -- 172.21.15.47:0/565862951 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94e0163a80 con 0x7f94e0162810 2024-04-26T23:19:50.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.570+0000 7f94dcff9640 1 -- 172.21.15.47:0/565862951 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f94d8005d10 con 0x7f94e0162810 2024-04-26T23:19:50.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.570+0000 7f94dcff9640 1 --2- 172.21.15.47:0/565862951 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f94c006c8f0 0x7f94c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:50.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.570+0000 7f94dcff9640 1 -- 172.21.15.47:0/565862951 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f94d808ca90 con 0x7f94e0162810 2024-04-26T23:19:50.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.570+0000 7f94dffff640 1 --2- 172.21.15.47:0/565862951 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f94c006c8f0 0x7f94c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:50.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.571+0000 7f94dffff640 1 --2- 172.21.15.47:0/565862951 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f94c006c8f0 0x7f94c006edb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f94c8034000 tx=0x7f94c802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:50.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.574+0000 7f94dcff9640 1 -- 172.21.15.47:0/565862951 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f94e0163a80 con 0x7f94e0162810 2024-04-26T23:19:50.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.801+0000 7f94e67cf640 1 -- 172.21.15.47:0/565862951 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f94e00694f0 con 0x7f94e0162810 2024-04-26T23:19:50.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.801+0000 7f94dcff9640 1 -- 172.21.15.47:0/565862951 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) ==== 53+0+73 (secure 0 0 0) 0x7f94d8022030 con 0x7f94e0162810 2024-04-26T23:19:50.803 INFO:teuthology.orchestra.run.smithi047.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T23:19:50.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.803+0000 7f94e67cf640 1 -- 172.21.15.47:0/565862951 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f94c006c8f0 msgr2=0x7f94c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:50.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.803+0000 7f94e67cf640 1 --2- 172.21.15.47:0/565862951 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f94c006c8f0 0x7f94c006edb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f94c8034000 tx=0x7f94c802e000 comp rx=0 tx=0).stop 2024-04-26T23:19:50.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.803+0000 7f94e67cf640 1 -- 172.21.15.47:0/565862951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94e0162810 msgr2=0x7f94e006d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:50.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.803+0000 7f94e67cf640 1 --2- 172.21.15.47:0/565862951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94e0162810 0x7f94e006d890 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f94d800efc0 tx=0x7f94d800c490 comp rx=0 tx=0).stop 2024-04-26T23:19:50.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.803+0000 7f94e67cf640 1 -- 172.21.15.47:0/565862951 shutdown_connections 2024-04-26T23:19:50.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.803+0000 7f94e67cf640 1 --2- 172.21.15.47:0/565862951 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f94c006c8f0 0x7f94c006edb0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:50.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.803+0000 7f94e67cf640 1 --2- 172.21.15.47:0/565862951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94e0162810 0x7f94e006d890 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:50.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.803+0000 7f94e67cf640 1 --2- 172.21.15.47:0/565862951 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94e00d1b30 0x7f94e006d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:50.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.803+0000 7f94e67cf640 1 --2- 172.21.15.47:0/565862951 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94e00b2340 0x7f94e014bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:50.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.803+0000 7f94e67cf640 1 -- 172.21.15.47:0/565862951 >> 172.21.15.47:0/565862951 conn(0x7f94e00d2da0 msgr2=0x7f94e00cd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:50.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.804+0000 7f94e67cf640 1 -- 172.21.15.47:0/565862951 shutdown_connections 2024-04-26T23:19:50.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:50.804+0000 7f94e67cf640 1 -- 172.21.15.47:0/565862951 wait complete. 2024-04-26T23:19:50.816 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T23:19:50.816 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T23:19:50.816 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi047.front.sepia.ceph.com 2024-04-26T23:19:50.817 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T23:19:50.817 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T23:19:50.817 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T23:19:50.817 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T23:19:50.817 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T23:19:50.817 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:19:50.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:50.817 DEBUG:teuthology.orchestra.run.smithi047:> ip addr 2024-04-26T23:19:50.833 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T23:19:50.833 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T23:19:50.833 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T23:19:50.833 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:19:50.833 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2024-04-26T23:19:50.834 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:19:50.834 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:19:50.834 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:19:50.834 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f0 2024-04-26T23:19:50.834 INFO:teuthology.orchestra.run.smithi047.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:19:50.834 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:19:50.834 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f1 2024-04-26T23:19:50.834 INFO:teuthology.orchestra.run.smithi047.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T23:19:50.834 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2024-04-26T23:19:50.834 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f0 2024-04-26T23:19:50.834 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-26T23:19:50.834 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42608sec preferred_lft 42608sec 2024-04-26T23:19:50.834 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link noprefixroute 2024-04-26T23:19:50.834 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:19:50.835 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:19:50.835 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2024-04-26T23:19:50.835 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f1 2024-04-26T23:19:50.835 INFO:teuthology.orchestra.run.smithi047.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T23:19:50.835 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 32:8d:47:b1:d0:77 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:19:50.835 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T23:19:50.835 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:19:50.835 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::7c6d:6aff:fe00:4b30/64 scope link 2024-04-26T23:19:50.835 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:19:50.835 INFO:teuthology.orchestra.run.smithi047.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T23:19:50.835 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 32:8d:47:b1:d0:77 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T23:19:50.835 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::308d:47ff:feb1:d077/64 scope link 2024-04-26T23:19:50.835 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:19:50.836 INFO:teuthology.orchestra.run.smithi047.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T23:19:50.836 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether b2:9e:42:6a:5b:5d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T23:19:50.836 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::b09e:42ff:fe6a:5b5d/64 scope link 2024-04-26T23:19:50.836 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:19:50.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:50.836 DEBUG:teuthology.orchestra.run.smithi047:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T23:19:50.896 INFO:teuthology.orchestra.run.smithi047.stdout:1 2024-04-26T23:19:50.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:50.898 DEBUG:teuthology.orchestra.run.smithi047:> route 2024-04-26T23:19:50.955 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2024-04-26T23:19:50.955 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T23:19:50.955 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T23:19:50.955 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T23:19:50.955 INFO:teuthology.orchestra.run.smithi047.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T23:19:50.956 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:19:50.956 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-04-26T23:19:50.956 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T23:19:50.956 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T23:19:50.956 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T23:19:50.956 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-04-26T23:19:51.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:19:51.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:19:51.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:51.082 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-04-26T23:19:51.097 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T23:19:51.097 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T23:19:51.097 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:19:51.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:51.097 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set brx.0 up 2024-04-26T23:19:51.163 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:19:51.164 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T23:19:51.219 INFO:teuthology.orchestra.run.smithi047.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:19:51.219 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:19:51.219 DEBUG:teuthology.orchestra.run.smithi047:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T23:19:51.273 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T23:19:51.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:51.274 DEBUG:teuthology.orchestra.run.smithi047:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:19:51.356 INFO:teuthology.orchestra.run.smithi047.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:19:51.357 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:19:51.357 INFO:teuthology.orchestra.run.smithi047.stdout:invalid new device string format 2024-04-26T23:19:51.365 INFO:teuthology.orchestra.run.smithi047.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:19:51.365 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:19:51.365 INFO:teuthology.orchestra.run.smithi047.stdout:invalid new device string format 2024-04-26T23:19:51.365 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: resolved to: "172.21.15.47:3300,172.21.15.203:3300,172.21.15.203:3301" 2024-04-26T23:19:51.365 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:19:51.370 INFO:teuthology.orchestra.run.smithi047.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:19:51.371 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:19:51.371 INFO:teuthology.orchestra.run.smithi047.stdout:invalid new device string format 2024-04-26T23:19:51.371 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: resolved to: "172.21.15.47:3300,172.21.15.203:3300,172.21.15.203:3301" 2024-04-26T23:19:51.371 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:19:51.371 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: resolved to: "172.21.15.47:6789,172.21.15.203:6789,172.21.15.203:6790" 2024-04-26T23:19:51.371 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.47:6789,172.21.15.203:6789,172.21.15.203:6790:/ 2024-04-26T23:19:51.371 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=399006d3-4db1-43e6-ba81-a79a276eabfd" will pass to kernel 2024-04-26T23:19:51.372 INFO:teuthology.orchestra.run.smithi047.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T23:19:51.372 INFO:teuthology.orchestra.run.smithi047.stdout: You just mounted a file system that supports labels which does not 2024-04-26T23:19:51.372 INFO:teuthology.orchestra.run.smithi047.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T23:19:51.372 INFO:teuthology.orchestra.run.smithi047.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T23:19:51.372 INFO:teuthology.orchestra.run.smithi047.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T23:19:51.373 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T23:19:51.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:51.373 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T23:19:51.439 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T23:19:51.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:19:51.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:19:51.543 INFO:teuthology.orchestra.run.smithi047.stdout:client14816 2024-04-26T23:19:51.544 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:19:51.544 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:19:51.560 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create data_new --pg_num_min 64 2024-04-26T23:19:51.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.704+0000 7f1aea3f0640 1 Processor -- start 2024-04-26T23:19:51.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.704+0000 7f1aea3f0640 1 -- start start 2024-04-26T23:19:51.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.704+0000 7f1aea3f0640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1ae415f3e0 0x7f1ae40d2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:51.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.704+0000 7f1aea3f0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ae40d31e0 0x7f1ae40d3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:51.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.704+0000 7f1aea3f0640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ae4160930 con 0x7f1ae415f7c0 2024-04-26T23:19:51.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.704+0000 7f1aea3f0640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f1ae415fd20 con 0x7f1ae415f3e0 2024-04-26T23:19:51.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.704+0000 7f1aea3f0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f1ae4160000 con 0x7f1ae40d31e0 2024-04-26T23:19:51.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.705+0000 7f1ae37fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ae40d31e0 0x7f1ae40d3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:51.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.705+0000 7f1ae37fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ae40d31e0 0x7f1ae40d3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.47:46974/0 (socket says 172.21.15.47:46974) 2024-04-26T23:19:51.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.705+0000 7f1ae3fff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1ae415f3e0 0x7f1ae40d2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:51.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.705+0000 7f1ae37fe640 1 -- 172.21.15.47:0/3698118739 learned_addr learned my addr 172.21.15.47:0/3698118739 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:51.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.705+0000 7f1ae898d640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f1ae415f7c0 0x7f1ae41620a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.47:41188/0 (socket says 172.21.15.47:41188) 2024-04-26T23:19:51.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.705+0000 7f1ae3fff640 1 -- 172.21.15.47:0/3698118739 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ae40d31e0 msgr2=0x7f1ae40d3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:51.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.705+0000 7f1ae3fff640 1 --2- 172.21.15.47:0/3698118739 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ae40d31e0 0x7f1ae40d3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:51.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.705+0000 7f1ae3fff640 1 -- 172.21.15.47:0/3698118739 >> v1:172.21.15.203:6789/0 conn(0x7f1ae415f7c0 legacy=0x7f1ae41620a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:51.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.705+0000 7f1ae3fff640 1 -- 172.21.15.47:0/3698118739 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ae41602e0 con 0x7f1ae415f3e0 2024-04-26T23:19:51.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.705+0000 7f1ae2ffd640 1 -- 172.21.15.47:0/3698118739 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2573841823 0 0) 0x7f1ae4160930 con 0x7f1ae415f7c0 2024-04-26T23:19:51.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.705+0000 7f1ae37fe640 1 --2- 172.21.15.47:0/3698118739 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ae40d31e0 0x7f1ae40d3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:19:51.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.706+0000 7f1ae3fff640 1 --2- 172.21.15.47:0/3698118739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ae415f3e0 0x7f1ae40d2ca0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1ad40097a0 tx=0x7f1ad402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=115ad4a17f106ea3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:51.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.706+0000 7f1ae2ffd640 1 -- 172.21.15.47:0/3698118739 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad4037070 con 0x7f1ae415f3e0 2024-04-26T23:19:51.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.706+0000 7f1ae2ffd640 1 -- 172.21.15.47:0/3698118739 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ad402ae50 con 0x7f1ae415f3e0 2024-04-26T23:19:51.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.706+0000 7f1ae2ffd640 1 -- 172.21.15.47:0/3698118739 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad4032660 con 0x7f1ae415f3e0 2024-04-26T23:19:51.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.707+0000 7f1aea3f0640 1 -- 172.21.15.47:0/3698118739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ae415f3e0 msgr2=0x7f1ae40d2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:51.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.707+0000 7f1aea3f0640 1 --2- 172.21.15.47:0/3698118739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ae415f3e0 0x7f1ae40d2ca0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1ad40097a0 tx=0x7f1ad402a100 comp rx=0 tx=0).stop 2024-04-26T23:19:51.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.707+0000 7f1aea3f0640 1 -- 172.21.15.47:0/3698118739 shutdown_connections 2024-04-26T23:19:51.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.707+0000 7f1aea3f0640 1 --2- 172.21.15.47:0/3698118739 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ae40d31e0 0x7f1ae40d3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:51.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.707+0000 7f1aea3f0640 1 --2- 172.21.15.47:0/3698118739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ae415f3e0 0x7f1ae40d2ca0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:51.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.707+0000 7f1aea3f0640 1 -- 172.21.15.47:0/3698118739 >> 172.21.15.47:0/3698118739 conn(0x7f1ae409e3c0 msgr2=0x7f1ae40d50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:51.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.707+0000 7f1aea3f0640 1 -- 172.21.15.47:0/3698118739 shutdown_connections 2024-04-26T23:19:51.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.707+0000 7f1aea3f0640 1 -- 172.21.15.47:0/3698118739 wait complete. 2024-04-26T23:19:51.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.708+0000 7f1aea3f0640 1 Processor -- start 2024-04-26T23:19:51.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.708+0000 7f1aea3f0640 1 -- start start 2024-04-26T23:19:51.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.709+0000 7f1aea3f0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ae40d31e0 0x7f1ae41f5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:51.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.709+0000 7f1aea3f0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ae415f3e0 0x7f1ae415bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:51.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.709+0000 7f1aea3f0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ae415f7c0 0x7f1ae40c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:51.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.709+0000 7f1aea3f0640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae40d3bd0 con 0x7f1ae40d31e0 2024-04-26T23:19:51.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.709+0000 7f1aea3f0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae4160d40 con 0x7f1ae415f7c0 2024-04-26T23:19:51.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.709+0000 7f1aea3f0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f1ae40d3d50 con 0x7f1ae415f3e0 2024-04-26T23:19:51.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.709+0000 7f1ae3fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ae40d31e0 0x7f1ae41f5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:51.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.709+0000 7f1ae3fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ae40d31e0 0x7f1ae41f5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37964/0 (socket says 172.21.15.47:37964) 2024-04-26T23:19:51.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.709+0000 7f1ae3fff640 1 -- 172.21.15.47:0/3691761653 learned_addr learned my addr 172.21.15.47:0/3691761653 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:51.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.709+0000 7f1ae37fe640 1 --2- 172.21.15.47:0/3691761653 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ae415f3e0 0x7f1ae415bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:51.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.709+0000 7f1ae898d640 1 --2- 172.21.15.47:0/3691761653 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ae415f7c0 0x7f1ae40c95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:51.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.710+0000 7f1ae3fff640 1 -- 172.21.15.47:0/3691761653 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ae415f3e0 msgr2=0x7f1ae415bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:51.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.710+0000 7f1ae3fff640 1 --2- 172.21.15.47:0/3691761653 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ae415f3e0 0x7f1ae415bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:51.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.710+0000 7f1ae3fff640 1 -- 172.21.15.47:0/3691761653 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ae415f7c0 msgr2=0x7f1ae40c95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:51.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.710+0000 7f1ae3fff640 1 --2- 172.21.15.47:0/3691761653 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ae415f7c0 0x7f1ae40c95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:51.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.710+0000 7f1ae3fff640 1 -- 172.21.15.47:0/3691761653 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ae415c060 con 0x7f1ae40d31e0 2024-04-26T23:19:51.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.710+0000 7f1ae3fff640 1 --2- 172.21.15.47:0/3691761653 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ae40d31e0 0x7f1ae41f5440 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4035040 tx=0x7f1ad40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:51.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.710+0000 7f1ae0ff9640 1 -- 172.21.15.47:0/3691761653 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad4037040 con 0x7f1ae40d31e0 2024-04-26T23:19:51.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.710+0000 7f1ae0ff9640 1 -- 172.21.15.47:0/3691761653 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ad4004030 con 0x7f1ae40d31e0 2024-04-26T23:19:51.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.710+0000 7f1aea3f0640 1 -- 172.21.15.47:0/3691761653 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ae40b8f40 con 0x7f1ae40d31e0 2024-04-26T23:19:51.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.710+0000 7f1ae0ff9640 1 -- 172.21.15.47:0/3691761653 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad4004420 con 0x7f1ae40d31e0 2024-04-26T23:19:51.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.710+0000 7f1aea3f0640 1 -- 172.21.15.47:0/3691761653 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ae40b9500 con 0x7f1ae40d31e0 2024-04-26T23:19:51.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.711+0000 7f1aea3f0640 1 -- 172.21.15.47:0/3691761653 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ae4069600 con 0x7f1ae40d31e0 2024-04-26T23:19:51.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.712+0000 7f1ae0ff9640 1 -- 172.21.15.47:0/3691761653 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1ad4003610 con 0x7f1ae40d31e0 2024-04-26T23:19:51.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.712+0000 7f1ae0ff9640 1 --2- 172.21.15.47:0/3691761653 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1ad006c8f0 0x7f1ad006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:51.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.712+0000 7f1ae0ff9640 1 -- 172.21.15.47:0/3691761653 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f1ad40ac860 con 0x7f1ae40d31e0 2024-04-26T23:19:51.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.712+0000 7f1ae37fe640 1 --2- 172.21.15.47:0/3691761653 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1ad006c8f0 0x7f1ad006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:51.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.713+0000 7f1ae37fe640 1 --2- 172.21.15.47:0/3691761653 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1ad006c8f0 0x7f1ad006edb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1ae40c8d00 tx=0x7f1ad8009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:51.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.717+0000 7f1ae0ff9640 1 -- 172.21.15.47:0/3691761653 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1ad407a540 con 0x7f1ae40d31e0 2024-04-26T23:19:51.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:51.906+0000 7f1aea3f0640 1 -- 172.21.15.47:0/3691761653 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "data_new", "pg_num_min": 64} v 0) -- 0x7f1ae40d2680 con 0x7f1ae40d31e0 2024-04-26T23:19:52.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.767+0000 7f1ae0ff9640 1 -- 172.21.15.47:0/3691761653 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "data_new", "pg_num_min": 64}]=0 pool 'data_new' created v39) ==== 124+0+0 (secure 0 0 0) 0x7f1ad407e840 con 0x7f1ae40d31e0 2024-04-26T23:19:52.770 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'data_new' created 2024-04-26T23:19:52.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.771+0000 7f1aea3f0640 1 -- 172.21.15.47:0/3691761653 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1ad006c8f0 msgr2=0x7f1ad006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:52.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.771+0000 7f1aea3f0640 1 --2- 172.21.15.47:0/3691761653 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1ad006c8f0 0x7f1ad006edb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1ae40c8d00 tx=0x7f1ad8009240 comp rx=0 tx=0).stop 2024-04-26T23:19:52.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.771+0000 7f1aea3f0640 1 -- 172.21.15.47:0/3691761653 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ae40d31e0 msgr2=0x7f1ae41f5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:52.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.771+0000 7f1aea3f0640 1 --2- 172.21.15.47:0/3691761653 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ae40d31e0 0x7f1ae41f5440 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4035040 tx=0x7f1ad40046c0 comp rx=0 tx=0).stop 2024-04-26T23:19:52.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.771+0000 7f1aea3f0640 1 -- 172.21.15.47:0/3691761653 shutdown_connections 2024-04-26T23:19:52.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.771+0000 7f1aea3f0640 1 --2- 172.21.15.47:0/3691761653 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1ad006c8f0 0x7f1ad006edb0 secure :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1ae40c8d00 tx=0x7f1ad8009240 comp rx=0 tx=0).stop 2024-04-26T23:19:52.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.771+0000 7f1aea3f0640 1 --2- 172.21.15.47:0/3691761653 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ae415f7c0 0x7f1ae40c95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:52.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.771+0000 7f1aea3f0640 1 --2- 172.21.15.47:0/3691761653 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ae415f3e0 0x7f1ae415bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:52.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.771+0000 7f1aea3f0640 1 --2- 172.21.15.47:0/3691761653 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ae40d31e0 0x7f1ae41f5440 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:52.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.771+0000 7f1aea3f0640 1 -- 172.21.15.47:0/3691761653 >> 172.21.15.47:0/3691761653 conn(0x7f1ae409e3c0 msgr2=0x7f1ae415d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:52.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.772+0000 7f1aea3f0640 1 -- 172.21.15.47:0/3691761653 shutdown_connections 2024-04-26T23:19:52.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.772+0000 7f1aea3f0640 1 -- 172.21.15.47:0/3691761653 wait complete. 2024-04-26T23:19:52.790 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs data_new 2024-04-26T23:19:52.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.893+0000 7f4255cd1640 1 Processor -- start 2024-04-26T23:19:52.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.893+0000 7f4255cd1640 1 -- start start 2024-04-26T23:19:52.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.894+0000 7f4255cd1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f42500d3bc0 0x7f42500d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:52.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.894+0000 7f4255cd1640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f42500b5f20 0x7f42500b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:52.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.894+0000 7f4255cd1640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42500b7e50 con 0x7f4250162880 2024-04-26T23:19:52.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.894+0000 7f4255cd1640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f42500d0580 con 0x7f42500b5f20 2024-04-26T23:19:52.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.894+0000 7f4255cd1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f42500d1800 con 0x7f42500d3bc0 2024-04-26T23:19:52.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.894+0000 7f424effd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f42500d3bc0 0x7f42500d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:52.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.894+0000 7f424ffff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f42500b5f20 0x7f42500b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:52.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.894+0000 7f424f7fe640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f4250162880 0x7f42500d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.47:41202/0 (socket says 172.21.15.47:41202) 2024-04-26T23:19:52.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.894+0000 7f424ffff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f42500b5f20 0x7f42500b8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.47:40846/0 (socket says 172.21.15.47:40846) 2024-04-26T23:19:52.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.894+0000 7f424f7fe640 1 -- 172.21.15.47:0/1332232703 learned_addr learned my addr 172.21.15.47:0/1332232703 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:52.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.894+0000 7f424effd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f42500d3bc0 0x7f42500d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.47:46992/0 (socket says 172.21.15.47:46992) 2024-04-26T23:19:52.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.895+0000 7f424effd640 1 -- 172.21.15.47:0/1332232703 >> v2:172.21.15.203:3300/0 conn(0x7f42500b5f20 msgr2=0x7f42500b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:52.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.895+0000 7f424effd640 1 --2- 172.21.15.47:0/1332232703 >> v2:172.21.15.203:3300/0 conn(0x7f42500b5f20 0x7f42500b8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:52.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.895+0000 7f424effd640 1 -- 172.21.15.47:0/1332232703 >> v1:172.21.15.203:6789/0 conn(0x7f4250162880 legacy=0x7f42500d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:52.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.895+0000 7f424e7fc640 1 -- 172.21.15.47:0/1332232703 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1075259328 0 0) 0x7f42500b7e50 con 0x7f4250162880 2024-04-26T23:19:52.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.895+0000 7f424effd640 1 -- 172.21.15.47:0/1332232703 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42500d07d0 con 0x7f42500d3bc0 2024-04-26T23:19:52.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.895+0000 7f424ffff640 1 --2- 172.21.15.47:0/1332232703 >> v2:172.21.15.203:3300/0 conn(0x7f42500b5f20 0x7f42500b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:19:52.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.895+0000 7f424effd640 1 --2- 172.21.15.47:0/1332232703 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f42500d3bc0 0x7f42500d2cb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f42440097a0 tx=0x7f424402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e1602cf35d19a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:52.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.896+0000 7f424e7fc640 1 -- 172.21.15.47:0/1332232703 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4244037070 con 0x7f42500d3bc0 2024-04-26T23:19:52.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.896+0000 7f424e7fc640 1 -- 172.21.15.47:0/1332232703 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4244032440 con 0x7f42500d3bc0 2024-04-26T23:19:52.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.896+0000 7f424e7fc640 1 -- 172.21.15.47:0/1332232703 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4244032810 con 0x7f42500d3bc0 2024-04-26T23:19:52.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.896+0000 7f4255cd1640 1 -- 172.21.15.47:0/1332232703 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f42500d3bc0 msgr2=0x7f42500d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:52.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.896+0000 7f4255cd1640 1 --2- 172.21.15.47:0/1332232703 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f42500d3bc0 0x7f42500d2cb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f42440097a0 tx=0x7f424402a180 comp rx=0 tx=0).stop 2024-04-26T23:19:52.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.896+0000 7f4255cd1640 1 -- 172.21.15.47:0/1332232703 shutdown_connections 2024-04-26T23:19:52.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.896+0000 7f4255cd1640 1 --2- 172.21.15.47:0/1332232703 >> v2:172.21.15.203:3300/0 conn(0x7f42500b5f20 0x7f42500b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:52.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.896+0000 7f4255cd1640 1 --2- 172.21.15.47:0/1332232703 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f42500d3bc0 0x7f42500d2cb0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:52.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.896+0000 7f4255cd1640 1 -- 172.21.15.47:0/1332232703 >> 172.21.15.47:0/1332232703 conn(0x7f42500d4f60 msgr2=0x7f42500d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:52.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.896+0000 7f4255cd1640 1 -- 172.21.15.47:0/1332232703 shutdown_connections 2024-04-26T23:19:52.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.896+0000 7f4255cd1640 1 -- 172.21.15.47:0/1332232703 wait complete. 2024-04-26T23:19:52.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.897+0000 7f4255cd1640 1 Processor -- start 2024-04-26T23:19:52.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.898+0000 7f4255cd1640 1 -- start start 2024-04-26T23:19:52.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.898+0000 7f4255cd1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f42500b5f20 0x7f42500b8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:52.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.898+0000 7f4255cd1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f42500d3bc0 0x7f42500b91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:52.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.898+0000 7f4255cd1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4250162880 0x7f42500b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:52.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.898+0000 7f4255cd1640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4250162c60 con 0x7f4250162880 2024-04-26T23:19:52.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.898+0000 7f4255cd1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f42500d0d90 con 0x7f42500b5f20 2024-04-26T23:19:52.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.898+0000 7f4255cd1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f4250162de0 con 0x7f42500d3bc0 2024-04-26T23:19:52.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.898+0000 7f424ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4250162880 0x7f42500b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:52.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.899+0000 7f424ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4250162880 0x7f42500b3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37968/0 (socket says 172.21.15.47:37968) 2024-04-26T23:19:52.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.899+0000 7f424ffff640 1 -- 172.21.15.47:0/2599744776 learned_addr learned my addr 172.21.15.47:0/2599744776 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:52.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.899+0000 7f424f7fe640 1 --2- 172.21.15.47:0/2599744776 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f42500b5f20 0x7f42500b8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:52.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.899+0000 7f424effd640 1 --2- 172.21.15.47:0/2599744776 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f42500d3bc0 0x7f42500b91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:52.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.899+0000 7f424ffff640 1 -- 172.21.15.47:0/2599744776 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f42500d3bc0 msgr2=0x7f42500b91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:52.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.899+0000 7f424ffff640 1 --2- 172.21.15.47:0/2599744776 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f42500d3bc0 0x7f42500b91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:52.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.899+0000 7f424ffff640 1 -- 172.21.15.47:0/2599744776 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f42500b5f20 msgr2=0x7f42500b8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:52.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.899+0000 7f424ffff640 1 --2- 172.21.15.47:0/2599744776 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f42500b5f20 0x7f42500b8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:52.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.899+0000 7f424ffff640 1 -- 172.21.15.47:0/2599744776 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42500b9a20 con 0x7f4250162880 2024-04-26T23:19:52.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.899+0000 7f424effd640 1 --2- 172.21.15.47:0/2599744776 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f42500d3bc0 0x7f42500b91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:19:52.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.899+0000 7f424ffff640 1 --2- 172.21.15.47:0/2599744776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4250162880 0x7f42500b3fe0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f424800efc0 tx=0x7f424800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:52.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.899+0000 7f4237fff640 1 -- 172.21.15.47:0/2599744776 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4248018030 con 0x7f4250162880 2024-04-26T23:19:52.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.899+0000 7f4237fff640 1 -- 172.21.15.47:0/2599744776 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f424800cd90 con 0x7f4250162880 2024-04-26T23:19:52.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.899+0000 7f4237fff640 1 -- 172.21.15.47:0/2599744776 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4248003bb0 con 0x7f4250162880 2024-04-26T23:19:52.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.899+0000 7f4255cd1640 1 -- 172.21.15.47:0/2599744776 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42500b9d00 con 0x7f4250162880 2024-04-26T23:19:52.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.899+0000 7f4255cd1640 1 -- 172.21.15.47:0/2599744776 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42500b9730 con 0x7f4250162880 2024-04-26T23:19:52.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.900+0000 7f4255cd1640 1 -- 172.21.15.47:0/2599744776 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42500b9ed0 con 0x7f4250162880 2024-04-26T23:19:52.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.901+0000 7f4237fff640 1 -- 172.21.15.47:0/2599744776 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f424801b070 con 0x7f4250162880 2024-04-26T23:19:52.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.902+0000 7f4237fff640 1 --2- 172.21.15.47:0/2599744776 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f423c06c8f0 0x7f423c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:52.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.902+0000 7f4237fff640 1 -- 172.21.15.47:0/2599744776 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6464+0+0 (secure 0 0 0) 0x7f424808c750 con 0x7f4250162880 2024-04-26T23:19:52.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.902+0000 7f424f7fe640 1 --2- 172.21.15.47:0/2599744776 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f423c06c8f0 0x7f423c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:52.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.902+0000 7f424f7fe640 1 --2- 172.21.15.47:0/2599744776 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f423c06c8f0 0x7f423c06edb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4240000f60 tx=0x7f4240007b50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:52.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:52.906+0000 7f4237fff640 1 -- 172.21.15.47:0/2599744776 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4248010040 con 0x7f4250162880 2024-04-26T23:19:53.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.133+0000 7f4255cd1640 1 -- 172.21.15.47:0/2599744776 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "data_new"} v 0) -- 0x7f42500694f0 con 0x7f4250162880 2024-04-26T23:19:53.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.780+0000 7f4237fff640 1 -- 172.21.15.47:0/2599744776 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "data_new"}]=0 added data pool 7 to fsmap v19) ==== 131+0+0 (secure 0 0 0) 0x7f424805a2d0 con 0x7f4250162880 2024-04-26T23:19:53.784 INFO:teuthology.orchestra.run.smithi047.stderr:added data pool 7 to fsmap 2024-04-26T23:19:53.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.786+0000 7f4255cd1640 1 -- 172.21.15.47:0/2599744776 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f423c06c8f0 msgr2=0x7f423c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:53.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.786+0000 7f4255cd1640 1 --2- 172.21.15.47:0/2599744776 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f423c06c8f0 0x7f423c06edb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4240000f60 tx=0x7f4240007b50 comp rx=0 tx=0).stop 2024-04-26T23:19:53.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.786+0000 7f4255cd1640 1 -- 172.21.15.47:0/2599744776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4250162880 msgr2=0x7f42500b3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:53.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.786+0000 7f4255cd1640 1 --2- 172.21.15.47:0/2599744776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4250162880 0x7f42500b3fe0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f424800efc0 tx=0x7f424800c490 comp rx=0 tx=0).stop 2024-04-26T23:19:53.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.786+0000 7f4255cd1640 1 -- 172.21.15.47:0/2599744776 shutdown_connections 2024-04-26T23:19:53.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.786+0000 7f4255cd1640 1 --2- 172.21.15.47:0/2599744776 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f423c06c8f0 0x7f423c06edb0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:53.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.786+0000 7f4255cd1640 1 --2- 172.21.15.47:0/2599744776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4250162880 0x7f42500b3fe0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:53.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.786+0000 7f4255cd1640 1 --2- 172.21.15.47:0/2599744776 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f42500d3bc0 0x7f42500b91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:53.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.786+0000 7f4255cd1640 1 --2- 172.21.15.47:0/2599744776 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f42500b5f20 0x7f42500b8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:53.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.786+0000 7f4255cd1640 1 -- 172.21.15.47:0/2599744776 >> 172.21.15.47:0/2599744776 conn(0x7f42500d4f60 msgr2=0x7f42500d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:53.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.787+0000 7f4255cd1640 1 -- 172.21.15.47:0/2599744776 shutdown_connections 2024-04-26T23:19:53.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.787+0000 7f4255cd1640 1 -- 172.21.15.47:0/2599744776 wait complete. 2024-04-26T23:19:53.802 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:19:53.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.906+0000 7f9d0b71b640 1 Processor -- start 2024-04-26T23:19:53.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.906+0000 7f9d0b71b640 1 -- start start 2024-04-26T23:19:53.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.907+0000 7f9d0b71b640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9d0406c240 0x7f9d0406c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:53.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.907+0000 7f9d0b71b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9d040aed50 0x7f9d040af1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:53.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.907+0000 7f9d0b71b640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d040b4480 con 0x7f9d0415f1d0 2024-04-26T23:19:53.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.907+0000 7f9d0b71b640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f9d0406cbe0 con 0x7f9d040aed50 2024-04-26T23:19:53.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.907+0000 7f9d0b71b640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f9d040ae8c0 con 0x7f9d0406c240 2024-04-26T23:19:53.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.907+0000 7f9d09cb8640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9d040aed50 0x7f9d040af1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:53.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.907+0000 7f9d09cb8640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9d040aed50 0x7f9d040af1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37972/0 (socket says 172.21.15.47:37972) 2024-04-26T23:19:53.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.907+0000 7f9d09cb8640 1 -- 172.21.15.47:0/4273984856 learned_addr learned my addr 172.21.15.47:0/4273984856 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:53.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.907+0000 7f9d08cb6640 1 --2- 172.21.15.47:0/4273984856 >> v2:172.21.15.203:3300/0 conn(0x7f9d0406c240 0x7f9d0406c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:53.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.907+0000 7f9d09cb8640 1 -- 172.21.15.47:0/4273984856 >> v2:172.21.15.203:3300/0 conn(0x7f9d0406c240 msgr2=0x7f9d0406c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:53.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.907+0000 7f9d09cb8640 1 --2- 172.21.15.47:0/4273984856 >> v2:172.21.15.203:3300/0 conn(0x7f9d0406c240 0x7f9d0406c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:53.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.907+0000 7f9d09cb8640 1 -- 172.21.15.47:0/4273984856 >> v1:172.21.15.203:6789/0 conn(0x7f9d0415f1d0 legacy=0x7f9d0406f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:53.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.907+0000 7f9d09cb8640 1 -- 172.21.15.47:0/4273984856 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d040b4720 con 0x7f9d040aed50 2024-04-26T23:19:53.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.908+0000 7f9d09cb8640 1 --2- 172.21.15.47:0/4273984856 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d040aed50 0x7f9d040af1e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f9d04074060 tx=0x7f9cfc02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15cbf401aad1e697 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:53.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.908+0000 7f9cfbfff640 1 -- 172.21.15.47:0/4273984856 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc036070 con 0x7f9d040aed50 2024-04-26T23:19:53.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.908+0000 7f9cfbfff640 1 -- 172.21.15.47:0/4273984856 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cfc031600 con 0x7f9d040aed50 2024-04-26T23:19:53.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.908+0000 7f9cfbfff640 1 -- 172.21.15.47:0/4273984856 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc0319d0 con 0x7f9d040aed50 2024-04-26T23:19:53.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.909+0000 7f9d0b71b640 1 -- 172.21.15.47:0/4273984856 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d040aed50 msgr2=0x7f9d040af1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:53.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.909+0000 7f9d0b71b640 1 --2- 172.21.15.47:0/4273984856 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d040aed50 0x7f9d040af1e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f9d04074060 tx=0x7f9cfc02a730 comp rx=0 tx=0).stop 2024-04-26T23:19:53.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.909+0000 7f9d0b71b640 1 -- 172.21.15.47:0/4273984856 shutdown_connections 2024-04-26T23:19:53.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.909+0000 7f9d0b71b640 1 --2- 172.21.15.47:0/4273984856 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d040aed50 0x7f9d040af1e0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:53.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.909+0000 7f9d0b71b640 1 --2- 172.21.15.47:0/4273984856 >> v2:172.21.15.203:3300/0 conn(0x7f9d0406c240 0x7f9d0406c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:53.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.909+0000 7f9d0b71b640 1 -- 172.21.15.47:0/4273984856 >> 172.21.15.47:0/4273984856 conn(0x7f9d040b3180 msgr2=0x7f9d040b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:53.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.909+0000 7f9d0b71b640 1 -- 172.21.15.47:0/4273984856 shutdown_connections 2024-04-26T23:19:53.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.909+0000 7f9d0b71b640 1 -- 172.21.15.47:0/4273984856 wait complete. 2024-04-26T23:19:53.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.910+0000 7f9d0b71b640 1 Processor -- start 2024-04-26T23:19:53.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.910+0000 7f9d0b71b640 1 -- start start 2024-04-26T23:19:53.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.911+0000 7f9d0b71b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d0406c240 0x7f9d040cfed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:53.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.911+0000 7f9d0b71b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9d040aed50 0x7f9d040c7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:53.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.911+0000 7f9d0b71b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d0415f1d0 0x7f9d0414be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:53.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.911+0000 7f9d0b71b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9d040b4fc0 con 0x7f9d0406c240 2024-04-26T23:19:53.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.911+0000 7f9d0b71b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9d040b4ce0 con 0x7f9d0415f1d0 2024-04-26T23:19:53.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.911+0000 7f9d0b71b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f9d04075ea0 con 0x7f9d040aed50 2024-04-26T23:19:53.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.911+0000 7f9d094b7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d0406c240 0x7f9d040cfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:53.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.911+0000 7f9d094b7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d0406c240 0x7f9d040cfed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37988/0 (socket says 172.21.15.47:37988) 2024-04-26T23:19:53.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.911+0000 7f9d094b7640 1 -- 172.21.15.47:0/1853094035 learned_addr learned my addr 172.21.15.47:0/1853094035 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:53.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.911+0000 7f9d094b7640 1 -- 172.21.15.47:0/1853094035 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9d040aed50 msgr2=0x7f9d040c7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:53.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.911+0000 7f9d094b7640 1 --2- 172.21.15.47:0/1853094035 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9d040aed50 0x7f9d040c7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:53.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.911+0000 7f9d094b7640 1 -- 172.21.15.47:0/1853094035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d0415f1d0 msgr2=0x7f9d0414be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:53.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.911+0000 7f9d09cb8640 1 --2- 172.21.15.47:0/1853094035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d0415f1d0 0x7f9d0414be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:53.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.911+0000 7f9d094b7640 1 --2- 172.21.15.47:0/1853094035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d0415f1d0 0x7f9d0414be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:53.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.911+0000 7f9d094b7640 1 -- 172.21.15.47:0/1853094035 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d0414de90 con 0x7f9d0406c240 2024-04-26T23:19:53.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.911+0000 7f9d094b7640 1 --2- 172.21.15.47:0/1853094035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d0406c240 0x7f9d040cfed0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f9d0000c400 tx=0x7f9d0000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:53.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.912+0000 7f9cf9ffb640 1 -- 172.21.15.47:0/1853094035 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d000052d0 con 0x7f9d0406c240 2024-04-26T23:19:53.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.912+0000 7f9cf9ffb640 1 -- 172.21.15.47:0/1853094035 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d00005470 con 0x7f9d0406c240 2024-04-26T23:19:53.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.912+0000 7f9d0b71b640 1 -- 172.21.15.47:0/1853094035 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d040d0410 con 0x7f9d0406c240 2024-04-26T23:19:53.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.912+0000 7f9cf9ffb640 1 -- 172.21.15.47:0/1853094035 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d00007260 con 0x7f9d0406c240 2024-04-26T23:19:53.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.912+0000 7f9d0b71b640 1 -- 172.21.15.47:0/1853094035 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d041f8c20 con 0x7f9d0406c240 2024-04-26T23:19:53.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.913+0000 7f9d0b71b640 1 -- 172.21.15.47:0/1853094035 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d040694f0 con 0x7f9d0406c240 2024-04-26T23:19:53.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.913+0000 7f9cf9ffb640 1 -- 172.21.15.47:0/1853094035 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9d00007400 con 0x7f9d0406c240 2024-04-26T23:19:53.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.913+0000 7f9cf9ffb640 1 --2- 172.21.15.47:0/1853094035 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9ce806c8f0 0x7f9ce806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:53.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.914+0000 7f9cf9ffb640 1 -- 172.21.15.47:0/1853094035 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6496+0+0 (secure 0 0 0) 0x7f9d0008b400 con 0x7f9d0406c240 2024-04-26T23:19:53.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.914+0000 7f9d08cb6640 1 --2- 172.21.15.47:0/1853094035 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9ce806c8f0 0x7f9ce806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:53.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.914+0000 7f9d08cb6640 1 --2- 172.21.15.47:0/1853094035 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9ce806c8f0 0x7f9ce806edb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4006fd0 tx=0x7f9cf4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:53.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:53.918+0000 7f9cf9ffb640 1 -- 172.21.15.47:0/1853094035 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9d0414de90 con 0x7f9d0406c240 2024-04-26T23:19:54.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.140+0000 7f9d0b71b640 1 -- 172.21.15.47:0/1853094035 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9d041629f0 con 0x7f9d0406c240 2024-04-26T23:19:54.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.140+0000 7f9cf9ffb640 1 -- 172.21.15.47:0/1853094035 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+5191 (secure 0 0 0) 0x7f9d00059520 con 0x7f9d0406c240 2024-04-26T23:19:54.143 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:19:54.143 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5080,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/1970124054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1970124054},{"type":"v1","addr":"172.21.15.203:6836","nonce":1970124054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5310,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/889573201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":889573201},{"type":"v1","addr":"172.21.15.47:6834","nonce":889573201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5316,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3898612760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":3898612760},{"type":"v1","addr":"172.21.15.47:6835","nonce":3898612760}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:19:44.091353+0000","modified":"2024-04-26T23:19:53.135935+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":14738},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14738":{"gid":14738,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/2526477051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":2526477051},{"type":"v1","addr":"172.21.15.203:6837","nonce":2526477051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":14738,"qdb_cluster":[14738]},"id":2}]} 2024-04-26T23:19:54.143 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 19 2024-04-26T23:19:54.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.142+0000 7f9d0b71b640 1 -- 172.21.15.47:0/1853094035 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9ce806c8f0 msgr2=0x7f9ce806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:54.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.142+0000 7f9d0b71b640 1 --2- 172.21.15.47:0/1853094035 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9ce806c8f0 0x7f9ce806edb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4006fd0 tx=0x7f9cf4008040 comp rx=0 tx=0).stop 2024-04-26T23:19:54.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.142+0000 7f9d0b71b640 1 -- 172.21.15.47:0/1853094035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d0406c240 msgr2=0x7f9d040cfed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:54.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.142+0000 7f9d0b71b640 1 --2- 172.21.15.47:0/1853094035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d0406c240 0x7f9d040cfed0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f9d0000c400 tx=0x7f9d0000c8c0 comp rx=0 tx=0).stop 2024-04-26T23:19:54.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.142+0000 7f9d0b71b640 1 -- 172.21.15.47:0/1853094035 shutdown_connections 2024-04-26T23:19:54.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.142+0000 7f9d0b71b640 1 --2- 172.21.15.47:0/1853094035 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9ce806c8f0 0x7f9ce806edb0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:54.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.142+0000 7f9d0b71b640 1 --2- 172.21.15.47:0/1853094035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d0415f1d0 0x7f9d0414be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:54.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.142+0000 7f9d0b71b640 1 --2- 172.21.15.47:0/1853094035 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9d040aed50 0x7f9d040c7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:54.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.142+0000 7f9d0b71b640 1 --2- 172.21.15.47:0/1853094035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d0406c240 0x7f9d040cfed0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:54.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.142+0000 7f9d0b71b640 1 -- 172.21.15.47:0/1853094035 >> 172.21.15.47:0/1853094035 conn(0x7f9d040b3180 msgr2=0x7f9d0406e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:54.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.142+0000 7f9d0b71b640 1 -- 172.21.15.47:0/1853094035 shutdown_connections 2024-04-26T23:19:54.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.142+0000 7f9d0b71b640 1 -- 172.21.15.47:0/1853094035 wait complete. 2024-04-26T23:19:54.155 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:19:54.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.254+0000 7f533f015640 1 Processor -- start 2024-04-26T23:19:54.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.254+0000 7f533f015640 1 -- start start 2024-04-26T23:19:54.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.254+0000 7f533f015640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f53380b5f20 0x7f53380d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:54.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.254+0000 7f533f015640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53380b7e50 con 0x7f53380d3bc0 2024-04-26T23:19:54.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.254+0000 7f533f015640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53380d1800 con 0x7f5338162880 2024-04-26T23:19:54.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.254+0000 7f533f015640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f53380d0580 con 0x7f53380b5f20 2024-04-26T23:19:54.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.255+0000 7f533d5b2640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f53380b5f20 0x7f53380d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:54.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.255+0000 7f5337fff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f53380d3bc0 0x7f53380b8300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44712/0 (socket says 172.21.15.47:44712) 2024-04-26T23:19:54.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.255+0000 7f5337fff640 1 -- 172.21.15.47:0/1840256776 learned_addr learned my addr 172.21.15.47:0/1840256776 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:54.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.255+0000 7f533d5b2640 1 -- 172.21.15.47:0/1840256776 >> v1:172.21.15.203:6789/0 conn(0x7f5338162880 legacy=0x7f53380d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:54.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.255+0000 7f533d5b2640 1 -- 172.21.15.47:0/1840256776 >> v1:172.21.15.47:6789/0 conn(0x7f53380d3bc0 legacy=0x7f53380b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:54.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.255+0000 7f533d5b2640 1 -- 172.21.15.47:0/1840256776 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53380d07d0 con 0x7f53380b5f20 2024-04-26T23:19:54.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.255+0000 7f53377fe640 1 -- 172.21.15.47:0/1840256776 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3743814247 0 0) 0x7f53380b7e50 con 0x7f53380d3bc0 2024-04-26T23:19:54.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.256+0000 7f533d5b2640 1 --2- 172.21.15.47:0/1840256776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53380b5f20 0x7f53380d2cb0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f5330009d90 tx=0x7f533002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0fac392da933229 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:54.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.256+0000 7f53377fe640 1 -- 172.21.15.47:0/1840256776 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5330007740 con 0x7f53380b5f20 2024-04-26T23:19:54.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.256+0000 7f53377fe640 1 -- 172.21.15.47:0/1840256776 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53300078e0 con 0x7f53380b5f20 2024-04-26T23:19:54.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.256+0000 7f53377fe640 1 -- 172.21.15.47:0/1840256776 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5330007cb0 con 0x7f53380b5f20 2024-04-26T23:19:54.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.256+0000 7f533f015640 1 -- 172.21.15.47:0/1840256776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53380b5f20 msgr2=0x7f53380d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:54.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.256+0000 7f533f015640 1 --2- 172.21.15.47:0/1840256776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53380b5f20 0x7f53380d2cb0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f5330009d90 tx=0x7f533002a6f0 comp rx=0 tx=0).stop 2024-04-26T23:19:54.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.256+0000 7f533f015640 1 -- 172.21.15.47:0/1840256776 shutdown_connections 2024-04-26T23:19:54.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.256+0000 7f533f015640 1 --2- 172.21.15.47:0/1840256776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53380b5f20 0x7f53380d2cb0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:54.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.256+0000 7f533f015640 1 -- 172.21.15.47:0/1840256776 >> 172.21.15.47:0/1840256776 conn(0x7f53380d4f60 msgr2=0x7f53380d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:54.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.256+0000 7f533f015640 1 -- 172.21.15.47:0/1840256776 shutdown_connections 2024-04-26T23:19:54.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.256+0000 7f533f015640 1 -- 172.21.15.47:0/1840256776 wait complete. 2024-04-26T23:19:54.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.258+0000 7f533f015640 1 Processor -- start 2024-04-26T23:19:54.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.258+0000 7f533f015640 1 -- start start 2024-04-26T23:19:54.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.258+0000 7f533f015640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53380b5f20 0x7f53380b2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:54.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.258+0000 7f533f015640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53380d3bc0 0x7f53381f55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:54.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.258+0000 7f533f015640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5338162880 0x7f53381f81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:54.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.258+0000 7f533f015640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5338162c60 con 0x7f53380b5f20 2024-04-26T23:19:54.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.258+0000 7f533f015640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f53380d0d90 con 0x7f5338162880 2024-04-26T23:19:54.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.258+0000 7f533f015640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f5338162de0 con 0x7f53380d3bc0 2024-04-26T23:19:54.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.259+0000 7f533cdb1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53380b5f20 0x7f53380b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:54.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.259+0000 7f533cdb1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53380b5f20 0x7f53380b2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38004/0 (socket says 172.21.15.47:38004) 2024-04-26T23:19:54.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.259+0000 7f533cdb1640 1 -- 172.21.15.47:0/3425279094 learned_addr learned my addr 172.21.15.47:0/3425279094 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:54.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.259+0000 7f533d5b2640 1 --2- 172.21.15.47:0/3425279094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5338162880 0x7f53381f81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:54.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.259+0000 7f5337fff640 1 --2- 172.21.15.47:0/3425279094 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53380d3bc0 0x7f53381f55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:54.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.259+0000 7f533cdb1640 1 -- 172.21.15.47:0/3425279094 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53380d3bc0 msgr2=0x7f53381f55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:54.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.259+0000 7f533cdb1640 1 --2- 172.21.15.47:0/3425279094 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53380d3bc0 0x7f53381f55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:54.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.259+0000 7f533cdb1640 1 -- 172.21.15.47:0/3425279094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5338162880 msgr2=0x7f53381f81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:54.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.259+0000 7f533cdb1640 1 --2- 172.21.15.47:0/3425279094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5338162880 0x7f53381f81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:54.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.259+0000 7f533cdb1640 1 -- 172.21.15.47:0/3425279094 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53381f4b80 con 0x7f53380b5f20 2024-04-26T23:19:54.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.259+0000 7f533d5b2640 1 --2- 172.21.15.47:0/3425279094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5338162880 0x7f53381f81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:19:54.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.259+0000 7f533cdb1640 1 --2- 172.21.15.47:0/3425279094 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53380b5f20 0x7f53380b2560 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f532c00c430 tx=0x7f532c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:54.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.259+0000 7f53357fa640 1 -- 172.21.15.47:0/3425279094 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532c017070 con 0x7f53380b5f20 2024-04-26T23:19:54.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.259+0000 7f53357fa640 1 -- 172.21.15.47:0/3425279094 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f532c0052d0 con 0x7f53380b5f20 2024-04-26T23:19:54.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.259+0000 7f533f015640 1 -- 172.21.15.47:0/3425279094 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53381f8c40 con 0x7f53380b5f20 2024-04-26T23:19:54.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.259+0000 7f53357fa640 1 -- 172.21.15.47:0/3425279094 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532c003830 con 0x7f53380b5f20 2024-04-26T23:19:54.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.259+0000 7f533f015640 1 -- 172.21.15.47:0/3425279094 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53381f4df0 con 0x7f53380b5f20 2024-04-26T23:19:54.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.260+0000 7f533f015640 1 -- 172.21.15.47:0/3425279094 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53380694f0 con 0x7f53380b5f20 2024-04-26T23:19:54.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.261+0000 7f53357fa640 1 -- 172.21.15.47:0/3425279094 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f532c01a070 con 0x7f53380b5f20 2024-04-26T23:19:54.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.261+0000 7f53357fa640 1 --2- 172.21.15.47:0/3425279094 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f531c06c8f0 0x7f531c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:54.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.261+0000 7f53357fa640 1 -- 172.21.15.47:0/3425279094 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6496+0+0 (secure 0 0 0) 0x7f532c08b110 con 0x7f53380b5f20 2024-04-26T23:19:54.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.261+0000 7f5337fff640 1 --2- 172.21.15.47:0/3425279094 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f531c06c8f0 0x7f531c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:54.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.262+0000 7f5337fff640 1 --2- 172.21.15.47:0/3425279094 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f531c06c8f0 0x7f531c06edb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5324007fd0 tx=0x7f5324009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:54.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.265+0000 7f53357fa640 1 -- 172.21.15.47:0/3425279094 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f53380d1800 con 0x7f53380b5f20 2024-04-26T23:19:54.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.450+0000 7f533f015640 1 -- 172.21.15.47:0/3425279094 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f53380cf390 con 0x7f53380b5f20 2024-04-26T23:19:54.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.451+0000 7f53357fa640 1 -- 172.21.15.47:0/3425279094 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+18731 (secure 0 0 0) 0x7f532c0592c0 con 0x7f53380b5f20 2024-04-26T23:19:54.454 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:19:54.454 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":40,"fsid":"399006d3-4db1-43e6-ba81-a79a276eabfd","created":"2024-04-26T23:18:06.731211+0000","modified":"2024-04-26T23:19:53.135719+0000","last_up_change":"2024-04-26T23:18:17.808736+0000","last_in_change":"2024-04-26T23:18:08.945859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:18:16.937492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:19:42.351747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-26T23:19:43.037763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2024-04-26T23:19:51.911275+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ee091567-fea0-4a14-bf4e-0e16c20bb2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6819","nonce":326520779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6821","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6823","nonce":326520779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6829","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6831","nonce":326520779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6825","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6827","nonce":326520779}]},"public_addr":"172.21.15.47:6819/326520779","cluster_addr":"172.21.15.47:6823/326520779","heartbeat_back_addr":"172.21.15.47:6831/326520779","heartbeat_front_addr":"172.21.15.47:6827/326520779","state":["exists","up"]},{"osd":1,"uuid":"c14440a0-bba5-4931-a6db-b736c08e869b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6809","nonce":1912610931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6811","nonce":1912610931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6815","nonce":1912610931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6813","nonce":1912610931}]},"public_addr":"172.21.15.47:6809/1912610931","cluster_addr":"172.21.15.47:6811/1912610931","heartbeat_back_addr":"172.21.15.47:6815/1912610931","heartbeat_front_addr":"172.21.15.47:6813/1912610931","state":["exists","up"]},{"osd":2,"uuid":"3940f8de-c9b2-4cd3-a911-9c301729eb30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6801","nonce":2569869305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6803","nonce":2569869305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6807","nonce":2569869305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6805","nonce":2569869305}]},"public_addr":"172.21.15.47:6801/2569869305","cluster_addr":"172.21.15.47:6803/2569869305","heartbeat_back_addr":"172.21.15.47:6807/2569869305","heartbeat_front_addr":"172.21.15.47:6805/2569869305","state":["exists","up"]},{"osd":3,"uuid":"1286a67e-8958-426a-8c2e-bc85498c619c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6818","nonce":2791411683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6822","nonce":2791411683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6830","nonce":2791411683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6826","nonce":2791411683}]},"public_addr":"172.21.15.47:6818/2791411683","cluster_addr":"172.21.15.47:6822/2791411683","heartbeat_back_addr":"172.21.15.47:6830/2791411683","heartbeat_front_addr":"172.21.15.47:6826/2791411683","state":["exists","up"]},{"osd":4,"uuid":"d8ee2c20-e5b7-46c2-9999-b234070126cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6809","nonce":441313104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6811","nonce":441313104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6815","nonce":441313104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6813","nonce":441313104}]},"public_addr":"172.21.15.203:6809/441313104","cluster_addr":"172.21.15.203:6811/441313104","heartbeat_back_addr":"172.21.15.203:6815/441313104","heartbeat_front_addr":"172.21.15.203:6813/441313104","state":["exists","up"]},{"osd":5,"uuid":"b453a8f0-f324-4840-9ae0-d67bdadb9f8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6801","nonce":56838444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6803","nonce":56838444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6807","nonce":56838444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6805","nonce":56838444}]},"public_addr":"172.21.15.203:6801/56838444","cluster_addr":"172.21.15.203:6803/56838444","heartbeat_back_addr":"172.21.15.203:6807/56838444","heartbeat_front_addr":"172.21.15.203:6805/56838444","state":["exists","up"]},{"osd":6,"uuid":"a6d31951-2d4b-4e8d-a510-6283b01dc158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6825","nonce":2738517080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6827","nonce":2738517080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6831","nonce":2738517080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6829","nonce":2738517080}]},"public_addr":"172.21.15.203:6825/2738517080","cluster_addr":"172.21.15.203:6827/2738517080","heartbeat_back_addr":"172.21.15.203:6831/2738517080","heartbeat_front_addr":"172.21.15.203:6829/2738517080","state":["exists","up"]},{"osd":7,"uuid":"7119b8c0-fe96-402d-9e75-7b509d9d44d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6817","nonce":1315565058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6819","nonce":1315565058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6823","nonce":1315565058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6821","nonce":1315565058}]},"public_addr":"172.21.15.203:6817/1315565058","cluster_addr":"172.21.15.203:6819/1315565058","heartbeat_back_addr":"172.21.15.203:6823/1315565058","heartbeat_front_addr":"172.21.15.203:6821/1315565058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.121921+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.137494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.084814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.111593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.072430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.125066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.097867+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.057061+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:19:54.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.454+0000 7f533f015640 1 -- 172.21.15.47:0/3425279094 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f531c06c8f0 msgr2=0x7f531c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:54.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.454+0000 7f533f015640 1 --2- 172.21.15.47:0/3425279094 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f531c06c8f0 0x7f531c06edb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5324007fd0 tx=0x7f5324009040 comp rx=0 tx=0).stop 2024-04-26T23:19:54.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.454+0000 7f533f015640 1 -- 172.21.15.47:0/3425279094 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53380b5f20 msgr2=0x7f53380b2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:54.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.454+0000 7f533f015640 1 --2- 172.21.15.47:0/3425279094 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53380b5f20 0x7f53380b2560 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f532c00c430 tx=0x7f532c00c8f0 comp rx=0 tx=0).stop 2024-04-26T23:19:54.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.454+0000 7f533f015640 1 -- 172.21.15.47:0/3425279094 shutdown_connections 2024-04-26T23:19:54.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.454+0000 7f533f015640 1 --2- 172.21.15.47:0/3425279094 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f531c06c8f0 0x7f531c06edb0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:54.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.454+0000 7f533f015640 1 --2- 172.21.15.47:0/3425279094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5338162880 0x7f53381f81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:54.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.454+0000 7f533f015640 1 --2- 172.21.15.47:0/3425279094 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53380d3bc0 0x7f53381f55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:54.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.454+0000 7f533f015640 1 --2- 172.21.15.47:0/3425279094 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53380b5f20 0x7f53380b2560 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:54.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.454+0000 7f533f015640 1 -- 172.21.15.47:0/3425279094 >> 172.21.15.47:0/3425279094 conn(0x7f53380d4f60 msgr2=0x7f53380d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:54.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.454+0000 7f533f015640 1 -- 172.21.15.47:0/3425279094 shutdown_connections 2024-04-26T23:19:54.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.454+0000 7f533f015640 1 -- 172.21.15.47:0/3425279094 wait complete. 2024-04-26T23:19:54.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:54.467 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch layoutfile) 2024-04-26T23:19:54.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:54.488 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir layoutdir) 2024-04-26T23:19:54.545 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds 'allow rw' mon 'allow r' osd 'allow rw pool=cephfs_data,allow rw pool=data_new' 2024-04-26T23:19:54.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.692+0000 7f8f555a4640 1 Processor -- start 2024-04-26T23:19:54.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.692+0000 7f8f555a4640 1 -- start start 2024-04-26T23:19:54.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.692+0000 7f8f555a4640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8f5015f550 0x7f8f500d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:54.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.692+0000 7f8f555a4640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f500d1040 con 0x7f8f500d3bc0 2024-04-26T23:19:54.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.692+0000 7f8f555a4640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f500c9680 con 0x7f8f5015f170 2024-04-26T23:19:54.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.692+0000 7f8f555a4640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f8f500c9430 con 0x7f8f5015f550 2024-04-26T23:19:54.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.693+0000 7f8f47fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8f5015f550 0x7f8f500d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:54.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.693+0000 7f8f4f7fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f8f500d3bc0 0x7f8f500ca440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44714/0 (socket says 172.21.15.47:44714) 2024-04-26T23:19:54.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.693+0000 7f8f4f7fe640 1 -- 172.21.15.47:0/1292477915 learned_addr learned my addr 172.21.15.47:0/1292477915 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:54.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.693+0000 7f8f47fff640 1 -- 172.21.15.47:0/1292477915 >> v1:172.21.15.203:6789/0 conn(0x7f8f5015f170 legacy=0x7f8f500d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:54.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.693+0000 7f8f47fff640 1 -- 172.21.15.47:0/1292477915 >> v1:172.21.15.47:6789/0 conn(0x7f8f500d3bc0 legacy=0x7f8f500ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:54.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.693+0000 7f8f47fff640 1 -- 172.21.15.47:0/1292477915 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f500c9960 con 0x7f8f5015f550 2024-04-26T23:19:54.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.694+0000 7f8f47fff640 1 --2- 172.21.15.47:0/1292477915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f5015f550 0x7f8f500d5a80 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8f500c95b0 tx=0x7f8f3802a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa7a2864c9fdd69b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:54.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.694+0000 7f8f4e7fc640 1 -- 172.21.15.47:0/1292477915 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f38037070 con 0x7f8f5015f550 2024-04-26T23:19:54.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.694+0000 7f8f4e7fc640 1 -- 172.21.15.47:0/1292477915 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f38032440 con 0x7f8f5015f550 2024-04-26T23:19:54.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.694+0000 7f8f4e7fc640 1 -- 172.21.15.47:0/1292477915 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f38032810 con 0x7f8f5015f550 2024-04-26T23:19:54.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.694+0000 7f8f555a4640 1 -- 172.21.15.47:0/1292477915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f5015f550 msgr2=0x7f8f500d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:54.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.694+0000 7f8f555a4640 1 --2- 172.21.15.47:0/1292477915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f5015f550 0x7f8f500d5a80 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8f500c95b0 tx=0x7f8f3802a200 comp rx=0 tx=0).stop 2024-04-26T23:19:54.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.694+0000 7f8f555a4640 1 -- 172.21.15.47:0/1292477915 shutdown_connections 2024-04-26T23:19:54.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.694+0000 7f8f555a4640 1 --2- 172.21.15.47:0/1292477915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f5015f550 0x7f8f500d5a80 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:54.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.694+0000 7f8f555a4640 1 -- 172.21.15.47:0/1292477915 >> 172.21.15.47:0/1292477915 conn(0x7f8f500d4f60 msgr2=0x7f8f500d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:54.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.694+0000 7f8f555a4640 1 -- 172.21.15.47:0/1292477915 shutdown_connections 2024-04-26T23:19:54.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.694+0000 7f8f555a4640 1 -- 172.21.15.47:0/1292477915 wait complete. 2024-04-26T23:19:54.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.696+0000 7f8f555a4640 1 Processor -- start 2024-04-26T23:19:54.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.696+0000 7f8f555a4640 1 -- start start 2024-04-26T23:19:54.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.696+0000 7f8f555a4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f500d3bc0 0x7f8f501f1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:54.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.696+0000 7f8f555a4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f5015f170 0x7f8f500b7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:54.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.696+0000 7f8f555a4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8f501f4a30 0x7f8f500ba0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:54.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.696+0000 7f8f555a4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8f500d2e30 con 0x7f8f500d3bc0 2024-04-26T23:19:54.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.696+0000 7f8f555a4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8f500d2cb0 con 0x7f8f5015f170 2024-04-26T23:19:54.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.696+0000 7f8f555a4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f8f500d2fb0 con 0x7f8f501f4a30 2024-04-26T23:19:54.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.697+0000 7f8f4effd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f500d3bc0 0x7f8f501f1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:54.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.697+0000 7f8f4effd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f500d3bc0 0x7f8f501f1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38024/0 (socket says 172.21.15.47:38024) 2024-04-26T23:19:54.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.697+0000 7f8f4effd640 1 -- 172.21.15.47:0/2419620942 learned_addr learned my addr 172.21.15.47:0/2419620942 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:54.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.697+0000 7f8f4f7fe640 1 --2- 172.21.15.47:0/2419620942 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8f501f4a30 0x7f8f500ba0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:54.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.697+0000 7f8f47fff640 1 --2- 172.21.15.47:0/2419620942 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f5015f170 0x7f8f500b7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:54.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.697+0000 7f8f4effd640 1 -- 172.21.15.47:0/2419620942 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8f501f4a30 msgr2=0x7f8f500ba0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:54.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.697+0000 7f8f4effd640 1 --2- 172.21.15.47:0/2419620942 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8f501f4a30 0x7f8f500ba0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:54.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.697+0000 7f8f4effd640 1 -- 172.21.15.47:0/2419620942 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f5015f170 msgr2=0x7f8f500b7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:54.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.697+0000 7f8f4effd640 1 --2- 172.21.15.47:0/2419620942 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f5015f170 0x7f8f500b7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:54.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.697+0000 7f8f4effd640 1 -- 172.21.15.47:0/2419620942 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f500b84d0 con 0x7f8f500d3bc0 2024-04-26T23:19:54.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.697+0000 7f8f4effd640 1 --2- 172.21.15.47:0/2419620942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f500d3bc0 0x7f8f501f1d50 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8f4000c430 tx=0x7f8f4000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:54.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.697+0000 7f8f477fe640 1 -- 172.21.15.47:0/2419620942 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f40011790 con 0x7f8f500d3bc0 2024-04-26T23:19:54.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.697+0000 7f8f477fe640 1 -- 172.21.15.47:0/2419620942 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f40017070 con 0x7f8f500d3bc0 2024-04-26T23:19:54.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.697+0000 7f8f555a4640 1 -- 172.21.15.47:0/2419620942 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f500b87b0 con 0x7f8f500d3bc0 2024-04-26T23:19:54.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.697+0000 7f8f477fe640 1 -- 172.21.15.47:0/2419620942 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f40011bb0 con 0x7f8f500d3bc0 2024-04-26T23:19:54.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.697+0000 7f8f555a4640 1 -- 172.21.15.47:0/2419620942 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f500b8d70 con 0x7f8f500d3bc0 2024-04-26T23:19:54.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.698+0000 7f8f555a4640 1 -- 172.21.15.47:0/2419620942 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f500694f0 con 0x7f8f500d3bc0 2024-04-26T23:19:54.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.699+0000 7f8f477fe640 1 -- 172.21.15.47:0/2419620942 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8f4001c460 con 0x7f8f500d3bc0 2024-04-26T23:19:54.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.699+0000 7f8f477fe640 1 --2- 172.21.15.47:0/2419620942 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f8f3006c8f0 0x7f8f3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:54.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.699+0000 7f8f477fe640 1 -- 172.21.15.47:0/2419620942 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6496+0+0 (secure 0 0 0) 0x7f8f4008bbd0 con 0x7f8f500d3bc0 2024-04-26T23:19:54.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.699+0000 7f8f47fff640 1 --2- 172.21.15.47:0/2419620942 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f8f3006c8f0 0x7f8f3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:54.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.700+0000 7f8f47fff640 1 --2- 172.21.15.47:0/2419620942 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f8f3006c8f0 0x7f8f3006edb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8f38034000 tx=0x7f8f3802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:54.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.703+0000 7f8f477fe640 1 -- 172.21.15.47:0/2419620942 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8f400596a0 con 0x7f8f500d3bc0 2024-04-26T23:19:54.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.963+0000 7f8f555a4640 1 -- 172.21.15.47:0/2419620942 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rw", "mon", "allow r", "osd", "allow rw pool=cephfs_data,allow rw pool=data_new"]} v 0) -- 0x7f8f50074590 con 0x7f8f500d3bc0 2024-04-26T23:19:54.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.965+0000 7f8f477fe640 1 -- 172.21.15.47:0/2419620942 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rw", "mon", "allow r", "osd", "allow rw pool=cephfs_data,allow rw pool=data_new"]}]=0 updated caps for client.0 v11) ==== 210+0+167 (secure 0 0 0) 0x7f8f4005d9a0 con 0x7f8f500d3bc0 2024-04-26T23:19:54.967 INFO:teuthology.orchestra.run.smithi047.stdout:[client.0] 2024-04-26T23:19:54.968 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQAKNixmA5QFHRAAT89WPyzDs0acPluyvU4svw== 2024-04-26T23:19:54.968 INFO:teuthology.orchestra.run.smithi047.stdout: caps mds = "allow rw" 2024-04-26T23:19:54.968 INFO:teuthology.orchestra.run.smithi047.stdout: caps mon = "allow r" 2024-04-26T23:19:54.968 INFO:teuthology.orchestra.run.smithi047.stdout: caps osd = "allow rw pool=cephfs_data,allow rw pool=data_new" 2024-04-26T23:19:54.968 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.0 2024-04-26T23:19:54.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.967+0000 7f8f555a4640 1 -- 172.21.15.47:0/2419620942 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f8f3006c8f0 msgr2=0x7f8f3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:54.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.967+0000 7f8f555a4640 1 --2- 172.21.15.47:0/2419620942 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f8f3006c8f0 0x7f8f3006edb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8f38034000 tx=0x7f8f3802e000 comp rx=0 tx=0).stop 2024-04-26T23:19:54.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.967+0000 7f8f555a4640 1 -- 172.21.15.47:0/2419620942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f500d3bc0 msgr2=0x7f8f501f1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:54.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.967+0000 7f8f555a4640 1 --2- 172.21.15.47:0/2419620942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f500d3bc0 0x7f8f501f1d50 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8f4000c430 tx=0x7f8f4000c8f0 comp rx=0 tx=0).stop 2024-04-26T23:19:54.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.967+0000 7f8f555a4640 1 -- 172.21.15.47:0/2419620942 shutdown_connections 2024-04-26T23:19:54.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.967+0000 7f8f555a4640 1 --2- 172.21.15.47:0/2419620942 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f8f3006c8f0 0x7f8f3006edb0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:54.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.967+0000 7f8f555a4640 1 --2- 172.21.15.47:0/2419620942 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8f501f4a30 0x7f8f500ba0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:54.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.967+0000 7f8f555a4640 1 --2- 172.21.15.47:0/2419620942 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f5015f170 0x7f8f500b7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:54.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.967+0000 7f8f555a4640 1 --2- 172.21.15.47:0/2419620942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f500d3bc0 0x7f8f501f1d50 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:54.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.967+0000 7f8f555a4640 1 -- 172.21.15.47:0/2419620942 >> 172.21.15.47:0/2419620942 conn(0x7f8f500d4f60 msgr2=0x7f8f500d1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:54.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.967+0000 7f8f555a4640 1 -- 172.21.15.47:0/2419620942 shutdown_connections 2024-04-26T23:19:54.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:54.967+0000 7f8f555a4640 1 -- 172.21.15.47:0/2419620942 wait complete. 2024-04-26T23:19:54.981 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:19:54.981 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:19:55.010 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:19:55.010 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:19:55.076 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:19:55.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.215+0000 7f35f061d640 1 Processor -- start 2024-04-26T23:19:55.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.215+0000 7f35f061d640 1 -- start start 2024-04-26T23:19:55.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.216+0000 7f35f061d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f35e80b2340 0x7f35e80cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:55.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.216+0000 7f35f061d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f35e80cd930 0x7f35e80d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:55.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.216+0000 7f35f061d640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35e80d10d0 con 0x7f35e8162810 2024-04-26T23:19:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.216+0000 7f35f061d640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f35e80d0750 con 0x7f35e80b2340 2024-04-26T23:19:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.216+0000 7f35f061d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f35e80d16a0 con 0x7f35e80cd930 2024-04-26T23:19:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.216+0000 7f35edbb8640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f35e80b2340 0x7f35e80cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.216+0000 7f35ee3b9640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f35e8162810 0x7f35e8070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44716/0 (socket says 172.21.15.47:44716) 2024-04-26T23:19:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.216+0000 7f35ee3b9640 1 -- 172.21.15.47:0/2162991258 learned_addr learned my addr 172.21.15.47:0/2162991258 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.216+0000 7f35eebba640 1 --2- 172.21.15.47:0/2162991258 >> v2:172.21.15.203:3300/0 conn(0x7f35e80cd930 0x7f35e80d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.216+0000 7f35ed3b7640 1 -- 172.21.15.47:0/2162991258 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3692790963 0 0) 0x7f35e80d10d0 con 0x7f35e8162810 2024-04-26T23:19:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.216+0000 7f35ed3b7640 1 -- 172.21.15.47:0/2162991258 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f35d0003520 con 0x7f35e8162810 2024-04-26T23:19:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.217+0000 7f35edbb8640 1 -- 172.21.15.47:0/2162991258 >> v2:172.21.15.203:3300/0 conn(0x7f35e80cd930 msgr2=0x7f35e80d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.217+0000 7f35edbb8640 1 --2- 172.21.15.47:0/2162991258 >> v2:172.21.15.203:3300/0 conn(0x7f35e80cd930 0x7f35e80d1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.217+0000 7f35edbb8640 1 -- 172.21.15.47:0/2162991258 >> v1:172.21.15.47:6789/0 conn(0x7f35e8162810 legacy=0x7f35e8070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.217+0000 7f35edbb8640 1 -- 172.21.15.47:0/2162991258 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35e80d1370 con 0x7f35e80b2340 2024-04-26T23:19:55.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.217+0000 7f35edbb8640 1 --2- 172.21.15.47:0/2162991258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35e80b2340 0x7f35e80cd3f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f35e80b2280 tx=0x7f35dc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88cbe893e2f74ea5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:55.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.217+0000 7f35ed3b7640 1 -- 172.21.15.47:0/2162991258 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35dc037070 con 0x7f35e80b2340 2024-04-26T23:19:55.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.217+0000 7f35ed3b7640 1 -- 172.21.15.47:0/2162991258 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35dc032440 con 0x7f35e80b2340 2024-04-26T23:19:55.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.217+0000 7f35ed3b7640 1 -- 172.21.15.47:0/2162991258 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35dc0327f0 con 0x7f35e80b2340 2024-04-26T23:19:55.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.218+0000 7f35f061d640 1 -- 172.21.15.47:0/2162991258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35e80b2340 msgr2=0x7f35e80cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:55.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.218+0000 7f35f061d640 1 --2- 172.21.15.47:0/2162991258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35e80b2340 0x7f35e80cd3f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f35e80b2280 tx=0x7f35dc02a1a0 comp rx=0 tx=0).stop 2024-04-26T23:19:55.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.218+0000 7f35f061d640 1 -- 172.21.15.47:0/2162991258 shutdown_connections 2024-04-26T23:19:55.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.218+0000 7f35f061d640 1 --2- 172.21.15.47:0/2162991258 >> v2:172.21.15.203:3300/0 conn(0x7f35e80cd930 0x7f35e80d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:55.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.218+0000 7f35f061d640 1 --2- 172.21.15.47:0/2162991258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35e80b2340 0x7f35e80cd3f0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:55.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.218+0000 7f35f061d640 1 -- 172.21.15.47:0/2162991258 >> 172.21.15.47:0/2162991258 conn(0x7f35e80d2da0 msgr2=0x7f35e80d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:55.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.218+0000 7f35f061d640 1 -- 172.21.15.47:0/2162991258 shutdown_connections 2024-04-26T23:19:55.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.218+0000 7f35f061d640 1 -- 172.21.15.47:0/2162991258 wait complete. 2024-04-26T23:19:55.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.219+0000 7f35f061d640 1 Processor -- start 2024-04-26T23:19:55.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.219+0000 7f35f061d640 1 -- start start 2024-04-26T23:19:55.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.220+0000 7f35f061d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f35e80b2340 0x7f35e80c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.220+0000 7f35f061d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35e80cd930 0x7f35e81493c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.220+0000 7f35f061d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35e8162810 0x7f35e80cf410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.220+0000 7f35f061d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f35e80d2070 con 0x7f35e80cd930 2024-04-26T23:19:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.220+0000 7f35f061d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f35e80d0470 con 0x7f35e80b2340 2024-04-26T23:19:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.220+0000 7f35f061d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f35e80d21f0 con 0x7f35e8162810 2024-04-26T23:19:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.220+0000 7f35edbb8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35e80cd930 0x7f35e81493c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.220+0000 7f35edbb8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35e80cd930 0x7f35e81493c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38042/0 (socket says 172.21.15.47:38042) 2024-04-26T23:19:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.220+0000 7f35edbb8640 1 -- 172.21.15.47:0/2159934962 learned_addr learned my addr 172.21.15.47:0/2159934962 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.220+0000 7f35ee3b9640 1 --2- 172.21.15.47:0/2159934962 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f35e80b2340 0x7f35e80c9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.220+0000 7f35eebba640 1 --2- 172.21.15.47:0/2159934962 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35e8162810 0x7f35e80cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.220+0000 7f35edbb8640 1 -- 172.21.15.47:0/2159934962 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35e8162810 msgr2=0x7f35e80cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.220+0000 7f35edbb8640 1 --2- 172.21.15.47:0/2159934962 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35e8162810 0x7f35e80cf410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.220+0000 7f35edbb8640 1 -- 172.21.15.47:0/2159934962 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f35e80b2340 msgr2=0x7f35e80c9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.220+0000 7f35edbb8640 1 --2- 172.21.15.47:0/2159934962 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f35e80b2340 0x7f35e80c9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:55.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.220+0000 7f35edbb8640 1 -- 172.21.15.47:0/2159934962 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35e80ca3b0 con 0x7f35e80cd930 2024-04-26T23:19:55.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.220+0000 7f35eebba640 1 --2- 172.21.15.47:0/2159934962 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35e8162810 0x7f35e80cf410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:19:55.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.221+0000 7f35edbb8640 1 --2- 172.21.15.47:0/2159934962 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35e80cd930 0x7f35e81493c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f35dc02a1e0 tx=0x7f35dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:55.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.221+0000 7f35e6ffd640 1 -- 172.21.15.47:0/2159934962 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35dc041030 con 0x7f35e80cd930 2024-04-26T23:19:55.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.221+0000 7f35e6ffd640 1 -- 172.21.15.47:0/2159934962 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35dc03d420 con 0x7f35e80cd930 2024-04-26T23:19:55.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.221+0000 7f35f061d640 1 -- 172.21.15.47:0/2159934962 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35e814bf40 con 0x7f35e80cd930 2024-04-26T23:19:55.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.221+0000 7f35e6ffd640 1 -- 172.21.15.47:0/2159934962 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35dc037040 con 0x7f35e80cd930 2024-04-26T23:19:55.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.221+0000 7f35f061d640 1 -- 172.21.15.47:0/2159934962 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35e814c500 con 0x7f35e80cd930 2024-04-26T23:19:55.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.222+0000 7f35f061d640 1 -- 172.21.15.47:0/2159934962 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35e80694f0 con 0x7f35e80cd930 2024-04-26T23:19:55.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.222+0000 7f35e6ffd640 1 -- 172.21.15.47:0/2159934962 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f35dc044070 con 0x7f35e80cd930 2024-04-26T23:19:55.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.223+0000 7f35e6ffd640 1 --2- 172.21.15.47:0/2159934962 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f35d406c8f0 0x7f35d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:55.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.223+0000 7f35e6ffd640 1 -- 172.21.15.47:0/2159934962 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f35dc0ac440 con 0x7f35e80cd930 2024-04-26T23:19:55.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.223+0000 7f35ee3b9640 1 --2- 172.21.15.47:0/2159934962 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f35d406c8f0 0x7f35d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:55.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.223+0000 7f35ee3b9640 1 --2- 172.21.15.47:0/2159934962 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f35d406c8f0 0x7f35d406edb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f35e0005880 tx=0x7f35e0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:55.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.227+0000 7f35e6ffd640 1 -- 172.21.15.47:0/2159934962 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f35dc07a570 con 0x7f35e80cd930 2024-04-26T23:19:55.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.415+0000 7f35f061d640 1 -- 172.21.15.47:0/2159934962 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f35e80b1f30 con 0x7f35e80cd930 2024-04-26T23:19:55.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.417+0000 7f35e6ffd640 1 -- 172.21.15.47:0/2159934962 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+18718 (secure 0 0 0) 0x7f35dc079e70 con 0x7f35e80cd930 2024-04-26T23:19:55.419 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:19:55.419 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":41,"fsid":"399006d3-4db1-43e6-ba81-a79a276eabfd","created":"2024-04-26T23:18:06.731211+0000","modified":"2024-04-26T23:19:54.781247+0000","last_up_change":"2024-04-26T23:18:17.808736+0000","last_in_change":"2024-04-26T23:18:08.945859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:18:16.937492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:19:42.351747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-26T23:19:43.037763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2024-04-26T23:19:51.911275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ee091567-fea0-4a14-bf4e-0e16c20bb2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6819","nonce":326520779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6821","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6823","nonce":326520779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6829","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6831","nonce":326520779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6825","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6827","nonce":326520779}]},"public_addr":"172.21.15.47:6819/326520779","cluster_addr":"172.21.15.47:6823/326520779","heartbeat_back_addr":"172.21.15.47:6831/326520779","heartbeat_front_addr":"172.21.15.47:6827/326520779","state":["exists","up"]},{"osd":1,"uuid":"c14440a0-bba5-4931-a6db-b736c08e869b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6809","nonce":1912610931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6811","nonce":1912610931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6815","nonce":1912610931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6813","nonce":1912610931}]},"public_addr":"172.21.15.47:6809/1912610931","cluster_addr":"172.21.15.47:6811/1912610931","heartbeat_back_addr":"172.21.15.47:6815/1912610931","heartbeat_front_addr":"172.21.15.47:6813/1912610931","state":["exists","up"]},{"osd":2,"uuid":"3940f8de-c9b2-4cd3-a911-9c301729eb30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6801","nonce":2569869305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6803","nonce":2569869305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6807","nonce":2569869305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6805","nonce":2569869305}]},"public_addr":"172.21.15.47:6801/2569869305","cluster_addr":"172.21.15.47:6803/2569869305","heartbeat_back_addr":"172.21.15.47:6807/2569869305","heartbeat_front_addr":"172.21.15.47:6805/2569869305","state":["exists","up"]},{"osd":3,"uuid":"1286a67e-8958-426a-8c2e-bc85498c619c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6818","nonce":2791411683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6822","nonce":2791411683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6830","nonce":2791411683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6826","nonce":2791411683}]},"public_addr":"172.21.15.47:6818/2791411683","cluster_addr":"172.21.15.47:6822/2791411683","heartbeat_back_addr":"172.21.15.47:6830/2791411683","heartbeat_front_addr":"172.21.15.47:6826/2791411683","state":["exists","up"]},{"osd":4,"uuid":"d8ee2c20-e5b7-46c2-9999-b234070126cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6809","nonce":441313104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6811","nonce":441313104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6815","nonce":441313104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6813","nonce":441313104}]},"public_addr":"172.21.15.203:6809/441313104","cluster_addr":"172.21.15.203:6811/441313104","heartbeat_back_addr":"172.21.15.203:6815/441313104","heartbeat_front_addr":"172.21.15.203:6813/441313104","state":["exists","up"]},{"osd":5,"uuid":"b453a8f0-f324-4840-9ae0-d67bdadb9f8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6801","nonce":56838444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6803","nonce":56838444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6807","nonce":56838444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6805","nonce":56838444}]},"public_addr":"172.21.15.203:6801/56838444","cluster_addr":"172.21.15.203:6803/56838444","heartbeat_back_addr":"172.21.15.203:6807/56838444","heartbeat_front_addr":"172.21.15.203:6805/56838444","state":["exists","up"]},{"osd":6,"uuid":"a6d31951-2d4b-4e8d-a510-6283b01dc158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6825","nonce":2738517080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6827","nonce":2738517080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6831","nonce":2738517080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6829","nonce":2738517080}]},"public_addr":"172.21.15.203:6825/2738517080","cluster_addr":"172.21.15.203:6827/2738517080","heartbeat_back_addr":"172.21.15.203:6831/2738517080","heartbeat_front_addr":"172.21.15.203:6829/2738517080","state":["exists","up"]},{"osd":7,"uuid":"7119b8c0-fe96-402d-9e75-7b509d9d44d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6817","nonce":1315565058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6819","nonce":1315565058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6823","nonce":1315565058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6821","nonce":1315565058}]},"public_addr":"172.21.15.203:6817/1315565058","cluster_addr":"172.21.15.203:6819/1315565058","heartbeat_back_addr":"172.21.15.203:6823/1315565058","heartbeat_front_addr":"172.21.15.203:6821/1315565058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.121921+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.137494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.084814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.111593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.072430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.125066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.097867+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.057061+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:19:55.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.418+0000 7f35f061d640 1 -- 172.21.15.47:0/2159934962 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f35d406c8f0 msgr2=0x7f35d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:55.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.418+0000 7f35f061d640 1 --2- 172.21.15.47:0/2159934962 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f35d406c8f0 0x7f35d406edb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f35e0005880 tx=0x7f35e0009040 comp rx=0 tx=0).stop 2024-04-26T23:19:55.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.418+0000 7f35f061d640 1 -- 172.21.15.47:0/2159934962 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35e80cd930 msgr2=0x7f35e81493c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:55.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.418+0000 7f35f061d640 1 --2- 172.21.15.47:0/2159934962 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35e80cd930 0x7f35e81493c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f35dc02a1e0 tx=0x7f35dc0046c0 comp rx=0 tx=0).stop 2024-04-26T23:19:55.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.418+0000 7f35f061d640 1 -- 172.21.15.47:0/2159934962 shutdown_connections 2024-04-26T23:19:55.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.418+0000 7f35f061d640 1 --2- 172.21.15.47:0/2159934962 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f35d406c8f0 0x7f35d406edb0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:55.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.418+0000 7f35f061d640 1 --2- 172.21.15.47:0/2159934962 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35e8162810 0x7f35e80cf410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:55.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.418+0000 7f35f061d640 1 --2- 172.21.15.47:0/2159934962 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35e80cd930 0x7f35e81493c0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:55.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.418+0000 7f35f061d640 1 --2- 172.21.15.47:0/2159934962 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f35e80b2340 0x7f35e80c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:55.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.418+0000 7f35f061d640 1 -- 172.21.15.47:0/2159934962 >> 172.21.15.47:0/2159934962 conn(0x7f35e80d2da0 msgr2=0x7f35e80b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:55.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.419+0000 7f35f061d640 1 -- 172.21.15.47:0/2159934962 shutdown_connections 2024-04-26T23:19:55.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.419+0000 7f35f061d640 1 -- 172.21.15.47:0/2159934962 wait complete. 2024-04-26T23:19:55.432 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:18:16.937492+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:19:42.351747+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:19:43.037763+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'data_new', 'create_time': '2024-04-26T23:19:51.911275+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:19:55.432 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:19:55.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.536+0000 7f6199007640 1 Processor -- start 2024-04-26T23:19:55.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.536+0000 7f6199007640 1 -- start start 2024-04-26T23:19:55.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.537+0000 7f6199007640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f619415f170 0x7f61940d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:55.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.537+0000 7f6199007640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f61940d3bc0 0x7f61940d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:55.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.537+0000 7f6199007640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61940d1040 con 0x7f619415f550 2024-04-26T23:19:55.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.537+0000 7f6199007640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f61940c9430 con 0x7f61940d3bc0 2024-04-26T23:19:55.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.537+0000 7f6199007640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f61940c9680 con 0x7f619415f170 2024-04-26T23:19:55.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.537+0000 7f619359e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f61940d3bc0 0x7f61940d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:55.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.537+0000 7f619359e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f61940d3bc0 0x7f61940d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38048/0 (socket says 172.21.15.47:38048) 2024-04-26T23:19:55.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.537+0000 7f619359e640 1 -- 172.21.15.47:0/224055198 learned_addr learned my addr 172.21.15.47:0/224055198 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:55.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.537+0000 7f6192d9d640 1 --2- 172.21.15.47:0/224055198 >> v2:172.21.15.203:3300/0 conn(0x7f619415f170 0x7f61940d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:55.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.538+0000 7f6191d9b640 1 -- 172.21.15.47:0/224055198 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1546517720 0 0) 0x7f6188001110 con 0x7f619415f550 2024-04-26T23:19:55.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.538+0000 7f619359e640 1 -- 172.21.15.47:0/224055198 >> v2:172.21.15.203:3300/0 conn(0x7f619415f170 msgr2=0x7f61940d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:55.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.538+0000 7f619359e640 1 --2- 172.21.15.47:0/224055198 >> v2:172.21.15.203:3300/0 conn(0x7f619415f170 0x7f61940d5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:55.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.538+0000 7f619359e640 1 -- 172.21.15.47:0/224055198 >> v1:172.21.15.47:6789/0 conn(0x7f619415f550 legacy=0x7f61940d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:55.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.538+0000 7f619359e640 1 -- 172.21.15.47:0/224055198 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61940c9960 con 0x7f61940d3bc0 2024-04-26T23:19:55.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.538+0000 7f619359e640 1 --2- 172.21.15.47:0/224055198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61940d3bc0 0x7f61940d2cb0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f6184009e50 tx=0x7f618402a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b6128f5a20d1d71 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:55.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.539+0000 7f6191d9b640 1 -- 172.21.15.47:0/224055198 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6184036070 con 0x7f61940d3bc0 2024-04-26T23:19:55.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.539+0000 7f6191d9b640 1 -- 172.21.15.47:0/224055198 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6184031600 con 0x7f61940d3bc0 2024-04-26T23:19:55.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.539+0000 7f6191d9b640 1 -- 172.21.15.47:0/224055198 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61840319d0 con 0x7f61940d3bc0 2024-04-26T23:19:55.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.539+0000 7f6199007640 1 -- 172.21.15.47:0/224055198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61940d3bc0 msgr2=0x7f61940d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:55.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.539+0000 7f6199007640 1 --2- 172.21.15.47:0/224055198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61940d3bc0 0x7f61940d2cb0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f6184009e50 tx=0x7f618402a760 comp rx=0 tx=0).stop 2024-04-26T23:19:55.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.540+0000 7f6199007640 1 -- 172.21.15.47:0/224055198 shutdown_connections 2024-04-26T23:19:55.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.540+0000 7f6199007640 1 --2- 172.21.15.47:0/224055198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61940d3bc0 0x7f61940d2cb0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:55.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.540+0000 7f6199007640 1 --2- 172.21.15.47:0/224055198 >> v2:172.21.15.203:3300/0 conn(0x7f619415f170 0x7f61940d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:55.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.540+0000 7f6199007640 1 -- 172.21.15.47:0/224055198 >> 172.21.15.47:0/224055198 conn(0x7f61940d4f60 msgr2=0x7f61940d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:55.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.540+0000 7f6199007640 1 -- 172.21.15.47:0/224055198 shutdown_connections 2024-04-26T23:19:55.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.540+0000 7f6199007640 1 -- 172.21.15.47:0/224055198 wait complete. 2024-04-26T23:19:55.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.541+0000 7f6199007640 1 Processor -- start 2024-04-26T23:19:55.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.541+0000 7f6199007640 1 -- start start 2024-04-26T23:19:55.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.542+0000 7f6199007640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61940d3bc0 0x7f61941f4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:55.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.542+0000 7f6199007640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f619415f170 0x7f6194157fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:55.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.542+0000 7f6199007640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61941f8120 0x7f61940b9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:55.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.542+0000 7f6199007640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f61940ca5c0 con 0x7f61940d3bc0 2024-04-26T23:19:55.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.542+0000 7f6199007640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f61940ca440 con 0x7f61941f8120 2024-04-26T23:19:55.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.542+0000 7f6199007640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f61940ca740 con 0x7f619415f170 2024-04-26T23:19:55.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.542+0000 7f6192d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61940d3bc0 0x7f61941f4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:55.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.542+0000 7f6192d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61940d3bc0 0x7f61941f4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38060/0 (socket says 172.21.15.47:38060) 2024-04-26T23:19:55.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.542+0000 7f6192d9d640 1 -- 172.21.15.47:0/2863732473 learned_addr learned my addr 172.21.15.47:0/2863732473 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:55.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.542+0000 7f619359e640 1 --2- 172.21.15.47:0/2863732473 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61941f8120 0x7f61940b9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:55.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.543+0000 7f619259c640 1 --2- 172.21.15.47:0/2863732473 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f619415f170 0x7f6194157fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:55.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.543+0000 7f6192d9d640 1 -- 172.21.15.47:0/2863732473 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f619415f170 msgr2=0x7f6194157fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:55.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.543+0000 7f6192d9d640 1 --2- 172.21.15.47:0/2863732473 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f619415f170 0x7f6194157fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:55.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.543+0000 7f6192d9d640 1 -- 172.21.15.47:0/2863732473 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61941f8120 msgr2=0x7f61940b9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:55.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.543+0000 7f6192d9d640 1 --2- 172.21.15.47:0/2863732473 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61941f8120 0x7f61940b9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:55.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.543+0000 7f6192d9d640 1 -- 172.21.15.47:0/2863732473 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61941f8b70 con 0x7f61940d3bc0 2024-04-26T23:19:55.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.543+0000 7f619259c640 1 --2- 172.21.15.47:0/2863732473 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f619415f170 0x7f6194157fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:19:55.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.543+0000 7f6192d9d640 1 --2- 172.21.15.47:0/2863732473 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61940d3bc0 0x7f61941f4b80 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f618c00e9b0 tx=0x7f618c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:55.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.543+0000 7f617b7fe640 1 -- 172.21.15.47:0/2863732473 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f618c017070 con 0x7f61940d3bc0 2024-04-26T23:19:55.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.543+0000 7f617b7fe640 1 -- 172.21.15.47:0/2863732473 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f618c00c9a0 con 0x7f61940d3bc0 2024-04-26T23:19:55.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.543+0000 7f6199007640 1 -- 172.21.15.47:0/2863732473 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61941f8e50 con 0x7f61940d3bc0 2024-04-26T23:19:55.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.543+0000 7f617b7fe640 1 -- 172.21.15.47:0/2863732473 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f618c00cd90 con 0x7f61940d3bc0 2024-04-26T23:19:55.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.543+0000 7f6199007640 1 -- 172.21.15.47:0/2863732473 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6194158520 con 0x7f61940d3bc0 2024-04-26T23:19:55.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.544+0000 7f6199007640 1 -- 172.21.15.47:0/2863732473 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61940694f0 con 0x7f61940d3bc0 2024-04-26T23:19:55.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.545+0000 7f617b7fe640 1 -- 172.21.15.47:0/2863732473 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f618c01a070 con 0x7f61940d3bc0 2024-04-26T23:19:55.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.545+0000 7f617b7fe640 1 --2- 172.21.15.47:0/2863732473 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f618006c8f0 0x7f618006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:55.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.545+0000 7f617b7fe640 1 -- 172.21.15.47:0/2863732473 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f618c08c180 con 0x7f61940d3bc0 2024-04-26T23:19:55.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.545+0000 7f619259c640 1 --2- 172.21.15.47:0/2863732473 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f618006c8f0 0x7f618006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:55.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.546+0000 7f619259c640 1 --2- 172.21.15.47:0/2863732473 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f618006c8f0 0x7f618006edb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6194158b90 tx=0x7f6188009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:55.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.549+0000 7f617b7fe640 1 -- 172.21.15.47:0/2863732473 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f618c059c50 con 0x7f61940d3bc0 2024-04-26T23:19:55.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.734+0000 7f6199007640 1 -- 172.21.15.47:0/2863732473 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f61941f9020 con 0x7f61940d3bc0 2024-04-26T23:19:55.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.735+0000 7f617b7fe640 1 -- 172.21.15.47:0/2863732473 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) ==== 93+0+10 (secure 0 0 0) 0x7f618c05df50 con 0x7f61940d3bc0 2024-04-26T23:19:55.737 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-26T23:19:55.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.737+0000 7f6199007640 1 -- 172.21.15.47:0/2863732473 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f618006c8f0 msgr2=0x7f618006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:55.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.737+0000 7f6199007640 1 --2- 172.21.15.47:0/2863732473 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f618006c8f0 0x7f618006edb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6194158b90 tx=0x7f6188009040 comp rx=0 tx=0).stop 2024-04-26T23:19:55.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.737+0000 7f6199007640 1 -- 172.21.15.47:0/2863732473 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61940d3bc0 msgr2=0x7f61941f4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:55.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.737+0000 7f6199007640 1 --2- 172.21.15.47:0/2863732473 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61940d3bc0 0x7f61941f4b80 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f618c00e9b0 tx=0x7f618c00ee70 comp rx=0 tx=0).stop 2024-04-26T23:19:55.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.737+0000 7f6199007640 1 -- 172.21.15.47:0/2863732473 shutdown_connections 2024-04-26T23:19:55.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.737+0000 7f6199007640 1 --2- 172.21.15.47:0/2863732473 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f618006c8f0 0x7f618006edb0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:55.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.737+0000 7f6199007640 1 --2- 172.21.15.47:0/2863732473 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61941f8120 0x7f61940b9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:55.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.737+0000 7f6199007640 1 --2- 172.21.15.47:0/2863732473 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f619415f170 0x7f6194157fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:55.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.737+0000 7f6199007640 1 --2- 172.21.15.47:0/2863732473 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61940d3bc0 0x7f61941f4b80 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:55.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.737+0000 7f6199007640 1 -- 172.21.15.47:0/2863732473 >> 172.21.15.47:0/2863732473 conn(0x7f61940d4f60 msgr2=0x7f61940d1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:55.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.737+0000 7f6199007640 1 -- 172.21.15.47:0/2863732473 shutdown_connections 2024-04-26T23:19:55.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.737+0000 7f6199007640 1 -- 172.21.15.47:0/2863732473 wait complete. 2024-04-26T23:19:55.750 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:19:55.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.853+0000 7f9a7450a640 1 Processor -- start 2024-04-26T23:19:55.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.853+0000 7f9a7450a640 1 -- start start 2024-04-26T23:19:55.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.854+0000 7f9a7450a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9a6c071260 0x7f9a6c15e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:55.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.854+0000 7f9a7450a640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a6c0d2390 con 0x7f9a6c070350 2024-04-26T23:19:55.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.854+0000 7f9a7450a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a6c06d3f0 con 0x7f9a6c0cfde0 2024-04-26T23:19:55.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.854+0000 7f9a7450a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f9a6c0afab0 con 0x7f9a6c071260 2024-04-26T23:19:55.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.854+0000 7f9a71aa5640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9a6c070350 0x7f9a6c070b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44738/0 (socket says 172.21.15.47:44738) 2024-04-26T23:19:55.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.854+0000 7f9a71aa5640 1 -- 172.21.15.47:0/381295069 learned_addr learned my addr 172.21.15.47:0/381295069 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:55.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.854+0000 7f9a72aa7640 1 --2- 172.21.15.47:0/381295069 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9a6c071260 0x7f9a6c15e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:55.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.854+0000 7f9a712a4640 1 -- 172.21.15.47:0/381295069 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3609485878 0 0) 0x7f9a6c0d2390 con 0x7f9a6c070350 2024-04-26T23:19:55.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.854+0000 7f9a712a4640 1 -- 172.21.15.47:0/381295069 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a58003520 con 0x7f9a6c070350 2024-04-26T23:19:55.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.854+0000 7f9a712a4640 1 -- 172.21.15.47:0/381295069 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1473171463 0 0) 0x7f9a6c06d3f0 con 0x7f9a6c0cfde0 2024-04-26T23:19:55.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.855+0000 7f9a712a4640 1 -- 172.21.15.47:0/381295069 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a6c0d2390 con 0x7f9a6c0cfde0 2024-04-26T23:19:55.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.855+0000 7f9a712a4640 1 -- 172.21.15.47:0/381295069 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 682282830 0 0) 0x7f9a58003520 con 0x7f9a6c070350 2024-04-26T23:19:55.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.855+0000 7f9a712a4640 1 -- 172.21.15.47:0/381295069 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9a6c071260 msgr2=0x7f9a6c15e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:55.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.855+0000 7f9a712a4640 1 --2- 172.21.15.47:0/381295069 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9a6c071260 0x7f9a6c15e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:55.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.855+0000 7f9a712a4640 1 -- 172.21.15.47:0/381295069 >> v1:172.21.15.203:6789/0 conn(0x7f9a6c0cfde0 legacy=0x7f9a6c0d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:55.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.855+0000 7f9a712a4640 1 -- 172.21.15.47:0/381295069 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a6c06d6d0 con 0x7f9a6c070350 2024-04-26T23:19:55.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.855+0000 7f9a712a4640 1 -- 172.21.15.47:0/381295069 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7f9a5c001670 con 0x7f9a6c070350 2024-04-26T23:19:55.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.855+0000 7f9a712a4640 1 -- 172.21.15.47:0/381295069 >> v1:172.21.15.47:6789/0 conn(0x7f9a6c070350 legacy=0x7f9a6c070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:55.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.855+0000 7f9a712a4640 1 --2- 172.21.15.47:0/381295069 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9a6c071260 0x7f9a58003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:55.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.855+0000 7f9a712a4640 1 --2- 172.21.15.47:0/381295069 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a580040d0 0x7f9a58004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:55.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.855+0000 7f9a712a4640 1 --2- 172.21.15.47:0/381295069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a58004d40 0x7f9a580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:55.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.855+0000 7f9a712a4640 1 -- 172.21.15.47:0/381295069 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5800bb50 con 0x7f9a58004d40 2024-04-26T23:19:55.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.855+0000 7f9a712a4640 1 -- 172.21.15.47:0/381295069 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5800bcd0 con 0x7f9a580040d0 2024-04-26T23:19:55.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.855+0000 7f9a712a4640 1 -- 172.21.15.47:0/381295069 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f9a5800bea0 con 0x7f9a6c071260 2024-04-26T23:19:55.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.855+0000 7f9a72aa7640 1 --2- 172.21.15.47:0/381295069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a58004d40 0x7f9a580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:55.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.855+0000 7f9a722a6640 1 --2- 172.21.15.47:0/381295069 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9a6c071260 0x7f9a58003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:55.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.855+0000 7f9a71aa5640 1 --2- 172.21.15.47:0/381295069 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a580040d0 0x7f9a58004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:55.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.855+0000 7f9a72aa7640 1 -- 172.21.15.47:0/381295069 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9a6c071260 msgr2=0x7f9a58003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:55.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.855+0000 7f9a72aa7640 1 --2- 172.21.15.47:0/381295069 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9a6c071260 0x7f9a58003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:55.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.855+0000 7f9a72aa7640 1 -- 172.21.15.47:0/381295069 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a580040d0 msgr2=0x7f9a58004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:55.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.855+0000 7f9a72aa7640 1 --2- 172.21.15.47:0/381295069 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a580040d0 0x7f9a58004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:55.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.855+0000 7f9a72aa7640 1 -- 172.21.15.47:0/381295069 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a58003520 con 0x7f9a58004d40 2024-04-26T23:19:55.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.856+0000 7f9a72aa7640 1 --2- 172.21.15.47:0/381295069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a58004d40 0x7f9a580051c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c0c6880 tx=0x7f9a58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=682c6abe5777c805 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:55.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.856+0000 7f9a712a4640 1 -- 172.21.15.47:0/381295069 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a64037070 con 0x7f9a58004d40 2024-04-26T23:19:55.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.856+0000 7f9a712a4640 1 -- 172.21.15.47:0/381295069 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a6402e050 con 0x7f9a58004d40 2024-04-26T23:19:55.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.856+0000 7f9a712a4640 1 -- 172.21.15.47:0/381295069 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a64009c10 con 0x7f9a58004d40 2024-04-26T23:19:55.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.856+0000 7f9a7450a640 1 -- 172.21.15.47:0/381295069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a58004d40 msgr2=0x7f9a580051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:55.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.856+0000 7f9a7450a640 1 --2- 172.21.15.47:0/381295069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a58004d40 0x7f9a580051c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c0c6880 tx=0x7f9a58005830 comp rx=0 tx=0).stop 2024-04-26T23:19:55.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.856+0000 7f9a72aa7640 1 -- 172.21.15.47:0/381295069 reap_dead start 2024-04-26T23:19:55.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.857+0000 7f9a7450a640 1 -- 172.21.15.47:0/381295069 shutdown_connections 2024-04-26T23:19:55.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.857+0000 7f9a7450a640 1 -- 172.21.15.47:0/381295069 >> 172.21.15.47:0/381295069 conn(0x7f9a6c0d2d80 msgr2=0x7f9a6c0d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:55.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.857+0000 7f9a7450a640 1 -- 172.21.15.47:0/381295069 shutdown_connections 2024-04-26T23:19:55.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.857+0000 7f9a7450a640 1 -- 172.21.15.47:0/381295069 wait complete. 2024-04-26T23:19:55.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.858+0000 7f9a7450a640 1 Processor -- start 2024-04-26T23:19:55.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.858+0000 7f9a7450a640 1 -- start start 2024-04-26T23:19:55.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.858+0000 7f9a7450a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9a58004d40 0x7f9a6c159a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:55.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.858+0000 7f9a7450a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a6c09d260 0x7f9a6c09fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:55.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.858+0000 7f9a7450a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a6c09d6e0 0x7f9a6c0a0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:55.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.858+0000 7f9a7450a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9a6c06f1c0 con 0x7f9a6c09d6e0 2024-04-26T23:19:55.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.858+0000 7f9a7450a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9a6c06dd20 con 0x7f9a6c09d260 2024-04-26T23:19:55.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.858+0000 7f9a7450a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f9a6c06f340 con 0x7f9a58004d40 2024-04-26T23:19:55.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.859+0000 7f9a72aa7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a6c09d6e0 0x7f9a6c0a0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:55.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.859+0000 7f9a72aa7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a6c09d6e0 0x7f9a6c0a0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38088/0 (socket says 172.21.15.47:38088) 2024-04-26T23:19:55.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.859+0000 7f9a72aa7640 1 -- 172.21.15.47:0/1899766945 learned_addr learned my addr 172.21.15.47:0/1899766945 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:55.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.859+0000 7f9a71aa5640 1 --2- 172.21.15.47:0/1899766945 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a6c09d260 0x7f9a6c09fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:55.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.859+0000 7f9a722a6640 1 --2- 172.21.15.47:0/1899766945 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9a58004d40 0x7f9a6c159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:55.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.859+0000 7f9a72aa7640 1 -- 172.21.15.47:0/1899766945 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9a58004d40 msgr2=0x7f9a6c159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:55.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.859+0000 7f9a72aa7640 1 --2- 172.21.15.47:0/1899766945 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9a58004d40 0x7f9a6c159a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:55.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.859+0000 7f9a72aa7640 1 -- 172.21.15.47:0/1899766945 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a6c09d260 msgr2=0x7f9a6c09fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:55.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.859+0000 7f9a72aa7640 1 --2- 172.21.15.47:0/1899766945 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a6c09d260 0x7f9a6c09fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:55.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.859+0000 7f9a72aa7640 1 -- 172.21.15.47:0/1899766945 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a6c15d450 con 0x7f9a6c09d6e0 2024-04-26T23:19:55.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.859+0000 7f9a71aa5640 1 --2- 172.21.15.47:0/1899766945 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a6c09d260 0x7f9a6c09fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:19:55.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.859+0000 7f9a72aa7640 1 --2- 172.21.15.47:0/1899766945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a6c09d6e0 0x7f9a6c0a0220 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f9a64002230 tx=0x7f9a64009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:55.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.859+0000 7f9a62ffd640 1 -- 172.21.15.47:0/1899766945 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a64037040 con 0x7f9a6c09d6e0 2024-04-26T23:19:55.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.859+0000 7f9a62ffd640 1 -- 172.21.15.47:0/1899766945 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a64044070 con 0x7f9a6c09d6e0 2024-04-26T23:19:55.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.859+0000 7f9a7450a640 1 -- 172.21.15.47:0/1899766945 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a6c15d6c0 con 0x7f9a6c09d6e0 2024-04-26T23:19:55.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.859+0000 7f9a62ffd640 1 -- 172.21.15.47:0/1899766945 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a64009c10 con 0x7f9a6c09d6e0 2024-04-26T23:19:55.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.859+0000 7f9a7450a640 1 -- 172.21.15.47:0/1899766945 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a6c15dc80 con 0x7f9a6c09d6e0 2024-04-26T23:19:55.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.860+0000 7f9a7450a640 1 -- 172.21.15.47:0/1899766945 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a6c0694f0 con 0x7f9a6c09d6e0 2024-04-26T23:19:55.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.861+0000 7f9a62ffd640 1 -- 172.21.15.47:0/1899766945 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9a64005d10 con 0x7f9a6c09d6e0 2024-04-26T23:19:55.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.861+0000 7f9a62ffd640 1 --2- 172.21.15.47:0/1899766945 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9a5006c8f0 0x7f9a5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:55.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.862+0000 7f9a62ffd640 1 -- 172.21.15.47:0/1899766945 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f9a640ac920 con 0x7f9a6c09d6e0 2024-04-26T23:19:55.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.862+0000 7f9a722a6640 1 --2- 172.21.15.47:0/1899766945 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9a5006c8f0 0x7f9a5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:55.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.862+0000 7f9a722a6640 1 --2- 172.21.15.47:0/1899766945 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9a5006c8f0 0x7f9a5006edb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9a680035f0 tx=0x7f9a68009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:55.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:55.866+0000 7f9a62ffd640 1 -- 172.21.15.47:0/1899766945 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9a6407a9c0 con 0x7f9a6c09d6e0 2024-04-26T23:19:56.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.055+0000 7f9a7450a640 1 -- 172.21.15.47:0/1899766945 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9a6c159fb0 con 0x7f9a6c09d6e0 2024-04-26T23:19:56.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.056+0000 7f9a62ffd640 1 -- 172.21.15.47:0/1899766945 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v41) ==== 104+0+11 (secure 0 0 0) 0x7f9a6407a2c0 con 0x7f9a6c09d6e0 2024-04-26T23:19:56.058 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-26T23:19:56.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.057+0000 7f9a7450a640 1 -- 172.21.15.47:0/1899766945 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9a5006c8f0 msgr2=0x7f9a5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:56.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.057+0000 7f9a7450a640 1 --2- 172.21.15.47:0/1899766945 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9a5006c8f0 0x7f9a5006edb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9a680035f0 tx=0x7f9a68009040 comp rx=0 tx=0).stop 2024-04-26T23:19:56.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.057+0000 7f9a7450a640 1 -- 172.21.15.47:0/1899766945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a6c09d6e0 msgr2=0x7f9a6c0a0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:56.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.057+0000 7f9a7450a640 1 --2- 172.21.15.47:0/1899766945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a6c09d6e0 0x7f9a6c0a0220 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f9a64002230 tx=0x7f9a64009750 comp rx=0 tx=0).stop 2024-04-26T23:19:56.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.058+0000 7f9a7450a640 1 -- 172.21.15.47:0/1899766945 shutdown_connections 2024-04-26T23:19:56.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.058+0000 7f9a7450a640 1 --2- 172.21.15.47:0/1899766945 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9a5006c8f0 0x7f9a5006edb0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:56.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.058+0000 7f9a7450a640 1 --2- 172.21.15.47:0/1899766945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a6c09d6e0 0x7f9a6c0a0220 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:56.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.058+0000 7f9a7450a640 1 --2- 172.21.15.47:0/1899766945 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a6c09d260 0x7f9a6c09fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:56.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.058+0000 7f9a7450a640 1 --2- 172.21.15.47:0/1899766945 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9a58004d40 0x7f9a6c159a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:56.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.058+0000 7f9a7450a640 1 -- 172.21.15.47:0/1899766945 >> 172.21.15.47:0/1899766945 conn(0x7f9a6c0d2d80 msgr2=0x7f9a6c0c7280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:56.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.058+0000 7f9a7450a640 1 -- 172.21.15.47:0/1899766945 shutdown_connections 2024-04-26T23:19:56.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.058+0000 7f9a7450a640 1 -- 172.21.15.47:0/1899766945 wait complete. 2024-04-26T23:19:56.070 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:19:56.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.171+0000 7f8b41dda640 1 Processor -- start 2024-04-26T23:19:56.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.171+0000 7f8b41dda640 1 -- start start 2024-04-26T23:19:56.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.172+0000 7f8b41dda640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8b3c0cbd30 0x7f8b3c0ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:56.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.172+0000 7f8b41dda640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8b3c0722e0 0x7f8b3c0ade60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:56.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.172+0000 7f8b41dda640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b3c0cd350 con 0x7f8b3c15d0e0 2024-04-26T23:19:56.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.172+0000 7f8b41dda640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f8b3c0cd630 con 0x7f8b3c0722e0 2024-04-26T23:19:56.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.172+0000 7f8b41dda640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f8b3c06f4f0 con 0x7f8b3c0cbd30 2024-04-26T23:19:56.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.172+0000 7f8b3bfff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8b3c0722e0 0x7f8b3c0ade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:56.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.172+0000 7f8b3bfff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8b3c0722e0 0x7f8b3c0ade60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38100/0 (socket says 172.21.15.47:38100) 2024-04-26T23:19:56.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.172+0000 7f8b3bfff640 1 -- 172.21.15.47:0/1895226549 learned_addr learned my addr 172.21.15.47:0/1895226549 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:56.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.172+0000 7f8b3affd640 1 --2- 172.21.15.47:0/1895226549 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8b3c0cbd30 0x7f8b3c0ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:56.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.172+0000 7f8b3bfff640 1 -- 172.21.15.47:0/1895226549 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8b3c0cbd30 msgr2=0x7f8b3c0ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:56.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.172+0000 7f8b3bfff640 1 --2- 172.21.15.47:0/1895226549 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8b3c0cbd30 0x7f8b3c0ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:56.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.172+0000 7f8b3bfff640 1 -- 172.21.15.47:0/1895226549 >> v1:172.21.15.203:6789/0 conn(0x7f8b3c15d0e0 legacy=0x7f8b3c0cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:56.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.172+0000 7f8b3bfff640 1 -- 172.21.15.47:0/1895226549 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b3c06f7d0 con 0x7f8b3c0722e0 2024-04-26T23:19:56.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.173+0000 7f8b3bfff640 1 --2- 172.21.15.47:0/1895226549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b3c0722e0 0x7f8b3c0ade60 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f8b34009e50 tx=0x7f8b3402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7b46cec2de7ef9d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:56.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.173+0000 7f8b3a7fc640 1 -- 172.21.15.47:0/1895226549 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b34036070 con 0x7f8b3c0722e0 2024-04-26T23:19:56.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.173+0000 7f8b3a7fc640 1 -- 172.21.15.47:0/1895226549 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b34031600 con 0x7f8b3c0722e0 2024-04-26T23:19:56.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.173+0000 7f8b3a7fc640 1 -- 172.21.15.47:0/1895226549 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b340319d0 con 0x7f8b3c0722e0 2024-04-26T23:19:56.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.174+0000 7f8b41dda640 1 -- 172.21.15.47:0/1895226549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b3c0722e0 msgr2=0x7f8b3c0ade60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:56.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.174+0000 7f8b41dda640 1 --2- 172.21.15.47:0/1895226549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b3c0722e0 0x7f8b3c0ade60 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f8b34009e50 tx=0x7f8b3402a710 comp rx=0 tx=0).stop 2024-04-26T23:19:56.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.174+0000 7f8b41dda640 1 -- 172.21.15.47:0/1895226549 shutdown_connections 2024-04-26T23:19:56.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.174+0000 7f8b41dda640 1 --2- 172.21.15.47:0/1895226549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b3c0722e0 0x7f8b3c0ade60 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:56.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.174+0000 7f8b41dda640 1 --2- 172.21.15.47:0/1895226549 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8b3c0cbd30 0x7f8b3c0ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:56.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.174+0000 7f8b41dda640 1 -- 172.21.15.47:0/1895226549 >> 172.21.15.47:0/1895226549 conn(0x7f8b3c0d0fd0 msgr2=0x7f8b3c0d13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:56.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.174+0000 7f8b41dda640 1 -- 172.21.15.47:0/1895226549 shutdown_connections 2024-04-26T23:19:56.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.174+0000 7f8b41dda640 1 -- 172.21.15.47:0/1895226549 wait complete. 2024-04-26T23:19:56.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.175+0000 7f8b41dda640 1 Processor -- start 2024-04-26T23:19:56.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.175+0000 7f8b41dda640 1 -- start start 2024-04-26T23:19:56.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.176+0000 7f8b41dda640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8b3c0722e0 0x7f8b3c0c75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:56.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.176+0000 7f8b41dda640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b3c0cbd30 0x7f8b3c09ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:56.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.176+0000 7f8b41dda640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b3c15d0e0 0x7f8b3c09bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:56.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.176+0000 7f8b41dda640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3c06fcf0 con 0x7f8b3c0cbd30 2024-04-26T23:19:56.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.176+0000 7f8b41dda640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3c0ae3a0 con 0x7f8b3c15d0e0 2024-04-26T23:19:56.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.176+0000 7f8b41dda640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f8b3c15d730 con 0x7f8b3c0722e0 2024-04-26T23:19:56.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.176+0000 7f8b3affd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b3c0cbd30 0x7f8b3c09ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:56.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.176+0000 7f8b3affd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b3c0cbd30 0x7f8b3c09ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38102/0 (socket says 172.21.15.47:38102) 2024-04-26T23:19:56.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.176+0000 7f8b3affd640 1 -- 172.21.15.47:0/1933784702 learned_addr learned my addr 172.21.15.47:0/1933784702 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:56.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.176+0000 7f8b3bfff640 1 --2- 172.21.15.47:0/1933784702 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b3c15d0e0 0x7f8b3c09bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:56.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.176+0000 7f8b3b7fe640 1 --2- 172.21.15.47:0/1933784702 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8b3c0722e0 0x7f8b3c0c75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:56.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.176+0000 7f8b3affd640 1 -- 172.21.15.47:0/1933784702 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8b3c0722e0 msgr2=0x7f8b3c0c75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:56.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.176+0000 7f8b3affd640 1 --2- 172.21.15.47:0/1933784702 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8b3c0722e0 0x7f8b3c0c75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:56.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.176+0000 7f8b3affd640 1 -- 172.21.15.47:0/1933784702 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b3c15d0e0 msgr2=0x7f8b3c09bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:56.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.176+0000 7f8b3affd640 1 --2- 172.21.15.47:0/1933784702 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b3c15d0e0 0x7f8b3c09bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:56.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.176+0000 7f8b3affd640 1 -- 172.21.15.47:0/1933784702 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b3c1f5b60 con 0x7f8b3c0cbd30 2024-04-26T23:19:56.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.176+0000 7f8b3affd640 1 --2- 172.21.15.47:0/1933784702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b3c0cbd30 0x7f8b3c09ba50 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f8b3000e9b0 tx=0x7f8b3000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:56.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.177+0000 7f8b23fff640 1 -- 172.21.15.47:0/1933784702 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b30017070 con 0x7f8b3c0cbd30 2024-04-26T23:19:56.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.177+0000 7f8b23fff640 1 -- 172.21.15.47:0/1933784702 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b3000c9a0 con 0x7f8b3c0cbd30 2024-04-26T23:19:56.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.177+0000 7f8b41dda640 1 -- 172.21.15.47:0/1933784702 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b3c1f5e40 con 0x7f8b3c0cbd30 2024-04-26T23:19:56.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.177+0000 7f8b23fff640 1 -- 172.21.15.47:0/1933784702 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3000cd90 con 0x7f8b3c0cbd30 2024-04-26T23:19:56.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.177+0000 7f8b41dda640 1 -- 172.21.15.47:0/1933784702 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b3c0cf4f0 con 0x7f8b3c0cbd30 2024-04-26T23:19:56.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.178+0000 7f8b41dda640 1 -- 172.21.15.47:0/1933784702 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b3c0cf2e0 con 0x7f8b3c0cbd30 2024-04-26T23:19:56.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.178+0000 7f8b23fff640 1 -- 172.21.15.47:0/1933784702 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8b3001a070 con 0x7f8b3c0cbd30 2024-04-26T23:19:56.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.179+0000 7f8b23fff640 1 --2- 172.21.15.47:0/1933784702 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f8b2806c8f0 0x7f8b2806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:56.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.179+0000 7f8b23fff640 1 -- 172.21.15.47:0/1933784702 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f8b3008beb0 con 0x7f8b3c0cbd30 2024-04-26T23:19:56.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.179+0000 7f8b3b7fe640 1 --2- 172.21.15.47:0/1933784702 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f8b2806c8f0 0x7f8b2806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:56.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.179+0000 7f8b3b7fe640 1 --2- 172.21.15.47:0/1933784702 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f8b2806c8f0 0x7f8b2806edb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c007fd0 tx=0x7f8b2c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:56.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.183+0000 7f8b23fff640 1 -- 172.21.15.47:0/1933784702 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8b3005a060 con 0x7f8b3c0cbd30 2024-04-26T23:19:56.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.367+0000 7f8b41dda640 1 -- 172.21.15.47:0/1933784702 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8b3c0694f0 con 0x7f8b3c0cbd30 2024-04-26T23:19:56.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.367+0000 7f8b23fff640 1 -- 172.21.15.47:0/1933784702 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v41) ==== 100+0+11 (secure 0 0 0) 0x7f8b30059960 con 0x7f8b3c0cbd30 2024-04-26T23:19:56.369 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-26T23:19:56.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.370+0000 7f8b41dda640 1 -- 172.21.15.47:0/1933784702 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f8b2806c8f0 msgr2=0x7f8b2806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:56.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.370+0000 7f8b41dda640 1 --2- 172.21.15.47:0/1933784702 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f8b2806c8f0 0x7f8b2806edb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c007fd0 tx=0x7f8b2c009040 comp rx=0 tx=0).stop 2024-04-26T23:19:56.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.370+0000 7f8b41dda640 1 -- 172.21.15.47:0/1933784702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b3c0cbd30 msgr2=0x7f8b3c09ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:56.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.370+0000 7f8b41dda640 1 --2- 172.21.15.47:0/1933784702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b3c0cbd30 0x7f8b3c09ba50 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f8b3000e9b0 tx=0x7f8b3000ee70 comp rx=0 tx=0).stop 2024-04-26T23:19:56.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.370+0000 7f8b41dda640 1 -- 172.21.15.47:0/1933784702 shutdown_connections 2024-04-26T23:19:56.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.370+0000 7f8b41dda640 1 --2- 172.21.15.47:0/1933784702 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f8b2806c8f0 0x7f8b2806edb0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:56.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.370+0000 7f8b41dda640 1 --2- 172.21.15.47:0/1933784702 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b3c15d0e0 0x7f8b3c09bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:56.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.370+0000 7f8b41dda640 1 --2- 172.21.15.47:0/1933784702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b3c0cbd30 0x7f8b3c09ba50 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:56.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.370+0000 7f8b41dda640 1 --2- 172.21.15.47:0/1933784702 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8b3c0722e0 0x7f8b3c0c75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:56.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.370+0000 7f8b41dda640 1 -- 172.21.15.47:0/1933784702 >> 172.21.15.47:0/1933784702 conn(0x7f8b3c0d0fd0 msgr2=0x7f8b3c0ae790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:56.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.370+0000 7f8b41dda640 1 -- 172.21.15.47:0/1933784702 shutdown_connections 2024-04-26T23:19:56.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.370+0000 7f8b41dda640 1 -- 172.21.15.47:0/1933784702 wait complete. 2024-04-26T23:19:56.383 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2024-04-26T23:19:56.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.485+0000 7f9cb042a640 1 Processor -- start 2024-04-26T23:19:56.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.485+0000 7f9cb042a640 1 -- start start 2024-04-26T23:19:56.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.486+0000 7f9cb042a640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9ca815d250 0x7f9ca80d0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:56.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.486+0000 7f9cb042a640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9ca80d1720 0x7f9ca8070970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:56.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.486+0000 7f9cb042a640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ca80701e0 con 0x7f9ca80d39f0 2024-04-26T23:19:56.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.486+0000 7f9cb042a640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f9ca815d630 con 0x7f9ca815d250 2024-04-26T23:19:56.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.486+0000 7f9cb042a640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f9ca80704e0 con 0x7f9ca80d1720 2024-04-26T23:19:56.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.486+0000 7f9cae1c6640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9ca815d250 0x7f9ca80d0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:56.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.486+0000 7f9cad9c5640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9ca80d39f0 0x7f9ca80d1010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44744/0 (socket says 172.21.15.47:44744) 2024-04-26T23:19:56.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.486+0000 7f9cad9c5640 1 -- 172.21.15.47:0/1305658362 learned_addr learned my addr 172.21.15.47:0/1305658362 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:56.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.486+0000 7f9cae1c6640 1 -- 172.21.15.47:0/1305658362 >> v2:172.21.15.203:3300/0 conn(0x7f9ca80d1720 msgr2=0x7f9ca8070970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:56.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.486+0000 7f9cae9c7640 1 --2- 172.21.15.47:0/1305658362 >> v2:172.21.15.203:3300/0 conn(0x7f9ca80d1720 0x7f9ca8070970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:56.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.486+0000 7f9cad1c4640 1 -- 172.21.15.47:0/1305658362 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1388582396 0 0) 0x7f9ca80701e0 con 0x7f9ca80d39f0 2024-04-26T23:19:56.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.486+0000 7f9cae1c6640 1 --2- 172.21.15.47:0/1305658362 >> v2:172.21.15.203:3300/0 conn(0x7f9ca80d1720 0x7f9ca8070970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:56.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.486+0000 7f9cae1c6640 1 -- 172.21.15.47:0/1305658362 >> v1:172.21.15.47:6789/0 conn(0x7f9ca80d39f0 legacy=0x7f9ca80d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:56.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.486+0000 7f9cae1c6640 1 -- 172.21.15.47:0/1305658362 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ca80ce370 con 0x7f9ca815d250 2024-04-26T23:19:56.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.486+0000 7f9cae9c7640 1 --2- 172.21.15.47:0/1305658362 >> v2:172.21.15.203:3300/0 conn(0x7f9ca80d1720 0x7f9ca8070970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:19:56.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.487+0000 7f9cae1c6640 1 --2- 172.21.15.47:0/1305658362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ca815d250 0x7f9ca80d0950 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9ca80b4800 tx=0x7f9ca402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23b2d29ed378a53b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:56.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.487+0000 7f9cad1c4640 1 -- 172.21.15.47:0/1305658362 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4037070 con 0x7f9ca815d250 2024-04-26T23:19:56.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.487+0000 7f9cad1c4640 1 -- 172.21.15.47:0/1305658362 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ca4032440 con 0x7f9ca815d250 2024-04-26T23:19:56.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.487+0000 7f9cad1c4640 1 -- 172.21.15.47:0/1305658362 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca402ae60 con 0x7f9ca815d250 2024-04-26T23:19:56.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.487+0000 7f9cb042a640 1 -- 172.21.15.47:0/1305658362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ca815d250 msgr2=0x7f9ca80d0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:56.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.487+0000 7f9cb042a640 1 --2- 172.21.15.47:0/1305658362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ca815d250 0x7f9ca80d0950 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9ca80b4800 tx=0x7f9ca402a110 comp rx=0 tx=0).stop 2024-04-26T23:19:56.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.488+0000 7f9cb042a640 1 -- 172.21.15.47:0/1305658362 shutdown_connections 2024-04-26T23:19:56.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.488+0000 7f9cb042a640 1 --2- 172.21.15.47:0/1305658362 >> v2:172.21.15.203:3300/0 conn(0x7f9ca80d1720 0x7f9ca8070970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:56.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.488+0000 7f9cb042a640 1 --2- 172.21.15.47:0/1305658362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ca815d250 0x7f9ca80d0950 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:56.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.488+0000 7f9cb042a640 1 -- 172.21.15.47:0/1305658362 >> 172.21.15.47:0/1305658362 conn(0x7f9ca80d2d80 msgr2=0x7f9ca80d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:56.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.488+0000 7f9cb042a640 1 -- 172.21.15.47:0/1305658362 shutdown_connections 2024-04-26T23:19:56.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.488+0000 7f9cb042a640 1 -- 172.21.15.47:0/1305658362 wait complete. 2024-04-26T23:19:56.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.489+0000 7f9cb042a640 1 Processor -- start 2024-04-26T23:19:56.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.489+0000 7f9cb042a640 1 -- start start 2024-04-26T23:19:56.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.489+0000 7f9cb042a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ca80d1720 0x7f9ca815a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:56.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.489+0000 7f9cb042a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ca80d39f0 0x7f9ca80ca320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:56.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.489+0000 7f9cb042a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9ca815d250 0x7f9ca80ca860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:56.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.489+0000 7f9cb042a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca80cebb0 con 0x7f9ca80d39f0 2024-04-26T23:19:56.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.489+0000 7f9cb042a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca80ce8d0 con 0x7f9ca80d1720 2024-04-26T23:19:56.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.489+0000 7f9cb042a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f9ca8070eb0 con 0x7f9ca815d250 2024-04-26T23:19:56.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.490+0000 7f9cad9c5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ca80d39f0 0x7f9ca80ca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:56.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.490+0000 7f9cad9c5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ca80d39f0 0x7f9ca80ca320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38116/0 (socket says 172.21.15.47:38116) 2024-04-26T23:19:56.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.490+0000 7f9cad9c5640 1 -- 172.21.15.47:0/3783903556 learned_addr learned my addr 172.21.15.47:0/3783903556 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:56.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.490+0000 7f9cae1c6640 1 --2- 172.21.15.47:0/3783903556 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ca80d1720 0x7f9ca815a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:56.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.490+0000 7f9cae9c7640 1 --2- 172.21.15.47:0/3783903556 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9ca815d250 0x7f9ca80ca860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:56.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.490+0000 7f9cad9c5640 1 -- 172.21.15.47:0/3783903556 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9ca815d250 msgr2=0x7f9ca80ca860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:56.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.490+0000 7f9cad9c5640 1 --2- 172.21.15.47:0/3783903556 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9ca815d250 0x7f9ca80ca860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:56.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.490+0000 7f9cad9c5640 1 -- 172.21.15.47:0/3783903556 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ca80d1720 msgr2=0x7f9ca815a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:56.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.490+0000 7f9cad9c5640 1 --2- 172.21.15.47:0/3783903556 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ca80d1720 0x7f9ca815a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:56.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.490+0000 7f9cad9c5640 1 -- 172.21.15.47:0/3783903556 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ca80c9950 con 0x7f9ca80d39f0 2024-04-26T23:19:56.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.490+0000 7f9cae9c7640 1 --2- 172.21.15.47:0/3783903556 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9ca815d250 0x7f9ca80ca860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:19:56.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.490+0000 7f9cad9c5640 1 --2- 172.21.15.47:0/3783903556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ca80d39f0 0x7f9ca80ca320 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9c9800c4d0 tx=0x7f9c9800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:56.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.490+0000 7f9c9effd640 1 -- 172.21.15.47:0/3783903556 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c98018070 con 0x7f9ca80d39f0 2024-04-26T23:19:56.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.490+0000 7f9c9effd640 1 -- 172.21.15.47:0/3783903556 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c98007500 con 0x7f9ca80d39f0 2024-04-26T23:19:56.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.490+0000 7f9c9effd640 1 -- 172.21.15.47:0/3783903556 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c98004a20 con 0x7f9ca80d39f0 2024-04-26T23:19:56.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.490+0000 7f9cb042a640 1 -- 172.21.15.47:0/3783903556 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ca80ccc30 con 0x7f9ca80d39f0 2024-04-26T23:19:56.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.490+0000 7f9cb042a640 1 -- 172.21.15.47:0/3783903556 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ca80c8690 con 0x7f9ca80d39f0 2024-04-26T23:19:56.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.491+0000 7f9cb042a640 1 -- 172.21.15.47:0/3783903556 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ca80694f0 con 0x7f9ca80d39f0 2024-04-26T23:19:56.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.492+0000 7f9c9effd640 1 -- 172.21.15.47:0/3783903556 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9c9801b070 con 0x7f9ca80d39f0 2024-04-26T23:19:56.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.492+0000 7f9c9effd640 1 --2- 172.21.15.47:0/3783903556 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9c8c06c8f0 0x7f9c8c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:56.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.492+0000 7f9c9effd640 1 -- 172.21.15.47:0/3783903556 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f9c9808b150 con 0x7f9ca80d39f0 2024-04-26T23:19:56.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.492+0000 7f9cae1c6640 1 --2- 172.21.15.47:0/3783903556 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9c8c06c8f0 0x7f9c8c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:56.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.493+0000 7f9cae1c6640 1 --2- 172.21.15.47:0/3783903556 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9c8c06c8f0 0x7f9c8c06edb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4034000 tx=0x7f9ca402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:56.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.496+0000 7f9c9effd640 1 -- 172.21.15.47:0/3783903556 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9c98059270 con 0x7f9ca80d39f0 2024-04-26T23:19:56.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.681+0000 7f9cb042a640 1 -- 172.21.15.47:0/3783903556 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) -- 0x7f9ca8160670 con 0x7f9ca80d39f0 2024-04-26T23:19:56.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.681+0000 7f9c9effd640 1 -- 172.21.15.47:0/3783903556 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v41) ==== 97+0+11 (secure 0 0 0) 0x7f9c98058b70 con 0x7f9ca80d39f0 2024-04-26T23:19:56.683 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-26T23:19:56.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.683+0000 7f9cb042a640 1 -- 172.21.15.47:0/3783903556 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9c8c06c8f0 msgr2=0x7f9c8c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:56.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.683+0000 7f9cb042a640 1 --2- 172.21.15.47:0/3783903556 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9c8c06c8f0 0x7f9c8c06edb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4034000 tx=0x7f9ca402e000 comp rx=0 tx=0).stop 2024-04-26T23:19:56.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.683+0000 7f9cb042a640 1 -- 172.21.15.47:0/3783903556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ca80d39f0 msgr2=0x7f9ca80ca320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:56.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.683+0000 7f9cb042a640 1 --2- 172.21.15.47:0/3783903556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ca80d39f0 0x7f9ca80ca320 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9c9800c4d0 tx=0x7f9c9800c990 comp rx=0 tx=0).stop 2024-04-26T23:19:56.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.683+0000 7f9cb042a640 1 -- 172.21.15.47:0/3783903556 shutdown_connections 2024-04-26T23:19:56.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.683+0000 7f9cb042a640 1 --2- 172.21.15.47:0/3783903556 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9c8c06c8f0 0x7f9c8c06edb0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:56.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.683+0000 7f9cb042a640 1 --2- 172.21.15.47:0/3783903556 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9ca815d250 0x7f9ca80ca860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:56.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.683+0000 7f9cb042a640 1 --2- 172.21.15.47:0/3783903556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ca80d39f0 0x7f9ca80ca320 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:56.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.683+0000 7f9cb042a640 1 --2- 172.21.15.47:0/3783903556 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ca80d1720 0x7f9ca815a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:56.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.683+0000 7f9cb042a640 1 -- 172.21.15.47:0/3783903556 >> 172.21.15.47:0/3783903556 conn(0x7f9ca80d2d80 msgr2=0x7f9ca80afab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:56.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.683+0000 7f9cb042a640 1 -- 172.21.15.47:0/3783903556 shutdown_connections 2024-04-26T23:19:56.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.683+0000 7f9cb042a640 1 -- 172.21.15.47:0/3783903556 wait complete. 2024-04-26T23:19:56.696 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T23:19:56.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.796+0000 7f31d44c8640 1 Processor -- start 2024-04-26T23:19:56.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.796+0000 7f31d44c8640 1 -- start start 2024-04-26T23:19:56.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.797+0000 7f31d44c8640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f31cc162880 0x7f31cc0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:56.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.797+0000 7f31d44c8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31cc0b5f20 0x7f31cc0d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:56.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.797+0000 7f31d44c8640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f31cc0d39f0 0x7f31cc0b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:56.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.797+0000 7f31d44c8640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f31cc0d0580 con 0x7f31cc0d39f0 2024-04-26T23:19:56.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.797+0000 7f31d44c8640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f31cc0b7e50 con 0x7f31cc162880 2024-04-26T23:19:56.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.797+0000 7f31d44c8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f31cc0d1800 con 0x7f31cc0b5f20 2024-04-26T23:19:56.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.797+0000 7f31d2a65640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f31cc0d39f0 0x7f31cc0b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:56.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.797+0000 7f31d2a65640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f31cc0d39f0 0x7f31cc0b69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38120/0 (socket says 172.21.15.47:38120) 2024-04-26T23:19:56.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.797+0000 7f31d2a65640 1 -- 172.21.15.47:0/3002648176 learned_addr learned my addr 172.21.15.47:0/3002648176 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:56.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.797+0000 7f31d1a63640 1 --2- 172.21.15.47:0/3002648176 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31cc0b5f20 0x7f31cc0d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:56.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.797+0000 7f31d2264640 1 --2- 172.21.15.47:0/3002648176 >> v2:172.21.15.203:3300/0 conn(0x7f31cc162880 0x7f31cc0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:56.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.798+0000 7f31d2a65640 1 -- 172.21.15.47:0/3002648176 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31cc0b5f20 msgr2=0x7f31cc0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:56.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.798+0000 7f31d2a65640 1 --2- 172.21.15.47:0/3002648176 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31cc0b5f20 0x7f31cc0d34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:56.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.798+0000 7f31d2a65640 1 -- 172.21.15.47:0/3002648176 >> v2:172.21.15.203:3300/0 conn(0x7f31cc162880 msgr2=0x7f31cc0d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:56.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.798+0000 7f31d2a65640 1 --2- 172.21.15.47:0/3002648176 >> v2:172.21.15.203:3300/0 conn(0x7f31cc162880 0x7f31cc0d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:56.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.798+0000 7f31d2a65640 1 -- 172.21.15.47:0/3002648176 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31cc0d07d0 con 0x7f31cc0d39f0 2024-04-26T23:19:56.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.798+0000 7f31d2a65640 1 --2- 172.21.15.47:0/3002648176 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31cc0d39f0 0x7f31cc0b69e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f31cc08d490 tx=0x7f31c402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a0c62317dd56bd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:56.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.798+0000 7f31d1262640 1 -- 172.21.15.47:0/3002648176 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c4036070 con 0x7f31cc0d39f0 2024-04-26T23:19:56.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.798+0000 7f31d1262640 1 -- 172.21.15.47:0/3002648176 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31c4031600 con 0x7f31cc0d39f0 2024-04-26T23:19:56.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.798+0000 7f31d1262640 1 -- 172.21.15.47:0/3002648176 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c40319d0 con 0x7f31cc0d39f0 2024-04-26T23:19:56.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.799+0000 7f31d44c8640 1 -- 172.21.15.47:0/3002648176 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31cc0d39f0 msgr2=0x7f31cc0b69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:56.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.799+0000 7f31d44c8640 1 --2- 172.21.15.47:0/3002648176 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31cc0d39f0 0x7f31cc0b69e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f31cc08d490 tx=0x7f31c402a6a0 comp rx=0 tx=0).stop 2024-04-26T23:19:56.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.799+0000 7f31d44c8640 1 -- 172.21.15.47:0/3002648176 shutdown_connections 2024-04-26T23:19:56.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.799+0000 7f31d44c8640 1 --2- 172.21.15.47:0/3002648176 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31cc0d39f0 0x7f31cc0b69e0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:56.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.799+0000 7f31d44c8640 1 --2- 172.21.15.47:0/3002648176 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31cc0b5f20 0x7f31cc0d34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:56.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.799+0000 7f31d44c8640 1 --2- 172.21.15.47:0/3002648176 >> v2:172.21.15.203:3300/0 conn(0x7f31cc162880 0x7f31cc0d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:56.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.799+0000 7f31d44c8640 1 -- 172.21.15.47:0/3002648176 >> 172.21.15.47:0/3002648176 conn(0x7f31cc0d4f60 msgr2=0x7f31cc0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:56.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.799+0000 7f31d44c8640 1 -- 172.21.15.47:0/3002648176 shutdown_connections 2024-04-26T23:19:56.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.799+0000 7f31d44c8640 1 -- 172.21.15.47:0/3002648176 wait complete. 2024-04-26T23:19:56.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.800+0000 7f31d44c8640 1 Processor -- start 2024-04-26T23:19:56.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.800+0000 7f31d44c8640 1 -- start start 2024-04-26T23:19:56.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.801+0000 7f31d44c8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31cc0b5f20 0x7f31cc0ba580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:56.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.801+0000 7f31d44c8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31cc0d39f0 0x7f31cc1f5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:56.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.801+0000 7f31d44c8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31cc162880 0x7f31cc0b5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:56.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.801+0000 7f31d44c8640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f31cc162c60 con 0x7f31cc0d39f0 2024-04-26T23:19:56.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.801+0000 7f31d44c8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f31cc0d0d90 con 0x7f31cc0b5f20 2024-04-26T23:19:56.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.801+0000 7f31d44c8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f31cc162de0 con 0x7f31cc162880 2024-04-26T23:19:56.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.801+0000 7f31d1a63640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31cc0d39f0 0x7f31cc1f5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:56.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.801+0000 7f31d1a63640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31cc0d39f0 0x7f31cc1f5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38124/0 (socket says 172.21.15.47:38124) 2024-04-26T23:19:56.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.801+0000 7f31d1a63640 1 -- 172.21.15.47:0/280738401 learned_addr learned my addr 172.21.15.47:0/280738401 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:56.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.801+0000 7f31d2264640 1 --2- 172.21.15.47:0/280738401 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31cc0b5f20 0x7f31cc0ba580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:56.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.801+0000 7f31d2a65640 1 --2- 172.21.15.47:0/280738401 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31cc162880 0x7f31cc0b5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:56.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.801+0000 7f31d1a63640 1 -- 172.21.15.47:0/280738401 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31cc162880 msgr2=0x7f31cc0b5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:56.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.801+0000 7f31d1a63640 1 --2- 172.21.15.47:0/280738401 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31cc162880 0x7f31cc0b5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:56.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.801+0000 7f31d1a63640 1 -- 172.21.15.47:0/280738401 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31cc0b5f20 msgr2=0x7f31cc0ba580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:56.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.801+0000 7f31d1a63640 1 --2- 172.21.15.47:0/280738401 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31cc0b5f20 0x7f31cc0ba580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:56.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.801+0000 7f31d1a63640 1 -- 172.21.15.47:0/280738401 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31cc0ce590 con 0x7f31cc0d39f0 2024-04-26T23:19:56.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.802+0000 7f31d1a63640 1 --2- 172.21.15.47:0/280738401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31cc0d39f0 0x7f31cc1f5080 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f31bc00e8a0 tx=0x7f31bc00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:56.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.802+0000 7f31c2ffd640 1 -- 172.21.15.47:0/280738401 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31bc017070 con 0x7f31cc0d39f0 2024-04-26T23:19:56.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.802+0000 7f31c2ffd640 1 -- 172.21.15.47:0/280738401 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31bc00ee90 con 0x7f31cc0d39f0 2024-04-26T23:19:56.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.802+0000 7f31d44c8640 1 -- 172.21.15.47:0/280738401 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31cc0ce870 con 0x7f31cc0d39f0 2024-04-26T23:19:56.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.802+0000 7f31c2ffd640 1 -- 172.21.15.47:0/280738401 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31bc00cb80 con 0x7f31cc0d39f0 2024-04-26T23:19:56.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.802+0000 7f31d44c8640 1 -- 172.21.15.47:0/280738401 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31cc0ce1f0 con 0x7f31cc0d39f0 2024-04-26T23:19:56.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.803+0000 7f31d44c8640 1 -- 172.21.15.47:0/280738401 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31cc0694f0 con 0x7f31cc0d39f0 2024-04-26T23:19:56.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.803+0000 7f31c2ffd640 1 -- 172.21.15.47:0/280738401 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f31bc01a070 con 0x7f31cc0d39f0 2024-04-26T23:19:56.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.804+0000 7f31c2ffd640 1 --2- 172.21.15.47:0/280738401 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f31b006c8f0 0x7f31b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:56.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.804+0000 7f31c2ffd640 1 -- 172.21.15.47:0/280738401 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f31bc08bbe0 con 0x7f31cc0d39f0 2024-04-26T23:19:56.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.804+0000 7f31d2264640 1 --2- 172.21.15.47:0/280738401 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f31b006c8f0 0x7f31b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:56.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.804+0000 7f31d2264640 1 --2- 172.21.15.47:0/280738401 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f31b006c8f0 0x7f31b006edb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f31c8009b40 tx=0x7f31c8009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:56.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:56.808+0000 7f31c2ffd640 1 -- 172.21.15.47:0/280738401 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f31bc059740 con 0x7f31cc0d39f0 2024-04-26T23:19:57.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.006+0000 7f31d44c8640 1 -- 172.21.15.47:0/280738401 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f31cc15fd60 con 0x7f31cc0d39f0 2024-04-26T23:19:57.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.006+0000 7f31c2ffd640 1 -- 172.21.15.47:0/280738401 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v41) ==== 80+0+0 (secure 0 0 0) 0x7f31bc05da40 con 0x7f31cc0d39f0 2024-04-26T23:19:57.008 INFO:teuthology.orchestra.run.smithi047.stderr:listed 0 entries 2024-04-26T23:19:57.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.008+0000 7f31d44c8640 1 -- 172.21.15.47:0/280738401 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f31b006c8f0 msgr2=0x7f31b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:57.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.008+0000 7f31d44c8640 1 --2- 172.21.15.47:0/280738401 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f31b006c8f0 0x7f31b006edb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f31c8009b40 tx=0x7f31c8009190 comp rx=0 tx=0).stop 2024-04-26T23:19:57.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.008+0000 7f31d44c8640 1 -- 172.21.15.47:0/280738401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31cc0d39f0 msgr2=0x7f31cc1f5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:57.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.008+0000 7f31d44c8640 1 --2- 172.21.15.47:0/280738401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31cc0d39f0 0x7f31cc1f5080 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f31bc00e8a0 tx=0x7f31bc00ed60 comp rx=0 tx=0).stop 2024-04-26T23:19:57.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.008+0000 7f31d44c8640 1 -- 172.21.15.47:0/280738401 shutdown_connections 2024-04-26T23:19:57.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.008+0000 7f31d44c8640 1 --2- 172.21.15.47:0/280738401 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f31b006c8f0 0x7f31b006edb0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:57.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.008+0000 7f31d44c8640 1 --2- 172.21.15.47:0/280738401 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31cc162880 0x7f31cc0b5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:57.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.008+0000 7f31d44c8640 1 --2- 172.21.15.47:0/280738401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31cc0d39f0 0x7f31cc1f5080 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:57.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.008+0000 7f31d44c8640 1 --2- 172.21.15.47:0/280738401 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31cc0b5f20 0x7f31cc0ba580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:57.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.008+0000 7f31d44c8640 1 -- 172.21.15.47:0/280738401 >> 172.21.15.47:0/280738401 conn(0x7f31cc0d4f60 msgr2=0x7f31cc160370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:57.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.008+0000 7f31d44c8640 1 -- 172.21.15.47:0/280738401 shutdown_connections 2024-04-26T23:19:57.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.008+0000 7f31d44c8640 1 -- 172.21.15.47:0/280738401 wait complete. 2024-04-26T23:19:57.021 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T23:19:57.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:57.021 DEBUG:teuthology.orchestra.run.smithi047:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T23:19:57.055 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi047.front.sepia.ceph.com 2024-04-26T23:19:57.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:57.055 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T23:19:57.115 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T23:19:57.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.250+0000 7ff4c5815640 1 Processor -- start 2024-04-26T23:19:57.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.250+0000 7ff4c5815640 1 -- start start 2024-04-26T23:19:57.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.250+0000 7ff4c5815640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4c015f2d0 0x7ff4c01604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:57.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.250+0000 7ff4c5815640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff4c006fea0 0x7ff4c00ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:57.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.250+0000 7ff4c5815640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4c0072f70 con 0x7ff4c0072700 2024-04-26T23:19:57.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.250+0000 7ff4c5815640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7ff4c0074010 con 0x7ff4c006fea0 2024-04-26T23:19:57.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.250+0000 7ff4c5815640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7ff4c0073250 con 0x7ff4c015f2d0 2024-04-26T23:19:57.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.251+0000 7ff4bf7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff4c006fea0 0x7ff4c00ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:57.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.251+0000 7ff4bf7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff4c006fea0 0x7ff4c00ce7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38138/0 (socket says 172.21.15.47:38138) 2024-04-26T23:19:57.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.251+0000 7ff4bf7fe640 1 -- 172.21.15.47:0/1065239547 learned_addr learned my addr 172.21.15.47:0/1065239547 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:57.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.251+0000 7ff4beffd640 1 --2- 172.21.15.47:0/1065239547 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4c015f2d0 0x7ff4c01604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:57.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.251+0000 7ff4bf7fe640 1 -- 172.21.15.47:0/1065239547 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4c015f2d0 msgr2=0x7ff4c01604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:57.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.251+0000 7ff4bf7fe640 1 --2- 172.21.15.47:0/1065239547 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4c015f2d0 0x7ff4c01604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:57.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.251+0000 7ff4bf7fe640 1 -- 172.21.15.47:0/1065239547 >> v1:172.21.15.203:6789/0 conn(0x7ff4c0072700 legacy=0x7ff4c006f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:57.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.251+0000 7ff4bf7fe640 1 -- 172.21.15.47:0/1065239547 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4c0073530 con 0x7ff4c006fea0 2024-04-26T23:19:57.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.251+0000 7ff4beffd640 1 --2- 172.21.15.47:0/1065239547 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4c015f2d0 0x7ff4c01604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:19:57.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.252+0000 7ff4bf7fe640 1 --2- 172.21.15.47:0/1065239547 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4c006fea0 0x7ff4c00ce7b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff4b800d150 tx=0x7ff4b802c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cd2bbda0b0d3702 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:57.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.252+0000 7ff4be7fc640 1 -- 172.21.15.47:0/1065239547 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b8009600 con 0x7ff4c006fea0 2024-04-26T23:19:57.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.252+0000 7ff4be7fc640 1 -- 172.21.15.47:0/1065239547 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4b8037070 con 0x7ff4c006fea0 2024-04-26T23:19:57.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.252+0000 7ff4be7fc640 1 -- 172.21.15.47:0/1065239547 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b80099c0 con 0x7ff4c006fea0 2024-04-26T23:19:57.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.252+0000 7ff4c5815640 1 -- 172.21.15.47:0/1065239547 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4c006fea0 msgr2=0x7ff4c00ce7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:57.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.252+0000 7ff4c5815640 1 --2- 172.21.15.47:0/1065239547 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4c006fea0 0x7ff4c00ce7b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff4b800d150 tx=0x7ff4b802c930 comp rx=0 tx=0).stop 2024-04-26T23:19:57.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.252+0000 7ff4c5815640 1 -- 172.21.15.47:0/1065239547 shutdown_connections 2024-04-26T23:19:57.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.252+0000 7ff4c5815640 1 --2- 172.21.15.47:0/1065239547 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4c006fea0 0x7ff4c00ce7b0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:57.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.252+0000 7ff4c5815640 1 --2- 172.21.15.47:0/1065239547 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4c015f2d0 0x7ff4c01604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:57.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.252+0000 7ff4c5815640 1 -- 172.21.15.47:0/1065239547 >> 172.21.15.47:0/1065239547 conn(0x7ff4c00d0be0 msgr2=0x7ff4c00d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:57.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.252+0000 7ff4c5815640 1 -- 172.21.15.47:0/1065239547 shutdown_connections 2024-04-26T23:19:57.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.252+0000 7ff4c5815640 1 -- 172.21.15.47:0/1065239547 wait complete. 2024-04-26T23:19:57.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.254+0000 7ff4c5815640 1 Processor -- start 2024-04-26T23:19:57.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.254+0000 7ff4c5815640 1 -- start start 2024-04-26T23:19:57.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.254+0000 7ff4c5815640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4c006fea0 0x7ff4c014f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:57.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.254+0000 7ff4c5815640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4c0072700 0x7ff4c01f51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:57.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.254+0000 7ff4c5815640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4c015f2d0 0x7ff4c01e66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:57.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.254+0000 7ff4c5815640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c015f6b0 con 0x7ff4c015f2d0 2024-04-26T23:19:57.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.254+0000 7ff4c5815640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c0073af0 con 0x7ff4c006fea0 2024-04-26T23:19:57.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.254+0000 7ff4c5815640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7ff4c0072b60 con 0x7ff4c0072700 2024-04-26T23:19:57.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.254+0000 7ff4bf7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4c015f2d0 0x7ff4c01e66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:57.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.254+0000 7ff4bf7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4c015f2d0 0x7ff4c01e66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38142/0 (socket says 172.21.15.47:38142) 2024-04-26T23:19:57.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.254+0000 7ff4bf7fe640 1 -- 172.21.15.47:0/3419211246 learned_addr learned my addr 172.21.15.47:0/3419211246 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:57.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.255+0000 7ff4beffd640 1 --2- 172.21.15.47:0/3419211246 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4c006fea0 0x7ff4c014f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:57.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.255+0000 7ff4b7fff640 1 --2- 172.21.15.47:0/3419211246 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4c0072700 0x7ff4c01f51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:57.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.255+0000 7ff4bf7fe640 1 -- 172.21.15.47:0/3419211246 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4c0072700 msgr2=0x7ff4c01f51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:57.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.255+0000 7ff4bf7fe640 1 --2- 172.21.15.47:0/3419211246 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4c0072700 0x7ff4c01f51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:57.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.255+0000 7ff4bf7fe640 1 -- 172.21.15.47:0/3419211246 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4c006fea0 msgr2=0x7ff4c014f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:57.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.255+0000 7ff4bf7fe640 1 --2- 172.21.15.47:0/3419211246 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4c006fea0 0x7ff4c014f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:57.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.255+0000 7ff4bf7fe640 1 -- 172.21.15.47:0/3419211246 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4c01f56f0 con 0x7ff4c015f2d0 2024-04-26T23:19:57.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.255+0000 7ff4beffd640 1 --2- 172.21.15.47:0/3419211246 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4c006fea0 0x7ff4c014f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:19:57.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.255+0000 7ff4bf7fe640 1 --2- 172.21.15.47:0/3419211246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4c015f2d0 0x7ff4c01e66a0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff4b800d150 tx=0x7ff4b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:57.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.255+0000 7ff4bcff9640 1 -- 172.21.15.47:0/3419211246 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b8009dc0 con 0x7ff4c015f2d0 2024-04-26T23:19:57.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.255+0000 7ff4bcff9640 1 -- 172.21.15.47:0/3419211246 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4b8047480 con 0x7ff4c015f2d0 2024-04-26T23:19:57.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.255+0000 7ff4c5815640 1 -- 172.21.15.47:0/3419211246 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4c00b3860 con 0x7ff4c015f2d0 2024-04-26T23:19:57.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.255+0000 7ff4bcff9640 1 -- 172.21.15.47:0/3419211246 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b8047870 con 0x7ff4c015f2d0 2024-04-26T23:19:57.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.255+0000 7ff4c5815640 1 -- 172.21.15.47:0/3419211246 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4c00b3b00 con 0x7ff4c015f2d0 2024-04-26T23:19:57.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.256+0000 7ff4c5815640 1 -- 172.21.15.47:0/3419211246 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4c00694f0 con 0x7ff4c015f2d0 2024-04-26T23:19:57.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.257+0000 7ff4bcff9640 1 -- 172.21.15.47:0/3419211246 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff4b8032420 con 0x7ff4c015f2d0 2024-04-26T23:19:57.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.257+0000 7ff4bcff9640 1 --2- 172.21.15.47:0/3419211246 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7ff4a006c8f0 0x7ff4a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:57.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.257+0000 7ff4bcff9640 1 -- 172.21.15.47:0/3419211246 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7ff4b80af120 con 0x7ff4c015f2d0 2024-04-26T23:19:57.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.257+0000 7ff4beffd640 1 --2- 172.21.15.47:0/3419211246 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7ff4a006c8f0 0x7ff4a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:57.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.258+0000 7ff4beffd640 1 --2- 172.21.15.47:0/3419211246 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7ff4a006c8f0 0x7ff4a006edb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0006fd0 tx=0x7ff4b0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:57.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.261+0000 7ff4bcff9640 1 -- 172.21.15.47:0/3419211246 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff4c01f56f0 con 0x7ff4c015f2d0 2024-04-26T23:19:57.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.485+0000 7ff4c5815640 1 -- 172.21.15.47:0/3419211246 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff4c015a920 con 0x7ff4c015f2d0 2024-04-26T23:19:57.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.486+0000 7ff4bcff9640 1 -- 172.21.15.47:0/3419211246 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v19) ==== 53+0+82 (secure 0 0 0) 0x7ff4b807d240 con 0x7ff4c015f2d0 2024-04-26T23:19:57.488 INFO:teuthology.orchestra.run.smithi047.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data data_new ] 2024-04-26T23:19:57.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.487+0000 7ff4c5815640 1 -- 172.21.15.47:0/3419211246 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7ff4a006c8f0 msgr2=0x7ff4a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:57.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.487+0000 7ff4c5815640 1 --2- 172.21.15.47:0/3419211246 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7ff4a006c8f0 0x7ff4a006edb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0006fd0 tx=0x7ff4b0008040 comp rx=0 tx=0).stop 2024-04-26T23:19:57.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.487+0000 7ff4c5815640 1 -- 172.21.15.47:0/3419211246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4c015f2d0 msgr2=0x7ff4c01e66a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:57.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.487+0000 7ff4c5815640 1 --2- 172.21.15.47:0/3419211246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4c015f2d0 0x7ff4c01e66a0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff4b800d150 tx=0x7ff4b8004a40 comp rx=0 tx=0).stop 2024-04-26T23:19:57.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.487+0000 7ff4c5815640 1 -- 172.21.15.47:0/3419211246 shutdown_connections 2024-04-26T23:19:57.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.487+0000 7ff4c5815640 1 --2- 172.21.15.47:0/3419211246 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7ff4a006c8f0 0x7ff4a006edb0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:57.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.487+0000 7ff4c5815640 1 --2- 172.21.15.47:0/3419211246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4c015f2d0 0x7ff4c01e66a0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:57.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.487+0000 7ff4c5815640 1 --2- 172.21.15.47:0/3419211246 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4c0072700 0x7ff4c01f51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:57.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.487+0000 7ff4c5815640 1 --2- 172.21.15.47:0/3419211246 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4c006fea0 0x7ff4c014f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:57.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.487+0000 7ff4c5815640 1 -- 172.21.15.47:0/3419211246 >> 172.21.15.47:0/3419211246 conn(0x7ff4c00d0be0 msgr2=0x7ff4c006ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:57.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.488+0000 7ff4c5815640 1 -- 172.21.15.47:0/3419211246 shutdown_connections 2024-04-26T23:19:57.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57.488+0000 7ff4c5815640 1 -- 172.21.15.47:0/3419211246 wait complete. 2024-04-26T23:19:57.500 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T23:19:57.500 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T23:19:57.500 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi047.front.sepia.ceph.com 2024-04-26T23:19:57.500 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T23:19:57.500 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T23:19:57.500 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T23:19:57.501 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T23:19:57.501 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T23:19:57.501 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:19:57.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:57.501 DEBUG:teuthology.orchestra.run.smithi047:> ip addr 2024-04-26T23:19:57.517 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T23:19:57.517 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T23:19:57.517 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T23:19:57.517 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:19:57.517 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2024-04-26T23:19:57.518 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:19:57.518 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:19:57.518 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:19:57.518 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f0 2024-04-26T23:19:57.518 INFO:teuthology.orchestra.run.smithi047.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:19:57.518 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:19:57.518 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f1 2024-04-26T23:19:57.518 INFO:teuthology.orchestra.run.smithi047.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T23:19:57.518 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2024-04-26T23:19:57.518 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f0 2024-04-26T23:19:57.518 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-26T23:19:57.518 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42601sec preferred_lft 42601sec 2024-04-26T23:19:57.518 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link noprefixroute 2024-04-26T23:19:57.518 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:19:57.518 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:19:57.519 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2024-04-26T23:19:57.519 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f1 2024-04-26T23:19:57.519 INFO:teuthology.orchestra.run.smithi047.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T23:19:57.519 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 32:8d:47:b1:d0:77 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:19:57.519 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T23:19:57.519 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:19:57.519 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::7c6d:6aff:fe00:4b30/64 scope link 2024-04-26T23:19:57.519 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:19:57.519 INFO:teuthology.orchestra.run.smithi047.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T23:19:57.519 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 32:8d:47:b1:d0:77 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T23:19:57.519 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::308d:47ff:feb1:d077/64 scope link 2024-04-26T23:19:57.519 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:19:57.519 INFO:teuthology.orchestra.run.smithi047.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T23:19:57.519 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether b2:9e:42:6a:5b:5d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T23:19:57.519 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::b09e:42ff:fe6a:5b5d/64 scope link 2024-04-26T23:19:57.520 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:19:57.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:57.520 DEBUG:teuthology.orchestra.run.smithi047:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T23:19:57.580 INFO:teuthology.orchestra.run.smithi047.stdout:1 2024-04-26T23:19:57.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:57.582 DEBUG:teuthology.orchestra.run.smithi047:> route 2024-04-26T23:19:57.639 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2024-04-26T23:19:57.639 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T23:19:57.639 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T23:19:57.639 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T23:19:57.639 INFO:teuthology.orchestra.run.smithi047.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T23:19:57.639 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:19:57.639 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-04-26T23:19:57.639 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T23:19:57.640 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T23:19:57.640 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T23:19:57.640 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-04-26T23:19:57.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:19:57.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:19:57.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:57.766 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-04-26T23:19:57.781 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T23:19:57.782 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T23:19:57.782 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:19:57.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:57.782 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set brx.0 up 2024-04-26T23:19:57.847 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:19:57.847 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T23:19:57.902 INFO:teuthology.orchestra.run.smithi047.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:19:57.902 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:19:57.902 DEBUG:teuthology.orchestra.run.smithi047:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T23:19:57.957 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T23:19:57.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:57.958 DEBUG:teuthology.orchestra.run.smithi047:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:19:58.040 INFO:teuthology.orchestra.run.smithi047.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:19:58.041 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:19:58.041 INFO:teuthology.orchestra.run.smithi047.stdout:invalid new device string format 2024-04-26T23:19:58.049 INFO:teuthology.orchestra.run.smithi047.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:19:58.049 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:19:58.049 INFO:teuthology.orchestra.run.smithi047.stdout:invalid new device string format 2024-04-26T23:19:58.049 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: resolved to: "172.21.15.47:3300,172.21.15.203:3300,172.21.15.203:3301" 2024-04-26T23:19:58.049 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:19:58.068 INFO:teuthology.orchestra.run.smithi047.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:19:58.068 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:19:58.068 INFO:teuthology.orchestra.run.smithi047.stdout:invalid new device string format 2024-04-26T23:19:58.068 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: resolved to: "172.21.15.47:3300,172.21.15.203:3300,172.21.15.203:3301" 2024-04-26T23:19:58.068 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:19:58.069 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: resolved to: "172.21.15.47:6789,172.21.15.203:6789,172.21.15.203:6790" 2024-04-26T23:19:58.069 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.47:6789,172.21.15.203:6789,172.21.15.203:6790:/ 2024-04-26T23:19:58.069 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=399006d3-4db1-43e6-ba81-a79a276eabfd" will pass to kernel 2024-04-26T23:19:58.069 INFO:teuthology.orchestra.run.smithi047.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T23:19:58.069 INFO:teuthology.orchestra.run.smithi047.stdout: You just mounted a file system that supports labels which does not 2024-04-26T23:19:58.069 INFO:teuthology.orchestra.run.smithi047.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T23:19:58.069 INFO:teuthology.orchestra.run.smithi047.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T23:19:58.069 INFO:teuthology.orchestra.run.smithi047.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T23:19:58.070 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T23:19:58.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:58.070 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T23:19:58.136 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T23:19:58.148 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:19:58.179 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:19:58.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:19:58.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:19:58.234 INFO:teuthology.orchestra.run.smithi047.stdout:client14861 2024-04-26T23:19:58.234 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:19:58.234 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:19:58.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:58.290 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.file.layout.pool -v data_new layoutfile) 2024-04-26T23:19:58.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T23:19:58.346 INFO:teuthology.orchestra.run.smithi047.stderr:setfattr: layoutfile: Permission denied 2024-04-26T23:19:58.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:19:58.347 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.layout.pool -v data_new layoutdir) 2024-04-26T23:19:58.402 INFO:teuthology.orchestra.run.smithi047.stderr:setfattr: layoutdir: Permission denied 2024-04-26T23:19:58.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T23:19:58.404 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:19:58.404 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:19:58.459 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:19:58.459 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:19:58.515 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:19:58.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.654+0000 7f998c872640 1 Processor -- start 2024-04-26T23:19:58.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.654+0000 7f998c872640 1 -- start start 2024-04-26T23:19:58.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.655+0000 7f998c872640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f99880b10b0 0x7f99880ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:58.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.655+0000 7f998c872640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f998806c950 0x7f998815e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:58.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.655+0000 7f998c872640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99880cfd80 con 0x7f99880b0380 2024-04-26T23:19:58.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.655+0000 7f998c872640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f99880af8e0 con 0x7f99880b10b0 2024-04-26T23:19:58.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.655+0000 7f998c872640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f99880d0020 con 0x7f998806c950 2024-04-26T23:19:58.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.655+0000 7f9985d9b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f99880b10b0 0x7f99880ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:58.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.655+0000 7f9985d9b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f99880b10b0 0x7f99880ccd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38146/0 (socket says 172.21.15.47:38146) 2024-04-26T23:19:58.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.655+0000 7f9985d9b640 1 -- 172.21.15.47:0/2696330793 learned_addr learned my addr 172.21.15.47:0/2696330793 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:58.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.655+0000 7f9986d9d640 1 --2- 172.21.15.47:0/2696330793 >> v2:172.21.15.203:3300/0 conn(0x7f998806c950 0x7f998815e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:58.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.655+0000 7f9985d9b640 1 -- 172.21.15.47:0/2696330793 >> v2:172.21.15.203:3300/0 conn(0x7f998806c950 msgr2=0x7f998815e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:58.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.655+0000 7f9985d9b640 1 --2- 172.21.15.47:0/2696330793 >> v2:172.21.15.203:3300/0 conn(0x7f998806c950 0x7f998815e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:58.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.655+0000 7f9985d9b640 1 -- 172.21.15.47:0/2696330793 >> v1:172.21.15.203:6789/0 conn(0x7f99880b0380 legacy=0x7f998806c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:58.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.655+0000 7f9985d9b640 1 -- 172.21.15.47:0/2696330793 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99880d0300 con 0x7f99880b10b0 2024-04-26T23:19:58.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.656+0000 7f9985d9b640 1 --2- 172.21.15.47:0/2696330793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99880b10b0 0x7f99880ccd50 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f9988072f00 tx=0x7f997002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d350bab264e2b858 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:58.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.656+0000 7f998559a640 1 -- 172.21.15.47:0/2696330793 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9970037070 con 0x7f99880b10b0 2024-04-26T23:19:58.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.656+0000 7f998559a640 1 -- 172.21.15.47:0/2696330793 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9970032440 con 0x7f99880b10b0 2024-04-26T23:19:58.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.656+0000 7f998559a640 1 -- 172.21.15.47:0/2696330793 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9970032810 con 0x7f99880b10b0 2024-04-26T23:19:58.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.657+0000 7f998c872640 1 -- 172.21.15.47:0/2696330793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99880b10b0 msgr2=0x7f99880ccd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:58.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.657+0000 7f998c872640 1 --2- 172.21.15.47:0/2696330793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99880b10b0 0x7f99880ccd50 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f9988072f00 tx=0x7f997002a1e0 comp rx=0 tx=0).stop 2024-04-26T23:19:58.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.657+0000 7f998c872640 1 -- 172.21.15.47:0/2696330793 shutdown_connections 2024-04-26T23:19:58.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.657+0000 7f998c872640 1 --2- 172.21.15.47:0/2696330793 >> v2:172.21.15.203:3300/0 conn(0x7f998806c950 0x7f998815e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:58.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.657+0000 7f998c872640 1 --2- 172.21.15.47:0/2696330793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99880b10b0 0x7f99880ccd50 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:58.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.657+0000 7f998c872640 1 -- 172.21.15.47:0/2696330793 >> 172.21.15.47:0/2696330793 conn(0x7f99880d1cf0 msgr2=0x7f99880d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:58.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.657+0000 7f998c872640 1 -- 172.21.15.47:0/2696330793 shutdown_connections 2024-04-26T23:19:58.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.657+0000 7f998c872640 1 -- 172.21.15.47:0/2696330793 wait complete. 2024-04-26T23:19:58.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.658+0000 7f998c872640 1 Processor -- start 2024-04-26T23:19:58.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.658+0000 7f998c872640 1 -- start start 2024-04-26T23:19:58.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.659+0000 7f998c872640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f998806c950 0x7f99880c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:58.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.659+0000 7f998c872640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99880b0380 0x7f9988161ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:58.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.659+0000 7f998c872640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f99880b10b0 0x7f99880cb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:58.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.659+0000 7f998c872640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f99880b0760 con 0x7f998806c950 2024-04-26T23:19:58.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.659+0000 7f998c872640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f99880d1980 con 0x7f99880b0380 2024-04-26T23:19:58.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.659+0000 7f998c872640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f99880d15c0 con 0x7f99880b10b0 2024-04-26T23:19:58.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.659+0000 7f998659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f998806c950 0x7f99880c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:58.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.659+0000 7f998659c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f998806c950 0x7f99880c9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38156/0 (socket says 172.21.15.47:38156) 2024-04-26T23:19:58.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.659+0000 7f998659c640 1 -- 172.21.15.47:0/2743401557 learned_addr learned my addr 172.21.15.47:0/2743401557 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:58.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.659+0000 7f9985d9b640 1 --2- 172.21.15.47:0/2743401557 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99880b0380 0x7f9988161ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:58.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.659+0000 7f9986d9d640 1 --2- 172.21.15.47:0/2743401557 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f99880b10b0 0x7f99880cb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:58.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.659+0000 7f998659c640 1 -- 172.21.15.47:0/2743401557 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f99880b10b0 msgr2=0x7f99880cb7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:58.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.659+0000 7f998659c640 1 --2- 172.21.15.47:0/2743401557 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f99880b10b0 0x7f99880cb7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:58.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.659+0000 7f998659c640 1 -- 172.21.15.47:0/2743401557 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99880b0380 msgr2=0x7f9988161ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:58.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.659+0000 7f998659c640 1 --2- 172.21.15.47:0/2743401557 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99880b0380 0x7f9988161ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:58.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.659+0000 7f998659c640 1 -- 172.21.15.47:0/2743401557 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99881623e0 con 0x7f998806c950 2024-04-26T23:19:58.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.660+0000 7f998659c640 1 --2- 172.21.15.47:0/2743401557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f998806c950 0x7f99880c9e10 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f998000c400 tx=0x7f998000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:58.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.660+0000 7f997effd640 1 -- 172.21.15.47:0/2743401557 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9980017070 con 0x7f998806c950 2024-04-26T23:19:58.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.660+0000 7f997effd640 1 -- 172.21.15.47:0/2743401557 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99800052d0 con 0x7f998806c950 2024-04-26T23:19:58.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.660+0000 7f998c872640 1 -- 172.21.15.47:0/2743401557 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99880cbce0 con 0x7f998806c950 2024-04-26T23:19:58.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.660+0000 7f997effd640 1 -- 172.21.15.47:0/2743401557 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9980003830 con 0x7f998806c950 2024-04-26T23:19:58.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.660+0000 7f998c872640 1 -- 172.21.15.47:0/2743401557 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99880caa20 con 0x7f998806c950 2024-04-26T23:19:58.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.661+0000 7f998c872640 1 -- 172.21.15.47:0/2743401557 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99880d0b00 con 0x7f998806c950 2024-04-26T23:19:58.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.661+0000 7f997effd640 1 -- 172.21.15.47:0/2743401557 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f998001a070 con 0x7f998806c950 2024-04-26T23:19:58.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.662+0000 7f997effd640 1 --2- 172.21.15.47:0/2743401557 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f996806c8f0 0x7f996806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:58.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.662+0000 7f997effd640 1 -- 172.21.15.47:0/2743401557 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f998008b1a0 con 0x7f998806c950 2024-04-26T23:19:58.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.662+0000 7f9985d9b640 1 --2- 172.21.15.47:0/2743401557 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f996806c8f0 0x7f996806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:58.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.663+0000 7f9985d9b640 1 --2- 172.21.15.47:0/2743401557 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f996806c8f0 0x7f996806edb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9970034000 tx=0x7f997002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:58.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.666+0000 7f997effd640 1 -- 172.21.15.47:0/2743401557 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f99880d0b00 con 0x7f998806c950 2024-04-26T23:19:58.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.851+0000 7f998c872640 1 -- 172.21.15.47:0/2743401557 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f99880694f0 con 0x7f998806c950 2024-04-26T23:19:58.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.853+0000 7f997effd640 1 -- 172.21.15.47:0/2743401557 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+18718 (secure 0 0 0) 0x7f99881623e0 con 0x7f998806c950 2024-04-26T23:19:58.855 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:19:58.855 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":41,"fsid":"399006d3-4db1-43e6-ba81-a79a276eabfd","created":"2024-04-26T23:18:06.731211+0000","modified":"2024-04-26T23:19:54.781247+0000","last_up_change":"2024-04-26T23:18:17.808736+0000","last_in_change":"2024-04-26T23:18:08.945859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:18:16.937492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:19:42.351747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-26T23:19:43.037763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2024-04-26T23:19:51.911275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ee091567-fea0-4a14-bf4e-0e16c20bb2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6819","nonce":326520779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6821","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6823","nonce":326520779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6829","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6831","nonce":326520779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6825","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6827","nonce":326520779}]},"public_addr":"172.21.15.47:6819/326520779","cluster_addr":"172.21.15.47:6823/326520779","heartbeat_back_addr":"172.21.15.47:6831/326520779","heartbeat_front_addr":"172.21.15.47:6827/326520779","state":["exists","up"]},{"osd":1,"uuid":"c14440a0-bba5-4931-a6db-b736c08e869b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6809","nonce":1912610931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6811","nonce":1912610931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6815","nonce":1912610931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6813","nonce":1912610931}]},"public_addr":"172.21.15.47:6809/1912610931","cluster_addr":"172.21.15.47:6811/1912610931","heartbeat_back_addr":"172.21.15.47:6815/1912610931","heartbeat_front_addr":"172.21.15.47:6813/1912610931","state":["exists","up"]},{"osd":2,"uuid":"3940f8de-c9b2-4cd3-a911-9c301729eb30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6801","nonce":2569869305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6803","nonce":2569869305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6807","nonce":2569869305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6805","nonce":2569869305}]},"public_addr":"172.21.15.47:6801/2569869305","cluster_addr":"172.21.15.47:6803/2569869305","heartbeat_back_addr":"172.21.15.47:6807/2569869305","heartbeat_front_addr":"172.21.15.47:6805/2569869305","state":["exists","up"]},{"osd":3,"uuid":"1286a67e-8958-426a-8c2e-bc85498c619c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6818","nonce":2791411683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6822","nonce":2791411683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6830","nonce":2791411683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6826","nonce":2791411683}]},"public_addr":"172.21.15.47:6818/2791411683","cluster_addr":"172.21.15.47:6822/2791411683","heartbeat_back_addr":"172.21.15.47:6830/2791411683","heartbeat_front_addr":"172.21.15.47:6826/2791411683","state":["exists","up"]},{"osd":4,"uuid":"d8ee2c20-e5b7-46c2-9999-b234070126cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6809","nonce":441313104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6811","nonce":441313104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6815","nonce":441313104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6813","nonce":441313104}]},"public_addr":"172.21.15.203:6809/441313104","cluster_addr":"172.21.15.203:6811/441313104","heartbeat_back_addr":"172.21.15.203:6815/441313104","heartbeat_front_addr":"172.21.15.203:6813/441313104","state":["exists","up"]},{"osd":5,"uuid":"b453a8f0-f324-4840-9ae0-d67bdadb9f8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6801","nonce":56838444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6803","nonce":56838444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6807","nonce":56838444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6805","nonce":56838444}]},"public_addr":"172.21.15.203:6801/56838444","cluster_addr":"172.21.15.203:6803/56838444","heartbeat_back_addr":"172.21.15.203:6807/56838444","heartbeat_front_addr":"172.21.15.203:6805/56838444","state":["exists","up"]},{"osd":6,"uuid":"a6d31951-2d4b-4e8d-a510-6283b01dc158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6825","nonce":2738517080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6827","nonce":2738517080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6831","nonce":2738517080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6829","nonce":2738517080}]},"public_addr":"172.21.15.203:6825/2738517080","cluster_addr":"172.21.15.203:6827/2738517080","heartbeat_back_addr":"172.21.15.203:6831/2738517080","heartbeat_front_addr":"172.21.15.203:6829/2738517080","state":["exists","up"]},{"osd":7,"uuid":"7119b8c0-fe96-402d-9e75-7b509d9d44d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6817","nonce":1315565058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6819","nonce":1315565058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6823","nonce":1315565058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6821","nonce":1315565058}]},"public_addr":"172.21.15.203:6817/1315565058","cluster_addr":"172.21.15.203:6819/1315565058","heartbeat_back_addr":"172.21.15.203:6823/1315565058","heartbeat_front_addr":"172.21.15.203:6821/1315565058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.121921+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.137494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.084814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.111593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.072430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.125066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.097867+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.057061+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:19:58.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.854+0000 7f998c872640 1 -- 172.21.15.47:0/2743401557 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f996806c8f0 msgr2=0x7f996806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:58.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.854+0000 7f998c872640 1 --2- 172.21.15.47:0/2743401557 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f996806c8f0 0x7f996806edb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9970034000 tx=0x7f997002e000 comp rx=0 tx=0).stop 2024-04-26T23:19:58.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.855+0000 7f998c872640 1 -- 172.21.15.47:0/2743401557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f998806c950 msgr2=0x7f99880c9e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:58.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.855+0000 7f998c872640 1 --2- 172.21.15.47:0/2743401557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f998806c950 0x7f99880c9e10 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f998000c400 tx=0x7f998000c8c0 comp rx=0 tx=0).stop 2024-04-26T23:19:58.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.855+0000 7f998c872640 1 -- 172.21.15.47:0/2743401557 shutdown_connections 2024-04-26T23:19:58.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.855+0000 7f998c872640 1 --2- 172.21.15.47:0/2743401557 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f996806c8f0 0x7f996806edb0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:58.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.855+0000 7f998c872640 1 --2- 172.21.15.47:0/2743401557 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f99880b10b0 0x7f99880cb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:58.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.855+0000 7f998c872640 1 --2- 172.21.15.47:0/2743401557 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99880b0380 0x7f9988161ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:58.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.855+0000 7f998c872640 1 --2- 172.21.15.47:0/2743401557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f998806c950 0x7f99880c9e10 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:58.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.855+0000 7f998c872640 1 -- 172.21.15.47:0/2743401557 >> 172.21.15.47:0/2743401557 conn(0x7f99880d1cf0 msgr2=0x7f99880ccc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:58.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.855+0000 7f998c872640 1 -- 172.21.15.47:0/2743401557 shutdown_connections 2024-04-26T23:19:58.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.855+0000 7f998c872640 1 -- 172.21.15.47:0/2743401557 wait complete. 2024-04-26T23:19:58.868 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:18:16.937492+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:19:42.351747+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:19:43.037763+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'data_new', 'create_time': '2024-04-26T23:19:51.911275+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:19:58.869 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:19:58.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.971+0000 7f9499852640 1 Processor -- start 2024-04-26T23:19:58.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.971+0000 7f9499852640 1 -- start start 2024-04-26T23:19:58.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.972+0000 7f9499852640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9494071e60 0x7f949406f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:58.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.972+0000 7f9499852640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f949406fe20 0x7f9494072240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:58.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.972+0000 7f9499852640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f949415d7c0 con 0x7f9494072780 2024-04-26T23:19:58.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.972+0000 7f9499852640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f949415daa0 con 0x7f949406fe20 2024-04-26T23:19:58.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.972+0000 7f9499852640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f949415dd80 con 0x7f9494071e60 2024-04-26T23:19:58.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.972+0000 7f94937fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9494072780 0x7f94940cb470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44752/0 (socket says 172.21.15.47:44752) 2024-04-26T23:19:58.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.972+0000 7f94937fe640 1 -- 172.21.15.47:0/3326066560 learned_addr learned my addr 172.21.15.47:0/3326066560 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:58.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.972+0000 7f94927fc640 1 --2- 172.21.15.47:0/3326066560 >> v2:172.21.15.203:3300/0 conn(0x7f949406fe20 0x7f9494072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:58.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.972+0000 7f9492ffd640 1 --2- 172.21.15.47:0/3326066560 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9494071e60 0x7f949406f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:58.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.972+0000 7f9491ffb640 1 -- 172.21.15.47:0/3326066560 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 591761933 0 0) 0x7f949415d7c0 con 0x7f9494072780 2024-04-26T23:19:58.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.972+0000 7f9491ffb640 1 -- 172.21.15.47:0/3326066560 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9480003520 con 0x7f9494072780 2024-04-26T23:19:58.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.973+0000 7f9492ffd640 1 -- 172.21.15.47:0/3326066560 >> v2:172.21.15.203:3300/0 conn(0x7f949406fe20 msgr2=0x7f9494072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:58.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.973+0000 7f9492ffd640 1 --2- 172.21.15.47:0/3326066560 >> v2:172.21.15.203:3300/0 conn(0x7f949406fe20 0x7f9494072240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:58.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.973+0000 7f9492ffd640 1 -- 172.21.15.47:0/3326066560 >> v1:172.21.15.47:6789/0 conn(0x7f9494072780 legacy=0x7f94940cb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:58.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.973+0000 7f9491ffb640 1 -- 172.21.15.47:0/3326066560 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4012228526 0 0) 0x7f9480003520 con 0x7f9494072780 2024-04-26T23:19:58.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.973+0000 7f9492ffd640 1 -- 172.21.15.47:0/3326066560 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f949415e060 con 0x7f9494071e60 2024-04-26T23:19:58.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.973+0000 7f94927fc640 1 --2- 172.21.15.47:0/3326066560 >> v2:172.21.15.203:3300/0 conn(0x7f949406fe20 0x7f9494072240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:19:58.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.973+0000 7f9492ffd640 1 --2- 172.21.15.47:0/3326066560 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9494071e60 0x7f949406f8e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f94840097a0 tx=0x7f948402a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=731388cfc2a9ba69 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:58.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.974+0000 7f9491ffb640 1 -- 172.21.15.47:0/3326066560 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9484037070 con 0x7f9494071e60 2024-04-26T23:19:58.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.974+0000 7f9491ffb640 1 -- 172.21.15.47:0/3326066560 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f948402aba0 con 0x7f9494071e60 2024-04-26T23:19:58.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.974+0000 7f9491ffb640 1 -- 172.21.15.47:0/3326066560 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948c0019f0 con 0x7f9494071e60 2024-04-26T23:19:58.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.974+0000 7f9499852640 1 -- 172.21.15.47:0/3326066560 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9494071e60 msgr2=0x7f949406f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:58.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.974+0000 7f9499852640 1 --2- 172.21.15.47:0/3326066560 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9494071e60 0x7f949406f8e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f94840097a0 tx=0x7f948402a110 comp rx=0 tx=0).stop 2024-04-26T23:19:58.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.974+0000 7f9499852640 1 -- 172.21.15.47:0/3326066560 shutdown_connections 2024-04-26T23:19:58.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.974+0000 7f9499852640 1 --2- 172.21.15.47:0/3326066560 >> v2:172.21.15.203:3300/0 conn(0x7f949406fe20 0x7f9494072240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:58.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.974+0000 7f9499852640 1 --2- 172.21.15.47:0/3326066560 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9494071e60 0x7f949406f8e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:58.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.974+0000 7f9499852640 1 -- 172.21.15.47:0/3326066560 >> 172.21.15.47:0/3326066560 conn(0x7f94940d0be0 msgr2=0x7f94940d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:58.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.974+0000 7f9499852640 1 -- 172.21.15.47:0/3326066560 shutdown_connections 2024-04-26T23:19:58.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.974+0000 7f9499852640 1 -- 172.21.15.47:0/3326066560 wait complete. 2024-04-26T23:19:58.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.975+0000 7f9499852640 1 Processor -- start 2024-04-26T23:19:58.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.976+0000 7f9499852640 1 -- start start 2024-04-26T23:19:58.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.976+0000 7f9499852640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f949406fe20 0x7f94940afe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:58.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.976+0000 7f9499852640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9494071e60 0x7f949415a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:58.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.976+0000 7f9499852640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9494072780 0x7f949415a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:58.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.976+0000 7f9499852640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f949415d3f0 con 0x7f949406fe20 2024-04-26T23:19:58.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.976+0000 7f9499852640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9494072c10 con 0x7f9494072780 2024-04-26T23:19:58.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.976+0000 7f9499852640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f949415d570 con 0x7f9494071e60 2024-04-26T23:19:58.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.976+0000 7f9492ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f949406fe20 0x7f94940afe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:58.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.976+0000 7f9492ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f949406fe20 0x7f94940afe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38158/0 (socket says 172.21.15.47:38158) 2024-04-26T23:19:58.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.976+0000 7f9492ffd640 1 -- 172.21.15.47:0/3004903597 learned_addr learned my addr 172.21.15.47:0/3004903597 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:58.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.977+0000 7f9492ffd640 1 -- 172.21.15.47:0/3004903597 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9494071e60 msgr2=0x7f949415a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:58.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.977+0000 7f9492ffd640 1 --2- 172.21.15.47:0/3004903597 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9494071e60 0x7f949415a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:58.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.977+0000 7f9492ffd640 1 -- 172.21.15.47:0/3004903597 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9494072780 msgr2=0x7f949415a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:58.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.977+0000 7f9492ffd640 1 --2- 172.21.15.47:0/3004903597 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9494072780 0x7f949415a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:58.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.977+0000 7f9492ffd640 1 -- 172.21.15.47:0/3004903597 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94941f6d70 con 0x7f949406fe20 2024-04-26T23:19:58.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.977+0000 7f9492ffd640 1 --2- 172.21.15.47:0/3004903597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f949406fe20 0x7f94940afe00 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f94840097a0 tx=0x7f94840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:58.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.977+0000 7f948bfff640 1 -- 172.21.15.47:0/3004903597 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9484037050 con 0x7f949406fe20 2024-04-26T23:19:58.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.977+0000 7f948bfff640 1 -- 172.21.15.47:0/3004903597 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9484043070 con 0x7f949406fe20 2024-04-26T23:19:58.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.977+0000 7f9499852640 1 -- 172.21.15.47:0/3004903597 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f949415ad00 con 0x7f949406fe20 2024-04-26T23:19:58.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.977+0000 7f948bfff640 1 -- 172.21.15.47:0/3004903597 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9484006e10 con 0x7f949406fe20 2024-04-26T23:19:58.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.977+0000 7f9499852640 1 -- 172.21.15.47:0/3004903597 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f949415b2c0 con 0x7f949406fe20 2024-04-26T23:19:58.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.978+0000 7f9499852640 1 -- 172.21.15.47:0/3004903597 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94940694f0 con 0x7f949406fe20 2024-04-26T23:19:58.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.979+0000 7f948bfff640 1 -- 172.21.15.47:0/3004903597 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f948403ca10 con 0x7f949406fe20 2024-04-26T23:19:58.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.979+0000 7f948bfff640 1 --2- 172.21.15.47:0/3004903597 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f947406c8f0 0x7f947406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:58.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.979+0000 7f948bfff640 1 -- 172.21.15.47:0/3004903597 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f94840ac850 con 0x7f949406fe20 2024-04-26T23:19:58.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.979+0000 7f94927fc640 1 --2- 172.21.15.47:0/3004903597 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f947406c8f0 0x7f947406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:58.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.980+0000 7f94927fc640 1 --2- 172.21.15.47:0/3004903597 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f947406c8f0 0x7f947406edb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f947c006e80 tx=0x7f947c006dd0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:58.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:58.983+0000 7f948bfff640 1 -- 172.21.15.47:0/3004903597 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f948407ab20 con 0x7f949406fe20 2024-04-26T23:19:59.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.169+0000 7f9499852640 1 -- 172.21.15.47:0/3004903597 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f949406ce60 con 0x7f949406fe20 2024-04-26T23:19:59.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.169+0000 7f948bfff640 1 -- 172.21.15.47:0/3004903597 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) ==== 93+0+10 (secure 0 0 0) 0x7f948407a420 con 0x7f949406fe20 2024-04-26T23:19:59.171 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-26T23:19:59.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.171+0000 7f9499852640 1 -- 172.21.15.47:0/3004903597 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f947406c8f0 msgr2=0x7f947406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:59.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.171+0000 7f9499852640 1 --2- 172.21.15.47:0/3004903597 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f947406c8f0 0x7f947406edb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f947c006e80 tx=0x7f947c006dd0 comp rx=0 tx=0).stop 2024-04-26T23:19:59.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.171+0000 7f9499852640 1 -- 172.21.15.47:0/3004903597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f949406fe20 msgr2=0x7f94940afe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:59.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.171+0000 7f9499852640 1 --2- 172.21.15.47:0/3004903597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f949406fe20 0x7f94940afe00 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f94840097a0 tx=0x7f94840046c0 comp rx=0 tx=0).stop 2024-04-26T23:19:59.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.171+0000 7f9499852640 1 -- 172.21.15.47:0/3004903597 shutdown_connections 2024-04-26T23:19:59.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.171+0000 7f9499852640 1 --2- 172.21.15.47:0/3004903597 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f947406c8f0 0x7f947406edb0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:59.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.171+0000 7f9499852640 1 --2- 172.21.15.47:0/3004903597 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9494072780 0x7f949415a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:59.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.171+0000 7f9499852640 1 --2- 172.21.15.47:0/3004903597 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9494071e60 0x7f949415a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:59.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.171+0000 7f9499852640 1 --2- 172.21.15.47:0/3004903597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f949406fe20 0x7f94940afe00 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:59.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.171+0000 7f9499852640 1 -- 172.21.15.47:0/3004903597 >> 172.21.15.47:0/3004903597 conn(0x7f94940d0be0 msgr2=0x7f949415ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:59.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.171+0000 7f9499852640 1 -- 172.21.15.47:0/3004903597 shutdown_connections 2024-04-26T23:19:59.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.171+0000 7f9499852640 1 -- 172.21.15.47:0/3004903597 wait complete. 2024-04-26T23:19:59.184 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:19:59.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.286+0000 7f76b24d1640 1 Processor -- start 2024-04-26T23:19:59.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.286+0000 7f76b24d1640 1 -- start start 2024-04-26T23:19:59.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.287+0000 7f76b24d1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76ac06c950 0x7f76ac160ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:59.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.287+0000 7f76b24d1640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f76ac070930 0x7f76ac0aed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:59.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.287+0000 7f76b24d1640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76ac0b0990 con 0x7f76ac15d100 2024-04-26T23:19:59.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.287+0000 7f76b24d1640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f76ac0af2d0 con 0x7f76ac070930 2024-04-26T23:19:59.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.287+0000 7f76b24d1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f76ac0b0c70 con 0x7f76ac06c950 2024-04-26T23:19:59.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.287+0000 7f76b0a6e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f76ac070930 0x7f76ac0aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:59.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.287+0000 7f76b0a6e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f76ac070930 0x7f76ac0aed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38162/0 (socket says 172.21.15.47:38162) 2024-04-26T23:19:59.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.287+0000 7f76b0a6e640 1 -- 172.21.15.47:0/2343107345 learned_addr learned my addr 172.21.15.47:0/2343107345 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:59.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.287+0000 7f76ab7fe640 1 --2- 172.21.15.47:0/2343107345 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76ac06c950 0x7f76ac160ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:59.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.287+0000 7f76b0a6e640 1 -- 172.21.15.47:0/2343107345 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76ac06c950 msgr2=0x7f76ac160ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:59.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.287+0000 7f76b0a6e640 1 --2- 172.21.15.47:0/2343107345 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76ac06c950 0x7f76ac160ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:59.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.287+0000 7f76b0a6e640 1 -- 172.21.15.47:0/2343107345 >> v1:172.21.15.203:6789/0 conn(0x7f76ac15d100 legacy=0x7f76ac06c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:59.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.287+0000 7f76b0a6e640 1 -- 172.21.15.47:0/2343107345 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76ac0b0f50 con 0x7f76ac070930 2024-04-26T23:19:59.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.288+0000 7f76b0a6e640 1 --2- 172.21.15.47:0/2343107345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76ac070930 0x7f76ac0aed90 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f76a4009e50 tx=0x7f76a402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe097c774972f7c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:59.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.288+0000 7f76aaffd640 1 -- 172.21.15.47:0/2343107345 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a4036070 con 0x7f76ac070930 2024-04-26T23:19:59.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.288+0000 7f76aaffd640 1 -- 172.21.15.47:0/2343107345 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76a4031600 con 0x7f76ac070930 2024-04-26T23:19:59.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.288+0000 7f76aaffd640 1 -- 172.21.15.47:0/2343107345 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a40319d0 con 0x7f76ac070930 2024-04-26T23:19:59.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.288+0000 7f76b24d1640 1 -- 172.21.15.47:0/2343107345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76ac070930 msgr2=0x7f76ac0aed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:59.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.288+0000 7f76b24d1640 1 --2- 172.21.15.47:0/2343107345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76ac070930 0x7f76ac0aed90 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f76a4009e50 tx=0x7f76a402a710 comp rx=0 tx=0).stop 2024-04-26T23:19:59.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.288+0000 7f76b24d1640 1 -- 172.21.15.47:0/2343107345 shutdown_connections 2024-04-26T23:19:59.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.288+0000 7f76b24d1640 1 --2- 172.21.15.47:0/2343107345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76ac070930 0x7f76ac0aed90 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:59.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.288+0000 7f76b24d1640 1 --2- 172.21.15.47:0/2343107345 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76ac06c950 0x7f76ac160ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:59.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.288+0000 7f76b24d1640 1 -- 172.21.15.47:0/2343107345 >> 172.21.15.47:0/2343107345 conn(0x7f76ac0b39b0 msgr2=0x7f76ac0b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:59.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.289+0000 7f76b24d1640 1 -- 172.21.15.47:0/2343107345 shutdown_connections 2024-04-26T23:19:59.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.289+0000 7f76b24d1640 1 -- 172.21.15.47:0/2343107345 wait complete. 2024-04-26T23:19:59.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.290+0000 7f76b24d1640 1 Processor -- start 2024-04-26T23:19:59.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.290+0000 7f76b24d1640 1 -- start start 2024-04-26T23:19:59.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.291+0000 7f76b24d1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76ac06c950 0x7f76ac151b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:59.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.291+0000 7f76b24d1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76ac070930 0x7f76ac15c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:59.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.291+0000 7f76b24d1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76ac15d100 0x7f76ac15f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:59.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.291+0000 7f76b24d1640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f76ac073160 con 0x7f76ac15d100 2024-04-26T23:19:59.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.291+0000 7f76b24d1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f76ac0b1510 con 0x7f76ac06c950 2024-04-26T23:19:59.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.291+0000 7f76b24d1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f76ac0732e0 con 0x7f76ac070930 2024-04-26T23:19:59.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.291+0000 7f76b0a6e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76ac15d100 0x7f76ac15f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:59.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.291+0000 7f76b0a6e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76ac15d100 0x7f76ac15f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38168/0 (socket says 172.21.15.47:38168) 2024-04-26T23:19:59.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.291+0000 7f76b0a6e640 1 -- 172.21.15.47:0/1718460778 learned_addr learned my addr 172.21.15.47:0/1718460778 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:59.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.291+0000 7f76abfff640 1 --2- 172.21.15.47:0/1718460778 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76ac06c950 0x7f76ac151b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:59.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.291+0000 7f76ab7fe640 1 --2- 172.21.15.47:0/1718460778 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76ac070930 0x7f76ac15c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:59.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.291+0000 7f76b0a6e640 1 -- 172.21.15.47:0/1718460778 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76ac070930 msgr2=0x7f76ac15c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:59.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.291+0000 7f76b0a6e640 1 --2- 172.21.15.47:0/1718460778 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76ac070930 0x7f76ac15c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:59.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.291+0000 7f76b0a6e640 1 -- 172.21.15.47:0/1718460778 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76ac06c950 msgr2=0x7f76ac151b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:59.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.291+0000 7f76b0a6e640 1 --2- 172.21.15.47:0/1718460778 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76ac06c950 0x7f76ac151b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:59.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.291+0000 7f76b0a6e640 1 -- 172.21.15.47:0/1718460778 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76ac0b5d30 con 0x7f76ac15d100 2024-04-26T23:19:59.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.291+0000 7f76abfff640 1 --2- 172.21.15.47:0/1718460778 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76ac06c950 0x7f76ac151b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:19:59.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.291+0000 7f76b0a6e640 1 --2- 172.21.15.47:0/1718460778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76ac15d100 0x7f76ac15f710 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f76a4009e50 tx=0x7f76a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:59.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.291+0000 7f76a8ff9640 1 -- 172.21.15.47:0/1718460778 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a4036050 con 0x7f76ac15d100 2024-04-26T23:19:59.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.292+0000 7f76a8ff9640 1 -- 172.21.15.47:0/1718460778 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76a4042070 con 0x7f76ac15d100 2024-04-26T23:19:59.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.292+0000 7f76b24d1640 1 -- 172.21.15.47:0/1718460778 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76ac0b6010 con 0x7f76ac15d100 2024-04-26T23:19:59.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.292+0000 7f76a8ff9640 1 -- 172.21.15.47:0/1718460778 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a4009510 con 0x7f76ac15d100 2024-04-26T23:19:59.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.292+0000 7f76b24d1640 1 -- 172.21.15.47:0/1718460778 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76ac0b65d0 con 0x7f76ac15d100 2024-04-26T23:19:59.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.292+0000 7f76b24d1640 1 -- 172.21.15.47:0/1718460778 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76ac0694f0 con 0x7f76ac15d100 2024-04-26T23:19:59.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.293+0000 7f76a8ff9640 1 -- 172.21.15.47:0/1718460778 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f76a4007bd0 con 0x7f76ac15d100 2024-04-26T23:19:59.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.293+0000 7f76a8ff9640 1 --2- 172.21.15.47:0/1718460778 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f769806c8f0 0x7f769806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:59.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.294+0000 7f76a8ff9640 1 -- 172.21.15.47:0/1718460778 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f76a40accd0 con 0x7f76ac15d100 2024-04-26T23:19:59.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.294+0000 7f76abfff640 1 --2- 172.21.15.47:0/1718460778 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f769806c8f0 0x7f769806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:59.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.294+0000 7f76abfff640 1 --2- 172.21.15.47:0/1718460778 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f769806c8f0 0x7f769806edb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f769c001660 tx=0x7f769c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:59.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.298+0000 7f76a8ff9640 1 -- 172.21.15.47:0/1718460778 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f76a407adf0 con 0x7f76ac15d100 2024-04-26T23:19:59.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.481+0000 7f76b24d1640 1 -- 172.21.15.47:0/1718460778 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f76ac0b0130 con 0x7f76ac15d100 2024-04-26T23:19:59.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.482+0000 7f76a8ff9640 1 -- 172.21.15.47:0/1718460778 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v41) ==== 104+0+11 (secure 0 0 0) 0x7f76a407a6f0 con 0x7f76ac15d100 2024-04-26T23:19:59.484 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-26T23:19:59.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.483+0000 7f76b24d1640 1 -- 172.21.15.47:0/1718460778 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f769806c8f0 msgr2=0x7f769806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:59.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.483+0000 7f76b24d1640 1 --2- 172.21.15.47:0/1718460778 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f769806c8f0 0x7f769806edb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f769c001660 tx=0x7f769c009040 comp rx=0 tx=0).stop 2024-04-26T23:19:59.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.484+0000 7f76b24d1640 1 -- 172.21.15.47:0/1718460778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76ac15d100 msgr2=0x7f76ac15f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:59.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.484+0000 7f76b24d1640 1 --2- 172.21.15.47:0/1718460778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76ac15d100 0x7f76ac15f710 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f76a4009e50 tx=0x7f76a4004a40 comp rx=0 tx=0).stop 2024-04-26T23:19:59.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.484+0000 7f76b24d1640 1 -- 172.21.15.47:0/1718460778 shutdown_connections 2024-04-26T23:19:59.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.484+0000 7f76b24d1640 1 --2- 172.21.15.47:0/1718460778 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f769806c8f0 0x7f769806edb0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:59.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.484+0000 7f76b24d1640 1 --2- 172.21.15.47:0/1718460778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76ac15d100 0x7f76ac15f710 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:59.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.484+0000 7f76b24d1640 1 --2- 172.21.15.47:0/1718460778 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76ac070930 0x7f76ac15c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:59.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.484+0000 7f76b24d1640 1 --2- 172.21.15.47:0/1718460778 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76ac06c950 0x7f76ac151b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:59.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.484+0000 7f76b24d1640 1 -- 172.21.15.47:0/1718460778 >> 172.21.15.47:0/1718460778 conn(0x7f76ac0b39b0 msgr2=0x7f76ac06fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:59.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.484+0000 7f76b24d1640 1 -- 172.21.15.47:0/1718460778 shutdown_connections 2024-04-26T23:19:59.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.484+0000 7f76b24d1640 1 -- 172.21.15.47:0/1718460778 wait complete. 2024-04-26T23:19:59.497 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:19:59.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.598+0000 7fb3a8d9d640 1 Processor -- start 2024-04-26T23:19:59.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.598+0000 7fb3a8d9d640 1 -- start start 2024-04-26T23:19:59.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.599+0000 7fb3a8d9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb3a4071260 0x7fb3a415e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:59.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.599+0000 7fb3a8d9d640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3a40d2390 con 0x7fb3a4070350 2024-04-26T23:19:59.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.599+0000 7fb3a8d9d640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3a406d3f0 con 0x7fb3a40cfde0 2024-04-26T23:19:59.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.599+0000 7fb3a8d9d640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fb3a40afab0 con 0x7fb3a4071260 2024-04-26T23:19:59.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.599+0000 7fb3a2d9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb3a4071260 0x7fb3a415e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:59.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.599+0000 7fb39bfff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb3a4070350 0x7fb3a4070b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44766/0 (socket says 172.21.15.47:44766) 2024-04-26T23:19:59.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.599+0000 7fb39bfff640 1 -- 172.21.15.47:0/4022029971 learned_addr learned my addr 172.21.15.47:0/4022029971 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:59.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.599+0000 7fb3a2d9d640 1 -- 172.21.15.47:0/4022029971 >> v1:172.21.15.203:6789/0 conn(0x7fb3a40cfde0 legacy=0x7fb3a40d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:59.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.599+0000 7fb3a2d9d640 1 -- 172.21.15.47:0/4022029971 >> v1:172.21.15.47:6789/0 conn(0x7fb3a4070350 legacy=0x7fb3a4070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:59.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.599+0000 7fb3a2d9d640 1 -- 172.21.15.47:0/4022029971 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3a406d6d0 con 0x7fb3a4071260 2024-04-26T23:19:59.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.600+0000 7fb3a2d9d640 1 --2- 172.21.15.47:0/4022029971 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3a4071260 0x7fb3a415e9f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fb39c009d90 tx=0x7fb39c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae23fc41c07f2f3f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:59.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.600+0000 7fb3a1d9b640 1 -- 172.21.15.47:0/4022029971 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb39c031600 con 0x7fb3a4071260 2024-04-26T23:19:59.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.600+0000 7fb3a1d9b640 1 -- 172.21.15.47:0/4022029971 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb39c0317a0 con 0x7fb3a4071260 2024-04-26T23:19:59.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.600+0000 7fb3a1d9b640 1 -- 172.21.15.47:0/4022029971 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb39c031b70 con 0x7fb3a4071260 2024-04-26T23:19:59.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.601+0000 7fb3a8d9d640 1 -- 172.21.15.47:0/4022029971 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3a4071260 msgr2=0x7fb3a415e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:59.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.601+0000 7fb3a8d9d640 1 --2- 172.21.15.47:0/4022029971 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3a4071260 0x7fb3a415e9f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fb39c009d90 tx=0x7fb39c02a610 comp rx=0 tx=0).stop 2024-04-26T23:19:59.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.601+0000 7fb3a8d9d640 1 -- 172.21.15.47:0/4022029971 shutdown_connections 2024-04-26T23:19:59.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.601+0000 7fb3a8d9d640 1 --2- 172.21.15.47:0/4022029971 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3a4071260 0x7fb3a415e9f0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:59.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.601+0000 7fb3a8d9d640 1 -- 172.21.15.47:0/4022029971 >> 172.21.15.47:0/4022029971 conn(0x7fb3a40d2d80 msgr2=0x7fb3a40d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:59.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.601+0000 7fb3a8d9d640 1 -- 172.21.15.47:0/4022029971 shutdown_connections 2024-04-26T23:19:59.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.601+0000 7fb3a8d9d640 1 -- 172.21.15.47:0/4022029971 wait complete. 2024-04-26T23:19:59.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.602+0000 7fb3a8d9d640 1 Processor -- start 2024-04-26T23:19:59.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.602+0000 7fb3a8d9d640 1 -- start start 2024-04-26T23:19:59.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.603+0000 7fb3a8d9d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3a4070350 0x7fb3a40c90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:59.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.603+0000 7fb3a8d9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3a4071260 0x7fb3a40c6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:59.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.603+0000 7fb3a8d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3a40cfde0 0x7fb3a41f8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:59.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.603+0000 7fb3a8d9d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a406f1c0 con 0x7fb3a40cfde0 2024-04-26T23:19:59.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.603+0000 7fb3a8d9d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a406dd20 con 0x7fb3a4071260 2024-04-26T23:19:59.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.603+0000 7fb3a8d9d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb3a406f340 con 0x7fb3a4070350 2024-04-26T23:19:59.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.603+0000 7fb3a2d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3a40cfde0 0x7fb3a41f8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:59.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.603+0000 7fb3a2d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3a40cfde0 0x7fb3a41f8f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38176/0 (socket says 172.21.15.47:38176) 2024-04-26T23:19:59.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.603+0000 7fb3a2d9d640 1 -- 172.21.15.47:0/2202996085 learned_addr learned my addr 172.21.15.47:0/2202996085 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:59.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.603+0000 7fb39bfff640 1 --2- 172.21.15.47:0/2202996085 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3a4071260 0x7fb3a40c6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:59.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.603+0000 7fb3a259c640 1 --2- 172.21.15.47:0/2202996085 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3a4070350 0x7fb3a40c90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:59.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.603+0000 7fb3a2d9d640 1 -- 172.21.15.47:0/2202996085 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3a4070350 msgr2=0x7fb3a40c90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:59.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.603+0000 7fb3a2d9d640 1 --2- 172.21.15.47:0/2202996085 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3a4070350 0x7fb3a40c90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:59.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.603+0000 7fb3a2d9d640 1 -- 172.21.15.47:0/2202996085 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3a4071260 msgr2=0x7fb3a40c6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:59.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.603+0000 7fb3a2d9d640 1 --2- 172.21.15.47:0/2202996085 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3a4071260 0x7fb3a40c6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:59.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.603+0000 7fb3a2d9d640 1 -- 172.21.15.47:0/2202996085 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3a41f55b0 con 0x7fb3a40cfde0 2024-04-26T23:19:59.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.603+0000 7fb3a2d9d640 1 --2- 172.21.15.47:0/2202996085 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3a40cfde0 0x7fb3a41f8f10 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb39c009d90 tx=0x7fb39c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:59.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.604+0000 7fb39affd640 1 -- 172.21.15.47:0/2202996085 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb39c040070 con 0x7fb3a40cfde0 2024-04-26T23:19:59.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.604+0000 7fb39affd640 1 -- 172.21.15.47:0/2202996085 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb39c030b60 con 0x7fb3a40cfde0 2024-04-26T23:19:59.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.604+0000 7fb3a8d9d640 1 -- 172.21.15.47:0/2202996085 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3a40c9810 con 0x7fb3a40cfde0 2024-04-26T23:19:59.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.604+0000 7fb39affd640 1 -- 172.21.15.47:0/2202996085 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb39c03c420 con 0x7fb3a40cfde0 2024-04-26T23:19:59.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.604+0000 7fb3a8d9d640 1 -- 172.21.15.47:0/2202996085 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3a415c0e0 con 0x7fb3a40cfde0 2024-04-26T23:19:59.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.605+0000 7fb3a8d9d640 1 -- 172.21.15.47:0/2202996085 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3a40d3bf0 con 0x7fb3a40cfde0 2024-04-26T23:19:59.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.605+0000 7fb39affd640 1 -- 172.21.15.47:0/2202996085 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb39c039040 con 0x7fb3a40cfde0 2024-04-26T23:19:59.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.606+0000 7fb39affd640 1 --2- 172.21.15.47:0/2202996085 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb38406c8f0 0x7fb38406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:59.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.606+0000 7fb39affd640 1 -- 172.21.15.47:0/2202996085 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7fb39c0ad130 con 0x7fb3a40cfde0 2024-04-26T23:19:59.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.606+0000 7fb3a259c640 1 --2- 172.21.15.47:0/2202996085 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb38406c8f0 0x7fb38406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:59.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.606+0000 7fb3a259c640 1 --2- 172.21.15.47:0/2202996085 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb38406c8f0 0x7fb38406edb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb394004f60 tx=0x7fb394007e30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:59.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.610+0000 7fb39affd640 1 -- 172.21.15.47:0/2202996085 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb3a40d3bf0 con 0x7fb3a40cfde0 2024-04-26T23:19:59.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.797+0000 7fb3a8d9d640 1 -- 172.21.15.47:0/2202996085 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb3a40694f0 con 0x7fb3a40cfde0 2024-04-26T23:19:59.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.798+0000 7fb39affd640 1 -- 172.21.15.47:0/2202996085 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v41) ==== 100+0+11 (secure 0 0 0) 0x7fb39c07ad50 con 0x7fb3a40cfde0 2024-04-26T23:19:59.800 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-26T23:19:59.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.800+0000 7fb3a8d9d640 1 -- 172.21.15.47:0/2202996085 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb38406c8f0 msgr2=0x7fb38406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:59.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.800+0000 7fb3a8d9d640 1 --2- 172.21.15.47:0/2202996085 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb38406c8f0 0x7fb38406edb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb394004f60 tx=0x7fb394007e30 comp rx=0 tx=0).stop 2024-04-26T23:19:59.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.800+0000 7fb3a8d9d640 1 -- 172.21.15.47:0/2202996085 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3a40cfde0 msgr2=0x7fb3a41f8f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:59.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.800+0000 7fb3a8d9d640 1 --2- 172.21.15.47:0/2202996085 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3a40cfde0 0x7fb3a41f8f10 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb39c009d90 tx=0x7fb39c004a40 comp rx=0 tx=0).stop 2024-04-26T23:19:59.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.800+0000 7fb3a8d9d640 1 -- 172.21.15.47:0/2202996085 shutdown_connections 2024-04-26T23:19:59.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.800+0000 7fb3a8d9d640 1 --2- 172.21.15.47:0/2202996085 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb38406c8f0 0x7fb38406edb0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:59.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.800+0000 7fb3a8d9d640 1 --2- 172.21.15.47:0/2202996085 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3a40cfde0 0x7fb3a41f8f10 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:59.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.800+0000 7fb3a8d9d640 1 --2- 172.21.15.47:0/2202996085 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3a4071260 0x7fb3a40c6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:59.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.800+0000 7fb3a8d9d640 1 --2- 172.21.15.47:0/2202996085 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3a4070350 0x7fb3a40c90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:59.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.800+0000 7fb3a8d9d640 1 -- 172.21.15.47:0/2202996085 >> 172.21.15.47:0/2202996085 conn(0x7fb3a40d2d80 msgr2=0x7fb3a40b1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:59.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.800+0000 7fb3a8d9d640 1 -- 172.21.15.47:0/2202996085 shutdown_connections 2024-04-26T23:19:59.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.800+0000 7fb3a8d9d640 1 -- 172.21.15.47:0/2202996085 wait complete. 2024-04-26T23:19:59.813 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2024-04-26T23:19:59.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.916+0000 7f5ef0e59640 1 Processor -- start 2024-04-26T23:19:59.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.916+0000 7f5ef0e59640 1 -- start start 2024-04-26T23:19:59.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.917+0000 7f5ef0e59640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5eec15f170 0x7f5eec0d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:59.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.917+0000 7f5ef0e59640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5eec0d1040 con 0x7f5eec15f550 2024-04-26T23:19:59.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.917+0000 7f5ef0e59640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5eec0c9680 con 0x7f5eec0d3bc0 2024-04-26T23:19:59.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.917+0000 7f5ef0e59640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f5eec0c9430 con 0x7f5eec15f170 2024-04-26T23:19:59.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.917+0000 7f5ee9d9b640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f5eec15f550 0x7f5eec0d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:48836/0 (socket says 172.21.15.47:48836) 2024-04-26T23:19:59.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.917+0000 7f5ee9d9b640 1 -- 172.21.15.47:0/3146533146 learned_addr learned my addr 172.21.15.47:0/3146533146 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:19:59.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.917+0000 7f5eea59c640 1 --2- 172.21.15.47:0/3146533146 >> v2:172.21.15.203:3300/0 conn(0x7f5eec15f170 0x7f5eec0d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:59.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.917+0000 7f5ee959a640 1 -- 172.21.15.47:0/3146533146 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1427717644 0 0) 0x7f5ee0001110 con 0x7f5eec15f550 2024-04-26T23:19:59.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.917+0000 7f5ee959a640 1 -- 172.21.15.47:0/3146533146 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ed4003520 con 0x7f5eec15f550 2024-04-26T23:19:59.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.918+0000 7f5ee959a640 1 -- 172.21.15.47:0/3146533146 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3791641832 0 0) 0x7f5eec0c9680 con 0x7f5eec0d3bc0 2024-04-26T23:19:59.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.918+0000 7f5ee959a640 1 -- 172.21.15.47:0/3146533146 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ee0001110 con 0x7f5eec0d3bc0 2024-04-26T23:19:59.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.918+0000 7f5ee959a640 1 -- 172.21.15.47:0/3146533146 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3010949134 0 0) 0x7f5ed4003520 con 0x7f5eec15f550 2024-04-26T23:19:59.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.918+0000 7f5ee959a640 1 -- 172.21.15.47:0/3146533146 >> v2:172.21.15.203:3300/0 conn(0x7f5eec15f170 msgr2=0x7f5eec0d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:59.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.918+0000 7f5ee959a640 1 --2- 172.21.15.47:0/3146533146 >> v2:172.21.15.203:3300/0 conn(0x7f5eec15f170 0x7f5eec0d5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:59.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.918+0000 7f5ee959a640 1 -- 172.21.15.47:0/3146533146 >> v1:172.21.15.203:6789/0 conn(0x7f5eec0d3bc0 legacy=0x7f5eec0ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:59.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.918+0000 7f5ee959a640 1 -- 172.21.15.47:0/3146533146 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eec0c9960 con 0x7f5eec15f550 2024-04-26T23:19:59.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.918+0000 7f5ee959a640 1 -- 172.21.15.47:0/3146533146 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7f5ee0001840 con 0x7f5eec15f550 2024-04-26T23:19:59.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.918+0000 7f5ee959a640 1 -- 172.21.15.47:0/3146533146 >> v1:172.21.15.47:6789/0 conn(0x7f5eec15f550 legacy=0x7f5eec0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:59.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.918+0000 7f5ee959a640 1 --2- 172.21.15.47:0/3146533146 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ed4003b40 0x7f5ed4003f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:59.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.918+0000 7f5ee959a640 1 --2- 172.21.15.47:0/3146533146 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ed4004800 0x7f5ed4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:59.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.918+0000 7f5ee959a640 1 --2- 172.21.15.47:0/3146533146 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ed40051c0 0x7f5ed400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:59.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.918+0000 7f5ee959a640 1 -- 172.21.15.47:0/3146533146 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5eec0c9430 con 0x7f5ed4004800 2024-04-26T23:19:59.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.918+0000 7f5ee959a640 1 -- 172.21.15.47:0/3146533146 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed400c060 con 0x7f5ed4003b40 2024-04-26T23:19:59.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.918+0000 7f5ee959a640 1 -- 172.21.15.47:0/3146533146 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f5ed400c260 con 0x7f5ed40051c0 2024-04-26T23:19:59.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.918+0000 7f5ee9d9b640 1 --2- 172.21.15.47:0/3146533146 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ed4004800 0x7f5ed4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:59.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.918+0000 7f5eea59c640 1 --2- 172.21.15.47:0/3146533146 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ed4003b40 0x7f5ed4003f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:59.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.918+0000 7f5eead9d640 1 --2- 172.21.15.47:0/3146533146 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ed40051c0 0x7f5ed400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:59.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.918+0000 7f5ee9d9b640 1 -- 172.21.15.47:0/3146533146 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ed40051c0 msgr2=0x7f5ed400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:59.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.918+0000 7f5ee9d9b640 1 --2- 172.21.15.47:0/3146533146 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ed40051c0 0x7f5ed400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:59.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.918+0000 7f5ee9d9b640 1 -- 172.21.15.47:0/3146533146 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ed4003b40 msgr2=0x7f5ed4003f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:19:59.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.918+0000 7f5ee9d9b640 1 --2- 172.21.15.47:0/3146533146 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ed4003b40 0x7f5ed4003f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:59.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.918+0000 7f5ee9d9b640 1 -- 172.21.15.47:0/3146533146 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ed4003520 con 0x7f5ed4004800 2024-04-26T23:19:59.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.918+0000 7f5eead9d640 1 -- 172.21.15.47:0/3146533146 reap_dead start 2024-04-26T23:19:59.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.919+0000 7f5ee9d9b640 1 --2- 172.21.15.47:0/3146533146 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ed4004800 0x7f5ed4004c80 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f5eec0c8990 tx=0x7f5ed4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19d9e07ef6dc3de1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:59.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.919+0000 7f5ee959a640 1 -- 172.21.15.47:0/3146533146 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0036070 con 0x7f5ed4004800 2024-04-26T23:19:59.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.919+0000 7f5ee959a640 1 -- 172.21.15.47:0/3146533146 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ee0005ad0 con 0x7f5ed4004800 2024-04-26T23:19:59.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.919+0000 7f5ee959a640 1 -- 172.21.15.47:0/3146533146 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee000c0e0 con 0x7f5ed4004800 2024-04-26T23:19:59.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.919+0000 7f5ef0e59640 1 -- 172.21.15.47:0/3146533146 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ed4004800 msgr2=0x7f5ed4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:59.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.919+0000 7f5ef0e59640 1 --2- 172.21.15.47:0/3146533146 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ed4004800 0x7f5ed4004c80 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f5eec0c8990 tx=0x7f5ed4005830 comp rx=0 tx=0).stop 2024-04-26T23:19:59.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.920+0000 7f5ef0e59640 1 -- 172.21.15.47:0/3146533146 shutdown_connections 2024-04-26T23:19:59.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.920+0000 7f5ef0e59640 1 --2- 172.21.15.47:0/3146533146 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ed4004800 0x7f5ed4004c80 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:59.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.920+0000 7f5ef0e59640 1 -- 172.21.15.47:0/3146533146 >> 172.21.15.47:0/3146533146 conn(0x7f5eec0d4f60 msgr2=0x7f5eec0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:19:59.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.920+0000 7f5ef0e59640 1 -- 172.21.15.47:0/3146533146 shutdown_connections 2024-04-26T23:19:59.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.920+0000 7f5ef0e59640 1 -- 172.21.15.47:0/3146533146 wait complete. 2024-04-26T23:19:59.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.921+0000 7f5ef0e59640 1 Processor -- start 2024-04-26T23:19:59.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.921+0000 7f5ef0e59640 1 -- start start 2024-04-26T23:19:59.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.922+0000 7f5ef0e59640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ed4004800 0x7f5eec0ba850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:59.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.922+0000 7f5ef0e59640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5eec14cb20 0x7f5eec14e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:59.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.922+0000 7f5ef0e59640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5eec14ee20 0x7f5eec14beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:59.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.922+0000 7f5ef0e59640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5eec0d5c00 con 0x7f5eec14cb20 2024-04-26T23:19:59.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.922+0000 7f5ef0e59640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5eec0d5a80 con 0x7f5ed4004800 2024-04-26T23:19:59.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.922+0000 7f5ef0e59640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f5eec0d5d80 con 0x7f5eec14ee20 2024-04-26T23:19:59.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.922+0000 7f5ee9d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5eec14cb20 0x7f5eec14e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:59.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.922+0000 7f5ee9d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5eec14cb20 0x7f5eec14e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57740/0 (socket says 172.21.15.47:57740) 2024-04-26T23:19:59.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.922+0000 7f5ee9d9b640 1 -- 172.21.15.47:0/2353568893 learned_addr learned my addr 172.21.15.47:0/2353568893 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:19:59.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.922+0000 7f5eea59c640 1 --2- 172.21.15.47:0/2353568893 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ed4004800 0x7f5eec0ba850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:59.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.922+0000 7f5eead9d640 1 --2- 172.21.15.47:0/2353568893 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5eec14ee20 0x7f5eec14beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:59.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.922+0000 7f5ee9d9b640 1 -- 172.21.15.47:0/2353568893 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5eec14ee20 msgr2=0x7f5eec14beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:59.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.922+0000 7f5ee9d9b640 1 --2- 172.21.15.47:0/2353568893 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5eec14ee20 0x7f5eec14beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:59.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.922+0000 7f5ee9d9b640 1 -- 172.21.15.47:0/2353568893 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ed4004800 msgr2=0x7f5eec0ba850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:19:59.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.922+0000 7f5ee9d9b640 1 --2- 172.21.15.47:0/2353568893 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ed4004800 0x7f5eec0ba850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:19:59.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.922+0000 7f5ee9d9b640 1 -- 172.21.15.47:0/2353568893 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eec14d300 con 0x7f5eec14cb20 2024-04-26T23:19:59.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.922+0000 7f5eead9d640 1 --2- 172.21.15.47:0/2353568893 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5eec14ee20 0x7f5eec14beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:19:59.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.922+0000 7f5ee9d9b640 1 --2- 172.21.15.47:0/2353568893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5eec14cb20 0x7f5eec14e3a0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0034040 tx=0x7f5eec0c9850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:59.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.922+0000 7f5edaffd640 1 -- 172.21.15.47:0/2353568893 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0041020 con 0x7f5eec14cb20 2024-04-26T23:19:59.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.922+0000 7f5edaffd640 1 -- 172.21.15.47:0/2353568893 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ee00049f0 con 0x7f5eec14cb20 2024-04-26T23:19:59.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.922+0000 7f5ef0e59640 1 -- 172.21.15.47:0/2353568893 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5eec14d5e0 con 0x7f5eec14cb20 2024-04-26T23:19:59.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.922+0000 7f5edaffd640 1 -- 172.21.15.47:0/2353568893 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0036030 con 0x7f5eec14cb20 2024-04-26T23:19:59.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.922+0000 7f5ef0e59640 1 -- 172.21.15.47:0/2353568893 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5eec14dba0 con 0x7f5eec14cb20 2024-04-26T23:19:59.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.923+0000 7f5ef0e59640 1 -- 172.21.15.47:0/2353568893 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5eec0694f0 con 0x7f5eec14cb20 2024-04-26T23:19:59.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.924+0000 7f5edaffd640 1 -- 172.21.15.47:0/2353568893 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5ee0044020 con 0x7f5eec14cb20 2024-04-26T23:19:59.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.924+0000 7f5edaffd640 1 --2- 172.21.15.47:0/2353568893 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f5ecc06c920 0x7f5ecc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:19:59.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.924+0000 7f5edaffd640 1 -- 172.21.15.47:0/2353568893 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f5ee00ac120 con 0x7f5eec14cb20 2024-04-26T23:19:59.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.924+0000 7f5eea59c640 1 --2- 172.21.15.47:0/2353568893 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f5ecc06c920 0x7f5ecc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:19:59.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.925+0000 7f5eea59c640 1 --2- 172.21.15.47:0/2353568893 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f5ecc06c920 0x7f5ecc06ede0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5eec0b9f00 tx=0x7f5edc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:19:59.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:19:59.929+0000 7f5edaffd640 1 -- 172.21.15.47:0/2353568893 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5eec0d1040 con 0x7f5eec14cb20 2024-04-26T23:20:00.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.113+0000 7f5ef0e59640 1 -- 172.21.15.47:0/2353568893 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) -- 0x7f5eec0694f0 con 0x7f5eec14cb20 2024-04-26T23:20:00.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.113+0000 7f5edaffd640 1 -- 172.21.15.47:0/2353568893 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v41) ==== 97+0+11 (secure 0 0 0) 0x7f5ee0079b70 con 0x7f5eec14cb20 2024-04-26T23:20:00.115 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-26T23:20:00.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.115+0000 7f5ef0e59640 1 -- 172.21.15.47:0/2353568893 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f5ecc06c920 msgr2=0x7f5ecc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:00.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.115+0000 7f5ef0e59640 1 --2- 172.21.15.47:0/2353568893 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f5ecc06c920 0x7f5ecc06ede0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5eec0b9f00 tx=0x7f5edc008040 comp rx=0 tx=0).stop 2024-04-26T23:20:00.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.115+0000 7f5ef0e59640 1 -- 172.21.15.47:0/2353568893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5eec14cb20 msgr2=0x7f5eec14e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:00.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.115+0000 7f5ef0e59640 1 --2- 172.21.15.47:0/2353568893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5eec14cb20 0x7f5eec14e3a0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0034040 tx=0x7f5eec0c9850 comp rx=0 tx=0).stop 2024-04-26T23:20:00.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.115+0000 7f5ef0e59640 1 -- 172.21.15.47:0/2353568893 shutdown_connections 2024-04-26T23:20:00.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.115+0000 7f5ef0e59640 1 --2- 172.21.15.47:0/2353568893 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f5ecc06c920 0x7f5ecc06ede0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:00.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.115+0000 7f5ef0e59640 1 --2- 172.21.15.47:0/2353568893 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5eec14ee20 0x7f5eec14beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:00.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.115+0000 7f5ef0e59640 1 --2- 172.21.15.47:0/2353568893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5eec14cb20 0x7f5eec14e3a0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:00.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.115+0000 7f5ef0e59640 1 --2- 172.21.15.47:0/2353568893 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ed4004800 0x7f5eec0ba850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:00.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.115+0000 7f5ef0e59640 1 -- 172.21.15.47:0/2353568893 >> 172.21.15.47:0/2353568893 conn(0x7f5eec0d4f60 msgr2=0x7f5eec0d1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:00.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.115+0000 7f5ef0e59640 1 -- 172.21.15.47:0/2353568893 shutdown_connections 2024-04-26T23:20:00.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.115+0000 7f5ef0e59640 1 -- 172.21.15.47:0/2353568893 wait complete. 2024-04-26T23:20:00.128 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T23:20:00.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.229+0000 7fd4f4704640 1 Processor -- start 2024-04-26T23:20:00.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.229+0000 7fd4f4704640 1 -- start start 2024-04-26T23:20:00.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.230+0000 7fd4f4704640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd4ec162880 0x7fd4ec0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:00.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.230+0000 7fd4f4704640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd4ec0b5f20 0x7fd4ec0d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:00.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.230+0000 7fd4f4704640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4ec0b7e50 con 0x7fd4ec0d39f0 2024-04-26T23:20:00.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.230+0000 7fd4f4704640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fd4ec0d3e40 con 0x7fd4ec162880 2024-04-26T23:20:00.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.230+0000 7fd4f4704640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fd4ec0d0580 con 0x7fd4ec0b5f20 2024-04-26T23:20:00.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.230+0000 7fd4f24a0640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd4ec162880 0x7fd4ec0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:00.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.230+0000 7fd4f24a0640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd4ec162880 0x7fd4ec0d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57754/0 (socket says 172.21.15.47:57754) 2024-04-26T23:20:00.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.230+0000 7fd4f24a0640 1 -- 172.21.15.47:0/558424040 learned_addr learned my addr 172.21.15.47:0/558424040 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:00.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.230+0000 7fd4f24a0640 1 -- 172.21.15.47:0/558424040 >> v2:172.21.15.203:3300/0 conn(0x7fd4ec0b5f20 msgr2=0x7fd4ec0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:00.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.230+0000 7fd4f24a0640 1 --2- 172.21.15.47:0/558424040 >> v2:172.21.15.203:3300/0 conn(0x7fd4ec0b5f20 0x7fd4ec0d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:00.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.230+0000 7fd4f24a0640 1 -- 172.21.15.47:0/558424040 >> v1:172.21.15.203:6789/0 conn(0x7fd4ec0d39f0 legacy=0x7fd4ec0b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:00.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.230+0000 7fd4f24a0640 1 -- 172.21.15.47:0/558424040 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4ec0d1800 con 0x7fd4ec162880 2024-04-26T23:20:00.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.231+0000 7fd4f24a0640 1 --2- 172.21.15.47:0/558424040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4ec162880 0x7fd4ec0d2cb0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec08d3d0 tx=0x7fd4e802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcb25c8850ec6e4c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:00.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.231+0000 7fd4f149e640 1 -- 172.21.15.47:0/558424040 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e8037070 con 0x7fd4ec162880 2024-04-26T23:20:00.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.231+0000 7fd4f149e640 1 -- 172.21.15.47:0/558424040 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4e8032440 con 0x7fd4ec162880 2024-04-26T23:20:00.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.231+0000 7fd4f149e640 1 -- 172.21.15.47:0/558424040 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e8032810 con 0x7fd4ec162880 2024-04-26T23:20:00.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.232+0000 7fd4f4704640 1 -- 172.21.15.47:0/558424040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4ec162880 msgr2=0x7fd4ec0d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:00.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.232+0000 7fd4f4704640 1 --2- 172.21.15.47:0/558424040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4ec162880 0x7fd4ec0d2cb0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec08d3d0 tx=0x7fd4e802a230 comp rx=0 tx=0).stop 2024-04-26T23:20:00.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.232+0000 7fd4f4704640 1 -- 172.21.15.47:0/558424040 shutdown_connections 2024-04-26T23:20:00.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.232+0000 7fd4f4704640 1 --2- 172.21.15.47:0/558424040 >> v2:172.21.15.203:3300/0 conn(0x7fd4ec0b5f20 0x7fd4ec0d34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:00.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.232+0000 7fd4f4704640 1 --2- 172.21.15.47:0/558424040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4ec162880 0x7fd4ec0d2cb0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:00.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.232+0000 7fd4f4704640 1 -- 172.21.15.47:0/558424040 >> 172.21.15.47:0/558424040 conn(0x7fd4ec0d4f60 msgr2=0x7fd4ec0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:00.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.232+0000 7fd4f4704640 1 -- 172.21.15.47:0/558424040 shutdown_connections 2024-04-26T23:20:00.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.232+0000 7fd4f4704640 1 -- 172.21.15.47:0/558424040 wait complete. 2024-04-26T23:20:00.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.233+0000 7fd4f4704640 1 Processor -- start 2024-04-26T23:20:00.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.233+0000 7fd4f4704640 1 -- start start 2024-04-26T23:20:00.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.234+0000 7fd4f4704640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4ec0b5f20 0x7fd4ec0b27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:00.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.234+0000 7fd4f4704640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4ec0d39f0 0x7fd4ec0cd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:00.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.234+0000 7fd4f4704640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd4ec162880 0x7fd4ec14f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:00.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.234+0000 7fd4f4704640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ec0d0d90 con 0x7fd4ec0d39f0 2024-04-26T23:20:00.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.234+0000 7fd4f4704640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ec0d0ab0 con 0x7fd4ec0b5f20 2024-04-26T23:20:00.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.234+0000 7fd4f4704640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd4ec162c60 con 0x7fd4ec162880 2024-04-26T23:20:00.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.234+0000 7fd4f1c9f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4ec0d39f0 0x7fd4ec0cd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:00.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.234+0000 7fd4f1c9f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4ec0d39f0 0x7fd4ec0cd9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57764/0 (socket says 172.21.15.47:57764) 2024-04-26T23:20:00.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.234+0000 7fd4f1c9f640 1 -- 172.21.15.47:0/2334696153 learned_addr learned my addr 172.21.15.47:0/2334696153 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:00.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.234+0000 7fd4f24a0640 1 --2- 172.21.15.47:0/2334696153 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4ec0b5f20 0x7fd4ec0b27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:00.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.234+0000 7fd4f2ca1640 1 --2- 172.21.15.47:0/2334696153 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd4ec162880 0x7fd4ec14f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:00.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.234+0000 7fd4f1c9f640 1 -- 172.21.15.47:0/2334696153 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd4ec162880 msgr2=0x7fd4ec14f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:00.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.234+0000 7fd4f1c9f640 1 --2- 172.21.15.47:0/2334696153 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd4ec162880 0x7fd4ec14f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:00.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.234+0000 7fd4f1c9f640 1 -- 172.21.15.47:0/2334696153 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4ec0b5f20 msgr2=0x7fd4ec0b27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:00.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.234+0000 7fd4f1c9f640 1 --2- 172.21.15.47:0/2334696153 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4ec0b5f20 0x7fd4ec0b27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:00.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.234+0000 7fd4f1c9f640 1 -- 172.21.15.47:0/2334696153 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4ec14f8e0 con 0x7fd4ec0d39f0 2024-04-26T23:20:00.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.234+0000 7fd4f24a0640 1 --2- 172.21.15.47:0/2334696153 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4ec0b5f20 0x7fd4ec0b27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:00.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.234+0000 7fd4f1c9f640 1 --2- 172.21.15.47:0/2334696153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4ec0d39f0 0x7fd4ec0cd9b0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc00b440 tx=0x7fd4dc00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:00.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.235+0000 7fd4e2ffd640 1 -- 172.21.15.47:0/2334696153 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc015020 con 0x7fd4ec0d39f0 2024-04-26T23:20:00.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.235+0000 7fd4e2ffd640 1 -- 172.21.15.47:0/2334696153 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4dc011890 con 0x7fd4ec0d39f0 2024-04-26T23:20:00.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.235+0000 7fd4f4704640 1 -- 172.21.15.47:0/2334696153 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4ec0ce150 con 0x7fd4ec0d39f0 2024-04-26T23:20:00.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.235+0000 7fd4e2ffd640 1 -- 172.21.15.47:0/2334696153 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc011c80 con 0x7fd4ec0d39f0 2024-04-26T23:20:00.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.235+0000 7fd4f4704640 1 -- 172.21.15.47:0/2334696153 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4ec0ce710 con 0x7fd4ec0d39f0 2024-04-26T23:20:00.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.236+0000 7fd4f4704640 1 -- 172.21.15.47:0/2334696153 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4ec0694f0 con 0x7fd4ec0d39f0 2024-04-26T23:20:00.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.236+0000 7fd4e2ffd640 1 -- 172.21.15.47:0/2334696153 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd4dc018020 con 0x7fd4ec0d39f0 2024-04-26T23:20:00.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.237+0000 7fd4e2ffd640 1 --2- 172.21.15.47:0/2334696153 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd4d006c8f0 0x7fd4d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:00.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.237+0000 7fd4e2ffd640 1 -- 172.21.15.47:0/2334696153 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7fd4dc08bec0 con 0x7fd4ec0d39f0 2024-04-26T23:20:00.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.237+0000 7fd4f24a0640 1 --2- 172.21.15.47:0/2334696153 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd4d006c8f0 0x7fd4d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:00.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.237+0000 7fd4f24a0640 1 --2- 172.21.15.47:0/2334696153 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd4d006c8f0 0x7fd4d006edb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8034000 tx=0x7fd4e802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:00.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.241+0000 7fd4e2ffd640 1 -- 172.21.15.47:0/2334696153 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd4dc01f020 con 0x7fd4ec0d39f0 2024-04-26T23:20:00.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.444+0000 7fd4f4704640 1 -- 172.21.15.47:0/2334696153 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd4ec0ce910 con 0x7fd4ec0d39f0 2024-04-26T23:20:00.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.445+0000 7fd4e2ffd640 1 -- 172.21.15.47:0/2334696153 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v41) ==== 80+0+0 (secure 0 0 0) 0x7fd4ec14f8e0 con 0x7fd4ec0d39f0 2024-04-26T23:20:00.447 INFO:teuthology.orchestra.run.smithi047.stderr:listed 0 entries 2024-04-26T23:20:00.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.447+0000 7fd4f4704640 1 -- 172.21.15.47:0/2334696153 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd4d006c8f0 msgr2=0x7fd4d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:00.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.447+0000 7fd4f4704640 1 --2- 172.21.15.47:0/2334696153 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd4d006c8f0 0x7fd4d006edb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8034000 tx=0x7fd4e802e000 comp rx=0 tx=0).stop 2024-04-26T23:20:00.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.447+0000 7fd4f4704640 1 -- 172.21.15.47:0/2334696153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4ec0d39f0 msgr2=0x7fd4ec0cd9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:00.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.447+0000 7fd4f4704640 1 --2- 172.21.15.47:0/2334696153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4ec0d39f0 0x7fd4ec0cd9b0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc00b440 tx=0x7fd4dc00b900 comp rx=0 tx=0).stop 2024-04-26T23:20:00.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.447+0000 7fd4f4704640 1 -- 172.21.15.47:0/2334696153 shutdown_connections 2024-04-26T23:20:00.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.447+0000 7fd4f4704640 1 --2- 172.21.15.47:0/2334696153 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd4d006c8f0 0x7fd4d006edb0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:00.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.447+0000 7fd4f4704640 1 --2- 172.21.15.47:0/2334696153 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd4ec162880 0x7fd4ec14f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:00.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.447+0000 7fd4f4704640 1 --2- 172.21.15.47:0/2334696153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4ec0d39f0 0x7fd4ec0cd9b0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:00.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.447+0000 7fd4f4704640 1 --2- 172.21.15.47:0/2334696153 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4ec0b5f20 0x7fd4ec0b27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:00.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.447+0000 7fd4f4704640 1 -- 172.21.15.47:0/2334696153 >> 172.21.15.47:0/2334696153 conn(0x7fd4ec0d4f60 msgr2=0x7fd4ec160ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:00.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.447+0000 7fd4f4704640 1 -- 172.21.15.47:0/2334696153 shutdown_connections 2024-04-26T23:20:00.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.447+0000 7fd4f4704640 1 -- 172.21.15.47:0/2334696153 wait complete. 2024-04-26T23:20:00.460 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T23:20:00.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:20:00.460 DEBUG:teuthology.orchestra.run.smithi047:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T23:20:00.489 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi047.front.sepia.ceph.com 2024-04-26T23:20:00.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:20:00.490 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T23:20:00.545 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds 'allow rwp' mon 'allow r' osd 'allow rw pool=cephfs_data,allow rw pool=data_new' 2024-04-26T23:20:00.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.691+0000 7f853bd89640 1 Processor -- start 2024-04-26T23:20:00.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.691+0000 7f853bd89640 1 -- start start 2024-04-26T23:20:00.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.692+0000 7f853bd89640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f85340b0380 0x7f85340ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:00.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.692+0000 7f853bd89640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85340cfd80 con 0x7f85340b10b0 2024-04-26T23:20:00.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.692+0000 7f853bd89640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85340d0020 con 0x7f853406c950 2024-04-26T23:20:00.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.692+0000 7f853bd89640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f85340af8e0 con 0x7f85340b0380 2024-04-26T23:20:00.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.692+0000 7f8539b25640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f85340b0380 0x7f85340ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:00.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.692+0000 7f8539324640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f85340b10b0 0x7f853406c240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:48850/0 (socket says 172.21.15.47:48850) 2024-04-26T23:20:00.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.692+0000 7f8539324640 1 -- 172.21.15.47:0/4263324531 learned_addr learned my addr 172.21.15.47:0/4263324531 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:00.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.693+0000 7f8539b25640 1 -- 172.21.15.47:0/4263324531 >> v1:172.21.15.203:6789/0 conn(0x7f853406c950 legacy=0x7f853415e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:00.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.693+0000 7f8539b25640 1 -- 172.21.15.47:0/4263324531 >> v1:172.21.15.47:6789/0 conn(0x7f85340b10b0 legacy=0x7f853406c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:00.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.693+0000 7f8538b23640 1 -- 172.21.15.47:0/4263324531 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1237057976 0 0) 0x7f85340cfd80 con 0x7f85340b10b0 2024-04-26T23:20:00.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.693+0000 7f8539b25640 1 -- 172.21.15.47:0/4263324531 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85340d0300 con 0x7f85340b0380 2024-04-26T23:20:00.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.693+0000 7f8539b25640 1 --2- 172.21.15.47:0/4263324531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85340b0380 0x7f85340ccd50 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f85300097a0 tx=0x7f853002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c9c84923d186aa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:00.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.693+0000 7f8538b23640 1 -- 172.21.15.47:0/4263324531 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8530037070 con 0x7f85340b0380 2024-04-26T23:20:00.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.693+0000 7f8538b23640 1 -- 172.21.15.47:0/4263324531 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f853002ae50 con 0x7f85340b0380 2024-04-26T23:20:00.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.693+0000 7f8538b23640 1 -- 172.21.15.47:0/4263324531 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8530032660 con 0x7f85340b0380 2024-04-26T23:20:00.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.694+0000 7f853bd89640 1 -- 172.21.15.47:0/4263324531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85340b0380 msgr2=0x7f85340ccd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:00.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.694+0000 7f853bd89640 1 --2- 172.21.15.47:0/4263324531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85340b0380 0x7f85340ccd50 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f85300097a0 tx=0x7f853002a140 comp rx=0 tx=0).stop 2024-04-26T23:20:00.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.694+0000 7f853bd89640 1 -- 172.21.15.47:0/4263324531 shutdown_connections 2024-04-26T23:20:00.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.694+0000 7f853bd89640 1 --2- 172.21.15.47:0/4263324531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85340b0380 0x7f85340ccd50 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:00.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.694+0000 7f853bd89640 1 -- 172.21.15.47:0/4263324531 >> 172.21.15.47:0/4263324531 conn(0x7f85340d1cf0 msgr2=0x7f85340d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:00.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.694+0000 7f853bd89640 1 -- 172.21.15.47:0/4263324531 shutdown_connections 2024-04-26T23:20:00.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.694+0000 7f853bd89640 1 -- 172.21.15.47:0/4263324531 wait complete. 2024-04-26T23:20:00.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.695+0000 7f853bd89640 1 Processor -- start 2024-04-26T23:20:00.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.695+0000 7f853bd89640 1 -- start start 2024-04-26T23:20:00.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.696+0000 7f853bd89640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f853406c950 0x7f8534153b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:00.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.696+0000 7f853bd89640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85340b0380 0x7f85341540c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:00.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.696+0000 7f853bd89640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85340b10b0 0x7f853414d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:00.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.696+0000 7f853bd89640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f85340b0760 con 0x7f85340b0380 2024-04-26T23:20:00.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.696+0000 7f853bd89640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f85340d1980 con 0x7f853406c950 2024-04-26T23:20:00.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.696+0000 7f853bd89640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f85340d15c0 con 0x7f85340b10b0 2024-04-26T23:20:00.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.696+0000 7f8539324640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85340b0380 0x7f85341540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:00.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.696+0000 7f8539324640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85340b0380 0x7f85341540c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57776/0 (socket says 172.21.15.47:57776) 2024-04-26T23:20:00.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.696+0000 7f8539324640 1 -- 172.21.15.47:0/947755505 learned_addr learned my addr 172.21.15.47:0/947755505 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:00.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.696+0000 7f8539b25640 1 --2- 172.21.15.47:0/947755505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f853406c950 0x7f8534153b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:00.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.696+0000 7f853a326640 1 --2- 172.21.15.47:0/947755505 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85340b10b0 0x7f853414d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:00.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.696+0000 7f8539324640 1 -- 172.21.15.47:0/947755505 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85340b10b0 msgr2=0x7f853414d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:00.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.696+0000 7f8539324640 1 --2- 172.21.15.47:0/947755505 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85340b10b0 0x7f853414d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:00.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.696+0000 7f8539324640 1 -- 172.21.15.47:0/947755505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f853406c950 msgr2=0x7f8534153b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:00.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.696+0000 7f8539324640 1 --2- 172.21.15.47:0/947755505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f853406c950 0x7f8534153b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:00.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.697+0000 7f8539324640 1 -- 172.21.15.47:0/947755505 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85340c99c0 con 0x7f85340b0380 2024-04-26T23:20:00.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.697+0000 7f8539b25640 1 --2- 172.21.15.47:0/947755505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f853406c950 0x7f8534153b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:20:00.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.697+0000 7f8539324640 1 --2- 172.21.15.47:0/947755505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85340b0380 0x7f85341540c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f852400c4d0 tx=0x7f852400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:00.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.697+0000 7f852a7fc640 1 -- 172.21.15.47:0/947755505 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8524017070 con 0x7f85340b0380 2024-04-26T23:20:00.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.697+0000 7f852a7fc640 1 -- 172.21.15.47:0/947755505 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8524005d00 con 0x7f85340b0380 2024-04-26T23:20:00.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.697+0000 7f853bd89640 1 -- 172.21.15.47:0/947755505 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85340c9ca0 con 0x7f85340b0380 2024-04-26T23:20:00.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.697+0000 7f852a7fc640 1 -- 172.21.15.47:0/947755505 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85240072b0 con 0x7f85340b0380 2024-04-26T23:20:00.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.697+0000 7f853bd89640 1 -- 172.21.15.47:0/947755505 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85340ca260 con 0x7f85340b0380 2024-04-26T23:20:00.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.698+0000 7f853bd89640 1 -- 172.21.15.47:0/947755505 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85340694f0 con 0x7f85340b0380 2024-04-26T23:20:00.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.698+0000 7f852a7fc640 1 -- 172.21.15.47:0/947755505 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f852401a070 con 0x7f85340b0380 2024-04-26T23:20:00.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.699+0000 7f852a7fc640 1 --2- 172.21.15.47:0/947755505 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f851806c8f0 0x7f851806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:00.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.699+0000 7f852a7fc640 1 -- 172.21.15.47:0/947755505 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f852408b1f0 con 0x7f85340b0380 2024-04-26T23:20:00.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.699+0000 7f8539b25640 1 --2- 172.21.15.47:0/947755505 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f851806c8f0 0x7f851806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:00.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.700+0000 7f8539b25640 1 --2- 172.21.15.47:0/947755505 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f851806c8f0 0x7f851806edb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8530034000 tx=0x7f853002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:00.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.703+0000 7f852a7fc640 1 -- 172.21.15.47:0/947755505 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f85240593a0 con 0x7f85340b0380 2024-04-26T23:20:00.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.962+0000 7f853bd89640 1 -- 172.21.15.47:0/947755505 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rwp", "mon", "allow r", "osd", "allow rw pool=cephfs_data,allow rw pool=data_new"]} v 0) -- 0x7f8534069db0 con 0x7f85340b0380 2024-04-26T23:20:00.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.963+0000 7f852a7fc640 1 -- 172.21.15.47:0/947755505 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rwp", "mon", "allow r", "osd", "allow rw pool=cephfs_data,allow rw pool=data_new"]}]=0 updated caps for client.0 v12) ==== 211+0+168 (secure 0 0 0) 0x7f8524058ca0 con 0x7f85340b0380 2024-04-26T23:20:00.965 INFO:teuthology.orchestra.run.smithi047.stdout:[client.0] 2024-04-26T23:20:00.966 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQAKNixmA5QFHRAAT89WPyzDs0acPluyvU4svw== 2024-04-26T23:20:00.966 INFO:teuthology.orchestra.run.smithi047.stdout: caps mds = "allow rwp" 2024-04-26T23:20:00.966 INFO:teuthology.orchestra.run.smithi047.stdout: caps mon = "allow r" 2024-04-26T23:20:00.966 INFO:teuthology.orchestra.run.smithi047.stdout: caps osd = "allow rw pool=cephfs_data,allow rw pool=data_new" 2024-04-26T23:20:00.967 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.0 2024-04-26T23:20:00.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.965+0000 7f853bd89640 1 -- 172.21.15.47:0/947755505 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f851806c8f0 msgr2=0x7f851806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:00.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.965+0000 7f853bd89640 1 --2- 172.21.15.47:0/947755505 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f851806c8f0 0x7f851806edb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8530034000 tx=0x7f853002e000 comp rx=0 tx=0).stop 2024-04-26T23:20:00.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.965+0000 7f853bd89640 1 -- 172.21.15.47:0/947755505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85340b0380 msgr2=0x7f85341540c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:00.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.965+0000 7f853bd89640 1 --2- 172.21.15.47:0/947755505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85340b0380 0x7f85341540c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f852400c4d0 tx=0x7f852400c990 comp rx=0 tx=0).stop 2024-04-26T23:20:00.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.965+0000 7f853bd89640 1 -- 172.21.15.47:0/947755505 shutdown_connections 2024-04-26T23:20:00.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.965+0000 7f853bd89640 1 --2- 172.21.15.47:0/947755505 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f851806c8f0 0x7f851806edb0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:00.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.965+0000 7f853bd89640 1 --2- 172.21.15.47:0/947755505 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85340b10b0 0x7f853414d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:00.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.965+0000 7f853bd89640 1 --2- 172.21.15.47:0/947755505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85340b0380 0x7f85341540c0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:00.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.965+0000 7f853bd89640 1 --2- 172.21.15.47:0/947755505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f853406c950 0x7f8534153b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:00.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.966+0000 7f853bd89640 1 -- 172.21.15.47:0/947755505 >> 172.21.15.47:0/947755505 conn(0x7f85340d1cf0 msgr2=0x7f85340b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:00.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.966+0000 7f853bd89640 1 -- 172.21.15.47:0/947755505 shutdown_connections 2024-04-26T23:20:00.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:00.966+0000 7f853bd89640 1 -- 172.21.15.47:0/947755505 wait complete. 2024-04-26T23:20:00.978 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T23:20:01.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.077+0000 7f5030cc8640 1 Processor -- start 2024-04-26T23:20:01.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.077+0000 7f5030cc8640 1 -- start start 2024-04-26T23:20:01.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.078+0000 7f5030cc8640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f502c071e60 0x7f502c06f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:01.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.078+0000 7f5030cc8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f502c0cc480 0x7f502c166ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:01.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.078+0000 7f5030cc8640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f502c15d7c0 con 0x7f502c06fe20 2024-04-26T23:20:01.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.078+0000 7f5030cc8640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f502c15daa0 con 0x7f502c071e60 2024-04-26T23:20:01.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.078+0000 7f5030cc8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f502c15dd80 con 0x7f502c0cc480 2024-04-26T23:20:01.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.078+0000 7f502ad9d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f502c0cc480 0x7f502c166ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:01.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.078+0000 7f502a59c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f502c071e60 0x7f502c06f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:01.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.078+0000 7f502a59c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f502c071e60 0x7f502c06f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.47:40152/0 (socket says 172.21.15.47:40152) 2024-04-26T23:20:01.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.078+0000 7f502ad9d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f502c0cc480 0x7f502c166ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.47:60788/0 (socket says 172.21.15.47:60788) 2024-04-26T23:20:01.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.078+0000 7f502a59c640 1 -- 172.21.15.47:0/2731104236 learned_addr learned my addr 172.21.15.47:0/2731104236 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:01.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.078+0000 7f5029d9b640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f502c06fe20 0x7f502c072240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.47:43010/0 (socket says 172.21.15.47:43010) 2024-04-26T23:20:01.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.079+0000 7f502a59c640 1 -- 172.21.15.47:0/2731104236 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f502c0cc480 msgr2=0x7f502c166ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:01.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.079+0000 7f502a59c640 1 --2- 172.21.15.47:0/2731104236 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f502c0cc480 0x7f502c166ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:01.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.079+0000 7f502a59c640 1 -- 172.21.15.47:0/2731104236 >> v1:172.21.15.203:6789/0 conn(0x7f502c06fe20 legacy=0x7f502c072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:01.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.079+0000 7f502a59c640 1 -- 172.21.15.47:0/2731104236 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f502c15e060 con 0x7f502c071e60 2024-04-26T23:20:01.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.079+0000 7f502ad9d640 1 --2- 172.21.15.47:0/2731104236 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f502c0cc480 0x7f502c166ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:20:01.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.079+0000 7f502a59c640 1 --2- 172.21.15.47:0/2731104236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f502c071e60 0x7f502c06f8e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f501c009870 tx=0x7f501c02a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9fcdf6ae96fc605 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:01.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.080+0000 7f502959a640 1 -- 172.21.15.47:0/2731104236 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f501c037070 con 0x7f502c071e60 2024-04-26T23:20:01.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.080+0000 7f502959a640 1 -- 172.21.15.47:0/2731104236 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f501c032440 con 0x7f502c071e60 2024-04-26T23:20:01.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.080+0000 7f502959a640 1 -- 172.21.15.47:0/2731104236 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f501c032810 con 0x7f502c071e60 2024-04-26T23:20:01.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.080+0000 7f5030cc8640 1 -- 172.21.15.47:0/2731104236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f502c071e60 msgr2=0x7f502c06f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:01.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.080+0000 7f5030cc8640 1 --2- 172.21.15.47:0/2731104236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f502c071e60 0x7f502c06f8e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f501c009870 tx=0x7f501c02a270 comp rx=0 tx=0).stop 2024-04-26T23:20:01.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.080+0000 7f5030cc8640 1 -- 172.21.15.47:0/2731104236 shutdown_connections 2024-04-26T23:20:01.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.080+0000 7f5030cc8640 1 --2- 172.21.15.47:0/2731104236 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f502c0cc480 0x7f502c166ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:01.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.080+0000 7f5030cc8640 1 --2- 172.21.15.47:0/2731104236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f502c071e60 0x7f502c06f8e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:01.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.080+0000 7f5030cc8640 1 -- 172.21.15.47:0/2731104236 >> 172.21.15.47:0/2731104236 conn(0x7f502c0d0be0 msgr2=0x7f502c0d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:01.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.080+0000 7f5030cc8640 1 -- 172.21.15.47:0/2731104236 shutdown_connections 2024-04-26T23:20:01.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.080+0000 7f5030cc8640 1 -- 172.21.15.47:0/2731104236 wait complete. 2024-04-26T23:20:01.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.081+0000 7f5030cc8640 1 Processor -- start 2024-04-26T23:20:01.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.082+0000 7f5030cc8640 1 -- start start 2024-04-26T23:20:01.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.082+0000 7f5030cc8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f502c06fe20 0x7f502c15c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:01.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.082+0000 7f5030cc8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f502c071e60 0x7f502c1f3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:01.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.082+0000 7f5030cc8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f502c0cc480 0x7f502c0c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:01.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.082+0000 7f5030cc8640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f502c15d570 con 0x7f502c0cc480 2024-04-26T23:20:01.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.082+0000 7f5030cc8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f502c15d3f0 con 0x7f502c06fe20 2024-04-26T23:20:01.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.082+0000 7f5030cc8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f502c072950 con 0x7f502c071e60 2024-04-26T23:20:01.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.082+0000 7f502ad9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f502c0cc480 0x7f502c0c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:01.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.082+0000 7f502ad9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f502c0cc480 0x7f502c0c9d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57786/0 (socket says 172.21.15.47:57786) 2024-04-26T23:20:01.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.082+0000 7f502ad9d640 1 -- 172.21.15.47:0/1459836670 learned_addr learned my addr 172.21.15.47:0/1459836670 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:01.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.083+0000 7f502a59c640 1 --2- 172.21.15.47:0/1459836670 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f502c06fe20 0x7f502c15c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:01.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.083+0000 7f5029d9b640 1 --2- 172.21.15.47:0/1459836670 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f502c071e60 0x7f502c1f3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:01.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.083+0000 7f502ad9d640 1 -- 172.21.15.47:0/1459836670 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f502c071e60 msgr2=0x7f502c1f3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:01.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.083+0000 7f502ad9d640 1 --2- 172.21.15.47:0/1459836670 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f502c071e60 0x7f502c1f3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:01.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.083+0000 7f502ad9d640 1 -- 172.21.15.47:0/1459836670 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f502c06fe20 msgr2=0x7f502c15c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:01.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.083+0000 7f502ad9d640 1 --2- 172.21.15.47:0/1459836670 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f502c06fe20 0x7f502c15c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:01.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.083+0000 7f502ad9d640 1 -- 172.21.15.47:0/1459836670 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f502c0c8c90 con 0x7f502c0cc480 2024-04-26T23:20:01.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.083+0000 7f502ad9d640 1 --2- 172.21.15.47:0/1459836670 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f502c0cc480 0x7f502c0c9d10 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f502400ef90 tx=0x7f502400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:01.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.083+0000 7f5022ffd640 1 -- 172.21.15.47:0/1459836670 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5024018020 con 0x7f502c0cc480 2024-04-26T23:20:01.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.083+0000 7f5022ffd640 1 -- 172.21.15.47:0/1459836670 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5024014480 con 0x7f502c0cc480 2024-04-26T23:20:01.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.083+0000 7f5022ffd640 1 -- 172.21.15.47:0/1459836670 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5024014850 con 0x7f502c0cc480 2024-04-26T23:20:01.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.083+0000 7f5030cc8640 1 -- 172.21.15.47:0/1459836670 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f502c0c8f50 con 0x7f502c0cc480 2024-04-26T23:20:01.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.083+0000 7f5030cc8640 1 -- 172.21.15.47:0/1459836670 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f502c0c9510 con 0x7f502c0cc480 2024-04-26T23:20:01.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.084+0000 7f5030cc8640 1 -- 172.21.15.47:0/1459836670 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f502c0694f0 con 0x7f502c0cc480 2024-04-26T23:20:01.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.085+0000 7f5022ffd640 1 -- 172.21.15.47:0/1459836670 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5024005d10 con 0x7f502c0cc480 2024-04-26T23:20:01.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.085+0000 7f5022ffd640 1 --2- 172.21.15.47:0/1459836670 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f500c06c8f0 0x7f500c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:01.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.085+0000 7f5022ffd640 1 -- 172.21.15.47:0/1459836670 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f502408c9b0 con 0x7f502c0cc480 2024-04-26T23:20:01.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.085+0000 7f502a59c640 1 --2- 172.21.15.47:0/1459836670 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f500c06c8f0 0x7f500c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:01.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.086+0000 7f502a59c640 1 --2- 172.21.15.47:0/1459836670 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f500c06c8f0 0x7f500c06edb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f501c034000 tx=0x7f501c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:01.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.089+0000 7f5022ffd640 1 -- 172.21.15.47:0/1459836670 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5024022030 con 0x7f502c0cc480 2024-04-26T23:20:01.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.316+0000 7f5030cc8640 1 -- 172.21.15.47:0/1459836670 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f502c159490 con 0x7f502c0cc480 2024-04-26T23:20:01.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.317+0000 7f5022ffd640 1 -- 172.21.15.47:0/1459836670 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v19) ==== 53+0+82 (secure 0 0 0) 0x7f5024010040 con 0x7f502c0cc480 2024-04-26T23:20:01.319 INFO:teuthology.orchestra.run.smithi047.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data data_new ] 2024-04-26T23:20:01.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.319+0000 7f5030cc8640 1 -- 172.21.15.47:0/1459836670 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f500c06c8f0 msgr2=0x7f500c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:01.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.319+0000 7f5030cc8640 1 --2- 172.21.15.47:0/1459836670 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f500c06c8f0 0x7f500c06edb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f501c034000 tx=0x7f501c02e000 comp rx=0 tx=0).stop 2024-04-26T23:20:01.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.319+0000 7f5030cc8640 1 -- 172.21.15.47:0/1459836670 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f502c0cc480 msgr2=0x7f502c0c9d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:01.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.319+0000 7f5030cc8640 1 --2- 172.21.15.47:0/1459836670 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f502c0cc480 0x7f502c0c9d10 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f502400ef90 tx=0x7f502400c490 comp rx=0 tx=0).stop 2024-04-26T23:20:01.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.319+0000 7f5030cc8640 1 -- 172.21.15.47:0/1459836670 shutdown_connections 2024-04-26T23:20:01.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.319+0000 7f5030cc8640 1 --2- 172.21.15.47:0/1459836670 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f500c06c8f0 0x7f500c06edb0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:01.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.319+0000 7f5030cc8640 1 --2- 172.21.15.47:0/1459836670 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f502c0cc480 0x7f502c0c9d10 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:01.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.319+0000 7f5030cc8640 1 --2- 172.21.15.47:0/1459836670 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f502c071e60 0x7f502c1f3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:01.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.319+0000 7f5030cc8640 1 --2- 172.21.15.47:0/1459836670 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f502c06fe20 0x7f502c15c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:01.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.319+0000 7f5030cc8640 1 -- 172.21.15.47:0/1459836670 >> 172.21.15.47:0/1459836670 conn(0x7f502c0d0be0 msgr2=0x7f502c15e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:01.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.319+0000 7f5030cc8640 1 -- 172.21.15.47:0/1459836670 shutdown_connections 2024-04-26T23:20:01.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01.319+0000 7f5030cc8640 1 -- 172.21.15.47:0/1459836670 wait complete. 2024-04-26T23:20:01.332 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T23:20:01.332 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T23:20:01.332 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi047.front.sepia.ceph.com 2024-04-26T23:20:01.332 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T23:20:01.332 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T23:20:01.332 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T23:20:01.332 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T23:20:01.332 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T23:20:01.333 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:20:01.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:20:01.333 DEBUG:teuthology.orchestra.run.smithi047:> ip addr 2024-04-26T23:20:01.349 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T23:20:01.349 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T23:20:01.349 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T23:20:01.349 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:20:01.349 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2024-04-26T23:20:01.349 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:20:01.349 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:20:01.349 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:20:01.349 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f0 2024-04-26T23:20:01.349 INFO:teuthology.orchestra.run.smithi047.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:20:01.350 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:20:01.350 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f1 2024-04-26T23:20:01.350 INFO:teuthology.orchestra.run.smithi047.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T23:20:01.350 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2024-04-26T23:20:01.350 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f0 2024-04-26T23:20:01.350 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-26T23:20:01.350 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42598sec preferred_lft 42598sec 2024-04-26T23:20:01.350 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link noprefixroute 2024-04-26T23:20:01.350 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:20:01.350 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:20:01.350 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2024-04-26T23:20:01.350 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f1 2024-04-26T23:20:01.350 INFO:teuthology.orchestra.run.smithi047.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T23:20:01.351 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 32:8d:47:b1:d0:77 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:20:01.351 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T23:20:01.351 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:20:01.351 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::7c6d:6aff:fe00:4b30/64 scope link 2024-04-26T23:20:01.351 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:20:01.351 INFO:teuthology.orchestra.run.smithi047.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T23:20:01.351 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 32:8d:47:b1:d0:77 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T23:20:01.351 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::308d:47ff:feb1:d077/64 scope link 2024-04-26T23:20:01.351 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:20:01.351 INFO:teuthology.orchestra.run.smithi047.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T23:20:01.351 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether b2:9e:42:6a:5b:5d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T23:20:01.351 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::b09e:42ff:fe6a:5b5d/64 scope link 2024-04-26T23:20:01.351 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:20:01.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:20:01.352 DEBUG:teuthology.orchestra.run.smithi047:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T23:20:01.412 INFO:teuthology.orchestra.run.smithi047.stdout:1 2024-04-26T23:20:01.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:20:01.414 DEBUG:teuthology.orchestra.run.smithi047:> route 2024-04-26T23:20:01.470 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2024-04-26T23:20:01.470 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T23:20:01.470 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T23:20:01.470 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T23:20:01.470 INFO:teuthology.orchestra.run.smithi047.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T23:20:01.470 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:20:01.471 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-04-26T23:20:01.471 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T23:20:01.471 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T23:20:01.471 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T23:20:01.471 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-04-26T23:20:01.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:20:01.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:20:01.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:20:01.597 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-04-26T23:20:01.612 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T23:20:01.612 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T23:20:01.613 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:20:01.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:20:01.613 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set brx.0 up 2024-04-26T23:20:01.677 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:20:01.678 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T23:20:01.733 INFO:teuthology.orchestra.run.smithi047.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:20:01.733 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:20:01.734 DEBUG:teuthology.orchestra.run.smithi047:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T23:20:01.789 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T23:20:01.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:20:01.790 DEBUG:teuthology.orchestra.run.smithi047:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:20:01.872 INFO:teuthology.orchestra.run.smithi047.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:20:01.872 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:20:01.873 INFO:teuthology.orchestra.run.smithi047.stdout:invalid new device string format 2024-04-26T23:20:01.881 INFO:teuthology.orchestra.run.smithi047.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:20:01.881 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:20:01.881 INFO:teuthology.orchestra.run.smithi047.stdout:invalid new device string format 2024-04-26T23:20:01.881 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: resolved to: "172.21.15.47:3300,172.21.15.203:3300,172.21.15.203:3301" 2024-04-26T23:20:01.881 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:20:01.886 INFO:teuthology.orchestra.run.smithi047.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:20:01.886 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:20:01.887 INFO:teuthology.orchestra.run.smithi047.stdout:invalid new device string format 2024-04-26T23:20:01.887 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: resolved to: "172.21.15.47:3300,172.21.15.203:3300,172.21.15.203:3301" 2024-04-26T23:20:01.887 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:20:01.887 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: resolved to: "172.21.15.47:6789,172.21.15.203:6789,172.21.15.203:6790" 2024-04-26T23:20:01.887 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.47:6789,172.21.15.203:6789,172.21.15.203:6790:/ 2024-04-26T23:20:01.887 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=399006d3-4db1-43e6-ba81-a79a276eabfd" will pass to kernel 2024-04-26T23:20:01.888 INFO:teuthology.orchestra.run.smithi047.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T23:20:01.888 INFO:teuthology.orchestra.run.smithi047.stdout: You just mounted a file system that supports labels which does not 2024-04-26T23:20:01.888 INFO:teuthology.orchestra.run.smithi047.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T23:20:01.888 INFO:teuthology.orchestra.run.smithi047.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T23:20:01.888 INFO:teuthology.orchestra.run.smithi047.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T23:20:01.889 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T23:20:01.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:20:01.890 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T23:20:01.955 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T23:20:02.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:20:02.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:20:02.052 INFO:teuthology.orchestra.run.smithi047.stdout:client5287 2024-04-26T23:20:02.052 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:20:02.052 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:20:02.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:20:02.068 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.file.layout.pool -v data_new layoutfile) 2024-04-26T23:20:02.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:20:02.124 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.layout.pool -v data_new layoutdir) 2024-04-26T23:20:02.184 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:20:02.184 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:20:02.241 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:20:02.242 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:20:02.295 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:20:02.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.435+0000 7f103430a640 1 Processor -- start 2024-04-26T23:20:02.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.435+0000 7f103430a640 1 -- start start 2024-04-26T23:20:02.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.436+0000 7f103430a640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f102c162810 0x7f102c070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:02.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.436+0000 7f103430a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f102c0d1b30 0x7f102c0d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:02.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.436+0000 7f103430a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f102c0d10d0 con 0x7f102c0b2340 2024-04-26T23:20:02.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.436+0000 7f103430a640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f102c0d0750 con 0x7f102c162810 2024-04-26T23:20:02.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.436+0000 7f103430a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f102c0d16a0 con 0x7f102c0d1b30 2024-04-26T23:20:02.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.436+0000 7f10320a6640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f102c162810 0x7f102c070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:02.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.436+0000 7f10328a7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f102c0d1b30 0x7f102c0d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:02.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.436+0000 7f10320a6640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f102c162810 0x7f102c070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.47:40162/0 (socket says 172.21.15.47:40162) 2024-04-26T23:20:02.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.436+0000 7f10318a5640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f102c0b2340 0x7f102c0cd3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.47:43020/0 (socket says 172.21.15.47:43020) 2024-04-26T23:20:02.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.436+0000 7f10320a6640 1 -- 172.21.15.47:0/2994379638 learned_addr learned my addr 172.21.15.47:0/2994379638 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:02.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.436+0000 7f10328a7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f102c0d1b30 0x7f102c0d1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.47:60810/0 (socket says 172.21.15.47:60810) 2024-04-26T23:20:02.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.436+0000 7f10328a7640 1 -- 172.21.15.47:0/2994379638 >> v2:172.21.15.203:3300/0 conn(0x7f102c162810 msgr2=0x7f102c070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:02.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.436+0000 7f10328a7640 1 --2- 172.21.15.47:0/2994379638 >> v2:172.21.15.203:3300/0 conn(0x7f102c162810 0x7f102c070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:02.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.436+0000 7f10328a7640 1 -- 172.21.15.47:0/2994379638 >> v1:172.21.15.203:6789/0 conn(0x7f102c0b2340 legacy=0x7f102c0cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:02.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.436+0000 7f10328a7640 1 -- 172.21.15.47:0/2994379638 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f102c0d1370 con 0x7f102c0d1b30 2024-04-26T23:20:02.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.436+0000 7f10310a4640 1 -- 172.21.15.47:0/2994379638 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3106999206 0 0) 0x7f102c0d10d0 con 0x7f102c0b2340 2024-04-26T23:20:02.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.436+0000 7f10320a6640 1 --2- 172.21.15.47:0/2994379638 >> v2:172.21.15.203:3300/0 conn(0x7f102c162810 0x7f102c070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:20:02.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.437+0000 7f10328a7640 1 --2- 172.21.15.47:0/2994379638 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f102c0d1b30 0x7f102c0d1fc0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1028009d90 tx=0x7f102802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cf2518ebdc9dacf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:02.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.437+0000 7f10310a4640 1 -- 172.21.15.47:0/2994379638 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1028007740 con 0x7f102c0d1b30 2024-04-26T23:20:02.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.437+0000 7f10310a4640 1 -- 172.21.15.47:0/2994379638 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10280078e0 con 0x7f102c0d1b30 2024-04-26T23:20:02.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.437+0000 7f10310a4640 1 -- 172.21.15.47:0/2994379638 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1028007cb0 con 0x7f102c0d1b30 2024-04-26T23:20:02.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.438+0000 7f103430a640 1 -- 172.21.15.47:0/2994379638 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f102c0d1b30 msgr2=0x7f102c0d1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:02.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.438+0000 7f103430a640 1 --2- 172.21.15.47:0/2994379638 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f102c0d1b30 0x7f102c0d1fc0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1028009d90 tx=0x7f102802a660 comp rx=0 tx=0).stop 2024-04-26T23:20:02.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.438+0000 7f103430a640 1 -- 172.21.15.47:0/2994379638 shutdown_connections 2024-04-26T23:20:02.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.438+0000 7f103430a640 1 --2- 172.21.15.47:0/2994379638 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f102c0d1b30 0x7f102c0d1fc0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:02.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.438+0000 7f103430a640 1 --2- 172.21.15.47:0/2994379638 >> v2:172.21.15.203:3300/0 conn(0x7f102c162810 0x7f102c070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:02.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.438+0000 7f103430a640 1 -- 172.21.15.47:0/2994379638 >> 172.21.15.47:0/2994379638 conn(0x7f102c0d2da0 msgr2=0x7f102c0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:02.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.438+0000 7f103430a640 1 -- 172.21.15.47:0/2994379638 shutdown_connections 2024-04-26T23:20:02.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.438+0000 7f103430a640 1 -- 172.21.15.47:0/2994379638 wait complete. 2024-04-26T23:20:02.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.439+0000 7f103430a640 1 Processor -- start 2024-04-26T23:20:02.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.439+0000 7f103430a640 1 -- start start 2024-04-26T23:20:02.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.440+0000 7f103430a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f102c0b2340 0x7f102c1f4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:02.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.440+0000 7f103430a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f102c0d1b30 0x7f102c06e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:02.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.440+0000 7f103430a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f102c162810 0x7f102c06ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:02.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.440+0000 7f103430a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f102c070890 con 0x7f102c0b2340 2024-04-26T23:20:02.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.440+0000 7f103430a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f102c0d0470 con 0x7f102c0d1b30 2024-04-26T23:20:02.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.440+0000 7f103430a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f102c0cdb00 con 0x7f102c162810 2024-04-26T23:20:02.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.440+0000 7f10320a6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f102c0b2340 0x7f102c1f4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:02.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.440+0000 7f10320a6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f102c0b2340 0x7f102c1f4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57800/0 (socket says 172.21.15.47:57800) 2024-04-26T23:20:02.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.440+0000 7f10320a6640 1 -- 172.21.15.47:0/1374543557 learned_addr learned my addr 172.21.15.47:0/1374543557 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:02.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.440+0000 7f10318a5640 1 --2- 172.21.15.47:0/1374543557 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f102c0d1b30 0x7f102c06e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:02.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.440+0000 7f10328a7640 1 --2- 172.21.15.47:0/1374543557 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f102c162810 0x7f102c06ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:02.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.440+0000 7f10320a6640 1 -- 172.21.15.47:0/1374543557 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f102c162810 msgr2=0x7f102c06ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:02.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.440+0000 7f10320a6640 1 --2- 172.21.15.47:0/1374543557 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f102c162810 0x7f102c06ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:02.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.440+0000 7f10320a6640 1 -- 172.21.15.47:0/1374543557 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f102c0d1b30 msgr2=0x7f102c06e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:02.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.440+0000 7f10320a6640 1 --2- 172.21.15.47:0/1374543557 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f102c0d1b30 0x7f102c06e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:02.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.440+0000 7f10320a6640 1 -- 172.21.15.47:0/1374543557 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f102c0ca1a0 con 0x7f102c0b2340 2024-04-26T23:20:02.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.441+0000 7f10320a6640 1 --2- 172.21.15.47:0/1374543557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f102c0b2340 0x7f102c1f4360 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f102400e9b0 tx=0x7f102400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:02.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.441+0000 7f1022ffd640 1 -- 172.21.15.47:0/1374543557 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1024017070 con 0x7f102c0b2340 2024-04-26T23:20:02.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.441+0000 7f1022ffd640 1 -- 172.21.15.47:0/1374543557 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f102400c9a0 con 0x7f102c0b2340 2024-04-26T23:20:02.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.441+0000 7f103430a640 1 -- 172.21.15.47:0/1374543557 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f102c0ca480 con 0x7f102c0b2340 2024-04-26T23:20:02.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.441+0000 7f1022ffd640 1 -- 172.21.15.47:0/1374543557 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102400cd50 con 0x7f102c0b2340 2024-04-26T23:20:02.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.441+0000 7f103430a640 1 -- 172.21.15.47:0/1374543557 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f102c1f48a0 con 0x7f102c0b2340 2024-04-26T23:20:02.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.442+0000 7f103430a640 1 -- 172.21.15.47:0/1374543557 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f102c0694f0 con 0x7f102c0b2340 2024-04-26T23:20:02.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.442+0000 7f1022ffd640 1 -- 172.21.15.47:0/1374543557 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f102401a070 con 0x7f102c0b2340 2024-04-26T23:20:02.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.443+0000 7f1022ffd640 1 --2- 172.21.15.47:0/1374543557 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f101006c8f0 0x7f101006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:02.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.443+0000 7f1022ffd640 1 -- 172.21.15.47:0/1374543557 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f102408bd50 con 0x7f102c0b2340 2024-04-26T23:20:02.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.443+0000 7f10318a5640 1 --2- 172.21.15.47:0/1374543557 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f101006c8f0 0x7f101006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:02.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.444+0000 7f10318a5640 1 --2- 172.21.15.47:0/1374543557 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f101006c8f0 0x7f101006edb0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f102c15aa10 tx=0x7f101c007d50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:02.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.447+0000 7f1022ffd640 1 -- 172.21.15.47:0/1374543557 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1024059710 con 0x7f102c0b2340 2024-04-26T23:20:02.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.636+0000 7f103430a640 1 -- 172.21.15.47:0/1374543557 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f102c0cab40 con 0x7f102c0b2340 2024-04-26T23:20:02.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.637+0000 7f1022ffd640 1 -- 172.21.15.47:0/1374543557 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+18718 (secure 0 0 0) 0x7f102405da10 con 0x7f102c0b2340 2024-04-26T23:20:02.639 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:20:02.640 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":41,"fsid":"399006d3-4db1-43e6-ba81-a79a276eabfd","created":"2024-04-26T23:18:06.731211+0000","modified":"2024-04-26T23:19:54.781247+0000","last_up_change":"2024-04-26T23:18:17.808736+0000","last_in_change":"2024-04-26T23:18:08.945859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:18:16.937492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:19:42.351747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-26T23:19:43.037763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2024-04-26T23:19:51.911275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ee091567-fea0-4a14-bf4e-0e16c20bb2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6819","nonce":326520779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6821","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6823","nonce":326520779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6829","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6831","nonce":326520779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6825","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6827","nonce":326520779}]},"public_addr":"172.21.15.47:6819/326520779","cluster_addr":"172.21.15.47:6823/326520779","heartbeat_back_addr":"172.21.15.47:6831/326520779","heartbeat_front_addr":"172.21.15.47:6827/326520779","state":["exists","up"]},{"osd":1,"uuid":"c14440a0-bba5-4931-a6db-b736c08e869b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6809","nonce":1912610931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6811","nonce":1912610931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6815","nonce":1912610931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6813","nonce":1912610931}]},"public_addr":"172.21.15.47:6809/1912610931","cluster_addr":"172.21.15.47:6811/1912610931","heartbeat_back_addr":"172.21.15.47:6815/1912610931","heartbeat_front_addr":"172.21.15.47:6813/1912610931","state":["exists","up"]},{"osd":2,"uuid":"3940f8de-c9b2-4cd3-a911-9c301729eb30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6801","nonce":2569869305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6803","nonce":2569869305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6807","nonce":2569869305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6805","nonce":2569869305}]},"public_addr":"172.21.15.47:6801/2569869305","cluster_addr":"172.21.15.47:6803/2569869305","heartbeat_back_addr":"172.21.15.47:6807/2569869305","heartbeat_front_addr":"172.21.15.47:6805/2569869305","state":["exists","up"]},{"osd":3,"uuid":"1286a67e-8958-426a-8c2e-bc85498c619c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6818","nonce":2791411683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6822","nonce":2791411683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6830","nonce":2791411683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6826","nonce":2791411683}]},"public_addr":"172.21.15.47:6818/2791411683","cluster_addr":"172.21.15.47:6822/2791411683","heartbeat_back_addr":"172.21.15.47:6830/2791411683","heartbeat_front_addr":"172.21.15.47:6826/2791411683","state":["exists","up"]},{"osd":4,"uuid":"d8ee2c20-e5b7-46c2-9999-b234070126cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6809","nonce":441313104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6811","nonce":441313104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6815","nonce":441313104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6813","nonce":441313104}]},"public_addr":"172.21.15.203:6809/441313104","cluster_addr":"172.21.15.203:6811/441313104","heartbeat_back_addr":"172.21.15.203:6815/441313104","heartbeat_front_addr":"172.21.15.203:6813/441313104","state":["exists","up"]},{"osd":5,"uuid":"b453a8f0-f324-4840-9ae0-d67bdadb9f8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6801","nonce":56838444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6803","nonce":56838444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6807","nonce":56838444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6805","nonce":56838444}]},"public_addr":"172.21.15.203:6801/56838444","cluster_addr":"172.21.15.203:6803/56838444","heartbeat_back_addr":"172.21.15.203:6807/56838444","heartbeat_front_addr":"172.21.15.203:6805/56838444","state":["exists","up"]},{"osd":6,"uuid":"a6d31951-2d4b-4e8d-a510-6283b01dc158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6825","nonce":2738517080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6827","nonce":2738517080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6831","nonce":2738517080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6829","nonce":2738517080}]},"public_addr":"172.21.15.203:6825/2738517080","cluster_addr":"172.21.15.203:6827/2738517080","heartbeat_back_addr":"172.21.15.203:6831/2738517080","heartbeat_front_addr":"172.21.15.203:6829/2738517080","state":["exists","up"]},{"osd":7,"uuid":"7119b8c0-fe96-402d-9e75-7b509d9d44d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6817","nonce":1315565058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6819","nonce":1315565058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6823","nonce":1315565058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6821","nonce":1315565058}]},"public_addr":"172.21.15.203:6817/1315565058","cluster_addr":"172.21.15.203:6819/1315565058","heartbeat_back_addr":"172.21.15.203:6823/1315565058","heartbeat_front_addr":"172.21.15.203:6821/1315565058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.121921+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.137494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.084814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.111593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.072430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.125066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.097867+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.057061+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:20:02.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.639+0000 7f103430a640 1 -- 172.21.15.47:0/1374543557 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f101006c8f0 msgr2=0x7f101006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:02.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.639+0000 7f103430a640 1 --2- 172.21.15.47:0/1374543557 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f101006c8f0 0x7f101006edb0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f102c15aa10 tx=0x7f101c007d50 comp rx=0 tx=0).stop 2024-04-26T23:20:02.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.639+0000 7f103430a640 1 -- 172.21.15.47:0/1374543557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f102c0b2340 msgr2=0x7f102c1f4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:02.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.639+0000 7f103430a640 1 --2- 172.21.15.47:0/1374543557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f102c0b2340 0x7f102c1f4360 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f102400e9b0 tx=0x7f102400ee70 comp rx=0 tx=0).stop 2024-04-26T23:20:02.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.639+0000 7f103430a640 1 -- 172.21.15.47:0/1374543557 shutdown_connections 2024-04-26T23:20:02.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.639+0000 7f103430a640 1 --2- 172.21.15.47:0/1374543557 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f101006c8f0 0x7f101006edb0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:02.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.639+0000 7f103430a640 1 --2- 172.21.15.47:0/1374543557 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f102c162810 0x7f102c06ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:02.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.639+0000 7f103430a640 1 --2- 172.21.15.47:0/1374543557 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f102c0d1b30 0x7f102c06e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:02.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.639+0000 7f103430a640 1 --2- 172.21.15.47:0/1374543557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f102c0b2340 0x7f102c1f4360 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:02.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.639+0000 7f103430a640 1 -- 172.21.15.47:0/1374543557 >> 172.21.15.47:0/1374543557 conn(0x7f102c0d2da0 msgr2=0x7f102c0b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:02.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.639+0000 7f103430a640 1 -- 172.21.15.47:0/1374543557 shutdown_connections 2024-04-26T23:20:02.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.639+0000 7f103430a640 1 -- 172.21.15.47:0/1374543557 wait complete. 2024-04-26T23:20:02.652 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:18:16.937492+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:19:42.351747+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:19:43.037763+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'data_new', 'create_time': '2024-04-26T23:19:51.911275+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:20:02.653 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:20:02.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.755+0000 7efc13189640 1 Processor -- start 2024-04-26T23:20:02.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.755+0000 7efc13189640 1 -- start start 2024-04-26T23:20:02.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.755+0000 7efc13189640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7efc0c0b40d0 0x7efc0c1625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:02.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.755+0000 7efc13189640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7efc0c162b30 0x7efc0c0b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:02.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.755+0000 7efc13189640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc0c0d0a10 con 0x7efc0c0d1a00 2024-04-26T23:20:02.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.755+0000 7efc13189640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7efc0c0d0cf0 con 0x7efc0c0b40d0 2024-04-26T23:20:02.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.755+0000 7efc13189640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7efc0c0d0fd0 con 0x7efc0c162b30 2024-04-26T23:20:02.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.756+0000 7efc0bfff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7efc0c0b40d0 0x7efc0c1625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:02.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.756+0000 7efc0bfff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7efc0c0b40d0 0x7efc0c1625f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57810/0 (socket says 172.21.15.47:57810) 2024-04-26T23:20:02.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.756+0000 7efc0bfff640 1 -- 172.21.15.47:0/3447695915 learned_addr learned my addr 172.21.15.47:0/3447695915 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:02.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.756+0000 7efc11726640 1 --2- 172.21.15.47:0/3447695915 >> v2:172.21.15.203:3300/0 conn(0x7efc0c162b30 0x7efc0c0b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:02.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.756+0000 7efc0bfff640 1 -- 172.21.15.47:0/3447695915 >> v2:172.21.15.203:3300/0 conn(0x7efc0c162b30 msgr2=0x7efc0c0b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:02.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.756+0000 7efc0bfff640 1 --2- 172.21.15.47:0/3447695915 >> v2:172.21.15.203:3300/0 conn(0x7efc0c162b30 0x7efc0c0b4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:02.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.756+0000 7efc0bfff640 1 -- 172.21.15.47:0/3447695915 >> v1:172.21.15.203:6789/0 conn(0x7efc0c0d1a00 legacy=0x7efc0c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:02.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.756+0000 7efc0bfff640 1 -- 172.21.15.47:0/3447695915 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc0c162f80 con 0x7efc0c0b40d0 2024-04-26T23:20:02.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.757+0000 7efc0bfff640 1 --2- 172.21.15.47:0/3447695915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc0c0b40d0 0x7efc0c1625f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7efc0c0b69a0 tx=0x7efbf802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11de5f9df535d1a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:02.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.757+0000 7efc0b7fe640 1 -- 172.21.15.47:0/3447695915 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf8037070 con 0x7efc0c0b40d0 2024-04-26T23:20:02.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.757+0000 7efc0b7fe640 1 -- 172.21.15.47:0/3447695915 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efbf8032440 con 0x7efc0c0b40d0 2024-04-26T23:20:02.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.757+0000 7efc0b7fe640 1 -- 172.21.15.47:0/3447695915 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf8032810 con 0x7efc0c0b40d0 2024-04-26T23:20:02.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.757+0000 7efc13189640 1 -- 172.21.15.47:0/3447695915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc0c0b40d0 msgr2=0x7efc0c1625f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:02.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.757+0000 7efc13189640 1 --2- 172.21.15.47:0/3447695915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc0c0b40d0 0x7efc0c1625f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7efc0c0b69a0 tx=0x7efbf802a1a0 comp rx=0 tx=0).stop 2024-04-26T23:20:02.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.757+0000 7efc13189640 1 -- 172.21.15.47:0/3447695915 shutdown_connections 2024-04-26T23:20:02.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.757+0000 7efc13189640 1 --2- 172.21.15.47:0/3447695915 >> v2:172.21.15.203:3300/0 conn(0x7efc0c162b30 0x7efc0c0b4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:02.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.757+0000 7efc13189640 1 --2- 172.21.15.47:0/3447695915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc0c0b40d0 0x7efc0c1625f0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:02.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.757+0000 7efc13189640 1 -- 172.21.15.47:0/3447695915 >> 172.21.15.47:0/3447695915 conn(0x7efc0c0d4fd0 msgr2=0x7efc0c0d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:02.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.757+0000 7efc13189640 1 -- 172.21.15.47:0/3447695915 shutdown_connections 2024-04-26T23:20:02.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.757+0000 7efc13189640 1 -- 172.21.15.47:0/3447695915 wait complete. 2024-04-26T23:20:02.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.758+0000 7efc13189640 1 Processor -- start 2024-04-26T23:20:02.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.758+0000 7efc13189640 1 -- start start 2024-04-26T23:20:02.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.759+0000 7efc13189640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc0c0b40d0 0x7efc0c0b3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:02.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.759+0000 7efc13189640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc0c0d1a00 0x7efc0c0ba110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:02.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.759+0000 7efc13189640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efc0c162b30 0x7efc0c14f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:02.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.759+0000 7efc13189640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7efc0c0d2670 con 0x7efc0c0b40d0 2024-04-26T23:20:02.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.759+0000 7efc13189640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7efc0c0d24f0 con 0x7efc0c0d1a00 2024-04-26T23:20:02.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.759+0000 7efc13189640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7efc0c0b5040 con 0x7efc0c162b30 2024-04-26T23:20:02.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.759+0000 7efc10f25640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc0c0b40d0 0x7efc0c0b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:02.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.760+0000 7efc10f25640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc0c0b40d0 0x7efc0c0b3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57822/0 (socket says 172.21.15.47:57822) 2024-04-26T23:20:02.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.760+0000 7efc10f25640 1 -- 172.21.15.47:0/2020142266 learned_addr learned my addr 172.21.15.47:0/2020142266 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:02.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.760+0000 7efc0bfff640 1 --2- 172.21.15.47:0/2020142266 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc0c0d1a00 0x7efc0c0ba110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:02.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.760+0000 7efc11726640 1 --2- 172.21.15.47:0/2020142266 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efc0c162b30 0x7efc0c14f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:02.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.760+0000 7efc10f25640 1 -- 172.21.15.47:0/2020142266 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efc0c162b30 msgr2=0x7efc0c14f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:02.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.760+0000 7efc10f25640 1 --2- 172.21.15.47:0/2020142266 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efc0c162b30 0x7efc0c14f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:02.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.760+0000 7efc10f25640 1 -- 172.21.15.47:0/2020142266 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc0c0d1a00 msgr2=0x7efc0c0ba110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:02.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.760+0000 7efc10f25640 1 --2- 172.21.15.47:0/2020142266 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc0c0d1a00 0x7efc0c0ba110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:02.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.760+0000 7efc10f25640 1 -- 172.21.15.47:0/2020142266 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc0c0cd300 con 0x7efc0c0b40d0 2024-04-26T23:20:02.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.760+0000 7efc10f25640 1 --2- 172.21.15.47:0/2020142266 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc0c0b40d0 0x7efc0c0b3950 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7efc0000c430 tx=0x7efc0000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:02.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.760+0000 7efc097fa640 1 -- 172.21.15.47:0/2020142266 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc00011790 con 0x7efc0c0b40d0 2024-04-26T23:20:02.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.760+0000 7efc097fa640 1 -- 172.21.15.47:0/2020142266 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc00017070 con 0x7efc0c0b40d0 2024-04-26T23:20:02.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.760+0000 7efc13189640 1 -- 172.21.15.47:0/2020142266 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc0c0cd5e0 con 0x7efc0c0b40d0 2024-04-26T23:20:02.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.760+0000 7efc097fa640 1 -- 172.21.15.47:0/2020142266 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc00011b70 con 0x7efc0c0b40d0 2024-04-26T23:20:02.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.760+0000 7efc13189640 1 -- 172.21.15.47:0/2020142266 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc0c0b9bf0 con 0x7efc0c0b40d0 2024-04-26T23:20:02.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.761+0000 7efc13189640 1 -- 172.21.15.47:0/2020142266 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc0c0694f0 con 0x7efc0c0b40d0 2024-04-26T23:20:02.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.762+0000 7efc097fa640 1 -- 172.21.15.47:0/2020142266 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efc00011d10 con 0x7efc0c0b40d0 2024-04-26T23:20:02.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.762+0000 7efc097fa640 1 --2- 172.21.15.47:0/2020142266 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7efbf006c8f0 0x7efbf006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:02.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.762+0000 7efc097fa640 1 -- 172.21.15.47:0/2020142266 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7efc0008bb80 con 0x7efc0c0b40d0 2024-04-26T23:20:02.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.762+0000 7efc0bfff640 1 --2- 172.21.15.47:0/2020142266 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7efbf006c8f0 0x7efbf006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:02.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.763+0000 7efc0bfff640 1 --2- 172.21.15.47:0/2020142266 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7efbf006c8f0 0x7efbf006edb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7efbf8034000 tx=0x7efbf802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:02.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.766+0000 7efc097fa640 1 -- 172.21.15.47:0/2020142266 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efc00059650 con 0x7efc0c0b40d0 2024-04-26T23:20:02.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.950+0000 7efc13189640 1 -- 172.21.15.47:0/2020142266 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efc0c069e00 con 0x7efc0c0b40d0 2024-04-26T23:20:02.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.951+0000 7efc097fa640 1 -- 172.21.15.47:0/2020142266 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) ==== 93+0+10 (secure 0 0 0) 0x7efc0005d950 con 0x7efc0c0b40d0 2024-04-26T23:20:02.953 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-26T23:20:02.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.952+0000 7efc13189640 1 -- 172.21.15.47:0/2020142266 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7efbf006c8f0 msgr2=0x7efbf006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:02.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.953+0000 7efc13189640 1 --2- 172.21.15.47:0/2020142266 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7efbf006c8f0 0x7efbf006edb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7efbf8034000 tx=0x7efbf802e000 comp rx=0 tx=0).stop 2024-04-26T23:20:02.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.953+0000 7efc13189640 1 -- 172.21.15.47:0/2020142266 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc0c0b40d0 msgr2=0x7efc0c0b3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:02.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.953+0000 7efc13189640 1 --2- 172.21.15.47:0/2020142266 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc0c0b40d0 0x7efc0c0b3950 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7efc0000c430 tx=0x7efc0000c8f0 comp rx=0 tx=0).stop 2024-04-26T23:20:02.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.953+0000 7efc13189640 1 -- 172.21.15.47:0/2020142266 shutdown_connections 2024-04-26T23:20:02.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.953+0000 7efc13189640 1 --2- 172.21.15.47:0/2020142266 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7efbf006c8f0 0x7efbf006edb0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:02.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.953+0000 7efc13189640 1 --2- 172.21.15.47:0/2020142266 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efc0c162b30 0x7efc0c14f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:02.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.953+0000 7efc13189640 1 --2- 172.21.15.47:0/2020142266 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc0c0d1a00 0x7efc0c0ba110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:02.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.953+0000 7efc13189640 1 --2- 172.21.15.47:0/2020142266 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc0c0b40d0 0x7efc0c0b3950 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:02.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.953+0000 7efc13189640 1 -- 172.21.15.47:0/2020142266 >> 172.21.15.47:0/2020142266 conn(0x7efc0c0d4fd0 msgr2=0x7efc0c0d1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:02.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.953+0000 7efc13189640 1 -- 172.21.15.47:0/2020142266 shutdown_connections 2024-04-26T23:20:02.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:02.953+0000 7efc13189640 1 -- 172.21.15.47:0/2020142266 wait complete. 2024-04-26T23:20:02.966 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:20:03.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.067+0000 7fbe31c93640 1 Processor -- start 2024-04-26T23:20:03.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.067+0000 7fbe31c93640 1 -- start start 2024-04-26T23:20:03.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.068+0000 7fbe31c93640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe2c15f170 0x7fbe2c0d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:03.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.068+0000 7fbe31c93640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbe2c0d3bc0 0x7fbe2c0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:03.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.068+0000 7fbe31c93640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe2c0d1040 con 0x7fbe2c15f550 2024-04-26T23:20:03.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.068+0000 7fbe31c93640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fbe2c0c9430 con 0x7fbe2c0d3bc0 2024-04-26T23:20:03.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.068+0000 7fbe31c93640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fbe2c0c9680 con 0x7fbe2c15f170 2024-04-26T23:20:03.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.068+0000 7fbe2bfff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbe2c0d3bc0 0x7fbe2c0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:03.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.068+0000 7fbe2affd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fbe2c15f550 0x7fbe2c0d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:48854/0 (socket says 172.21.15.47:48854) 2024-04-26T23:20:03.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.068+0000 7fbe2affd640 1 -- 172.21.15.47:0/333898504 learned_addr learned my addr 172.21.15.47:0/333898504 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:03.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.068+0000 7fbe2bfff640 1 -- 172.21.15.47:0/333898504 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe2c15f170 msgr2=0x7fbe2c0d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:03.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.068+0000 7fbe2b7fe640 1 --2- 172.21.15.47:0/333898504 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe2c15f170 0x7fbe2c0d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:03.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.068+0000 7fbe2a7fc640 1 -- 172.21.15.47:0/333898504 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3101197294 0 0) 0x7fbe20001110 con 0x7fbe2c15f550 2024-04-26T23:20:03.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.068+0000 7fbe2bfff640 1 --2- 172.21.15.47:0/333898504 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe2c15f170 0x7fbe2c0d5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:03.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.068+0000 7fbe2bfff640 1 -- 172.21.15.47:0/333898504 >> v1:172.21.15.47:6789/0 conn(0x7fbe2c15f550 legacy=0x7fbe2c0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:03.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.068+0000 7fbe2bfff640 1 -- 172.21.15.47:0/333898504 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe2c0c9960 con 0x7fbe2c0d3bc0 2024-04-26T23:20:03.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.069+0000 7fbe2bfff640 1 --2- 172.21.15.47:0/333898504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe2c0d3bc0 0x7fbe2c0d2cb0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fbe24009d90 tx=0x7fbe2402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9ccf3fa761a7146 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:03.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.069+0000 7fbe2a7fc640 1 -- 172.21.15.47:0/333898504 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe24007740 con 0x7fbe2c0d3bc0 2024-04-26T23:20:03.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.069+0000 7fbe2a7fc640 1 -- 172.21.15.47:0/333898504 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe240078e0 con 0x7fbe2c0d3bc0 2024-04-26T23:20:03.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.069+0000 7fbe2a7fc640 1 -- 172.21.15.47:0/333898504 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe24007cb0 con 0x7fbe2c0d3bc0 2024-04-26T23:20:03.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.070+0000 7fbe31c93640 1 -- 172.21.15.47:0/333898504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe2c0d3bc0 msgr2=0x7fbe2c0d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:03.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.070+0000 7fbe31c93640 1 --2- 172.21.15.47:0/333898504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe2c0d3bc0 0x7fbe2c0d2cb0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fbe24009d90 tx=0x7fbe2402a660 comp rx=0 tx=0).stop 2024-04-26T23:20:03.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.070+0000 7fbe31c93640 1 -- 172.21.15.47:0/333898504 shutdown_connections 2024-04-26T23:20:03.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.070+0000 7fbe31c93640 1 --2- 172.21.15.47:0/333898504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe2c0d3bc0 0x7fbe2c0d2cb0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:03.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.070+0000 7fbe31c93640 1 --2- 172.21.15.47:0/333898504 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe2c15f170 0x7fbe2c0d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:03.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.070+0000 7fbe31c93640 1 -- 172.21.15.47:0/333898504 >> 172.21.15.47:0/333898504 conn(0x7fbe2c0d4f60 msgr2=0x7fbe2c0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:03.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.070+0000 7fbe31c93640 1 -- 172.21.15.47:0/333898504 shutdown_connections 2024-04-26T23:20:03.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.070+0000 7fbe31c93640 1 -- 172.21.15.47:0/333898504 wait complete. 2024-04-26T23:20:03.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.071+0000 7fbe31c93640 1 Processor -- start 2024-04-26T23:20:03.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.071+0000 7fbe31c93640 1 -- start start 2024-04-26T23:20:03.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.072+0000 7fbe31c93640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe2c0d3bc0 0x7fbe2c0ce830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:03.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.072+0000 7fbe31c93640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe2c15f170 0x7fbe2c15b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:03.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.072+0000 7fbe31c93640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe2c158020 0x7fbe2c158460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:03.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.072+0000 7fbe31c93640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2c0ca5c0 con 0x7fbe2c158020 2024-04-26T23:20:03.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.072+0000 7fbe31c93640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2c0ca440 con 0x7fbe2c15f170 2024-04-26T23:20:03.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.072+0000 7fbe31c93640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fbe2c0ca740 con 0x7fbe2c0d3bc0 2024-04-26T23:20:03.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.072+0000 7fbe2bfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe2c158020 0x7fbe2c158460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:03.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.072+0000 7fbe2bfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe2c158020 0x7fbe2c158460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57838/0 (socket says 172.21.15.47:57838) 2024-04-26T23:20:03.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.072+0000 7fbe2bfff640 1 -- 172.21.15.47:0/2494791149 learned_addr learned my addr 172.21.15.47:0/2494791149 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:03.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.072+0000 7fbe2affd640 1 --2- 172.21.15.47:0/2494791149 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe2c15f170 0x7fbe2c15b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:03.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.072+0000 7fbe2b7fe640 1 --2- 172.21.15.47:0/2494791149 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe2c0d3bc0 0x7fbe2c0ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:03.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.072+0000 7fbe2bfff640 1 -- 172.21.15.47:0/2494791149 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe2c0d3bc0 msgr2=0x7fbe2c0ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:03.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.072+0000 7fbe2bfff640 1 --2- 172.21.15.47:0/2494791149 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe2c0d3bc0 0x7fbe2c0ce830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:03.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.072+0000 7fbe2bfff640 1 -- 172.21.15.47:0/2494791149 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe2c15f170 msgr2=0x7fbe2c15b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:03.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.072+0000 7fbe2bfff640 1 --2- 172.21.15.47:0/2494791149 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe2c15f170 0x7fbe2c15b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:03.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.072+0000 7fbe2bfff640 1 -- 172.21.15.47:0/2494791149 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe2c1589a0 con 0x7fbe2c158020 2024-04-26T23:20:03.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.072+0000 7fbe2bfff640 1 --2- 172.21.15.47:0/2494791149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe2c158020 0x7fbe2c158460 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fbe24009eb0 tx=0x7fbe24004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:03.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.073+0000 7fbe13fff640 1 -- 172.21.15.47:0/2494791149 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe24040070 con 0x7fbe2c158020 2024-04-26T23:20:03.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.073+0000 7fbe13fff640 1 -- 172.21.15.47:0/2494791149 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe24031c90 con 0x7fbe2c158020 2024-04-26T23:20:03.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.073+0000 7fbe31c93640 1 -- 172.21.15.47:0/2494791149 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe2c0cce20 con 0x7fbe2c158020 2024-04-26T23:20:03.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.073+0000 7fbe13fff640 1 -- 172.21.15.47:0/2494791149 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe24045680 con 0x7fbe2c158020 2024-04-26T23:20:03.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.073+0000 7fbe31c93640 1 -- 172.21.15.47:0/2494791149 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe2c0cd3e0 con 0x7fbe2c158020 2024-04-26T23:20:03.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.074+0000 7fbe31c93640 1 -- 172.21.15.47:0/2494791149 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe2c0694f0 con 0x7fbe2c158020 2024-04-26T23:20:03.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.074+0000 7fbe13fff640 1 -- 172.21.15.47:0/2494791149 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbe24039040 con 0x7fbe2c158020 2024-04-26T23:20:03.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.074+0000 7fbe13fff640 1 --2- 172.21.15.47:0/2494791149 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fbe1806c8f0 0x7fbe1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:03.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.075+0000 7fbe13fff640 1 -- 172.21.15.47:0/2494791149 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7fbe240ad8c0 con 0x7fbe2c158020 2024-04-26T23:20:03.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.075+0000 7fbe2b7fe640 1 --2- 172.21.15.47:0/2494791149 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fbe1806c8f0 0x7fbe1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:03.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.075+0000 7fbe2b7fe640 1 --2- 172.21.15.47:0/2494791149 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fbe1806c8f0 0x7fbe1806edb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c006fd0 tx=0x7fbe1c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:03.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.079+0000 7fbe13fff640 1 -- 172.21.15.47:0/2494791149 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbe24043360 con 0x7fbe2c158020 2024-04-26T23:20:03.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.262+0000 7fbe31c93640 1 -- 172.21.15.47:0/2494791149 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbe2c06eb40 con 0x7fbe2c158020 2024-04-26T23:20:03.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.262+0000 7fbe13fff640 1 -- 172.21.15.47:0/2494791149 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v41) ==== 104+0+11 (secure 0 0 0) 0x7fbe2407ba20 con 0x7fbe2c158020 2024-04-26T23:20:03.264 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-26T23:20:03.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.264+0000 7fbe31c93640 1 -- 172.21.15.47:0/2494791149 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fbe1806c8f0 msgr2=0x7fbe1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:03.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.264+0000 7fbe31c93640 1 --2- 172.21.15.47:0/2494791149 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fbe1806c8f0 0x7fbe1806edb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c006fd0 tx=0x7fbe1c008040 comp rx=0 tx=0).stop 2024-04-26T23:20:03.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.264+0000 7fbe31c93640 1 -- 172.21.15.47:0/2494791149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe2c158020 msgr2=0x7fbe2c158460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:03.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.264+0000 7fbe31c93640 1 --2- 172.21.15.47:0/2494791149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe2c158020 0x7fbe2c158460 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fbe24009eb0 tx=0x7fbe24004a40 comp rx=0 tx=0).stop 2024-04-26T23:20:03.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.264+0000 7fbe31c93640 1 -- 172.21.15.47:0/2494791149 shutdown_connections 2024-04-26T23:20:03.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.264+0000 7fbe31c93640 1 --2- 172.21.15.47:0/2494791149 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fbe1806c8f0 0x7fbe1806edb0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:03.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.264+0000 7fbe31c93640 1 --2- 172.21.15.47:0/2494791149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe2c158020 0x7fbe2c158460 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:03.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.264+0000 7fbe31c93640 1 --2- 172.21.15.47:0/2494791149 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe2c15f170 0x7fbe2c15b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:03.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.264+0000 7fbe31c93640 1 --2- 172.21.15.47:0/2494791149 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe2c0d3bc0 0x7fbe2c0ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:03.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.264+0000 7fbe31c93640 1 -- 172.21.15.47:0/2494791149 >> 172.21.15.47:0/2494791149 conn(0x7fbe2c0d4f60 msgr2=0x7fbe2c06d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:03.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.264+0000 7fbe31c93640 1 -- 172.21.15.47:0/2494791149 shutdown_connections 2024-04-26T23:20:03.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.264+0000 7fbe31c93640 1 -- 172.21.15.47:0/2494791149 wait complete. 2024-04-26T23:20:03.277 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:20:03.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.378+0000 7f2154d9d640 1 Processor -- start 2024-04-26T23:20:03.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.378+0000 7f2154d9d640 1 -- start start 2024-04-26T23:20:03.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.379+0000 7f2154d9d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f215015a9f0 0x7f21500b4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:03.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.379+0000 7f2154d9d640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21500ce060 con 0x7f2150161510 2024-04-26T23:20:03.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.379+0000 7f2154d9d640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21500ce340 con 0x7f21500726e0 2024-04-26T23:20:03.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.379+0000 7f2154d9d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f21500ce620 con 0x7f215015a9f0 2024-04-26T23:20:03.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.380+0000 7f214dd9b640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f2150161510 0x7f215015dbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:48860/0 (socket says 172.21.15.47:48860) 2024-04-26T23:20:03.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.380+0000 7f214dd9b640 1 -- 172.21.15.47:0/3235932829 learned_addr learned my addr 172.21.15.47:0/3235932829 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:03.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.380+0000 7f214e59c640 1 --2- 172.21.15.47:0/3235932829 >> v2:172.21.15.203:3300/0 conn(0x7f215015a9f0 0x7f21500b4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:03.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.380+0000 7f214d59a640 1 -- 172.21.15.47:0/3235932829 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2365843613 0 0) 0x7f21500ce060 con 0x7f2150161510 2024-04-26T23:20:03.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.380+0000 7f214d59a640 1 -- 172.21.15.47:0/3235932829 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2138003520 con 0x7f2150161510 2024-04-26T23:20:03.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.380+0000 7f214d59a640 1 -- 172.21.15.47:0/3235932829 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 324416245 0 0) 0x7f21500ce340 con 0x7f21500726e0 2024-04-26T23:20:03.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.380+0000 7f214d59a640 1 -- 172.21.15.47:0/3235932829 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f21500ce060 con 0x7f21500726e0 2024-04-26T23:20:03.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.380+0000 7f214d59a640 1 -- 172.21.15.47:0/3235932829 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 985902647 0 0) 0x7f2138003520 con 0x7f2150161510 2024-04-26T23:20:03.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.380+0000 7f214e59c640 1 -- 172.21.15.47:0/3235932829 >> v1:172.21.15.203:6789/0 conn(0x7f21500726e0 legacy=0x7f215006c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:03.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.380+0000 7f214e59c640 1 -- 172.21.15.47:0/3235932829 >> v1:172.21.15.47:6789/0 conn(0x7f2150161510 legacy=0x7f215015dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:03.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.380+0000 7f214e59c640 1 -- 172.21.15.47:0/3235932829 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f215015d6b0 con 0x7f215015a9f0 2024-04-26T23:20:03.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.381+0000 7f214e59c640 1 --2- 172.21.15.47:0/3235932829 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f215015a9f0 0x7f21500b4480 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f21400097a0 tx=0x7f214002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=679887397f4b35a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:03.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.381+0000 7f214d59a640 1 -- 172.21.15.47:0/3235932829 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2140037070 con 0x7f215015a9f0 2024-04-26T23:20:03.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.381+0000 7f214d59a640 1 -- 172.21.15.47:0/3235932829 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f214002ac50 con 0x7f215015a9f0 2024-04-26T23:20:03.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.381+0000 7f214d59a640 1 -- 172.21.15.47:0/3235932829 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2144001670 con 0x7f215015a9f0 2024-04-26T23:20:03.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.382+0000 7f2154d9d640 1 -- 172.21.15.47:0/3235932829 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f215015a9f0 msgr2=0x7f21500b4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:03.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.382+0000 7f2154d9d640 1 --2- 172.21.15.47:0/3235932829 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f215015a9f0 0x7f21500b4480 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f21400097a0 tx=0x7f214002a1c0 comp rx=0 tx=0).stop 2024-04-26T23:20:03.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.382+0000 7f2154d9d640 1 -- 172.21.15.47:0/3235932829 shutdown_connections 2024-04-26T23:20:03.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.382+0000 7f2154d9d640 1 --2- 172.21.15.47:0/3235932829 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f215015a9f0 0x7f21500b4480 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:03.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.382+0000 7f2154d9d640 1 -- 172.21.15.47:0/3235932829 >> 172.21.15.47:0/3235932829 conn(0x7f21500b3180 msgr2=0x7f21500b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:03.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.382+0000 7f2154d9d640 1 -- 172.21.15.47:0/3235932829 shutdown_connections 2024-04-26T23:20:03.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.382+0000 7f2154d9d640 1 -- 172.21.15.47:0/3235932829 wait complete. 2024-04-26T23:20:03.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.383+0000 7f2154d9d640 1 Processor -- start 2024-04-26T23:20:03.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.383+0000 7f2154d9d640 1 -- start start 2024-04-26T23:20:03.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.384+0000 7f2154d9d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f21500726e0 0x7f2150146ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:03.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.384+0000 7f2154d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f215015a9f0 0x7f21500c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:03.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.384+0000 7f2154d9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2150161510 0x7f21500c7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:03.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.384+0000 7f2154d9d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f215015e2e0 con 0x7f215015a9f0 2024-04-26T23:20:03.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.384+0000 7f2154d9d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f215015add0 con 0x7f2150161510 2024-04-26T23:20:03.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.384+0000 7f2154d9d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f215015e460 con 0x7f21500726e0 2024-04-26T23:20:03.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.384+0000 7f214dd9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f215015a9f0 0x7f21500c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:03.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.384+0000 7f214dd9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f215015a9f0 0x7f21500c6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57854/0 (socket says 172.21.15.47:57854) 2024-04-26T23:20:03.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.384+0000 7f214dd9b640 1 -- 172.21.15.47:0/2795974628 learned_addr learned my addr 172.21.15.47:0/2795974628 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:03.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.384+0000 7f214e59c640 1 --2- 172.21.15.47:0/2795974628 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f21500726e0 0x7f2150146ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:03.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.384+0000 7f214ed9d640 1 --2- 172.21.15.47:0/2795974628 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2150161510 0x7f21500c7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:03.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.384+0000 7f214dd9b640 1 -- 172.21.15.47:0/2795974628 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f21500726e0 msgr2=0x7f2150146ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:03.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.384+0000 7f214dd9b640 1 --2- 172.21.15.47:0/2795974628 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f21500726e0 0x7f2150146ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:03.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.384+0000 7f214dd9b640 1 -- 172.21.15.47:0/2795974628 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2150161510 msgr2=0x7f21500c7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:03.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.384+0000 7f214dd9b640 1 --2- 172.21.15.47:0/2795974628 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2150161510 0x7f21500c7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:03.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.384+0000 7f214dd9b640 1 -- 172.21.15.47:0/2795974628 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f215014d4f0 con 0x7f215015a9f0 2024-04-26T23:20:03.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.384+0000 7f214dd9b640 1 --2- 172.21.15.47:0/2795974628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f215015a9f0 0x7f21500c6dd0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f214400c5c0 tx=0x7f214400ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:03.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.385+0000 7f213effd640 1 -- 172.21.15.47:0/2795974628 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2144017070 con 0x7f215015a9f0 2024-04-26T23:20:03.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.385+0000 7f213effd640 1 -- 172.21.15.47:0/2795974628 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2144005ce0 con 0x7f215015a9f0 2024-04-26T23:20:03.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.385+0000 7f2154d9d640 1 -- 172.21.15.47:0/2795974628 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21500c79e0 con 0x7f215015a9f0 2024-04-26T23:20:03.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.385+0000 7f213effd640 1 -- 172.21.15.47:0/2795974628 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21440072b0 con 0x7f215015a9f0 2024-04-26T23:20:03.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.385+0000 7f2154d9d640 1 -- 172.21.15.47:0/2795974628 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2150147710 con 0x7f215015a9f0 2024-04-26T23:20:03.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.386+0000 7f2154d9d640 1 -- 172.21.15.47:0/2795974628 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2150068fc0 con 0x7f215015a9f0 2024-04-26T23:20:03.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.386+0000 7f213effd640 1 -- 172.21.15.47:0/2795974628 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f214401a070 con 0x7f215015a9f0 2024-04-26T23:20:03.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.387+0000 7f213effd640 1 --2- 172.21.15.47:0/2795974628 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f213006c8f0 0x7f213006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:03.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.387+0000 7f213effd640 1 -- 172.21.15.47:0/2795974628 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f214408b3b0 con 0x7f215015a9f0 2024-04-26T23:20:03.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.387+0000 7f214e59c640 1 --2- 172.21.15.47:0/2795974628 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f213006c8f0 0x7f213006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:03.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.387+0000 7f214e59c640 1 --2- 172.21.15.47:0/2795974628 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f213006c8f0 0x7f213006edb0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2140034000 tx=0x7f214002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:03.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.391+0000 7f213effd640 1 -- 172.21.15.47:0/2795974628 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f215014d4f0 con 0x7f215015a9f0 2024-04-26T23:20:03.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.575+0000 7f2154d9d640 1 -- 172.21.15.47:0/2795974628 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2150075ef0 con 0x7f215015a9f0 2024-04-26T23:20:03.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.575+0000 7f213effd640 1 -- 172.21.15.47:0/2795974628 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v41) ==== 100+0+11 (secure 0 0 0) 0x7f21440594d0 con 0x7f215015a9f0 2024-04-26T23:20:03.578 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-26T23:20:03.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.577+0000 7f2154d9d640 1 -- 172.21.15.47:0/2795974628 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f213006c8f0 msgr2=0x7f213006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:03.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.577+0000 7f2154d9d640 1 --2- 172.21.15.47:0/2795974628 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f213006c8f0 0x7f213006edb0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2140034000 tx=0x7f214002e000 comp rx=0 tx=0).stop 2024-04-26T23:20:03.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.577+0000 7f2154d9d640 1 -- 172.21.15.47:0/2795974628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f215015a9f0 msgr2=0x7f21500c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:03.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.577+0000 7f2154d9d640 1 --2- 172.21.15.47:0/2795974628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f215015a9f0 0x7f21500c6dd0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f214400c5c0 tx=0x7f214400ca80 comp rx=0 tx=0).stop 2024-04-26T23:20:03.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.577+0000 7f2154d9d640 1 -- 172.21.15.47:0/2795974628 shutdown_connections 2024-04-26T23:20:03.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.577+0000 7f2154d9d640 1 --2- 172.21.15.47:0/2795974628 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f213006c8f0 0x7f213006edb0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:03.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.577+0000 7f2154d9d640 1 --2- 172.21.15.47:0/2795974628 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2150161510 0x7f21500c7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:03.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.577+0000 7f2154d9d640 1 --2- 172.21.15.47:0/2795974628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f215015a9f0 0x7f21500c6dd0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:03.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.577+0000 7f2154d9d640 1 --2- 172.21.15.47:0/2795974628 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f21500726e0 0x7f2150146ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:03.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.577+0000 7f2154d9d640 1 -- 172.21.15.47:0/2795974628 >> 172.21.15.47:0/2795974628 conn(0x7f21500b3180 msgr2=0x7f2150073400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:03.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.577+0000 7f2154d9d640 1 -- 172.21.15.47:0/2795974628 shutdown_connections 2024-04-26T23:20:03.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.577+0000 7f2154d9d640 1 -- 172.21.15.47:0/2795974628 wait complete. 2024-04-26T23:20:03.590 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2024-04-26T23:20:03.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.692+0000 7fa455346640 1 Processor -- start 2024-04-26T23:20:03.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.692+0000 7fa455346640 1 -- start start 2024-04-26T23:20:03.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.693+0000 7fa455346640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa4500cc480 0x7fa450072240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:03.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.693+0000 7fa455346640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa450072780 0x7fa450166ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:03.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.693+0000 7fa455346640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa45015d7c0 con 0x7fa450071e60 2024-04-26T23:20:03.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.693+0000 7fa455346640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fa45015daa0 con 0x7fa450072780 2024-04-26T23:20:03.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.693+0000 7fa455346640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fa45015dd80 con 0x7fa4500cc480 2024-04-26T23:20:03.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.693+0000 7fa44f7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa450072780 0x7fa450166ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:03.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.693+0000 7fa44effd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa450071e60 0x7fa45006f8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:48862/0 (socket says 172.21.15.47:48862) 2024-04-26T23:20:03.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.693+0000 7fa44effd640 1 -- 172.21.15.47:0/3309364768 learned_addr learned my addr 172.21.15.47:0/3309364768 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:03.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.693+0000 7fa44e7fc640 1 --2- 172.21.15.47:0/3309364768 >> v2:172.21.15.203:3300/0 conn(0x7fa4500cc480 0x7fa450072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:03.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.693+0000 7fa44f7fe640 1 -- 172.21.15.47:0/3309364768 >> v2:172.21.15.203:3300/0 conn(0x7fa4500cc480 msgr2=0x7fa450072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:03.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.693+0000 7fa44f7fe640 1 --2- 172.21.15.47:0/3309364768 >> v2:172.21.15.203:3300/0 conn(0x7fa4500cc480 0x7fa450072240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:03.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.693+0000 7fa44f7fe640 1 -- 172.21.15.47:0/3309364768 >> v1:172.21.15.47:6789/0 conn(0x7fa450071e60 legacy=0x7fa45006f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:03.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.694+0000 7fa44dffb640 1 -- 172.21.15.47:0/3309364768 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2433542770 0 0) 0x7fa45015d7c0 con 0x7fa450071e60 2024-04-26T23:20:03.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.694+0000 7fa44f7fe640 1 -- 172.21.15.47:0/3309364768 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa45015e060 con 0x7fa450072780 2024-04-26T23:20:03.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.694+0000 7fa44f7fe640 1 --2- 172.21.15.47:0/3309364768 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa450072780 0x7fa450166ad0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa4500b3820 tx=0x7fa44802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b47e9f7b78391288 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:03.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.694+0000 7fa44dffb640 1 -- 172.21.15.47:0/3309364768 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa448007740 con 0x7fa450072780 2024-04-26T23:20:03.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.694+0000 7fa44dffb640 1 -- 172.21.15.47:0/3309364768 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4480078e0 con 0x7fa450072780 2024-04-26T23:20:03.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.694+0000 7fa44dffb640 1 -- 172.21.15.47:0/3309364768 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa448007cb0 con 0x7fa450072780 2024-04-26T23:20:03.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.695+0000 7fa455346640 1 -- 172.21.15.47:0/3309364768 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa450072780 msgr2=0x7fa450166ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:03.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.695+0000 7fa455346640 1 --2- 172.21.15.47:0/3309364768 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa450072780 0x7fa450166ad0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa4500b3820 tx=0x7fa44802a680 comp rx=0 tx=0).stop 2024-04-26T23:20:03.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.695+0000 7fa455346640 1 -- 172.21.15.47:0/3309364768 shutdown_connections 2024-04-26T23:20:03.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.695+0000 7fa455346640 1 --2- 172.21.15.47:0/3309364768 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa450072780 0x7fa450166ad0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:03.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.695+0000 7fa455346640 1 --2- 172.21.15.47:0/3309364768 >> v2:172.21.15.203:3300/0 conn(0x7fa4500cc480 0x7fa450072240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:03.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.695+0000 7fa455346640 1 -- 172.21.15.47:0/3309364768 >> 172.21.15.47:0/3309364768 conn(0x7fa4500d0be0 msgr2=0x7fa4500d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:03.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.695+0000 7fa455346640 1 -- 172.21.15.47:0/3309364768 shutdown_connections 2024-04-26T23:20:03.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.695+0000 7fa455346640 1 -- 172.21.15.47:0/3309364768 wait complete. 2024-04-26T23:20:03.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.696+0000 7fa455346640 1 Processor -- start 2024-04-26T23:20:03.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.696+0000 7fa455346640 1 -- start start 2024-04-26T23:20:03.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.697+0000 7fa455346640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa450071e60 0x7fa450152ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:03.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.697+0000 7fa455346640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa450072780 0x7fa4500b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:03.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.697+0000 7fa455346640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4500cc480 0x7fa4500b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:03.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.697+0000 7fa455346640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa45015d3f0 con 0x7fa450071e60 2024-04-26T23:20:03.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.697+0000 7fa455346640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa450072bd0 con 0x7fa4500cc480 2024-04-26T23:20:03.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.697+0000 7fa455346640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa45015d570 con 0x7fa450072780 2024-04-26T23:20:03.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.697+0000 7fa44effd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa450071e60 0x7fa450152ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:03.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.697+0000 7fa44effd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa450071e60 0x7fa450152ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57866/0 (socket says 172.21.15.47:57866) 2024-04-26T23:20:03.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.697+0000 7fa44effd640 1 -- 172.21.15.47:0/378139022 learned_addr learned my addr 172.21.15.47:0/378139022 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:03.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.697+0000 7fa44f7fe640 1 --2- 172.21.15.47:0/378139022 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4500cc480 0x7fa4500b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:03.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.697+0000 7fa44e7fc640 1 --2- 172.21.15.47:0/378139022 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa450072780 0x7fa4500b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:03.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.697+0000 7fa44effd640 1 -- 172.21.15.47:0/378139022 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa450072780 msgr2=0x7fa4500b48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:03.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.697+0000 7fa44effd640 1 --2- 172.21.15.47:0/378139022 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa450072780 0x7fa4500b48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:03.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.697+0000 7fa44effd640 1 -- 172.21.15.47:0/378139022 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4500cc480 msgr2=0x7fa4500b4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:03.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.697+0000 7fa44effd640 1 --2- 172.21.15.47:0/378139022 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4500cc480 0x7fa4500b4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:03.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.697+0000 7fa44effd640 1 -- 172.21.15.47:0/378139022 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4501562f0 con 0x7fa450071e60 2024-04-26T23:20:03.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.697+0000 7fa44e7fc640 1 --2- 172.21.15.47:0/378139022 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa450072780 0x7fa4500b48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:03.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.698+0000 7fa44effd640 1 --2- 172.21.15.47:0/378139022 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa450071e60 0x7fa450152ad0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa44000c500 tx=0x7fa44000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:03.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.698+0000 7fa4377fe640 1 -- 172.21.15.47:0/378139022 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa440018070 con 0x7fa450071e60 2024-04-26T23:20:03.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.698+0000 7fa4377fe640 1 -- 172.21.15.47:0/378139022 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa440007580 con 0x7fa450071e60 2024-04-26T23:20:03.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.698+0000 7fa455346640 1 -- 172.21.15.47:0/378139022 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa450150480 con 0x7fa450071e60 2024-04-26T23:20:03.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.698+0000 7fa4377fe640 1 -- 172.21.15.47:0/378139022 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa440004a20 con 0x7fa450071e60 2024-04-26T23:20:03.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.698+0000 7fa455346640 1 -- 172.21.15.47:0/378139022 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa450153580 con 0x7fa450071e60 2024-04-26T23:20:03.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.699+0000 7fa455346640 1 -- 172.21.15.47:0/378139022 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4500694f0 con 0x7fa450071e60 2024-04-26T23:20:03.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.699+0000 7fa4377fe640 1 -- 172.21.15.47:0/378139022 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa44001b070 con 0x7fa450071e60 2024-04-26T23:20:03.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.700+0000 7fa4377fe640 1 --2- 172.21.15.47:0/378139022 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa43c06c8f0 0x7fa43c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:03.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.700+0000 7fa4377fe640 1 -- 172.21.15.47:0/378139022 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7fa44008b130 con 0x7fa450071e60 2024-04-26T23:20:03.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.700+0000 7fa44e7fc640 1 --2- 172.21.15.47:0/378139022 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa43c06c8f0 0x7fa43c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:03.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.701+0000 7fa44e7fc640 1 --2- 172.21.15.47:0/378139022 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa43c06c8f0 0x7fa43c06edb0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa444006fd0 tx=0x7fa444008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:03.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.704+0000 7fa4377fe640 1 -- 172.21.15.47:0/378139022 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa4400592e0 con 0x7fa450071e60 2024-04-26T23:20:03.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.894+0000 7fa455346640 1 -- 172.21.15.47:0/378139022 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) -- 0x7fa45015a620 con 0x7fa450071e60 2024-04-26T23:20:03.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.894+0000 7fa4377fe640 1 -- 172.21.15.47:0/378139022 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v41) ==== 97+0+11 (secure 0 0 0) 0x7fa440058be0 con 0x7fa450071e60 2024-04-26T23:20:03.896 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-26T23:20:03.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.896+0000 7fa455346640 1 -- 172.21.15.47:0/378139022 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa43c06c8f0 msgr2=0x7fa43c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:03.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.896+0000 7fa455346640 1 --2- 172.21.15.47:0/378139022 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa43c06c8f0 0x7fa43c06edb0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa444006fd0 tx=0x7fa444008040 comp rx=0 tx=0).stop 2024-04-26T23:20:03.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.896+0000 7fa455346640 1 -- 172.21.15.47:0/378139022 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa450071e60 msgr2=0x7fa450152ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:03.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.896+0000 7fa455346640 1 --2- 172.21.15.47:0/378139022 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa450071e60 0x7fa450152ad0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa44000c500 tx=0x7fa44000c9c0 comp rx=0 tx=0).stop 2024-04-26T23:20:03.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.896+0000 7fa455346640 1 -- 172.21.15.47:0/378139022 shutdown_connections 2024-04-26T23:20:03.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.896+0000 7fa455346640 1 --2- 172.21.15.47:0/378139022 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa43c06c8f0 0x7fa43c06edb0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:03.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.896+0000 7fa455346640 1 --2- 172.21.15.47:0/378139022 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4500cc480 0x7fa4500b4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:03.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.896+0000 7fa455346640 1 --2- 172.21.15.47:0/378139022 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa450072780 0x7fa4500b48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:03.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.896+0000 7fa455346640 1 --2- 172.21.15.47:0/378139022 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa450071e60 0x7fa450152ad0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:03.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.896+0000 7fa455346640 1 -- 172.21.15.47:0/378139022 >> 172.21.15.47:0/378139022 conn(0x7fa4500d0be0 msgr2=0x7fa45015ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:03.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.896+0000 7fa455346640 1 -- 172.21.15.47:0/378139022 shutdown_connections 2024-04-26T23:20:03.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:03.896+0000 7fa455346640 1 -- 172.21.15.47:0/378139022 wait complete. 2024-04-26T23:20:03.909 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T23:20:04.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.009+0000 7f6b7f474640 1 Processor -- start 2024-04-26T23:20:04.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.009+0000 7f6b7f474640 1 -- start start 2024-04-26T23:20:04.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.010+0000 7f6b7f474640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6b78162810 0x7f6b78070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:04.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.010+0000 7f6b7f474640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b780d1b30 0x7f6b780d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:04.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.010+0000 7f6b7f474640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b780d10d0 con 0x7f6b780b2340 2024-04-26T23:20:04.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.010+0000 7f6b7f474640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f6b780d0750 con 0x7f6b78162810 2024-04-26T23:20:04.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.010+0000 7f6b7f474640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f6b780d16a0 con 0x7f6b780d1b30 2024-04-26T23:20:04.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.010+0000 7f6b7d210640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6b78162810 0x7f6b78070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:04.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.010+0000 7f6b7ca0f640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f6b780b2340 0x7f6b780cd3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:48872/0 (socket says 172.21.15.47:48872) 2024-04-26T23:20:04.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.010+0000 7f6b7ca0f640 1 -- 172.21.15.47:0/1478147086 learned_addr learned my addr 172.21.15.47:0/1478147086 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:04.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.010+0000 7f6b7d210640 1 -- 172.21.15.47:0/1478147086 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b780d1b30 msgr2=0x7f6b780d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:04.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.010+0000 7f6b7d210640 1 --2- 172.21.15.47:0/1478147086 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b780d1b30 0x7f6b780d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:04.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.010+0000 7f6b7d210640 1 -- 172.21.15.47:0/1478147086 >> v1:172.21.15.47:6789/0 conn(0x7f6b780b2340 legacy=0x7f6b780cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:04.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.010+0000 7f6b6ffff640 1 -- 172.21.15.47:0/1478147086 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 439326821 0 0) 0x7f6b780d10d0 con 0x7f6b780b2340 2024-04-26T23:20:04.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.011+0000 7f6b7d210640 1 -- 172.21.15.47:0/1478147086 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b780d1370 con 0x7f6b78162810 2024-04-26T23:20:04.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.011+0000 7f6b7d210640 1 --2- 172.21.15.47:0/1478147086 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b78162810 0x7f6b78070350 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f6b740097a0 tx=0x7f6b7402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11ecff5a752d73b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:04.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.011+0000 7f6b6ffff640 1 -- 172.21.15.47:0/1478147086 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b74037070 con 0x7f6b78162810 2024-04-26T23:20:04.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.011+0000 7f6b6ffff640 1 -- 172.21.15.47:0/1478147086 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b74032440 con 0x7f6b78162810 2024-04-26T23:20:04.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.011+0000 7f6b6ffff640 1 -- 172.21.15.47:0/1478147086 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b74032810 con 0x7f6b78162810 2024-04-26T23:20:04.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.012+0000 7f6b7f474640 1 -- 172.21.15.47:0/1478147086 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b78162810 msgr2=0x7f6b78070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:04.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.012+0000 7f6b7f474640 1 --2- 172.21.15.47:0/1478147086 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b78162810 0x7f6b78070350 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f6b740097a0 tx=0x7f6b7402a180 comp rx=0 tx=0).stop 2024-04-26T23:20:04.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.012+0000 7f6b7f474640 1 -- 172.21.15.47:0/1478147086 shutdown_connections 2024-04-26T23:20:04.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.012+0000 7f6b7f474640 1 --2- 172.21.15.47:0/1478147086 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b780d1b30 0x7f6b780d1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:04.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.012+0000 7f6b7f474640 1 --2- 172.21.15.47:0/1478147086 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b78162810 0x7f6b78070350 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:04.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.012+0000 7f6b7f474640 1 -- 172.21.15.47:0/1478147086 >> 172.21.15.47:0/1478147086 conn(0x7f6b780d2da0 msgr2=0x7f6b780d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:04.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.012+0000 7f6b7f474640 1 -- 172.21.15.47:0/1478147086 shutdown_connections 2024-04-26T23:20:04.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.012+0000 7f6b7f474640 1 -- 172.21.15.47:0/1478147086 wait complete. 2024-04-26T23:20:04.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.013+0000 7f6b7f474640 1 Processor -- start 2024-04-26T23:20:04.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.013+0000 7f6b7f474640 1 -- start start 2024-04-26T23:20:04.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.014+0000 7f6b7f474640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b780b2340 0x7f6b781500e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:04.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.014+0000 7f6b7f474640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b780d1b30 0x7f6b780c8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:04.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.014+0000 7f6b7f474640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b78162810 0x7f6b780c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:04.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.014+0000 7f6b7f474640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6b78070890 con 0x7f6b780b2340 2024-04-26T23:20:04.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.014+0000 7f6b7f474640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6b780d0470 con 0x7f6b780d1b30 2024-04-26T23:20:04.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.014+0000 7f6b7f474640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f6b780cdb00 con 0x7f6b78162810 2024-04-26T23:20:04.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.014+0000 7f6b7d210640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b780b2340 0x7f6b781500e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:04.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.014+0000 7f6b7d210640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b780b2340 0x7f6b781500e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57894/0 (socket says 172.21.15.47:57894) 2024-04-26T23:20:04.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.014+0000 7f6b7d210640 1 -- 172.21.15.47:0/3463088543 learned_addr learned my addr 172.21.15.47:0/3463088543 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:04.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.014+0000 7f6b7ca0f640 1 --2- 172.21.15.47:0/3463088543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b780d1b30 0x7f6b780c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:04.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.014+0000 7f6b7da11640 1 --2- 172.21.15.47:0/3463088543 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b78162810 0x7f6b780c8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:04.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.014+0000 7f6b7d210640 1 -- 172.21.15.47:0/3463088543 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b78162810 msgr2=0x7f6b780c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:04.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.014+0000 7f6b7d210640 1 --2- 172.21.15.47:0/3463088543 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b78162810 0x7f6b780c8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:04.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.014+0000 7f6b7d210640 1 -- 172.21.15.47:0/3463088543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b780d1b30 msgr2=0x7f6b780c8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:04.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.014+0000 7f6b7d210640 1 --2- 172.21.15.47:0/3463088543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b780d1b30 0x7f6b780c8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:04.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.014+0000 7f6b7d210640 1 -- 172.21.15.47:0/3463088543 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b780cbbd0 con 0x7f6b780b2340 2024-04-26T23:20:04.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.014+0000 7f6b7ca0f640 1 --2- 172.21.15.47:0/3463088543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b780d1b30 0x7f6b780c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:04.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.014+0000 7f6b7da11640 1 --2- 172.21.15.47:0/3463088543 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b78162810 0x7f6b780c8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:04.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.015+0000 7f6b7d210640 1 --2- 172.21.15.47:0/3463088543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b780b2340 0x7f6b781500e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f6b74035040 tx=0x7f6b740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:04.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.015+0000 7f6b6dffb640 1 -- 172.21.15.47:0/3463088543 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b74037040 con 0x7f6b780b2340 2024-04-26T23:20:04.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.015+0000 7f6b6dffb640 1 -- 172.21.15.47:0/3463088543 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b74004250 con 0x7f6b780b2340 2024-04-26T23:20:04.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.015+0000 7f6b7f474640 1 -- 172.21.15.47:0/3463088543 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b780cbeb0 con 0x7f6b780b2340 2024-04-26T23:20:04.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.015+0000 7f6b6dffb640 1 -- 172.21.15.47:0/3463088543 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b74032e20 con 0x7f6b780b2340 2024-04-26T23:20:04.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.015+0000 7f6b7f474640 1 -- 172.21.15.47:0/3463088543 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b780cc470 con 0x7f6b780b2340 2024-04-26T23:20:04.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.016+0000 7f6b7f474640 1 -- 172.21.15.47:0/3463088543 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b780694f0 con 0x7f6b780b2340 2024-04-26T23:20:04.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.016+0000 7f6b6dffb640 1 -- 172.21.15.47:0/3463088543 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6b74030770 con 0x7f6b780b2340 2024-04-26T23:20:04.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.017+0000 7f6b6dffb640 1 --2- 172.21.15.47:0/3463088543 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f6b5c06c920 0x7f6b5c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:04.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.017+0000 7f6b6dffb640 1 -- 172.21.15.47:0/3463088543 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f6b740ad130 con 0x7f6b780b2340 2024-04-26T23:20:04.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.017+0000 7f6b7ca0f640 1 --2- 172.21.15.47:0/3463088543 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f6b5c06c920 0x7f6b5c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:04.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.018+0000 7f6b7ca0f640 1 --2- 172.21.15.47:0/3463088543 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f6b5c06c920 0x7f6b5c06ede0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6b781f8850 tx=0x7f6b68009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:04.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.021+0000 7f6b6dffb640 1 -- 172.21.15.47:0/3463088543 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6b7407b370 con 0x7f6b780b2340 2024-04-26T23:20:04.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.218+0000 7f6b7f474640 1 -- 172.21.15.47:0/3463088543 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6b780b1870 con 0x7f6b780b2340 2024-04-26T23:20:04.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.219+0000 7f6b6dffb640 1 -- 172.21.15.47:0/3463088543 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v41) ==== 80+0+0 (secure 0 0 0) 0x7f6b74081030 con 0x7f6b780b2340 2024-04-26T23:20:04.221 INFO:teuthology.orchestra.run.smithi047.stderr:listed 0 entries 2024-04-26T23:20:04.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.221+0000 7f6b7f474640 1 -- 172.21.15.47:0/3463088543 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f6b5c06c920 msgr2=0x7f6b5c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:04.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.221+0000 7f6b7f474640 1 --2- 172.21.15.47:0/3463088543 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f6b5c06c920 0x7f6b5c06ede0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6b781f8850 tx=0x7f6b68009040 comp rx=0 tx=0).stop 2024-04-26T23:20:04.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.221+0000 7f6b7f474640 1 -- 172.21.15.47:0/3463088543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b780b2340 msgr2=0x7f6b781500e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:04.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.221+0000 7f6b7f474640 1 --2- 172.21.15.47:0/3463088543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b780b2340 0x7f6b781500e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f6b74035040 tx=0x7f6b740046c0 comp rx=0 tx=0).stop 2024-04-26T23:20:04.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.221+0000 7f6b7f474640 1 -- 172.21.15.47:0/3463088543 shutdown_connections 2024-04-26T23:20:04.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.221+0000 7f6b7f474640 1 --2- 172.21.15.47:0/3463088543 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f6b5c06c920 0x7f6b5c06ede0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:04.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.221+0000 7f6b7f474640 1 --2- 172.21.15.47:0/3463088543 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b78162810 0x7f6b780c8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:04.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.221+0000 7f6b7f474640 1 --2- 172.21.15.47:0/3463088543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b780d1b30 0x7f6b780c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:04.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.221+0000 7f6b7f474640 1 --2- 172.21.15.47:0/3463088543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b780b2340 0x7f6b781500e0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:04.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.221+0000 7f6b7f474640 1 -- 172.21.15.47:0/3463088543 >> 172.21.15.47:0/3463088543 conn(0x7f6b780d2da0 msgr2=0x7f6b780cd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:04.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.221+0000 7f6b7f474640 1 -- 172.21.15.47:0/3463088543 shutdown_connections 2024-04-26T23:20:04.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.221+0000 7f6b7f474640 1 -- 172.21.15.47:0/3463088543 wait complete. 2024-04-26T23:20:04.234 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T23:20:04.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:20:04.234 DEBUG:teuthology.orchestra.run.smithi047:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T23:20:04.262 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi047.front.sepia.ceph.com 2024-04-26T23:20:04.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:20:04.263 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T23:20:04.319 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2024-04-26T23:20:04.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.464+0000 7fbce4828640 1 Processor -- start 2024-04-26T23:20:04.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.464+0000 7fbce4828640 1 -- start start 2024-04-26T23:20:04.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.465+0000 7fbce4828640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbce00b40d0 0x7fbce01625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:04.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.465+0000 7fbce4828640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbce0162b30 0x7fbce00b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:04.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.465+0000 7fbce4828640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbce00d0a10 con 0x7fbce00d1a00 2024-04-26T23:20:04.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.465+0000 7fbce4828640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fbce00d0cf0 con 0x7fbce00b40d0 2024-04-26T23:20:04.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.465+0000 7fbce4828640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fbce00d0fd0 con 0x7fbce0162b30 2024-04-26T23:20:04.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.465+0000 7fbcddd9b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbce00b40d0 0x7fbce01625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:04.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.465+0000 7fbcde59c640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fbce00d1a00 0x7fbce00d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:48882/0 (socket says 172.21.15.47:48882) 2024-04-26T23:20:04.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.465+0000 7fbcde59c640 1 -- 172.21.15.47:0/1662357648 learned_addr learned my addr 172.21.15.47:0/1662357648 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:04.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.466+0000 7fbcddd9b640 1 -- 172.21.15.47:0/1662357648 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbce0162b30 msgr2=0x7fbce00b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:04.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.466+0000 7fbcded9d640 1 --2- 172.21.15.47:0/1662357648 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbce0162b30 0x7fbce00b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:04.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.466+0000 7fbcddd9b640 1 --2- 172.21.15.47:0/1662357648 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbce0162b30 0x7fbce00b4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:04.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.466+0000 7fbcddd9b640 1 -- 172.21.15.47:0/1662357648 >> v1:172.21.15.47:6789/0 conn(0x7fbce00d1a00 legacy=0x7fbce00d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:04.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.466+0000 7fbcdd59a640 1 -- 172.21.15.47:0/1662357648 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2449796434 0 0) 0x7fbce00d0a10 con 0x7fbce00d1a00 2024-04-26T23:20:04.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.466+0000 7fbcddd9b640 1 -- 172.21.15.47:0/1662357648 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbce0162f80 con 0x7fbce00b40d0 2024-04-26T23:20:04.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.466+0000 7fbcded9d640 1 --2- 172.21.15.47:0/1662357648 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbce0162b30 0x7fbce00b4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:04.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.466+0000 7fbcddd9b640 1 --2- 172.21.15.47:0/1662357648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbce00b40d0 0x7fbce01625f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fbcd40097a0 tx=0x7fbcd402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c07de63e116016 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:04.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.467+0000 7fbcdd59a640 1 -- 172.21.15.47:0/1662357648 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd4037070 con 0x7fbce00b40d0 2024-04-26T23:20:04.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.467+0000 7fbcdd59a640 1 -- 172.21.15.47:0/1662357648 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcd402ae50 con 0x7fbce00b40d0 2024-04-26T23:20:04.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.467+0000 7fbcdd59a640 1 -- 172.21.15.47:0/1662357648 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd4032660 con 0x7fbce00b40d0 2024-04-26T23:20:04.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.467+0000 7fbce4828640 1 -- 172.21.15.47:0/1662357648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbce00b40d0 msgr2=0x7fbce01625f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:04.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.467+0000 7fbce4828640 1 --2- 172.21.15.47:0/1662357648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbce00b40d0 0x7fbce01625f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fbcd40097a0 tx=0x7fbcd402a100 comp rx=0 tx=0).stop 2024-04-26T23:20:04.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.467+0000 7fbce4828640 1 -- 172.21.15.47:0/1662357648 shutdown_connections 2024-04-26T23:20:04.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.467+0000 7fbce4828640 1 --2- 172.21.15.47:0/1662357648 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbce0162b30 0x7fbce00b4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:04.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.467+0000 7fbce4828640 1 --2- 172.21.15.47:0/1662357648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbce00b40d0 0x7fbce01625f0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:04.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.467+0000 7fbce4828640 1 -- 172.21.15.47:0/1662357648 >> 172.21.15.47:0/1662357648 conn(0x7fbce00d4fd0 msgr2=0x7fbce00d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:04.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.467+0000 7fbce4828640 1 -- 172.21.15.47:0/1662357648 shutdown_connections 2024-04-26T23:20:04.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.467+0000 7fbce4828640 1 -- 172.21.15.47:0/1662357648 wait complete. 2024-04-26T23:20:04.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.469+0000 7fbce4828640 1 Processor -- start 2024-04-26T23:20:04.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.469+0000 7fbce4828640 1 -- start start 2024-04-26T23:20:04.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.469+0000 7fbce4828640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbce00b40d0 0x7fbce00c9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:04.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.469+0000 7fbce4828640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbce00d1a00 0x7fbce00b1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:04.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.469+0000 7fbce4828640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbce0162b30 0x7fbce00b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:04.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.469+0000 7fbce4828640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fbce00d2670 con 0x7fbce00d1a00 2024-04-26T23:20:04.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.469+0000 7fbce4828640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbce00d24f0 con 0x7fbce00b40d0 2024-04-26T23:20:04.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.469+0000 7fbce4828640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fbce00b5040 con 0x7fbce0162b30 2024-04-26T23:20:04.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.470+0000 7fbcddd9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbce00d1a00 0x7fbce00b1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:04.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.470+0000 7fbcddd9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbce00d1a00 0x7fbce00b1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57916/0 (socket says 172.21.15.47:57916) 2024-04-26T23:20:04.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.470+0000 7fbcddd9b640 1 -- 172.21.15.47:0/2047843998 learned_addr learned my addr 172.21.15.47:0/2047843998 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:04.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.470+0000 7fbcde59c640 1 --2- 172.21.15.47:0/2047843998 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbce00b40d0 0x7fbce00c9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:04.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.470+0000 7fbcded9d640 1 --2- 172.21.15.47:0/2047843998 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbce0162b30 0x7fbce00b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:04.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.470+0000 7fbcddd9b640 1 -- 172.21.15.47:0/2047843998 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbce0162b30 msgr2=0x7fbce00b21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:04.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.470+0000 7fbcddd9b640 1 --2- 172.21.15.47:0/2047843998 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbce0162b30 0x7fbce00b21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:04.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.470+0000 7fbcddd9b640 1 -- 172.21.15.47:0/2047843998 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbce00b40d0 msgr2=0x7fbce00c9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:04.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.470+0000 7fbcddd9b640 1 --2- 172.21.15.47:0/2047843998 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbce00b40d0 0x7fbce00c9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:04.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.470+0000 7fbcddd9b640 1 -- 172.21.15.47:0/2047843998 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbce00cc140 con 0x7fbce00d1a00 2024-04-26T23:20:04.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.470+0000 7fbcddd9b640 1 --2- 172.21.15.47:0/2047843998 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbce00d1a00 0x7fbce00b1c60 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fbcd40097a0 tx=0x7fbcd40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:04.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.470+0000 7fbcceffd640 1 -- 172.21.15.47:0/2047843998 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd4037040 con 0x7fbce00d1a00 2024-04-26T23:20:04.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.470+0000 7fbcceffd640 1 -- 172.21.15.47:0/2047843998 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcd4004250 con 0x7fbce00d1a00 2024-04-26T23:20:04.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.470+0000 7fbcceffd640 1 -- 172.21.15.47:0/2047843998 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd4003610 con 0x7fbce00d1a00 2024-04-26T23:20:04.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.470+0000 7fbce4828640 1 -- 172.21.15.47:0/2047843998 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbce00cc3b0 con 0x7fbce00d1a00 2024-04-26T23:20:04.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.470+0000 7fbce4828640 1 -- 172.21.15.47:0/2047843998 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbce00cc970 con 0x7fbce00d1a00 2024-04-26T23:20:04.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.471+0000 7fbce4828640 1 -- 172.21.15.47:0/2047843998 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbce00694f0 con 0x7fbce00d1a00 2024-04-26T23:20:04.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.472+0000 7fbcceffd640 1 -- 172.21.15.47:0/2047843998 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbcd40037b0 con 0x7fbce00d1a00 2024-04-26T23:20:04.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.472+0000 7fbcceffd640 1 --2- 172.21.15.47:0/2047843998 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fbcc006c8f0 0x7fbcc006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:04.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.472+0000 7fbcceffd640 1 -- 172.21.15.47:0/2047843998 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7fbcd40acc80 con 0x7fbce00d1a00 2024-04-26T23:20:04.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.472+0000 7fbcde59c640 1 --2- 172.21.15.47:0/2047843998 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fbcc006c8f0 0x7fbcc006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:04.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.473+0000 7fbcde59c640 1 --2- 172.21.15.47:0/2047843998 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fbcc006c8f0 0x7fbcc006edb0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80012e0 tx=0x7fbcd8001700 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:04.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.477+0000 7fbcceffd640 1 -- 172.21.15.47:0/2047843998 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbcd407ada0 con 0x7fbce00d1a00 2024-04-26T23:20:04.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.734+0000 7fbce4828640 1 -- 172.21.15.47:0/2047843998 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) -- 0x7fbce00d1210 con 0x7fbce00d1a00 2024-04-26T23:20:04.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.736+0000 7fbcceffd640 1 -- 172.21.15.47:0/2047843998 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v13) ==== 184+0+141 (secure 0 0 0) 0x7fbcd407a6a0 con 0x7fbce00d1a00 2024-04-26T23:20:04.738 INFO:teuthology.orchestra.run.smithi047.stdout:[client.0] 2024-04-26T23:20:04.738 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQAKNixmA5QFHRAAT89WPyzDs0acPluyvU4svw== 2024-04-26T23:20:04.739 INFO:teuthology.orchestra.run.smithi047.stdout: caps mds = "allow" 2024-04-26T23:20:04.739 INFO:teuthology.orchestra.run.smithi047.stdout: caps mon = "allow r" 2024-04-26T23:20:04.739 INFO:teuthology.orchestra.run.smithi047.stdout: caps osd = "allow rw pool=cephfs_data" 2024-04-26T23:20:04.739 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.0 2024-04-26T23:20:04.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.739+0000 7fbce4828640 1 -- 172.21.15.47:0/2047843998 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fbcc006c8f0 msgr2=0x7fbcc006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:04.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.739+0000 7fbce4828640 1 --2- 172.21.15.47:0/2047843998 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fbcc006c8f0 0x7fbcc006edb0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80012e0 tx=0x7fbcd8001700 comp rx=0 tx=0).stop 2024-04-26T23:20:04.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.739+0000 7fbce4828640 1 -- 172.21.15.47:0/2047843998 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbce00d1a00 msgr2=0x7fbce00b1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:04.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.739+0000 7fbce4828640 1 --2- 172.21.15.47:0/2047843998 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbce00d1a00 0x7fbce00b1c60 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fbcd40097a0 tx=0x7fbcd40046c0 comp rx=0 tx=0).stop 2024-04-26T23:20:04.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.739+0000 7fbce4828640 1 -- 172.21.15.47:0/2047843998 shutdown_connections 2024-04-26T23:20:04.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.739+0000 7fbce4828640 1 --2- 172.21.15.47:0/2047843998 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fbcc006c8f0 0x7fbcc006edb0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:04.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.739+0000 7fbce4828640 1 --2- 172.21.15.47:0/2047843998 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbce0162b30 0x7fbce00b21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:04.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.739+0000 7fbce4828640 1 --2- 172.21.15.47:0/2047843998 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbce00d1a00 0x7fbce00b1c60 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:04.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.739+0000 7fbce4828640 1 --2- 172.21.15.47:0/2047843998 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbce00b40d0 0x7fbce00c9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:04.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.739+0000 7fbce4828640 1 -- 172.21.15.47:0/2047843998 >> 172.21.15.47:0/2047843998 conn(0x7fbce00d4fd0 msgr2=0x7fbce006da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:04.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.739+0000 7fbce4828640 1 -- 172.21.15.47:0/2047843998 shutdown_connections 2024-04-26T23:20:04.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:04.739+0000 7fbce4828640 1 -- 172.21.15.47:0/2047843998 wait complete. 2024-04-26T23:20:04.752 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T23:20:04.752 DEBUG:teuthology.orchestra.run.smithi047:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:20:04.785 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:20:04.830 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T23:20:04.830 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:20:04.830 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:20:04.830 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:20:04.846 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=361 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T23:20:04.846 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:20:04.846 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-26T23:20:04.846 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:20:04.887 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056088,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:20:04.888 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:20:04.888 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:20:04.888 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:20:04.903 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=361 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T23:20:04.904 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:20:04.904 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-26T23:20:04.904 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:20:04.943 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056088,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:20:04.944 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:20:05.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.043+0000 7f9ed3452640 1 Processor -- start 2024-04-26T23:20:05.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.043+0000 7f9ed3452640 1 -- start start 2024-04-26T23:20:05.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.044+0000 7f9ed3452640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9ecc0d1380 0x7f9ecc0d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:05.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.044+0000 7f9ed3452640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9ecc070ae0 0x7f9ecc070f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:05.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.044+0000 7f9ed3452640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ecc0d1ce0 con 0x7f9ecc15d1e0 2024-04-26T23:20:05.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.044+0000 7f9ed3452640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f9ecc0cede0 con 0x7f9ecc0d1380 2024-04-26T23:20:05.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.044+0000 7f9ed3452640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f9ecc070650 con 0x7f9ecc070ae0 2024-04-26T23:20:05.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.044+0000 7f9ed09ed640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9ecc0d1380 0x7f9ecc0d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:05.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.044+0000 7f9ed19ef640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9ecc070ae0 0x7f9ecc070f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:05.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.044+0000 7f9ed11ee640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f9ecc15d1e0 0x7f9ecc0d09e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.47:43048/0 (socket says 172.21.15.47:43048) 2024-04-26T23:20:05.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.044+0000 7f9ed09ed640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9ecc0d1380 0x7f9ecc0d17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.47:40266/0 (socket says 172.21.15.47:40266) 2024-04-26T23:20:05.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.044+0000 7f9ed11ee640 1 -- 172.21.15.47:0/1225802592 learned_addr learned my addr 172.21.15.47:0/1225802592 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:05.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.045+0000 7f9ec3fff640 1 -- 172.21.15.47:0/1225802592 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3078011647 0 0) 0x7f9ecc0d1ce0 con 0x7f9ecc15d1e0 2024-04-26T23:20:05.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.045+0000 7f9ec3fff640 1 -- 172.21.15.47:0/1225802592 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9eb8003520 con 0x7f9ecc15d1e0 2024-04-26T23:20:05.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.045+0000 7f9ed09ed640 1 -- 172.21.15.47:0/1225802592 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9ecc070ae0 msgr2=0x7f9ecc070f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:05.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.045+0000 7f9ed09ed640 1 --2- 172.21.15.47:0/1225802592 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9ecc070ae0 0x7f9ecc070f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:05.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.045+0000 7f9ed09ed640 1 -- 172.21.15.47:0/1225802592 >> v1:172.21.15.203:6789/0 conn(0x7f9ecc15d1e0 legacy=0x7f9ecc0d09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:05.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.045+0000 7f9ed09ed640 1 -- 172.21.15.47:0/1225802592 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ecc0ce840 con 0x7f9ecc0d1380 2024-04-26T23:20:05.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.045+0000 7f9ed19ef640 1 --2- 172.21.15.47:0/1225802592 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9ecc070ae0 0x7f9ecc070f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:20:05.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.045+0000 7f9ed09ed640 1 --2- 172.21.15.47:0/1225802592 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ecc0d1380 0x7f9ecc0d17a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc0097a0 tx=0x7f9ebc02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe318defd5f8ae40 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:05.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.046+0000 7f9ec3fff640 1 -- 172.21.15.47:0/1225802592 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc037070 con 0x7f9ecc0d1380 2024-04-26T23:20:05.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.046+0000 7f9ec3fff640 1 -- 172.21.15.47:0/1225802592 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ebc032440 con 0x7f9ecc0d1380 2024-04-26T23:20:05.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.046+0000 7f9ec3fff640 1 -- 172.21.15.47:0/1225802592 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc032810 con 0x7f9ecc0d1380 2024-04-26T23:20:05.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.046+0000 7f9ed3452640 1 -- 172.21.15.47:0/1225802592 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ecc0d1380 msgr2=0x7f9ecc0d17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:05.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.046+0000 7f9ed3452640 1 --2- 172.21.15.47:0/1225802592 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ecc0d1380 0x7f9ecc0d17a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc0097a0 tx=0x7f9ebc02a180 comp rx=0 tx=0).stop 2024-04-26T23:20:05.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.046+0000 7f9ed3452640 1 -- 172.21.15.47:0/1225802592 shutdown_connections 2024-04-26T23:20:05.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.046+0000 7f9ed3452640 1 --2- 172.21.15.47:0/1225802592 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9ecc070ae0 0x7f9ecc070f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:05.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.046+0000 7f9ed3452640 1 --2- 172.21.15.47:0/1225802592 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ecc0d1380 0x7f9ecc0d17a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:05.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.046+0000 7f9ed3452640 1 -- 172.21.15.47:0/1225802592 >> 172.21.15.47:0/1225802592 conn(0x7f9ecc0d2da0 msgr2=0x7f9ecc0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:05.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.046+0000 7f9ed3452640 1 -- 172.21.15.47:0/1225802592 shutdown_connections 2024-04-26T23:20:05.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.046+0000 7f9ed3452640 1 -- 172.21.15.47:0/1225802592 wait complete. 2024-04-26T23:20:05.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.047+0000 7f9ed3452640 1 Processor -- start 2024-04-26T23:20:05.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.047+0000 7f9ed3452640 1 -- start start 2024-04-26T23:20:05.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.048+0000 7f9ed3452640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ecc070ae0 0x7f9ecc0c84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:05.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.048+0000 7f9ed3452640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ecc0d1380 0x7f9ecc14b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:05.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.048+0000 7f9ed3452640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9ecc15d1e0 0x7f9ecc0cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:05.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.048+0000 7f9ed3452640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc0d10f0 con 0x7f9ecc070ae0 2024-04-26T23:20:05.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.048+0000 7f9ed3452640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc0702e0 con 0x7f9ecc0d1380 2024-04-26T23:20:05.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.048+0000 7f9ed3452640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f9ecc0ce4c0 con 0x7f9ecc15d1e0 2024-04-26T23:20:05.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.048+0000 7f9ed11ee640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ecc070ae0 0x7f9ecc0c84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:05.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.048+0000 7f9ed11ee640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ecc070ae0 0x7f9ecc0c84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57920/0 (socket says 172.21.15.47:57920) 2024-04-26T23:20:05.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.048+0000 7f9ed11ee640 1 -- 172.21.15.47:0/2071153957 learned_addr learned my addr 172.21.15.47:0/2071153957 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:05.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.048+0000 7f9ed09ed640 1 --2- 172.21.15.47:0/2071153957 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ecc0d1380 0x7f9ecc14b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:05.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.048+0000 7f9ed19ef640 1 --2- 172.21.15.47:0/2071153957 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9ecc15d1e0 0x7f9ecc0cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:05.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.049+0000 7f9ed11ee640 1 -- 172.21.15.47:0/2071153957 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9ecc15d1e0 msgr2=0x7f9ecc0cb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:05.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.049+0000 7f9ed11ee640 1 --2- 172.21.15.47:0/2071153957 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9ecc15d1e0 0x7f9ecc0cb4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:05.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.049+0000 7f9ed11ee640 1 -- 172.21.15.47:0/2071153957 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ecc0d1380 msgr2=0x7f9ecc14b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:05.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.049+0000 7f9ed11ee640 1 --2- 172.21.15.47:0/2071153957 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ecc0d1380 0x7f9ecc14b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:05.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.049+0000 7f9ed11ee640 1 -- 172.21.15.47:0/2071153957 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ecc0c8ce0 con 0x7f9ecc070ae0 2024-04-26T23:20:05.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.049+0000 7f9ed11ee640 1 --2- 172.21.15.47:0/2071153957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ecc070ae0 0x7f9ecc0c84b0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f9ec800c3c0 tx=0x7f9ec800c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:05.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.049+0000 7f9ec1ffb640 1 -- 172.21.15.47:0/2071153957 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8017070 con 0x7f9ecc070ae0 2024-04-26T23:20:05.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.049+0000 7f9ec1ffb640 1 -- 172.21.15.47:0/2071153957 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ec8005ce0 con 0x7f9ecc070ae0 2024-04-26T23:20:05.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.049+0000 7f9ec1ffb640 1 -- 172.21.15.47:0/2071153957 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec80072d0 con 0x7f9ecc070ae0 2024-04-26T23:20:05.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.049+0000 7f9ed3452640 1 -- 172.21.15.47:0/2071153957 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ecc0c8fc0 con 0x7f9ecc070ae0 2024-04-26T23:20:05.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.049+0000 7f9ed3452640 1 -- 172.21.15.47:0/2071153957 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ecc0c9580 con 0x7f9ecc070ae0 2024-04-26T23:20:05.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.050+0000 7f9ed3452640 1 -- 172.21.15.47:0/2071153957 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ecc0694f0 con 0x7f9ecc070ae0 2024-04-26T23:20:05.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.051+0000 7f9ec1ffb640 1 -- 172.21.15.47:0/2071153957 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9ec801a070 con 0x7f9ecc070ae0 2024-04-26T23:20:05.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.051+0000 7f9ec1ffb640 1 --2- 172.21.15.47:0/2071153957 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9eb006c8f0 0x7f9eb006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:05.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.051+0000 7f9ec1ffb640 1 -- 172.21.15.47:0/2071153957 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f9ec808b120 con 0x7f9ecc070ae0 2024-04-26T23:20:05.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.052+0000 7f9ed09ed640 1 --2- 172.21.15.47:0/2071153957 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9eb006c8f0 0x7f9eb006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:05.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.052+0000 7f9ed09ed640 1 --2- 172.21.15.47:0/2071153957 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9eb006c8f0 0x7f9eb006edb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc034000 tx=0x7f9ebc02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:05.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.056+0000 7f9ec1ffb640 1 -- 172.21.15.47:0/2071153957 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9ec8058bf0 con 0x7f9ecc070ae0 2024-04-26T23:20:05.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.278+0000 7f9ed3452640 1 -- 172.21.15.47:0/2071153957 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9ecc0b2050 con 0x7f9ecc070ae0 2024-04-26T23:20:05.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.279+0000 7f9ec1ffb640 1 -- 172.21.15.47:0/2071153957 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+5191 (secure 0 0 0) 0x7f9ec805cef0 con 0x7f9ecc070ae0 2024-04-26T23:20:05.281 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:20:05.281 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5080,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/1970124054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1970124054},{"type":"v1","addr":"172.21.15.203:6836","nonce":1970124054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5310,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/889573201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":889573201},{"type":"v1","addr":"172.21.15.47:6834","nonce":889573201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5316,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3898612760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":3898612760},{"type":"v1","addr":"172.21.15.47:6835","nonce":3898612760}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:19:44.091353+0000","modified":"2024-04-26T23:19:53.135935+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":14738},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14738":{"gid":14738,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/2526477051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":2526477051},{"type":"v1","addr":"172.21.15.203:6837","nonce":2526477051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":14738,"qdb_cluster":[14738]},"id":2}]} 2024-04-26T23:20:05.282 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 19 2024-04-26T23:20:05.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.281+0000 7f9ed3452640 1 -- 172.21.15.47:0/2071153957 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9eb006c8f0 msgr2=0x7f9eb006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:05.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.281+0000 7f9ed3452640 1 --2- 172.21.15.47:0/2071153957 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9eb006c8f0 0x7f9eb006edb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc034000 tx=0x7f9ebc02e000 comp rx=0 tx=0).stop 2024-04-26T23:20:05.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.281+0000 7f9ed3452640 1 -- 172.21.15.47:0/2071153957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ecc070ae0 msgr2=0x7f9ecc0c84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:05.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.281+0000 7f9ed3452640 1 --2- 172.21.15.47:0/2071153957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ecc070ae0 0x7f9ecc0c84b0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f9ec800c3c0 tx=0x7f9ec800c7c0 comp rx=0 tx=0).stop 2024-04-26T23:20:05.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.281+0000 7f9ed3452640 1 -- 172.21.15.47:0/2071153957 shutdown_connections 2024-04-26T23:20:05.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.281+0000 7f9ed3452640 1 --2- 172.21.15.47:0/2071153957 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9eb006c8f0 0x7f9eb006edb0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:05.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.281+0000 7f9ed3452640 1 --2- 172.21.15.47:0/2071153957 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9ecc15d1e0 0x7f9ecc0cb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:05.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.281+0000 7f9ed3452640 1 --2- 172.21.15.47:0/2071153957 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ecc0d1380 0x7f9ecc14b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:05.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.281+0000 7f9ed3452640 1 --2- 172.21.15.47:0/2071153957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ecc070ae0 0x7f9ecc0c84b0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:05.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.281+0000 7f9ed3452640 1 -- 172.21.15.47:0/2071153957 >> 172.21.15.47:0/2071153957 conn(0x7f9ecc0d2da0 msgr2=0x7f9ecc0b2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:05.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.281+0000 7f9ed3452640 1 -- 172.21.15.47:0/2071153957 shutdown_connections 2024-04-26T23:20:05.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.281+0000 7f9ed3452640 1 -- 172.21.15.47:0/2071153957 wait complete. 2024-04-26T23:20:05.294 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:20:05.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.392+0000 7f2020178640 1 Processor -- start 2024-04-26T23:20:05.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.393+0000 7f2020178640 1 -- start start 2024-04-26T23:20:05.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.393+0000 7f2020178640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2018071e60 0x7f201806f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:05.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.393+0000 7f2020178640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f201806fe20 0x7f2018072240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:05.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.393+0000 7f2020178640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f201815d7c0 con 0x7f2018072780 2024-04-26T23:20:05.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.393+0000 7f2020178640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f201815daa0 con 0x7f201806fe20 2024-04-26T23:20:05.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.393+0000 7f2020178640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f201815dd80 con 0x7f2018071e60 2024-04-26T23:20:05.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.394+0000 7f201d713640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f201806fe20 0x7f2018072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:05.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.394+0000 7f201e715640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f2018072780 0x7f20180cb470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:48894/0 (socket says 172.21.15.47:48894) 2024-04-26T23:20:05.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.394+0000 7f201e715640 1 -- 172.21.15.47:0/3615281037 learned_addr learned my addr 172.21.15.47:0/3615281037 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:05.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.394+0000 7f201d713640 1 -- 172.21.15.47:0/3615281037 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2018071e60 msgr2=0x7f201806f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:05.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.394+0000 7f201df14640 1 --2- 172.21.15.47:0/3615281037 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2018071e60 0x7f201806f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:05.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.394+0000 7f201d713640 1 --2- 172.21.15.47:0/3615281037 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2018071e60 0x7f201806f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:05.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.394+0000 7f201cf12640 1 -- 172.21.15.47:0/3615281037 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 767382917 0 0) 0x7f201815d7c0 con 0x7f2018072780 2024-04-26T23:20:05.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.394+0000 7f201d713640 1 -- 172.21.15.47:0/3615281037 >> v1:172.21.15.47:6789/0 conn(0x7f2018072780 legacy=0x7f20180cb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:05.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.394+0000 7f201d713640 1 -- 172.21.15.47:0/3615281037 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f201815e060 con 0x7f201806fe20 2024-04-26T23:20:05.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.394+0000 7f201df14640 1 --2- 172.21.15.47:0/3615281037 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2018071e60 0x7f201806f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:05.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.395+0000 7f201d713640 1 --2- 172.21.15.47:0/3615281037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f201806fe20 0x7f2018072240 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f20080097a0 tx=0x7f200802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3116eb78eb5a298 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:05.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.395+0000 7f201cf12640 1 -- 172.21.15.47:0/3615281037 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2008037070 con 0x7f201806fe20 2024-04-26T23:20:05.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.395+0000 7f201cf12640 1 -- 172.21.15.47:0/3615281037 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2008032440 con 0x7f201806fe20 2024-04-26T23:20:05.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.395+0000 7f201cf12640 1 -- 172.21.15.47:0/3615281037 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2008032810 con 0x7f201806fe20 2024-04-26T23:20:05.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.395+0000 7f2020178640 1 -- 172.21.15.47:0/3615281037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f201806fe20 msgr2=0x7f2018072240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:05.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.395+0000 7f2020178640 1 --2- 172.21.15.47:0/3615281037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f201806fe20 0x7f2018072240 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f20080097a0 tx=0x7f200802a180 comp rx=0 tx=0).stop 2024-04-26T23:20:05.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.395+0000 7f2020178640 1 -- 172.21.15.47:0/3615281037 shutdown_connections 2024-04-26T23:20:05.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.395+0000 7f2020178640 1 --2- 172.21.15.47:0/3615281037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f201806fe20 0x7f2018072240 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:05.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.395+0000 7f2020178640 1 --2- 172.21.15.47:0/3615281037 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2018071e60 0x7f201806f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:05.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.395+0000 7f2020178640 1 -- 172.21.15.47:0/3615281037 >> 172.21.15.47:0/3615281037 conn(0x7f20180d0be0 msgr2=0x7f20180d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:05.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.395+0000 7f2020178640 1 -- 172.21.15.47:0/3615281037 shutdown_connections 2024-04-26T23:20:05.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.395+0000 7f2020178640 1 -- 172.21.15.47:0/3615281037 wait complete. 2024-04-26T23:20:05.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.397+0000 7f2020178640 1 Processor -- start 2024-04-26T23:20:05.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.397+0000 7f2020178640 1 -- start start 2024-04-26T23:20:05.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.397+0000 7f2020178640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f201806fe20 0x7f20180c7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:05.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.397+0000 7f2020178640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2018071e60 0x7f201814f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:05.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.397+0000 7f2020178640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2018072780 0x7f20180d0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:05.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.397+0000 7f2020178640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f201815d3f0 con 0x7f201806fe20 2024-04-26T23:20:05.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.397+0000 7f2020178640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2018072c10 con 0x7f2018071e60 2024-04-26T23:20:05.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.397+0000 7f2020178640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f201815d570 con 0x7f2018072780 2024-04-26T23:20:05.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.397+0000 7f201df14640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f201806fe20 0x7f20180c7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:05.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.397+0000 7f201df14640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f201806fe20 0x7f20180c7ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57948/0 (socket says 172.21.15.47:57948) 2024-04-26T23:20:05.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.397+0000 7f201df14640 1 -- 172.21.15.47:0/1595872982 learned_addr learned my addr 172.21.15.47:0/1595872982 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:05.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.398+0000 7f201d713640 1 --2- 172.21.15.47:0/1595872982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2018071e60 0x7f201814f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:05.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.398+0000 7f201e715640 1 --2- 172.21.15.47:0/1595872982 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2018072780 0x7f20180d0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:05.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.398+0000 7f201df14640 1 -- 172.21.15.47:0/1595872982 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2018072780 msgr2=0x7f20180d0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:05.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.398+0000 7f201df14640 1 --2- 172.21.15.47:0/1595872982 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2018072780 0x7f20180d0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:05.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.398+0000 7f201df14640 1 -- 172.21.15.47:0/1595872982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2018071e60 msgr2=0x7f201814f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:05.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.398+0000 7f201df14640 1 --2- 172.21.15.47:0/1595872982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2018071e60 0x7f201814f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:05.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.398+0000 7f201df14640 1 -- 172.21.15.47:0/1595872982 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20180d06c0 con 0x7f201806fe20 2024-04-26T23:20:05.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.398+0000 7f201d713640 1 --2- 172.21.15.47:0/1595872982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2018071e60 0x7f201814f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:05.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.398+0000 7f201e715640 1 --2- 172.21.15.47:0/1595872982 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2018072780 0x7f20180d0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:05.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.398+0000 7f201df14640 1 --2- 172.21.15.47:0/1595872982 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f201806fe20 0x7f20180c7ed0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f201400c7f0 tx=0x7f201400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:05.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.398+0000 7f200e7fc640 1 -- 172.21.15.47:0/1595872982 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2014015070 con 0x7f201806fe20 2024-04-26T23:20:05.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.398+0000 7f200e7fc640 1 -- 172.21.15.47:0/1595872982 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2014007a30 con 0x7f201806fe20 2024-04-26T23:20:05.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.398+0000 7f2020178640 1 -- 172.21.15.47:0/1595872982 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20180ce7b0 con 0x7f201806fe20 2024-04-26T23:20:05.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.398+0000 7f200e7fc640 1 -- 172.21.15.47:0/1595872982 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2014007e00 con 0x7f201806fe20 2024-04-26T23:20:05.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.398+0000 7f2020178640 1 -- 172.21.15.47:0/1595872982 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20180ced70 con 0x7f201806fe20 2024-04-26T23:20:05.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.399+0000 7f2020178640 1 -- 172.21.15.47:0/1595872982 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20180694f0 con 0x7f201806fe20 2024-04-26T23:20:05.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.399+0000 7f200e7fc640 1 -- 172.21.15.47:0/1595872982 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2014018070 con 0x7f201806fe20 2024-04-26T23:20:05.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.400+0000 7f200e7fc640 1 --2- 172.21.15.47:0/1595872982 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1ffc06c8f0 0x7f1ffc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:05.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.400+0000 7f200e7fc640 1 -- 172.21.15.47:0/1595872982 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f201408be20 con 0x7f201806fe20 2024-04-26T23:20:05.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.400+0000 7f201d713640 1 --2- 172.21.15.47:0/1595872982 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1ffc06c8f0 0x7f1ffc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:05.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.401+0000 7f201d713640 1 --2- 172.21.15.47:0/1595872982 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1ffc06c8f0 0x7f1ffc06edb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2008034000 tx=0x7f200802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:05.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.404+0000 7f200e7fc640 1 -- 172.21.15.47:0/1595872982 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2014059f40 con 0x7f201806fe20 2024-04-26T23:20:05.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.589+0000 7f2020178640 1 -- 172.21.15.47:0/1595872982 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f20180cc730 con 0x7f201806fe20 2024-04-26T23:20:05.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.590+0000 7f200e7fc640 1 -- 172.21.15.47:0/1595872982 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+18718 (secure 0 0 0) 0x7f2014059840 con 0x7f201806fe20 2024-04-26T23:20:05.592 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:20:05.593 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":41,"fsid":"399006d3-4db1-43e6-ba81-a79a276eabfd","created":"2024-04-26T23:18:06.731211+0000","modified":"2024-04-26T23:19:54.781247+0000","last_up_change":"2024-04-26T23:18:17.808736+0000","last_in_change":"2024-04-26T23:18:08.945859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:18:16.937492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:19:42.351747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-26T23:19:43.037763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2024-04-26T23:19:51.911275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ee091567-fea0-4a14-bf4e-0e16c20bb2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6819","nonce":326520779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6821","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6823","nonce":326520779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6829","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6831","nonce":326520779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6825","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6827","nonce":326520779}]},"public_addr":"172.21.15.47:6819/326520779","cluster_addr":"172.21.15.47:6823/326520779","heartbeat_back_addr":"172.21.15.47:6831/326520779","heartbeat_front_addr":"172.21.15.47:6827/326520779","state":["exists","up"]},{"osd":1,"uuid":"c14440a0-bba5-4931-a6db-b736c08e869b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6809","nonce":1912610931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6811","nonce":1912610931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6815","nonce":1912610931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6813","nonce":1912610931}]},"public_addr":"172.21.15.47:6809/1912610931","cluster_addr":"172.21.15.47:6811/1912610931","heartbeat_back_addr":"172.21.15.47:6815/1912610931","heartbeat_front_addr":"172.21.15.47:6813/1912610931","state":["exists","up"]},{"osd":2,"uuid":"3940f8de-c9b2-4cd3-a911-9c301729eb30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6801","nonce":2569869305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6803","nonce":2569869305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6807","nonce":2569869305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6805","nonce":2569869305}]},"public_addr":"172.21.15.47:6801/2569869305","cluster_addr":"172.21.15.47:6803/2569869305","heartbeat_back_addr":"172.21.15.47:6807/2569869305","heartbeat_front_addr":"172.21.15.47:6805/2569869305","state":["exists","up"]},{"osd":3,"uuid":"1286a67e-8958-426a-8c2e-bc85498c619c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6818","nonce":2791411683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6822","nonce":2791411683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6830","nonce":2791411683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6826","nonce":2791411683}]},"public_addr":"172.21.15.47:6818/2791411683","cluster_addr":"172.21.15.47:6822/2791411683","heartbeat_back_addr":"172.21.15.47:6830/2791411683","heartbeat_front_addr":"172.21.15.47:6826/2791411683","state":["exists","up"]},{"osd":4,"uuid":"d8ee2c20-e5b7-46c2-9999-b234070126cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6809","nonce":441313104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6811","nonce":441313104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6815","nonce":441313104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6813","nonce":441313104}]},"public_addr":"172.21.15.203:6809/441313104","cluster_addr":"172.21.15.203:6811/441313104","heartbeat_back_addr":"172.21.15.203:6815/441313104","heartbeat_front_addr":"172.21.15.203:6813/441313104","state":["exists","up"]},{"osd":5,"uuid":"b453a8f0-f324-4840-9ae0-d67bdadb9f8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6801","nonce":56838444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6803","nonce":56838444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6807","nonce":56838444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6805","nonce":56838444}]},"public_addr":"172.21.15.203:6801/56838444","cluster_addr":"172.21.15.203:6803/56838444","heartbeat_back_addr":"172.21.15.203:6807/56838444","heartbeat_front_addr":"172.21.15.203:6805/56838444","state":["exists","up"]},{"osd":6,"uuid":"a6d31951-2d4b-4e8d-a510-6283b01dc158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6825","nonce":2738517080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6827","nonce":2738517080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6831","nonce":2738517080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6829","nonce":2738517080}]},"public_addr":"172.21.15.203:6825/2738517080","cluster_addr":"172.21.15.203:6827/2738517080","heartbeat_back_addr":"172.21.15.203:6831/2738517080","heartbeat_front_addr":"172.21.15.203:6829/2738517080","state":["exists","up"]},{"osd":7,"uuid":"7119b8c0-fe96-402d-9e75-7b509d9d44d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6817","nonce":1315565058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6819","nonce":1315565058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6823","nonce":1315565058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6821","nonce":1315565058}]},"public_addr":"172.21.15.203:6817/1315565058","cluster_addr":"172.21.15.203:6819/1315565058","heartbeat_back_addr":"172.21.15.203:6823/1315565058","heartbeat_front_addr":"172.21.15.203:6821/1315565058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.121921+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.137494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.084814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.111593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.072430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.125066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.097867+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.057061+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:20:05.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.592+0000 7f2020178640 1 -- 172.21.15.47:0/1595872982 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1ffc06c8f0 msgr2=0x7f1ffc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:05.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.592+0000 7f2020178640 1 --2- 172.21.15.47:0/1595872982 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1ffc06c8f0 0x7f1ffc06edb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2008034000 tx=0x7f200802e000 comp rx=0 tx=0).stop 2024-04-26T23:20:05.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.592+0000 7f2020178640 1 -- 172.21.15.47:0/1595872982 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f201806fe20 msgr2=0x7f20180c7ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:05.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.592+0000 7f2020178640 1 --2- 172.21.15.47:0/1595872982 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f201806fe20 0x7f20180c7ed0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f201400c7f0 tx=0x7f201400cbf0 comp rx=0 tx=0).stop 2024-04-26T23:20:05.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.592+0000 7f2020178640 1 -- 172.21.15.47:0/1595872982 shutdown_connections 2024-04-26T23:20:05.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.592+0000 7f2020178640 1 --2- 172.21.15.47:0/1595872982 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1ffc06c8f0 0x7f1ffc06edb0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:05.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.592+0000 7f2020178640 1 --2- 172.21.15.47:0/1595872982 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2018072780 0x7f20180d0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:05.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.592+0000 7f2020178640 1 --2- 172.21.15.47:0/1595872982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2018071e60 0x7f201814f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:05.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.592+0000 7f2020178640 1 --2- 172.21.15.47:0/1595872982 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f201806fe20 0x7f20180c7ed0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:05.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.592+0000 7f2020178640 1 -- 172.21.15.47:0/1595872982 >> 172.21.15.47:0/1595872982 conn(0x7f20180d0be0 msgr2=0x7f2018159120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:05.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.592+0000 7f2020178640 1 -- 172.21.15.47:0/1595872982 shutdown_connections 2024-04-26T23:20:05.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.592+0000 7f2020178640 1 -- 172.21.15.47:0/1595872982 wait complete. 2024-04-26T23:20:05.605 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:18:16.937492+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:19:42.351747+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:19:43.037763+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'data_new', 'create_time': '2024-04-26T23:19:51.911275+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:20:05.606 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:20:05.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.708+0000 7fa0b9538640 1 Processor -- start 2024-04-26T23:20:05.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.708+0000 7fa0b9538640 1 -- start start 2024-04-26T23:20:05.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.709+0000 7fa0b9538640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa0b40cfde0 0x7fa0b40d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:05.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.709+0000 7fa0b9538640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0b4071260 0x7fa0b415e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:05.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.709+0000 7fa0b9538640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0b40d2390 con 0x7fa0b4070350 2024-04-26T23:20:05.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.709+0000 7fa0b9538640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fa0b40afab0 con 0x7fa0b40cfde0 2024-04-26T23:20:05.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.709+0000 7fa0b9538640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa0b406dd20 con 0x7fa0b4071260 2024-04-26T23:20:05.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.709+0000 7fa0b2ffd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa0b40cfde0 0x7fa0b40d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:05.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.709+0000 7fa0b2ffd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa0b40cfde0 0x7fa0b40d1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57954/0 (socket says 172.21.15.47:57954) 2024-04-26T23:20:05.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.709+0000 7fa0b2ffd640 1 -- 172.21.15.47:0/3634128007 learned_addr learned my addr 172.21.15.47:0/3634128007 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:05.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.709+0000 7fa0b37fe640 1 --2- 172.21.15.47:0/3634128007 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0b4071260 0x7fa0b415e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:05.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.709+0000 7fa0b2ffd640 1 -- 172.21.15.47:0/3634128007 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0b4071260 msgr2=0x7fa0b415e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:05.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.709+0000 7fa0b2ffd640 1 --2- 172.21.15.47:0/3634128007 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0b4071260 0x7fa0b415e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:05.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.709+0000 7fa0b2ffd640 1 -- 172.21.15.47:0/3634128007 >> v1:172.21.15.203:6789/0 conn(0x7fa0b4070350 legacy=0x7fa0b4070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:05.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.709+0000 7fa0b2ffd640 1 -- 172.21.15.47:0/3634128007 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b406d3f0 con 0x7fa0b40cfde0 2024-04-26T23:20:05.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.709+0000 7fa0b37fe640 1 --2- 172.21.15.47:0/3634128007 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0b4071260 0x7fa0b415e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:05.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.710+0000 7fa0b2ffd640 1 --2- 172.21.15.47:0/3634128007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0b40cfde0 0x7fa0b40d1b60 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4009870 tx=0x7fa0a402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7d65b96f4d81c1f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:05.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.710+0000 7fa0b27fc640 1 -- 172.21.15.47:0/3634128007 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a4037070 con 0x7fa0b40cfde0 2024-04-26T23:20:05.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.710+0000 7fa0b27fc640 1 -- 172.21.15.47:0/3634128007 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0a4032440 con 0x7fa0b40cfde0 2024-04-26T23:20:05.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.710+0000 7fa0b27fc640 1 -- 172.21.15.47:0/3634128007 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a4032810 con 0x7fa0b40cfde0 2024-04-26T23:20:05.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.710+0000 7fa0b9538640 1 -- 172.21.15.47:0/3634128007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0b40cfde0 msgr2=0x7fa0b40d1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:05.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.711+0000 7fa0b9538640 1 --2- 172.21.15.47:0/3634128007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0b40cfde0 0x7fa0b40d1b60 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4009870 tx=0x7fa0a402a180 comp rx=0 tx=0).stop 2024-04-26T23:20:05.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.711+0000 7fa0b9538640 1 -- 172.21.15.47:0/3634128007 shutdown_connections 2024-04-26T23:20:05.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.711+0000 7fa0b9538640 1 --2- 172.21.15.47:0/3634128007 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0b4071260 0x7fa0b415e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:05.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.711+0000 7fa0b9538640 1 --2- 172.21.15.47:0/3634128007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0b40cfde0 0x7fa0b40d1b60 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:05.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.711+0000 7fa0b9538640 1 -- 172.21.15.47:0/3634128007 >> 172.21.15.47:0/3634128007 conn(0x7fa0b40d2d80 msgr2=0x7fa0b40d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:05.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.711+0000 7fa0b9538640 1 -- 172.21.15.47:0/3634128007 shutdown_connections 2024-04-26T23:20:05.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.711+0000 7fa0b9538640 1 -- 172.21.15.47:0/3634128007 wait complete. 2024-04-26T23:20:05.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.712+0000 7fa0b9538640 1 Processor -- start 2024-04-26T23:20:05.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.712+0000 7fa0b9538640 1 -- start start 2024-04-26T23:20:05.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.712+0000 7fa0b9538640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0b4070350 0x7fa0b409eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:05.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.712+0000 7fa0b9538640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0b4071260 0x7fa0b40c9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:05.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.712+0000 7fa0b9538640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0b40cfde0 0x7fa0b415ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:05.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.712+0000 7fa0b9538640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b40d20a0 con 0x7fa0b40cfde0 2024-04-26T23:20:05.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.712+0000 7fa0b9538640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b406d9b0 con 0x7fa0b4070350 2024-04-26T23:20:05.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.712+0000 7fa0b9538640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa0b415ef30 con 0x7fa0b4071260 2024-04-26T23:20:05.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.713+0000 7fa0b37fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0b40cfde0 0x7fa0b415ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:05.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.713+0000 7fa0b37fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0b40cfde0 0x7fa0b415ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57960/0 (socket says 172.21.15.47:57960) 2024-04-26T23:20:05.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.713+0000 7fa0b37fe640 1 -- 172.21.15.47:0/773981039 learned_addr learned my addr 172.21.15.47:0/773981039 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:05.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.713+0000 7fa0b2ffd640 1 --2- 172.21.15.47:0/773981039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0b4070350 0x7fa0b409eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:05.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.713+0000 7fa0abfff640 1 --2- 172.21.15.47:0/773981039 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0b4071260 0x7fa0b40c9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:05.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.713+0000 7fa0b37fe640 1 -- 172.21.15.47:0/773981039 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0b4071260 msgr2=0x7fa0b40c9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:05.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.713+0000 7fa0b37fe640 1 --2- 172.21.15.47:0/773981039 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0b4071260 0x7fa0b40c9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:05.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.713+0000 7fa0b37fe640 1 -- 172.21.15.47:0/773981039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0b4070350 msgr2=0x7fa0b409eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:05.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.713+0000 7fa0b37fe640 1 --2- 172.21.15.47:0/773981039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0b4070350 0x7fa0b409eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:05.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.713+0000 7fa0b37fe640 1 -- 172.21.15.47:0/773981039 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b409cd60 con 0x7fa0b40cfde0 2024-04-26T23:20:05.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.713+0000 7fa0b37fe640 1 --2- 172.21.15.47:0/773981039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0b40cfde0 0x7fa0b415ceb0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac00ce70 tx=0x7fa0ac00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:05.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.713+0000 7fa0ab7fe640 1 -- 172.21.15.47:0/773981039 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac003b90 con 0x7fa0b40cfde0 2024-04-26T23:20:05.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.713+0000 7fa0ab7fe640 1 -- 172.21.15.47:0/773981039 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0ac003d50 con 0x7fa0b40cfde0 2024-04-26T23:20:05.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.713+0000 7fa0ab7fe640 1 -- 172.21.15.47:0/773981039 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac004260 con 0x7fa0b40cfde0 2024-04-26T23:20:05.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.713+0000 7fa0b9538640 1 -- 172.21.15.47:0/773981039 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0b409d040 con 0x7fa0b40cfde0 2024-04-26T23:20:05.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.713+0000 7fa0b9538640 1 -- 172.21.15.47:0/773981039 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0b409d600 con 0x7fa0b40cfde0 2024-04-26T23:20:05.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.714+0000 7fa0b9538640 1 -- 172.21.15.47:0/773981039 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0b40694f0 con 0x7fa0b40cfde0 2024-04-26T23:20:05.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.715+0000 7fa0ab7fe640 1 -- 172.21.15.47:0/773981039 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa0ac004400 con 0x7fa0b40cfde0 2024-04-26T23:20:05.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.715+0000 7fa0ab7fe640 1 --2- 172.21.15.47:0/773981039 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa09406c920 0x7fa09406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:05.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.715+0000 7fa0ab7fe640 1 -- 172.21.15.47:0/773981039 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7fa0ac08c670 con 0x7fa0b40cfde0 2024-04-26T23:20:05.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.715+0000 7fa0b2ffd640 1 --2- 172.21.15.47:0/773981039 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa09406c920 0x7fa09406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:05.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.716+0000 7fa0b2ffd640 1 --2- 172.21.15.47:0/773981039 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa09406c920 0x7fa09406ede0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4034000 tx=0x7fa0a402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:05.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.719+0000 7fa0ab7fe640 1 -- 172.21.15.47:0/773981039 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa0ac05a1d0 con 0x7fa0b40cfde0 2024-04-26T23:20:05.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.904+0000 7fa0b9538640 1 -- 172.21.15.47:0/773981039 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa0b40cf5e0 con 0x7fa0b40cfde0 2024-04-26T23:20:05.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.904+0000 7fa0ab7fe640 1 -- 172.21.15.47:0/773981039 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) ==== 93+0+10 (secure 0 0 0) 0x7fa0ac05e4d0 con 0x7fa0b40cfde0 2024-04-26T23:20:05.907 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-26T23:20:05.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.906+0000 7fa0b9538640 1 -- 172.21.15.47:0/773981039 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa09406c920 msgr2=0x7fa09406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:05.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.906+0000 7fa0b9538640 1 --2- 172.21.15.47:0/773981039 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa09406c920 0x7fa09406ede0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4034000 tx=0x7fa0a402e000 comp rx=0 tx=0).stop 2024-04-26T23:20:05.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.906+0000 7fa0b9538640 1 -- 172.21.15.47:0/773981039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0b40cfde0 msgr2=0x7fa0b415ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:05.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.906+0000 7fa0b9538640 1 --2- 172.21.15.47:0/773981039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0b40cfde0 0x7fa0b415ceb0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac00ce70 tx=0x7fa0ac00a710 comp rx=0 tx=0).stop 2024-04-26T23:20:05.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.906+0000 7fa0b9538640 1 -- 172.21.15.47:0/773981039 shutdown_connections 2024-04-26T23:20:05.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.906+0000 7fa0b9538640 1 --2- 172.21.15.47:0/773981039 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa09406c920 0x7fa09406ede0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:05.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.906+0000 7fa0b9538640 1 --2- 172.21.15.47:0/773981039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0b40cfde0 0x7fa0b415ceb0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:05.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.906+0000 7fa0b9538640 1 --2- 172.21.15.47:0/773981039 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0b4071260 0x7fa0b40c9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:05.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.906+0000 7fa0b9538640 1 --2- 172.21.15.47:0/773981039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0b4070350 0x7fa0b409eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:05.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.906+0000 7fa0b9538640 1 -- 172.21.15.47:0/773981039 >> 172.21.15.47:0/773981039 conn(0x7fa0b40d2d80 msgr2=0x7fa0b41625c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:05.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.906+0000 7fa0b9538640 1 -- 172.21.15.47:0/773981039 shutdown_connections 2024-04-26T23:20:05.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:05.906+0000 7fa0b9538640 1 -- 172.21.15.47:0/773981039 wait complete. 2024-04-26T23:20:05.920 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:20:06.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.022+0000 7f75620cf640 1 Processor -- start 2024-04-26T23:20:06.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.022+0000 7f75620cf640 1 -- start start 2024-04-26T23:20:06.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.023+0000 7f75620cf640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f755c15f170 0x7f755c0d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:06.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.023+0000 7f75620cf640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f755c0d3bc0 0x7f755c0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:06.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.023+0000 7f75620cf640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f755c0d1040 con 0x7f755c15f550 2024-04-26T23:20:06.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.023+0000 7f75620cf640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f755c0c9430 con 0x7f755c0d3bc0 2024-04-26T23:20:06.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.023+0000 7f75620cf640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f755c0c9680 con 0x7f755c15f170 2024-04-26T23:20:06.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.023+0000 7f755bfff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f755c0d3bc0 0x7f755c0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:06.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.023+0000 7f755affd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f755c15f550 0x7f755c0d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:48910/0 (socket says 172.21.15.47:48910) 2024-04-26T23:20:06.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.023+0000 7f755affd640 1 -- 172.21.15.47:0/3560518678 learned_addr learned my addr 172.21.15.47:0/3560518678 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:06.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.023+0000 7f755b7fe640 1 --2- 172.21.15.47:0/3560518678 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f755c15f170 0x7f755c0d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:06.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.023+0000 7f755bfff640 1 -- 172.21.15.47:0/3560518678 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f755c15f170 msgr2=0x7f755c0d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:06.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.023+0000 7f755bfff640 1 --2- 172.21.15.47:0/3560518678 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f755c15f170 0x7f755c0d5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:06.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.023+0000 7f755bfff640 1 -- 172.21.15.47:0/3560518678 >> v1:172.21.15.47:6789/0 conn(0x7f755c15f550 legacy=0x7f755c0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:06.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.023+0000 7f755a7fc640 1 -- 172.21.15.47:0/3560518678 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2320787928 0 0) 0x7f754c001110 con 0x7f755c15f550 2024-04-26T23:20:06.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.023+0000 7f755bfff640 1 -- 172.21.15.47:0/3560518678 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f755c0c9960 con 0x7f755c0d3bc0 2024-04-26T23:20:06.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.023+0000 7f755b7fe640 1 --2- 172.21.15.47:0/3560518678 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f755c15f170 0x7f755c0d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:06.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.024+0000 7f755bfff640 1 --2- 172.21.15.47:0/3560518678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f755c0d3bc0 0x7f755c0d2cb0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f7554009d90 tx=0x7f755402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b2e5b539423c669 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:06.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.024+0000 7f755a7fc640 1 -- 172.21.15.47:0/3560518678 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7554007740 con 0x7f755c0d3bc0 2024-04-26T23:20:06.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.024+0000 7f755a7fc640 1 -- 172.21.15.47:0/3560518678 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75540078e0 con 0x7f755c0d3bc0 2024-04-26T23:20:06.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.024+0000 7f755a7fc640 1 -- 172.21.15.47:0/3560518678 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7554007cb0 con 0x7f755c0d3bc0 2024-04-26T23:20:06.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.024+0000 7f75620cf640 1 -- 172.21.15.47:0/3560518678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f755c0d3bc0 msgr2=0x7f755c0d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:06.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.024+0000 7f75620cf640 1 --2- 172.21.15.47:0/3560518678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f755c0d3bc0 0x7f755c0d2cb0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f7554009d90 tx=0x7f755402a660 comp rx=0 tx=0).stop 2024-04-26T23:20:06.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.025+0000 7f75620cf640 1 -- 172.21.15.47:0/3560518678 shutdown_connections 2024-04-26T23:20:06.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.025+0000 7f75620cf640 1 --2- 172.21.15.47:0/3560518678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f755c0d3bc0 0x7f755c0d2cb0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:06.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.025+0000 7f75620cf640 1 --2- 172.21.15.47:0/3560518678 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f755c15f170 0x7f755c0d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:06.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.025+0000 7f75620cf640 1 -- 172.21.15.47:0/3560518678 >> 172.21.15.47:0/3560518678 conn(0x7f755c0d4f60 msgr2=0x7f755c0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:06.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.025+0000 7f75620cf640 1 -- 172.21.15.47:0/3560518678 shutdown_connections 2024-04-26T23:20:06.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.025+0000 7f75620cf640 1 -- 172.21.15.47:0/3560518678 wait complete. 2024-04-26T23:20:06.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.026+0000 7f75620cf640 1 Processor -- start 2024-04-26T23:20:06.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.026+0000 7f75620cf640 1 -- start start 2024-04-26T23:20:06.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.027+0000 7f75620cf640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f755c0d3bc0 0x7f755c0ce000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:06.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.027+0000 7f75620cf640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f755c15f170 0x7f755c0ce540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:06.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.027+0000 7f75620cf640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f755c1f4b80 0x7f755c1f5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:06.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.027+0000 7f75620cf640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f755c0ca5c0 con 0x7f755c0d3bc0 2024-04-26T23:20:06.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.027+0000 7f75620cf640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f755c0ca440 con 0x7f755c15f170 2024-04-26T23:20:06.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.027+0000 7f75620cf640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f755c0ca740 con 0x7f755c1f4b80 2024-04-26T23:20:06.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.027+0000 7f755b7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f755c0d3bc0 0x7f755c0ce000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:06.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.027+0000 7f755b7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f755c0d3bc0 0x7f755c0ce000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57988/0 (socket says 172.21.15.47:57988) 2024-04-26T23:20:06.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.027+0000 7f755b7fe640 1 -- 172.21.15.47:0/1727349729 learned_addr learned my addr 172.21.15.47:0/1727349729 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:06.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.027+0000 7f755b7fe640 1 -- 172.21.15.47:0/1727349729 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f755c1f4b80 msgr2=0x7f755c1f5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:06.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.027+0000 7f755bfff640 1 --2- 172.21.15.47:0/1727349729 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f755c1f4b80 0x7f755c1f5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:06.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.027+0000 7f755b7fe640 1 --2- 172.21.15.47:0/1727349729 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f755c1f4b80 0x7f755c1f5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:06.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.027+0000 7f755b7fe640 1 -- 172.21.15.47:0/1727349729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f755c15f170 msgr2=0x7f755c0ce540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:06.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.027+0000 7f755affd640 1 --2- 172.21.15.47:0/1727349729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f755c15f170 0x7f755c0ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:06.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.027+0000 7f755b7fe640 1 --2- 172.21.15.47:0/1727349729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f755c15f170 0x7f755c0ce540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:06.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.027+0000 7f755b7fe640 1 -- 172.21.15.47:0/1727349729 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f755c0cd240 con 0x7f755c0d3bc0 2024-04-26T23:20:06.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.027+0000 7f755b7fe640 1 --2- 172.21.15.47:0/1727349729 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f755c0d3bc0 0x7f755c0ce000 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f754400c850 tx=0x7f754400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:06.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.027+0000 7f7553fff640 1 -- 172.21.15.47:0/1727349729 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7544015070 con 0x7f755c0d3bc0 2024-04-26T23:20:06.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.027+0000 7f7553fff640 1 -- 172.21.15.47:0/1727349729 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75440079d0 con 0x7f755c0d3bc0 2024-04-26T23:20:06.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.027+0000 7f75620cf640 1 -- 172.21.15.47:0/1727349729 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f755c0cd520 con 0x7f755c0d3bc0 2024-04-26T23:20:06.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.027+0000 7f7553fff640 1 -- 172.21.15.47:0/1727349729 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f754400cea0 con 0x7f755c0d3bc0 2024-04-26T23:20:06.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.028+0000 7f75620cf640 1 -- 172.21.15.47:0/1727349729 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f755c15bb70 con 0x7f755c0d3bc0 2024-04-26T23:20:06.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.029+0000 7f75620cf640 1 -- 172.21.15.47:0/1727349729 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f755c0694f0 con 0x7f755c0d3bc0 2024-04-26T23:20:06.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.029+0000 7f7553fff640 1 -- 172.21.15.47:0/1727349729 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7544018070 con 0x7f755c0d3bc0 2024-04-26T23:20:06.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.029+0000 7f7553fff640 1 --2- 172.21.15.47:0/1727349729 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f753c06c920 0x7f753c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:06.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.030+0000 7f7553fff640 1 -- 172.21.15.47:0/1727349729 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f754408bd10 con 0x7f755c0d3bc0 2024-04-26T23:20:06.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.030+0000 7f755affd640 1 --2- 172.21.15.47:0/1727349729 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f753c06c920 0x7f753c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:06.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.030+0000 7f755affd640 1 --2- 172.21.15.47:0/1727349729 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f753c06c920 0x7f753c06ede0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f755c0d2580 tx=0x7f754c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:06.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.034+0000 7f7553fff640 1 -- 172.21.15.47:0/1727349729 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7544059e30 con 0x7f755c0d3bc0 2024-04-26T23:20:06.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.220+0000 7f75620cf640 1 -- 172.21.15.47:0/1727349729 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f755c09ff20 con 0x7f755c0d3bc0 2024-04-26T23:20:06.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.220+0000 7f7553fff640 1 -- 172.21.15.47:0/1727349729 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v41) ==== 104+0+11 (secure 0 0 0) 0x7f7544059730 con 0x7f755c0d3bc0 2024-04-26T23:20:06.222 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-26T23:20:06.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.222+0000 7f75620cf640 1 -- 172.21.15.47:0/1727349729 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f753c06c920 msgr2=0x7f753c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:06.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.222+0000 7f75620cf640 1 --2- 172.21.15.47:0/1727349729 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f753c06c920 0x7f753c06ede0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f755c0d2580 tx=0x7f754c009040 comp rx=0 tx=0).stop 2024-04-26T23:20:06.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.222+0000 7f75620cf640 1 -- 172.21.15.47:0/1727349729 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f755c0d3bc0 msgr2=0x7f755c0ce000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:06.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.222+0000 7f75620cf640 1 --2- 172.21.15.47:0/1727349729 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f755c0d3bc0 0x7f755c0ce000 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f754400c850 tx=0x7f754400cc50 comp rx=0 tx=0).stop 2024-04-26T23:20:06.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.222+0000 7f75620cf640 1 -- 172.21.15.47:0/1727349729 shutdown_connections 2024-04-26T23:20:06.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.222+0000 7f75620cf640 1 --2- 172.21.15.47:0/1727349729 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f753c06c920 0x7f753c06ede0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:06.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.222+0000 7f75620cf640 1 --2- 172.21.15.47:0/1727349729 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f755c1f4b80 0x7f755c1f5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:06.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.222+0000 7f75620cf640 1 --2- 172.21.15.47:0/1727349729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f755c15f170 0x7f755c0ce540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:06.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.222+0000 7f75620cf640 1 --2- 172.21.15.47:0/1727349729 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f755c0d3bc0 0x7f755c0ce000 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:06.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.222+0000 7f75620cf640 1 -- 172.21.15.47:0/1727349729 >> 172.21.15.47:0/1727349729 conn(0x7f755c0d4f60 msgr2=0x7f755c06d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:06.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.222+0000 7f75620cf640 1 -- 172.21.15.47:0/1727349729 shutdown_connections 2024-04-26T23:20:06.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.222+0000 7f75620cf640 1 -- 172.21.15.47:0/1727349729 wait complete. 2024-04-26T23:20:06.235 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:20:06.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.336+0000 7f276ee6d640 1 Processor -- start 2024-04-26T23:20:06.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.336+0000 7f276ee6d640 1 -- start start 2024-04-26T23:20:06.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.337+0000 7f276ee6d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f276815f550 0x7f27680d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:06.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.337+0000 7f276ee6d640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27680d1040 con 0x7f27680d3bc0 2024-04-26T23:20:06.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.337+0000 7f276ee6d640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27680c9680 con 0x7f276815f170 2024-04-26T23:20:06.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.337+0000 7f276ee6d640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f27680c9430 con 0x7f276815f550 2024-04-26T23:20:06.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.337+0000 7f2767fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f276815f550 0x7f27680d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:06.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.337+0000 7f2767fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f276815f550 0x7f27680d5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57996/0 (socket says 172.21.15.47:57996) 2024-04-26T23:20:06.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.337+0000 7f2767fff640 1 -- 172.21.15.47:0/829903201 learned_addr learned my addr 172.21.15.47:0/829903201 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:06.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.337+0000 7f276d40a640 1 --1- 172.21.15.47:0/829903201 >> v1:172.21.15.47:6789/0 conn(0x7f27680d3bc0 0x7f27680ca440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:48916/0 (socket says 172.21.15.47:48916) 2024-04-26T23:20:06.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.338+0000 7f2767fff640 1 -- 172.21.15.47:0/829903201 >> v1:172.21.15.203:6789/0 conn(0x7f276815f170 legacy=0x7f27680d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:06.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.338+0000 7f2767fff640 1 -- 172.21.15.47:0/829903201 >> v1:172.21.15.47:6789/0 conn(0x7f27680d3bc0 legacy=0x7f27680ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:06.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.338+0000 7f27677fe640 1 -- 172.21.15.47:0/829903201 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1318942843 0 0) 0x7f2760001490 con 0x7f27680d3bc0 2024-04-26T23:20:06.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.338+0000 7f2767fff640 1 -- 172.21.15.47:0/829903201 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27680c9960 con 0x7f276815f550 2024-04-26T23:20:06.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.338+0000 7f2767fff640 1 --2- 172.21.15.47:0/829903201 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f276815f550 0x7f27680d5a80 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f27680c95b0 tx=0x7f275c02a320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a36c13e78b930e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:06.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.338+0000 7f27677fe640 1 -- 172.21.15.47:0/829903201 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275c037070 con 0x7f276815f550 2024-04-26T23:20:06.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.339+0000 7f27677fe640 1 -- 172.21.15.47:0/829903201 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f275c032640 con 0x7f276815f550 2024-04-26T23:20:06.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.339+0000 7f27677fe640 1 -- 172.21.15.47:0/829903201 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275c032a10 con 0x7f276815f550 2024-04-26T23:20:06.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.339+0000 7f276ee6d640 1 -- 172.21.15.47:0/829903201 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f276815f550 msgr2=0x7f27680d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:06.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.339+0000 7f276ee6d640 1 --2- 172.21.15.47:0/829903201 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f276815f550 0x7f27680d5a80 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f27680c95b0 tx=0x7f275c02a320 comp rx=0 tx=0).stop 2024-04-26T23:20:06.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.339+0000 7f276ee6d640 1 -- 172.21.15.47:0/829903201 shutdown_connections 2024-04-26T23:20:06.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.339+0000 7f276ee6d640 1 --2- 172.21.15.47:0/829903201 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f276815f550 0x7f27680d5a80 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:06.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.339+0000 7f276ee6d640 1 -- 172.21.15.47:0/829903201 >> 172.21.15.47:0/829903201 conn(0x7f27680d4f60 msgr2=0x7f27680d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:06.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.339+0000 7f276ee6d640 1 -- 172.21.15.47:0/829903201 shutdown_connections 2024-04-26T23:20:06.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.339+0000 7f276ee6d640 1 -- 172.21.15.47:0/829903201 wait complete. 2024-04-26T23:20:06.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.340+0000 7f276ee6d640 1 Processor -- start 2024-04-26T23:20:06.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.340+0000 7f276ee6d640 1 -- start start 2024-04-26T23:20:06.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.341+0000 7f276ee6d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27680d3bc0 0x7f27680b64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:06.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.341+0000 7f276ee6d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f276815f170 0x7f276809ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:06.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.341+0000 7f276ee6d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27681521c0 0x7f2768152670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:06.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.341+0000 7f276ee6d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f27680d2e30 con 0x7f27680d3bc0 2024-04-26T23:20:06.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.341+0000 7f276ee6d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f27680d2cb0 con 0x7f276815f170 2024-04-26T23:20:06.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.341+0000 7f276ee6d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f27680d2fb0 con 0x7f27681521c0 2024-04-26T23:20:06.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.341+0000 7f276cc09640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27680d3bc0 0x7f27680b64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:06.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.341+0000 7f276cc09640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27680d3bc0 0x7f27680b64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58004/0 (socket says 172.21.15.47:58004) 2024-04-26T23:20:06.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.341+0000 7f276cc09640 1 -- 172.21.15.47:0/2155686854 learned_addr learned my addr 172.21.15.47:0/2155686854 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:06.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.342+0000 7f276d40a640 1 --2- 172.21.15.47:0/2155686854 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27681521c0 0x7f2768152670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:06.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.342+0000 7f2767fff640 1 --2- 172.21.15.47:0/2155686854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f276815f170 0x7f276809ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:06.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.342+0000 7f276cc09640 1 -- 172.21.15.47:0/2155686854 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27681521c0 msgr2=0x7f2768152670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:06.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.342+0000 7f276cc09640 1 --2- 172.21.15.47:0/2155686854 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27681521c0 0x7f2768152670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:06.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.342+0000 7f276cc09640 1 -- 172.21.15.47:0/2155686854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f276815f170 msgr2=0x7f276809ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:06.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.342+0000 7f276cc09640 1 --2- 172.21.15.47:0/2155686854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f276815f170 0x7f276809ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:06.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.342+0000 7f276cc09640 1 -- 172.21.15.47:0/2155686854 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f276809f2f0 con 0x7f27680d3bc0 2024-04-26T23:20:06.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.342+0000 7f276cc09640 1 --2- 172.21.15.47:0/2155686854 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27680d3bc0 0x7f27680b64b0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f275800c480 tx=0x7f275800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:06.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.342+0000 7f27657fa640 1 -- 172.21.15.47:0/2155686854 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27580052d0 con 0x7f27680d3bc0 2024-04-26T23:20:06.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.342+0000 7f27657fa640 1 -- 172.21.15.47:0/2155686854 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2758005470 con 0x7f27680d3bc0 2024-04-26T23:20:06.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.342+0000 7f276ee6d640 1 -- 172.21.15.47:0/2155686854 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f276809f5d0 con 0x7f27680d3bc0 2024-04-26T23:20:06.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.342+0000 7f27657fa640 1 -- 172.21.15.47:0/2155686854 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2758012870 con 0x7f27680d3bc0 2024-04-26T23:20:06.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.342+0000 7f276ee6d640 1 -- 172.21.15.47:0/2155686854 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2768154380 con 0x7f27680d3bc0 2024-04-26T23:20:06.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.343+0000 7f276ee6d640 1 -- 172.21.15.47:0/2155686854 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27680694f0 con 0x7f27680d3bc0 2024-04-26T23:20:06.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.344+0000 7f27657fa640 1 -- 172.21.15.47:0/2155686854 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2758012a10 con 0x7f27680d3bc0 2024-04-26T23:20:06.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.344+0000 7f27657fa640 1 --2- 172.21.15.47:0/2155686854 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f275406c8f0 0x7f275406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:06.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.344+0000 7f27657fa640 1 -- 172.21.15.47:0/2155686854 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f275808ba70 con 0x7f27680d3bc0 2024-04-26T23:20:06.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.344+0000 7f2767fff640 1 --2- 172.21.15.47:0/2155686854 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f275406c8f0 0x7f275406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:06.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.345+0000 7f2767fff640 1 --2- 172.21.15.47:0/2155686854 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f275406c8f0 0x7f275406edb0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f275c034000 tx=0x7f275c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:06.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.348+0000 7f27657fa640 1 -- 172.21.15.47:0/2155686854 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2758059c20 con 0x7f27680d3bc0 2024-04-26T23:20:06.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.534+0000 7f276ee6d640 1 -- 172.21.15.47:0/2155686854 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f27680cb4b0 con 0x7f27680d3bc0 2024-04-26T23:20:06.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.535+0000 7f27657fa640 1 -- 172.21.15.47:0/2155686854 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v41) ==== 100+0+11 (secure 0 0 0) 0x7f2758059520 con 0x7f27680d3bc0 2024-04-26T23:20:06.537 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-26T23:20:06.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.537+0000 7f276ee6d640 1 -- 172.21.15.47:0/2155686854 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f275406c8f0 msgr2=0x7f275406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:06.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.537+0000 7f276ee6d640 1 --2- 172.21.15.47:0/2155686854 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f275406c8f0 0x7f275406edb0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f275c034000 tx=0x7f275c02e000 comp rx=0 tx=0).stop 2024-04-26T23:20:06.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.537+0000 7f276ee6d640 1 -- 172.21.15.47:0/2155686854 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27680d3bc0 msgr2=0x7f27680b64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:06.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.537+0000 7f276ee6d640 1 --2- 172.21.15.47:0/2155686854 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27680d3bc0 0x7f27680b64b0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f275800c480 tx=0x7f275800c940 comp rx=0 tx=0).stop 2024-04-26T23:20:06.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.537+0000 7f276ee6d640 1 -- 172.21.15.47:0/2155686854 shutdown_connections 2024-04-26T23:20:06.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.537+0000 7f276ee6d640 1 --2- 172.21.15.47:0/2155686854 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f275406c8f0 0x7f275406edb0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:06.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.537+0000 7f276ee6d640 1 --2- 172.21.15.47:0/2155686854 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27681521c0 0x7f2768152670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:06.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.537+0000 7f276ee6d640 1 --2- 172.21.15.47:0/2155686854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f276815f170 0x7f276809ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:06.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.537+0000 7f276ee6d640 1 --2- 172.21.15.47:0/2155686854 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27680d3bc0 0x7f27680b64b0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:06.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.537+0000 7f276ee6d640 1 -- 172.21.15.47:0/2155686854 >> 172.21.15.47:0/2155686854 conn(0x7f27680d4f60 msgr2=0x7f27680ca340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:06.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.537+0000 7f276ee6d640 1 -- 172.21.15.47:0/2155686854 shutdown_connections 2024-04-26T23:20:06.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.537+0000 7f276ee6d640 1 -- 172.21.15.47:0/2155686854 wait complete. 2024-04-26T23:20:06.550 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2024-04-26T23:20:06.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.652+0000 7f1dbf3af640 1 Processor -- start 2024-04-26T23:20:06.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.652+0000 7f1dbf3af640 1 -- start start 2024-04-26T23:20:06.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.653+0000 7f1dbf3af640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1db815f550 0x7f1db80d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:06.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.653+0000 7f1dbf3af640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1db80d3bc0 0x7f1db80d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:06.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.653+0000 7f1dbf3af640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1db80d1040 con 0x7f1db815f170 2024-04-26T23:20:06.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.653+0000 7f1dbf3af640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f1db80c9430 con 0x7f1db80d3bc0 2024-04-26T23:20:06.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.653+0000 7f1dbf3af640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f1db80c9680 con 0x7f1db815f550 2024-04-26T23:20:06.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.653+0000 7f1dbd14b640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1db815f170 0x7f1db80d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:48918/0 (socket says 172.21.15.47:48918) 2024-04-26T23:20:06.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.653+0000 7f1dbd14b640 1 -- 172.21.15.47:0/3155165307 learned_addr learned my addr 172.21.15.47:0/3155165307 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:06.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.653+0000 7f1dbd94c640 1 --2- 172.21.15.47:0/3155165307 >> v2:172.21.15.203:3300/0 conn(0x7f1db80d3bc0 0x7f1db80d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:06.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.653+0000 7f1dbc94a640 1 --2- 172.21.15.47:0/3155165307 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1db815f550 0x7f1db80d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:06.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.653+0000 7f1daffff640 1 -- 172.21.15.47:0/3155165307 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3670020997 0 0) 0x7f1db4001110 con 0x7f1db815f170 2024-04-26T23:20:06.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.653+0000 7f1daffff640 1 -- 172.21.15.47:0/3155165307 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1da4003520 con 0x7f1db815f170 2024-04-26T23:20:06.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.653+0000 7f1dbd94c640 1 -- 172.21.15.47:0/3155165307 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1db815f550 msgr2=0x7f1db80d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:06.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.653+0000 7f1daffff640 1 -- 172.21.15.47:0/3155165307 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1294392490 0 0) 0x7f1da4003520 con 0x7f1db815f170 2024-04-26T23:20:06.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.653+0000 7f1dbd94c640 1 --2- 172.21.15.47:0/3155165307 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1db815f550 0x7f1db80d5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:06.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.653+0000 7f1dbd94c640 1 -- 172.21.15.47:0/3155165307 >> v1:172.21.15.47:6789/0 conn(0x7f1db815f170 legacy=0x7f1db80d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:06.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.653+0000 7f1dbd94c640 1 -- 172.21.15.47:0/3155165307 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1db80c9960 con 0x7f1db80d3bc0 2024-04-26T23:20:06.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.653+0000 7f1dbc94a640 1 --2- 172.21.15.47:0/3155165307 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1db815f550 0x7f1db80d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:20:06.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.654+0000 7f1dbd94c640 1 --2- 172.21.15.47:0/3155165307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1db80d3bc0 0x7f1db80d2cb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1db0009d90 tx=0x7f1db002a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4ec59f0de5b6b80 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:06.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.654+0000 7f1daffff640 1 -- 172.21.15.47:0/3155165307 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db4001840 con 0x7f1db80d3bc0 2024-04-26T23:20:06.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.654+0000 7f1daffff640 1 -- 172.21.15.47:0/3155165307 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1db0007540 con 0x7f1db80d3bc0 2024-04-26T23:20:06.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.654+0000 7f1daffff640 1 -- 172.21.15.47:0/3155165307 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db00078f0 con 0x7f1db80d3bc0 2024-04-26T23:20:06.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.655+0000 7f1dbf3af640 1 -- 172.21.15.47:0/3155165307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1db80d3bc0 msgr2=0x7f1db80d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:06.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.655+0000 7f1dbf3af640 1 --2- 172.21.15.47:0/3155165307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1db80d3bc0 0x7f1db80d2cb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1db0009d90 tx=0x7f1db002a5f0 comp rx=0 tx=0).stop 2024-04-26T23:20:06.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.655+0000 7f1dbf3af640 1 -- 172.21.15.47:0/3155165307 shutdown_connections 2024-04-26T23:20:06.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.655+0000 7f1dbf3af640 1 --2- 172.21.15.47:0/3155165307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1db80d3bc0 0x7f1db80d2cb0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:06.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.655+0000 7f1dbf3af640 1 --2- 172.21.15.47:0/3155165307 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1db815f550 0x7f1db80d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:06.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.655+0000 7f1dbf3af640 1 -- 172.21.15.47:0/3155165307 >> 172.21.15.47:0/3155165307 conn(0x7f1db80d4f60 msgr2=0x7f1db80d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:06.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.655+0000 7f1dbf3af640 1 -- 172.21.15.47:0/3155165307 shutdown_connections 2024-04-26T23:20:06.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.655+0000 7f1dbf3af640 1 -- 172.21.15.47:0/3155165307 wait complete. 2024-04-26T23:20:06.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.656+0000 7f1dbf3af640 1 Processor -- start 2024-04-26T23:20:06.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.656+0000 7f1dbf3af640 1 -- start start 2024-04-26T23:20:06.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.657+0000 7f1dbf3af640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1db80d3bc0 0x7f1db80b9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:06.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.657+0000 7f1dbf3af640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1db815f170 0x7f1db809f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:06.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.657+0000 7f1dbf3af640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1db809f790 0x7f1db815c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:06.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.657+0000 7f1dbf3af640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1db80ca5c0 con 0x7f1db809f790 2024-04-26T23:20:06.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.657+0000 7f1dbf3af640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1db80ca440 con 0x7f1db80d3bc0 2024-04-26T23:20:06.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.657+0000 7f1dbf3af640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f1db80ca740 con 0x7f1db815f170 2024-04-26T23:20:06.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.657+0000 7f1dbd94c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1db809f790 0x7f1db815c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:06.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.657+0000 7f1dbd94c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1db809f790 0x7f1db815c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58014/0 (socket says 172.21.15.47:58014) 2024-04-26T23:20:06.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.657+0000 7f1dbd94c640 1 -- 172.21.15.47:0/2806894409 learned_addr learned my addr 172.21.15.47:0/2806894409 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:06.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.657+0000 7f1dbd14b640 1 --2- 172.21.15.47:0/2806894409 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1db80d3bc0 0x7f1db80b9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:06.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.657+0000 7f1dbc94a640 1 --2- 172.21.15.47:0/2806894409 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1db815f170 0x7f1db809f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:06.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.657+0000 7f1dbd94c640 1 -- 172.21.15.47:0/2806894409 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1db815f170 msgr2=0x7f1db809f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:06.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.657+0000 7f1dbd94c640 1 --2- 172.21.15.47:0/2806894409 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1db815f170 0x7f1db809f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:06.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.657+0000 7f1dbd94c640 1 -- 172.21.15.47:0/2806894409 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1db80d3bc0 msgr2=0x7f1db80b9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:06.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.657+0000 7f1dbd94c640 1 --2- 172.21.15.47:0/2806894409 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1db80d3bc0 0x7f1db80b9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:06.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.657+0000 7f1dbd94c640 1 -- 172.21.15.47:0/2806894409 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1db809fc10 con 0x7f1db809f790 2024-04-26T23:20:06.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.657+0000 7f1dbd94c640 1 --2- 172.21.15.47:0/2806894409 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1db809f790 0x7f1db815c590 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f1db002abe0 tx=0x7f1db0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:06.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.658+0000 7f1dadffb640 1 -- 172.21.15.47:0/2806894409 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db0040070 con 0x7f1db809f790 2024-04-26T23:20:06.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.658+0000 7f1dadffb640 1 -- 172.21.15.47:0/2806894409 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1db0031c40 con 0x7f1db809f790 2024-04-26T23:20:06.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.658+0000 7f1dadffb640 1 -- 172.21.15.47:0/2806894409 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db0009450 con 0x7f1db809f790 2024-04-26T23:20:06.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.658+0000 7f1dbf3af640 1 -- 172.21.15.47:0/2806894409 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1db809fed0 con 0x7f1db809f790 2024-04-26T23:20:06.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.658+0000 7f1dbf3af640 1 -- 172.21.15.47:0/2806894409 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1db80a0490 con 0x7f1db809f790 2024-04-26T23:20:06.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.659+0000 7f1dbf3af640 1 -- 172.21.15.47:0/2806894409 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1db80694f0 con 0x7f1db809f790 2024-04-26T23:20:06.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.659+0000 7f1dadffb640 1 -- 172.21.15.47:0/2806894409 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1db0039040 con 0x7f1db809f790 2024-04-26T23:20:06.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.660+0000 7f1dadffb640 1 --2- 172.21.15.47:0/2806894409 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1d9c06c8f0 0x7f1d9c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:06.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.660+0000 7f1dadffb640 1 -- 172.21.15.47:0/2806894409 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f1db00ad0a0 con 0x7f1db809f790 2024-04-26T23:20:06.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.660+0000 7f1dbd14b640 1 --2- 172.21.15.47:0/2806894409 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1d9c06c8f0 0x7f1d9c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:06.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.660+0000 7f1dbd14b640 1 --2- 172.21.15.47:0/2806894409 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1d9c06c8f0 0x7f1d9c06edb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1db4005880 tx=0x7f1db40012d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:06.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.664+0000 7f1dadffb640 1 -- 172.21.15.47:0/2806894409 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1db007ab70 con 0x7f1db809f790 2024-04-26T23:20:06.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.848+0000 7f1dbf3af640 1 -- 172.21.15.47:0/2806894409 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) -- 0x7f1db80d4920 con 0x7f1db809f790 2024-04-26T23:20:06.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.848+0000 7f1dadffb640 1 -- 172.21.15.47:0/2806894409 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v41) ==== 97+0+11 (secure 0 0 0) 0x7f1db007ee70 con 0x7f1db809f790 2024-04-26T23:20:06.850 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-26T23:20:06.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.850+0000 7f1dbf3af640 1 -- 172.21.15.47:0/2806894409 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1d9c06c8f0 msgr2=0x7f1d9c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:06.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.850+0000 7f1dbf3af640 1 --2- 172.21.15.47:0/2806894409 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1d9c06c8f0 0x7f1d9c06edb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1db4005880 tx=0x7f1db40012d0 comp rx=0 tx=0).stop 2024-04-26T23:20:06.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.850+0000 7f1dbf3af640 1 -- 172.21.15.47:0/2806894409 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1db809f790 msgr2=0x7f1db815c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:06.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.850+0000 7f1dbf3af640 1 --2- 172.21.15.47:0/2806894409 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1db809f790 0x7f1db815c590 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f1db002abe0 tx=0x7f1db0004a40 comp rx=0 tx=0).stop 2024-04-26T23:20:06.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.850+0000 7f1dbf3af640 1 -- 172.21.15.47:0/2806894409 shutdown_connections 2024-04-26T23:20:06.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.850+0000 7f1dbf3af640 1 --2- 172.21.15.47:0/2806894409 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1d9c06c8f0 0x7f1d9c06edb0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:06.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.850+0000 7f1dbf3af640 1 --2- 172.21.15.47:0/2806894409 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1db809f790 0x7f1db815c590 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:06.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.850+0000 7f1dbf3af640 1 --2- 172.21.15.47:0/2806894409 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1db815f170 0x7f1db809f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:06.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.850+0000 7f1dbf3af640 1 --2- 172.21.15.47:0/2806894409 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1db80d3bc0 0x7f1db80b9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:06.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.850+0000 7f1dbf3af640 1 -- 172.21.15.47:0/2806894409 >> 172.21.15.47:0/2806894409 conn(0x7f1db80d4f60 msgr2=0x7f1db80d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:06.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.850+0000 7f1dbf3af640 1 -- 172.21.15.47:0/2806894409 shutdown_connections 2024-04-26T23:20:06.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.850+0000 7f1dbf3af640 1 -- 172.21.15.47:0/2806894409 wait complete. 2024-04-26T23:20:06.863 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:20:06.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.962+0000 7f4ec9662640 1 Processor -- start 2024-04-26T23:20:06.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.962+0000 7f4ec9662640 1 -- start start 2024-04-26T23:20:06.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.963+0000 7f4ec9662640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4ec40d3bc0 0x7f4ec40d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:06.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.963+0000 7f4ec9662640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ec40b5f20 0x7f4ec40b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:06.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.963+0000 7f4ec9662640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ec40b7e50 con 0x7f4ec4162880 2024-04-26T23:20:06.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.963+0000 7f4ec9662640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f4ec40d0580 con 0x7f4ec40d3bc0 2024-04-26T23:20:06.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.963+0000 7f4ec9662640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f4ec40d1800 con 0x7f4ec40b5f20 2024-04-26T23:20:06.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.963+0000 7f4ec2ffd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4ec4162880 0x7f4ec40d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:48934/0 (socket says 172.21.15.47:48934) 2024-04-26T23:20:06.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.963+0000 7f4ec2ffd640 1 -- 172.21.15.47:0/1926571099 learned_addr learned my addr 172.21.15.47:0/1926571099 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:06.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.963+0000 7f4ec37fe640 1 --2- 172.21.15.47:0/1926571099 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ec40b5f20 0x7f4ec40b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:06.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.963+0000 7f4ec1ffb640 1 -- 172.21.15.47:0/1926571099 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4237548741 0 0) 0x7f4ec40b7e50 con 0x7f4ec4162880 2024-04-26T23:20:06.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.963+0000 7f4ec1ffb640 1 -- 172.21.15.47:0/1926571099 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4eb0003520 con 0x7f4ec4162880 2024-04-26T23:20:06.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.963+0000 7f4ec27fc640 1 --2- 172.21.15.47:0/1926571099 >> v2:172.21.15.203:3300/0 conn(0x7f4ec40d3bc0 0x7f4ec40d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:06.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.964+0000 7f4ec1ffb640 1 -- 172.21.15.47:0/1926571099 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3385756141 0 0) 0x7f4eb0003520 con 0x7f4ec4162880 2024-04-26T23:20:06.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.964+0000 7f4ec1ffb640 1 -- 172.21.15.47:0/1926571099 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ec40b5f20 msgr2=0x7f4ec40b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:06.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.964+0000 7f4ec1ffb640 1 --2- 172.21.15.47:0/1926571099 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ec40b5f20 0x7f4ec40b8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:06.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.964+0000 7f4ec1ffb640 1 -- 172.21.15.47:0/1926571099 >> v2:172.21.15.203:3300/0 conn(0x7f4ec40d3bc0 msgr2=0x7f4ec40d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:06.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.964+0000 7f4ec1ffb640 1 --2- 172.21.15.47:0/1926571099 >> v2:172.21.15.203:3300/0 conn(0x7f4ec40d3bc0 0x7f4ec40d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:06.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.964+0000 7f4ec1ffb640 1 -- 172.21.15.47:0/1926571099 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec40d07d0 con 0x7f4ec4162880 2024-04-26T23:20:06.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.964+0000 7f4ec1ffb640 1 -- 172.21.15.47:0/1926571099 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7f4ebc001670 con 0x7f4ec4162880 2024-04-26T23:20:06.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.964+0000 7f4ec1ffb640 1 -- 172.21.15.47:0/1926571099 >> v1:172.21.15.47:6789/0 conn(0x7f4ec4162880 legacy=0x7f4ec40d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:06.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.964+0000 7f4ec1ffb640 1 --2- 172.21.15.47:0/1926571099 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4eb00039b0 0x7f4eb0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:06.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.964+0000 7f4ec1ffb640 1 --2- 172.21.15.47:0/1926571099 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ec40b5f20 0x7f4eb0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:06.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.964+0000 7f4ec1ffb640 1 --2- 172.21.15.47:0/1926571099 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4eb0004d40 0x7f4eb00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:06.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.964+0000 7f4ec1ffb640 1 -- 172.21.15.47:0/1926571099 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec40d0580 con 0x7f4eb0004d40 2024-04-26T23:20:06.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.964+0000 7f4ec1ffb640 1 -- 172.21.15.47:0/1926571099 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec40d1800 con 0x7f4eb00039b0 2024-04-26T23:20:06.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.964+0000 7f4ec1ffb640 1 -- 172.21.15.47:0/1926571099 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f4eb000b9c0 con 0x7f4ec40b5f20 2024-04-26T23:20:06.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.964+0000 7f4ec37fe640 1 --2- 172.21.15.47:0/1926571099 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4eb0004d40 0x7f4eb00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:06.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.964+0000 7f4ec2ffd640 1 --2- 172.21.15.47:0/1926571099 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4eb00039b0 0x7f4eb0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:06.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.964+0000 7f4ec27fc640 1 --2- 172.21.15.47:0/1926571099 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ec40b5f20 0x7f4eb0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:06.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.964+0000 7f4ec37fe640 1 -- 172.21.15.47:0/1926571099 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ec40b5f20 msgr2=0x7f4eb0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:06.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.964+0000 7f4ec37fe640 1 --2- 172.21.15.47:0/1926571099 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ec40b5f20 0x7f4eb0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:06.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.964+0000 7f4ec37fe640 1 -- 172.21.15.47:0/1926571099 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4eb00039b0 msgr2=0x7f4eb0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:06.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.964+0000 7f4ec37fe640 1 --2- 172.21.15.47:0/1926571099 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4eb00039b0 0x7f4eb0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:06.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.964+0000 7f4ec37fe640 1 -- 172.21.15.47:0/1926571099 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eb0003520 con 0x7f4eb0004d40 2024-04-26T23:20:06.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.965+0000 7f4ec37fe640 1 --2- 172.21.15.47:0/1926571099 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4eb0004d40 0x7f4eb00051c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4009fc0 tx=0x7f4eb0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61ac8c5087af6d34 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:06.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.965+0000 7f4ec1ffb640 1 -- 172.21.15.47:0/1926571099 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb4009c10 con 0x7f4eb0004d40 2024-04-26T23:20:06.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.965+0000 7f4ec1ffb640 1 -- 172.21.15.47:0/1926571099 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4eb4036070 con 0x7f4eb0004d40 2024-04-26T23:20:06.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.965+0000 7f4ec1ffb640 1 -- 172.21.15.47:0/1926571099 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb4031880 con 0x7f4eb0004d40 2024-04-26T23:20:06.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.965+0000 7f4ec9662640 1 -- 172.21.15.47:0/1926571099 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4eb0004d40 msgr2=0x7f4eb00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:06.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.965+0000 7f4ec9662640 1 --2- 172.21.15.47:0/1926571099 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4eb0004d40 0x7f4eb00051c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4009fc0 tx=0x7f4eb0005830 comp rx=0 tx=0).stop 2024-04-26T23:20:06.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.965+0000 7f4ec37fe640 1 -- 172.21.15.47:0/1926571099 reap_dead start 2024-04-26T23:20:06.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.966+0000 7f4ec9662640 1 -- 172.21.15.47:0/1926571099 shutdown_connections 2024-04-26T23:20:06.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.966+0000 7f4ec9662640 1 -- 172.21.15.47:0/1926571099 >> 172.21.15.47:0/1926571099 conn(0x7f4ec40d4f60 msgr2=0x7f4ec40d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:06.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.966+0000 7f4ec9662640 1 -- 172.21.15.47:0/1926571099 shutdown_connections 2024-04-26T23:20:06.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.966+0000 7f4ec9662640 1 -- 172.21.15.47:0/1926571099 wait complete. 2024-04-26T23:20:06.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.967+0000 7f4ec9662640 1 Processor -- start 2024-04-26T23:20:06.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.967+0000 7f4ec9662640 1 -- start start 2024-04-26T23:20:06.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.968+0000 7f4ec9662640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4eb0004d40 0x7f4ec40b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:06.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.968+0000 7f4ec9662640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ec40b45b0 0x7f4ec406e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:06.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.968+0000 7f4ec9662640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ec40cd120 0x7f4ec40cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:06.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.968+0000 7f4ec9662640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec4162c60 con 0x7f4ec40b45b0 2024-04-26T23:20:06.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.968+0000 7f4ec9662640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec40d0d90 con 0x7f4eb0004d40 2024-04-26T23:20:06.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.968+0000 7f4ec9662640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f4ec4162de0 con 0x7f4ec40cd120 2024-04-26T23:20:06.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.968+0000 7f4ec27fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ec40b45b0 0x7f4ec406e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:06.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.968+0000 7f4ec27fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ec40b45b0 0x7f4ec406e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58024/0 (socket says 172.21.15.47:58024) 2024-04-26T23:20:06.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.968+0000 7f4ec27fc640 1 -- 172.21.15.47:0/3130821402 learned_addr learned my addr 172.21.15.47:0/3130821402 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:06.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.968+0000 7f4ec37fe640 1 --2- 172.21.15.47:0/3130821402 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ec40cd120 0x7f4ec40cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:06.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.968+0000 7f4ec27fc640 1 -- 172.21.15.47:0/3130821402 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ec40cd120 msgr2=0x7f4ec40cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:06.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.968+0000 7f4ec2ffd640 1 --2- 172.21.15.47:0/3130821402 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4eb0004d40 0x7f4ec40b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:06.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.968+0000 7f4ec27fc640 1 --2- 172.21.15.47:0/3130821402 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ec40cd120 0x7f4ec40cd5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:06.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.968+0000 7f4ec27fc640 1 -- 172.21.15.47:0/3130821402 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4eb0004d40 msgr2=0x7f4ec40b1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:06.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.968+0000 7f4ec27fc640 1 --2- 172.21.15.47:0/3130821402 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4eb0004d40 0x7f4ec40b1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:06.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.968+0000 7f4ec27fc640 1 -- 172.21.15.47:0/3130821402 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec415e730 con 0x7f4ec40b45b0 2024-04-26T23:20:06.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.968+0000 7f4ec27fc640 1 --2- 172.21.15.47:0/3130821402 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ec40b45b0 0x7f4ec406e180 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4eac011a70 tx=0x7f4eac011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:06.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.968+0000 7f4ebb7fe640 1 -- 172.21.15.47:0/3130821402 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eac00fb50 con 0x7f4ec40b45b0 2024-04-26T23:20:06.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.968+0000 7f4ebb7fe640 1 -- 172.21.15.47:0/3130821402 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4eac00fcf0 con 0x7f4ec40b45b0 2024-04-26T23:20:06.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.968+0000 7f4ec9662640 1 -- 172.21.15.47:0/3130821402 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ec40b21c0 con 0x7f4ec40b45b0 2024-04-26T23:20:06.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.968+0000 7f4ebb7fe640 1 -- 172.21.15.47:0/3130821402 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eac00fea0 con 0x7f4ec40b45b0 2024-04-26T23:20:06.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.969+0000 7f4ec9662640 1 -- 172.21.15.47:0/3130821402 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ec40b2780 con 0x7f4ec40b45b0 2024-04-26T23:20:06.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.969+0000 7f4ec9662640 1 -- 172.21.15.47:0/3130821402 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ec40694f0 con 0x7f4ec40b45b0 2024-04-26T23:20:06.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.970+0000 7f4ebb7fe640 1 -- 172.21.15.47:0/3130821402 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4eac012040 con 0x7f4ec40b45b0 2024-04-26T23:20:06.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.970+0000 7f4ebb7fe640 1 --2- 172.21.15.47:0/3130821402 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4ea406c8f0 0x7f4ea406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:06.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.970+0000 7f4ebb7fe640 1 -- 172.21.15.47:0/3130821402 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f4eac08c550 con 0x7f4ec40b45b0 2024-04-26T23:20:06.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.971+0000 7f4ec2ffd640 1 --2- 172.21.15.47:0/3130821402 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4ea406c8f0 0x7f4ea406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:06.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.971+0000 7f4ec2ffd640 1 --2- 172.21.15.47:0/3130821402 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4ea406c8f0 0x7f4ea406edb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc001050 tx=0x7f4ebc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:06.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:06.975+0000 7f4ebb7fe640 1 -- 172.21.15.47:0/3130821402 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4ec415e730 con 0x7f4ec40b45b0 2024-04-26T23:20:07.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.198+0000 7f4ec9662640 1 -- 172.21.15.47:0/3130821402 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ec40d5510 con 0x7f4ec40b45b0 2024-04-26T23:20:07.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.199+0000 7f4ebb7fe640 1 -- 172.21.15.47:0/3130821402 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+5191 (secure 0 0 0) 0x7f4eac019030 con 0x7f4ec40b45b0 2024-04-26T23:20:07.201 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:20:07.201 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5080,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/1970124054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1970124054},{"type":"v1","addr":"172.21.15.203:6836","nonce":1970124054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5310,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/889573201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":889573201},{"type":"v1","addr":"172.21.15.47:6834","nonce":889573201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5316,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3898612760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":3898612760},{"type":"v1","addr":"172.21.15.47:6835","nonce":3898612760}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:19:44.091353+0000","modified":"2024-04-26T23:19:53.135935+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":14738},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14738":{"gid":14738,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/2526477051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":2526477051},{"type":"v1","addr":"172.21.15.203:6837","nonce":2526477051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":14738,"qdb_cluster":[14738]},"id":2}]} 2024-04-26T23:20:07.201 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 19 2024-04-26T23:20:07.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.201+0000 7f4ec9662640 1 -- 172.21.15.47:0/3130821402 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4ea406c8f0 msgr2=0x7f4ea406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:07.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.201+0000 7f4ec9662640 1 --2- 172.21.15.47:0/3130821402 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4ea406c8f0 0x7f4ea406edb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc001050 tx=0x7f4ebc009040 comp rx=0 tx=0).stop 2024-04-26T23:20:07.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.201+0000 7f4ec9662640 1 -- 172.21.15.47:0/3130821402 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ec40b45b0 msgr2=0x7f4ec406e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:07.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.201+0000 7f4ec9662640 1 --2- 172.21.15.47:0/3130821402 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ec40b45b0 0x7f4ec406e180 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4eac011a70 tx=0x7f4eac011f30 comp rx=0 tx=0).stop 2024-04-26T23:20:07.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.201+0000 7f4ec9662640 1 -- 172.21.15.47:0/3130821402 shutdown_connections 2024-04-26T23:20:07.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.201+0000 7f4ec9662640 1 --2- 172.21.15.47:0/3130821402 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4ea406c8f0 0x7f4ea406edb0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:07.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.201+0000 7f4ec9662640 1 --2- 172.21.15.47:0/3130821402 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ec40cd120 0x7f4ec40cd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:07.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.201+0000 7f4ec9662640 1 --2- 172.21.15.47:0/3130821402 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ec40b45b0 0x7f4ec406e180 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:07.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.201+0000 7f4ec9662640 1 --2- 172.21.15.47:0/3130821402 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4eb0004d40 0x7f4ec40b1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:07.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.201+0000 7f4ec9662640 1 -- 172.21.15.47:0/3130821402 >> 172.21.15.47:0/3130821402 conn(0x7f4ec40d4f60 msgr2=0x7f4ec41602a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:07.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.201+0000 7f4ec9662640 1 -- 172.21.15.47:0/3130821402 shutdown_connections 2024-04-26T23:20:07.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.201+0000 7f4ec9662640 1 -- 172.21.15.47:0/3130821402 wait complete. 2024-04-26T23:20:07.214 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:20:07.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.313+0000 7f9dce652640 1 Processor -- start 2024-04-26T23:20:07.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.313+0000 7f9dce652640 1 -- start start 2024-04-26T23:20:07.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.314+0000 7f9dce652640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9dc80b2340 0x7f9dc80cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:07.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.314+0000 7f9dce652640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9dc80d10d0 con 0x7f9dc80cd930 2024-04-26T23:20:07.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.314+0000 7f9dce652640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9dc80d1370 con 0x7f9dc8162810 2024-04-26T23:20:07.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.314+0000 7f9dce652640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f9dc80d0750 con 0x7f9dc80b2340 2024-04-26T23:20:07.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.314+0000 7f9dccbef640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9dc80cd930 0x7f9dc80d1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:48938/0 (socket says 172.21.15.47:48938) 2024-04-26T23:20:07.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.314+0000 7f9dccbef640 1 -- 172.21.15.47:0/2786156433 learned_addr learned my addr 172.21.15.47:0/2786156433 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:07.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.314+0000 7f9dc6ffd640 1 -- 172.21.15.47:0/2786156433 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3738121958 0 0) 0x7f9dc80d10d0 con 0x7f9dc80cd930 2024-04-26T23:20:07.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.314+0000 7f9dc77fe640 1 --2- 172.21.15.47:0/2786156433 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9dc80b2340 0x7f9dc80cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:07.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.314+0000 7f9dc6ffd640 1 -- 172.21.15.47:0/2786156433 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9db0003520 con 0x7f9dc80cd930 2024-04-26T23:20:07.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.314+0000 7f9dc6ffd640 1 -- 172.21.15.47:0/2786156433 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2507719909 0 0) 0x7f9db0003520 con 0x7f9dc80cd930 2024-04-26T23:20:07.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.314+0000 7f9dc6ffd640 1 -- 172.21.15.47:0/2786156433 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9dc80b2340 msgr2=0x7f9dc80cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:07.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.314+0000 7f9dc6ffd640 1 --2- 172.21.15.47:0/2786156433 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9dc80b2340 0x7f9dc80cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:07.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.314+0000 7f9dc6ffd640 1 -- 172.21.15.47:0/2786156433 >> v1:172.21.15.203:6789/0 conn(0x7f9dc8162810 legacy=0x7f9dc8070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:07.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.314+0000 7f9dc6ffd640 1 -- 172.21.15.47:0/2786156433 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dc80d01b0 con 0x7f9dc80cd930 2024-04-26T23:20:07.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.314+0000 7f9dc6ffd640 1 -- 172.21.15.47:0/2786156433 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7f9dc00019f0 con 0x7f9dc80cd930 2024-04-26T23:20:07.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.314+0000 7f9dc6ffd640 1 -- 172.21.15.47:0/2786156433 >> v1:172.21.15.47:6789/0 conn(0x7f9dc80cd930 legacy=0x7f9dc80d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:07.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.314+0000 7f9dc6ffd640 1 --2- 172.21.15.47:0/2786156433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9db00039e0 0x7f9db0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:07.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.314+0000 7f9dc6ffd640 1 --2- 172.21.15.47:0/2786156433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9db0004800 0x7f9db0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:07.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.314+0000 7f9dc6ffd640 1 --2- 172.21.15.47:0/2786156433 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9dc80b2340 0x7f9db00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:07.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.314+0000 7f9dc6ffd640 1 -- 172.21.15.47:0/2786156433 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc80d0750 con 0x7f9db00039e0 2024-04-26T23:20:07.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.314+0000 7f9dc6ffd640 1 -- 172.21.15.47:0/2786156433 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9db000ba70 con 0x7f9db0004800 2024-04-26T23:20:07.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.314+0000 7f9dc6ffd640 1 -- 172.21.15.47:0/2786156433 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f9db000bc70 con 0x7f9dc80b2340 2024-04-26T23:20:07.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.314+0000 7f9dc7fff640 1 --2- 172.21.15.47:0/2786156433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9db00039e0 0x7f9db0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:07.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.315+0000 7f9dc77fe640 1 --2- 172.21.15.47:0/2786156433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9db0004800 0x7f9db0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:07.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.315+0000 7f9dccbef640 1 --2- 172.21.15.47:0/2786156433 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9dc80b2340 0x7f9db00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:07.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.315+0000 7f9dc7fff640 1 -- 172.21.15.47:0/2786156433 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9dc80b2340 msgr2=0x7f9db00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:07.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.315+0000 7f9dc7fff640 1 --2- 172.21.15.47:0/2786156433 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9dc80b2340 0x7f9db00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:07.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.315+0000 7f9dc7fff640 1 -- 172.21.15.47:0/2786156433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9db0004800 msgr2=0x7f9db0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:07.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.315+0000 7f9dc7fff640 1 --2- 172.21.15.47:0/2786156433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9db0004800 0x7f9db0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:07.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.315+0000 7f9dc7fff640 1 -- 172.21.15.47:0/2786156433 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9db0003520 con 0x7f9db00039e0 2024-04-26T23:20:07.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.315+0000 7f9dccbef640 1 --2- 172.21.15.47:0/2786156433 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9dc80b2340 0x7f9db00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:20:07.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.316+0000 7f9dc7fff640 1 --2- 172.21.15.47:0/2786156433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9db00039e0 0x7f9db0003e20 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f9db8001720 tx=0x7f9db0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bff584e02003163 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:07.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.316+0000 7f9dc6ffd640 1 -- 172.21.15.47:0/2786156433 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db8001370 con 0x7f9db00039e0 2024-04-26T23:20:07.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.316+0000 7f9dc6ffd640 1 -- 172.21.15.47:0/2786156433 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9db80052d0 con 0x7f9db00039e0 2024-04-26T23:20:07.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.316+0000 7f9dc6ffd640 1 -- 172.21.15.47:0/2786156433 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db802ab90 con 0x7f9db00039e0 2024-04-26T23:20:07.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.316+0000 7f9dce652640 1 -- 172.21.15.47:0/2786156433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9db00039e0 msgr2=0x7f9db0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:07.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.316+0000 7f9dce652640 1 --2- 172.21.15.47:0/2786156433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9db00039e0 0x7f9db0003e20 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f9db8001720 tx=0x7f9db0005830 comp rx=0 tx=0).stop 2024-04-26T23:20:07.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.316+0000 7f9dccbef640 1 -- 172.21.15.47:0/2786156433 reap_dead start 2024-04-26T23:20:07.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.316+0000 7f9dce652640 1 -- 172.21.15.47:0/2786156433 shutdown_connections 2024-04-26T23:20:07.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.316+0000 7f9dce652640 1 -- 172.21.15.47:0/2786156433 >> 172.21.15.47:0/2786156433 conn(0x7f9dc80d2da0 msgr2=0x7f9dc80d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:07.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.316+0000 7f9dce652640 1 -- 172.21.15.47:0/2786156433 shutdown_connections 2024-04-26T23:20:07.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.316+0000 7f9dce652640 1 -- 172.21.15.47:0/2786156433 wait complete. 2024-04-26T23:20:07.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.317+0000 7f9dce652640 1 Processor -- start 2024-04-26T23:20:07.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.318+0000 7f9dce652640 1 -- start start 2024-04-26T23:20:07.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.318+0000 7f9dce652640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9db00039e0 0x7f9dc80c7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:07.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.318+0000 7f9dce652640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9dc8149070 0x7f9dc81e95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:07.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.318+0000 7f9dce652640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9dc815d690 0x7f9dc815db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:07.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.318+0000 7f9dce652640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc80d2240 con 0x7f9dc8149070 2024-04-26T23:20:07.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.318+0000 7f9dce652640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc80d16a0 con 0x7f9db00039e0 2024-04-26T23:20:07.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.318+0000 7f9dce652640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f9dc80d23c0 con 0x7f9dc815d690 2024-04-26T23:20:07.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.318+0000 7f9dc77fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9dc8149070 0x7f9dc81e95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:07.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.318+0000 7f9dc77fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9dc8149070 0x7f9dc81e95c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58038/0 (socket says 172.21.15.47:58038) 2024-04-26T23:20:07.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.318+0000 7f9dc77fe640 1 -- 172.21.15.47:0/2476143892 learned_addr learned my addr 172.21.15.47:0/2476143892 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:07.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.319+0000 7f9dc7fff640 1 --2- 172.21.15.47:0/2476143892 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9db00039e0 0x7f9dc80c7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:07.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.319+0000 7f9dccbef640 1 --2- 172.21.15.47:0/2476143892 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9dc815d690 0x7f9dc815db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:07.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.319+0000 7f9dc77fe640 1 -- 172.21.15.47:0/2476143892 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9dc815d690 msgr2=0x7f9dc815db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:07.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.319+0000 7f9dc77fe640 1 --2- 172.21.15.47:0/2476143892 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9dc815d690 0x7f9dc815db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:07.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.319+0000 7f9dc77fe640 1 -- 172.21.15.47:0/2476143892 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9db00039e0 msgr2=0x7f9dc80c7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:07.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.319+0000 7f9dc77fe640 1 --2- 172.21.15.47:0/2476143892 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9db00039e0 0x7f9dc80c7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:07.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.319+0000 7f9dc77fe640 1 -- 172.21.15.47:0/2476143892 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dc81e9b00 con 0x7f9dc8149070 2024-04-26T23:20:07.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.319+0000 7f9dccbef640 1 --2- 172.21.15.47:0/2476143892 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9dc815d690 0x7f9dc815db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:07.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.319+0000 7f9dc77fe640 1 --2- 172.21.15.47:0/2476143892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9dc8149070 0x7f9dc81e95c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc0119a0 tx=0x7f9dbc011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:07.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.319+0000 7f9dc4ff9640 1 -- 172.21.15.47:0/2476143892 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc00faf0 con 0x7f9dc8149070 2024-04-26T23:20:07.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.319+0000 7f9dc4ff9640 1 -- 172.21.15.47:0/2476143892 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dbc00fc90 con 0x7f9dc8149070 2024-04-26T23:20:07.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.319+0000 7f9dce652640 1 -- 172.21.15.47:0/2476143892 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dc80c7950 con 0x7f9dc8149070 2024-04-26T23:20:07.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.319+0000 7f9dc4ff9640 1 -- 172.21.15.47:0/2476143892 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc014440 con 0x7f9dc8149070 2024-04-26T23:20:07.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.319+0000 7f9dce652640 1 -- 172.21.15.47:0/2476143892 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dc80c7f10 con 0x7f9dc8149070 2024-04-26T23:20:07.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.320+0000 7f9dce652640 1 -- 172.21.15.47:0/2476143892 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dc80694f0 con 0x7f9dc8149070 2024-04-26T23:20:07.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.321+0000 7f9dc4ff9640 1 -- 172.21.15.47:0/2476143892 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9dbc012040 con 0x7f9dc8149070 2024-04-26T23:20:07.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.321+0000 7f9dc4ff9640 1 --2- 172.21.15.47:0/2476143892 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9db406c8f0 0x7f9db406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:07.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.321+0000 7f9dc4ff9640 1 -- 172.21.15.47:0/2476143892 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f9dbc08c3e0 con 0x7f9dc8149070 2024-04-26T23:20:07.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.321+0000 7f9dc7fff640 1 --2- 172.21.15.47:0/2476143892 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9db406c8f0 0x7f9db406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:07.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.322+0000 7f9dc7fff640 1 --2- 172.21.15.47:0/2476143892 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9db406c8f0 0x7f9db406edb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9db8032000 tx=0x7f9db802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:07.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.325+0000 7f9dc4ff9640 1 -- 172.21.15.47:0/2476143892 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9dbc019030 con 0x7f9dc8149070 2024-04-26T23:20:07.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.511+0000 7f9dce652640 1 -- 172.21.15.47:0/2476143892 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9dc80b1e60 con 0x7f9dc8149070 2024-04-26T23:20:07.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.513+0000 7f9dc4ff9640 1 -- 172.21.15.47:0/2476143892 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+18718 (secure 0 0 0) 0x7f9dbc059ec0 con 0x7f9dc8149070 2024-04-26T23:20:07.515 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:20:07.515 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":41,"fsid":"399006d3-4db1-43e6-ba81-a79a276eabfd","created":"2024-04-26T23:18:06.731211+0000","modified":"2024-04-26T23:19:54.781247+0000","last_up_change":"2024-04-26T23:18:17.808736+0000","last_in_change":"2024-04-26T23:18:08.945859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:18:16.937492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:19:42.351747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-26T23:19:43.037763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2024-04-26T23:19:51.911275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ee091567-fea0-4a14-bf4e-0e16c20bb2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6819","nonce":326520779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6821","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6823","nonce":326520779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6829","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6831","nonce":326520779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6825","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6827","nonce":326520779}]},"public_addr":"172.21.15.47:6819/326520779","cluster_addr":"172.21.15.47:6823/326520779","heartbeat_back_addr":"172.21.15.47:6831/326520779","heartbeat_front_addr":"172.21.15.47:6827/326520779","state":["exists","up"]},{"osd":1,"uuid":"c14440a0-bba5-4931-a6db-b736c08e869b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6809","nonce":1912610931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6811","nonce":1912610931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6815","nonce":1912610931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6813","nonce":1912610931}]},"public_addr":"172.21.15.47:6809/1912610931","cluster_addr":"172.21.15.47:6811/1912610931","heartbeat_back_addr":"172.21.15.47:6815/1912610931","heartbeat_front_addr":"172.21.15.47:6813/1912610931","state":["exists","up"]},{"osd":2,"uuid":"3940f8de-c9b2-4cd3-a911-9c301729eb30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6801","nonce":2569869305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6803","nonce":2569869305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6807","nonce":2569869305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6805","nonce":2569869305}]},"public_addr":"172.21.15.47:6801/2569869305","cluster_addr":"172.21.15.47:6803/2569869305","heartbeat_back_addr":"172.21.15.47:6807/2569869305","heartbeat_front_addr":"172.21.15.47:6805/2569869305","state":["exists","up"]},{"osd":3,"uuid":"1286a67e-8958-426a-8c2e-bc85498c619c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6818","nonce":2791411683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6822","nonce":2791411683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6830","nonce":2791411683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6826","nonce":2791411683}]},"public_addr":"172.21.15.47:6818/2791411683","cluster_addr":"172.21.15.47:6822/2791411683","heartbeat_back_addr":"172.21.15.47:6830/2791411683","heartbeat_front_addr":"172.21.15.47:6826/2791411683","state":["exists","up"]},{"osd":4,"uuid":"d8ee2c20-e5b7-46c2-9999-b234070126cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6809","nonce":441313104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6811","nonce":441313104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6815","nonce":441313104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6813","nonce":441313104}]},"public_addr":"172.21.15.203:6809/441313104","cluster_addr":"172.21.15.203:6811/441313104","heartbeat_back_addr":"172.21.15.203:6815/441313104","heartbeat_front_addr":"172.21.15.203:6813/441313104","state":["exists","up"]},{"osd":5,"uuid":"b453a8f0-f324-4840-9ae0-d67bdadb9f8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6801","nonce":56838444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6803","nonce":56838444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6807","nonce":56838444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6805","nonce":56838444}]},"public_addr":"172.21.15.203:6801/56838444","cluster_addr":"172.21.15.203:6803/56838444","heartbeat_back_addr":"172.21.15.203:6807/56838444","heartbeat_front_addr":"172.21.15.203:6805/56838444","state":["exists","up"]},{"osd":6,"uuid":"a6d31951-2d4b-4e8d-a510-6283b01dc158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6825","nonce":2738517080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6827","nonce":2738517080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6831","nonce":2738517080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6829","nonce":2738517080}]},"public_addr":"172.21.15.203:6825/2738517080","cluster_addr":"172.21.15.203:6827/2738517080","heartbeat_back_addr":"172.21.15.203:6831/2738517080","heartbeat_front_addr":"172.21.15.203:6829/2738517080","state":["exists","up"]},{"osd":7,"uuid":"7119b8c0-fe96-402d-9e75-7b509d9d44d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6817","nonce":1315565058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6819","nonce":1315565058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6823","nonce":1315565058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6821","nonce":1315565058}]},"public_addr":"172.21.15.203:6817/1315565058","cluster_addr":"172.21.15.203:6819/1315565058","heartbeat_back_addr":"172.21.15.203:6823/1315565058","heartbeat_front_addr":"172.21.15.203:6821/1315565058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.121921+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.137494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.084814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.111593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.072430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.125066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.097867+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.057061+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:20:07.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.515+0000 7f9dce652640 1 -- 172.21.15.47:0/2476143892 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9db406c8f0 msgr2=0x7f9db406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:07.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.515+0000 7f9dce652640 1 --2- 172.21.15.47:0/2476143892 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9db406c8f0 0x7f9db406edb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9db8032000 tx=0x7f9db802e000 comp rx=0 tx=0).stop 2024-04-26T23:20:07.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.515+0000 7f9dce652640 1 -- 172.21.15.47:0/2476143892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9dc8149070 msgr2=0x7f9dc81e95c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:07.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.515+0000 7f9dce652640 1 --2- 172.21.15.47:0/2476143892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9dc8149070 0x7f9dc81e95c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc0119a0 tx=0x7f9dbc011e60 comp rx=0 tx=0).stop 2024-04-26T23:20:07.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.515+0000 7f9dce652640 1 -- 172.21.15.47:0/2476143892 shutdown_connections 2024-04-26T23:20:07.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.515+0000 7f9dce652640 1 --2- 172.21.15.47:0/2476143892 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9db406c8f0 0x7f9db406edb0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:07.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.515+0000 7f9dce652640 1 --2- 172.21.15.47:0/2476143892 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9dc815d690 0x7f9dc815db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:07.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.515+0000 7f9dce652640 1 --2- 172.21.15.47:0/2476143892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9dc8149070 0x7f9dc81e95c0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:07.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.515+0000 7f9dce652640 1 --2- 172.21.15.47:0/2476143892 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9db00039e0 0x7f9dc80c7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:07.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.515+0000 7f9dce652640 1 -- 172.21.15.47:0/2476143892 >> 172.21.15.47:0/2476143892 conn(0x7f9dc80d2da0 msgr2=0x7f9dc80cf3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:07.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.515+0000 7f9dce652640 1 -- 172.21.15.47:0/2476143892 shutdown_connections 2024-04-26T23:20:07.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.515+0000 7f9dce652640 1 -- 172.21.15.47:0/2476143892 wait complete. 2024-04-26T23:20:07.528 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T23:20:07.528 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:20:07.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.628+0000 7f67a3d78640 1 Processor -- start 2024-04-26T23:20:07.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.628+0000 7f67a3d78640 1 -- start start 2024-04-26T23:20:07.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.628+0000 7f67a3d78640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f679c0d06f0 0x7f679c0d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:07.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.628+0000 7f67a3d78640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f679c0b2850 0x7f679c0b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:07.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.628+0000 7f67a3d78640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f679c0d0b70 con 0x7f679c15f3a0 2024-04-26T23:20:07.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.628+0000 7f67a3d78640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f679c0d0e20 con 0x7f679c0b2850 2024-04-26T23:20:07.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.628+0000 7f67a3d78640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f679c0b1d70 con 0x7f679c0d06f0 2024-04-26T23:20:07.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.629+0000 7f67a2315640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f679c0b2850 0x7f679c0b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:07.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.629+0000 7f67a1b14640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f679c15f3a0 0x7f679c0d1770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:48952/0 (socket says 172.21.15.47:48952) 2024-04-26T23:20:07.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.629+0000 7f67a1b14640 1 -- 172.21.15.47:0/517993156 learned_addr learned my addr 172.21.15.47:0/517993156 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:07.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.629+0000 7f67a1313640 1 --2- 172.21.15.47:0/517993156 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f679c0d06f0 0x7f679c0d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:07.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.629+0000 7f67a2315640 1 -- 172.21.15.47:0/517993156 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f679c0d06f0 msgr2=0x7f679c0d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:07.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.629+0000 7f67a2315640 1 --2- 172.21.15.47:0/517993156 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f679c0d06f0 0x7f679c0d1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:07.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.629+0000 7f67a2315640 1 -- 172.21.15.47:0/517993156 >> v1:172.21.15.47:6789/0 conn(0x7f679c15f3a0 legacy=0x7f679c0d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:07.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.629+0000 7f67a2315640 1 -- 172.21.15.47:0/517993156 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f679c0b2050 con 0x7f679c0b2850 2024-04-26T23:20:07.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.630+0000 7f67a2315640 1 --2- 172.21.15.47:0/517993156 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f679c0b2850 0x7f679c0b2cc0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f6794009d90 tx=0x7f679402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30e9a15bb11b2c6b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:07.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.630+0000 7f67a0b12640 1 -- 172.21.15.47:0/517993156 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6794007740 con 0x7f679c0b2850 2024-04-26T23:20:07.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.630+0000 7f67a0b12640 1 -- 172.21.15.47:0/517993156 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67940078e0 con 0x7f679c0b2850 2024-04-26T23:20:07.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.630+0000 7f67a0b12640 1 -- 172.21.15.47:0/517993156 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6794007cb0 con 0x7f679c0b2850 2024-04-26T23:20:07.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.630+0000 7f67a3d78640 1 -- 172.21.15.47:0/517993156 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f679c0b2850 msgr2=0x7f679c0b2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:07.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.630+0000 7f67a3d78640 1 --2- 172.21.15.47:0/517993156 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f679c0b2850 0x7f679c0b2cc0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f6794009d90 tx=0x7f679402a660 comp rx=0 tx=0).stop 2024-04-26T23:20:07.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.630+0000 7f67a3d78640 1 -- 172.21.15.47:0/517993156 shutdown_connections 2024-04-26T23:20:07.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.630+0000 7f67a3d78640 1 --2- 172.21.15.47:0/517993156 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f679c0b2850 0x7f679c0b2cc0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:07.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.630+0000 7f67a3d78640 1 --2- 172.21.15.47:0/517993156 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f679c0d06f0 0x7f679c0d1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:07.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.630+0000 7f67a3d78640 1 -- 172.21.15.47:0/517993156 >> 172.21.15.47:0/517993156 conn(0x7f679c0d4f90 msgr2=0x7f679c0d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:07.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.630+0000 7f67a3d78640 1 -- 172.21.15.47:0/517993156 shutdown_connections 2024-04-26T23:20:07.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.631+0000 7f67a3d78640 1 -- 172.21.15.47:0/517993156 wait complete. 2024-04-26T23:20:07.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.632+0000 7f67a3d78640 1 Processor -- start 2024-04-26T23:20:07.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.632+0000 7f67a3d78640 1 -- start start 2024-04-26T23:20:07.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.632+0000 7f67a3d78640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f679c0b2850 0x7f679c162040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:07.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.632+0000 7f67a3d78640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f679c0d06f0 0x7f679c159780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:07.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.632+0000 7f67a3d78640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f679c15f3a0 0x7f679c0b8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:07.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.632+0000 7f67a3d78640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f679c15f900 con 0x7f679c15f3a0 2024-04-26T23:20:07.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.632+0000 7f67a3d78640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f679c15f780 con 0x7f679c0d06f0 2024-04-26T23:20:07.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.632+0000 7f67a3d78640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f679c0d23c0 con 0x7f679c0b2850 2024-04-26T23:20:07.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.632+0000 7f67a2315640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f679c15f3a0 0x7f679c0b8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:07.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.632+0000 7f67a2315640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f679c15f3a0 0x7f679c0b8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58046/0 (socket says 172.21.15.47:58046) 2024-04-26T23:20:07.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.632+0000 7f67a2315640 1 -- 172.21.15.47:0/2127141322 learned_addr learned my addr 172.21.15.47:0/2127141322 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:07.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.633+0000 7f67a1313640 1 --2- 172.21.15.47:0/2127141322 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f679c0d06f0 0x7f679c159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:07.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.633+0000 7f67a2315640 1 -- 172.21.15.47:0/2127141322 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f679c0b2850 msgr2=0x7f679c162040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:07.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.633+0000 7f67a2315640 1 --2- 172.21.15.47:0/2127141322 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f679c0b2850 0x7f679c162040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:07.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.633+0000 7f67a2315640 1 -- 172.21.15.47:0/2127141322 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f679c0d06f0 msgr2=0x7f679c159780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:07.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.633+0000 7f67a2315640 1 --2- 172.21.15.47:0/2127141322 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f679c0d06f0 0x7f679c159780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:07.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.633+0000 7f67a2315640 1 -- 172.21.15.47:0/2127141322 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f679c0b8ad0 con 0x7f679c15f3a0 2024-04-26T23:20:07.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.633+0000 7f67a2315640 1 --2- 172.21.15.47:0/2127141322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f679c15f3a0 0x7f679c0b8590 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f679402ac50 tx=0x7f6794004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:07.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.633+0000 7f67927fc640 1 -- 172.21.15.47:0/2127141322 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f679403f070 con 0x7f679c15f3a0 2024-04-26T23:20:07.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.633+0000 7f67927fc640 1 -- 172.21.15.47:0/2127141322 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6794031bc0 con 0x7f679c15f3a0 2024-04-26T23:20:07.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.633+0000 7f67a3d78640 1 -- 172.21.15.47:0/2127141322 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f679c0b8da0 con 0x7f679c15f3a0 2024-04-26T23:20:07.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.633+0000 7f67927fc640 1 -- 172.21.15.47:0/2127141322 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6794044460 con 0x7f679c15f3a0 2024-04-26T23:20:07.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.633+0000 7f67a3d78640 1 -- 172.21.15.47:0/2127141322 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f679c0b9360 con 0x7f679c15f3a0 2024-04-26T23:20:07.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.634+0000 7f67a3d78640 1 -- 172.21.15.47:0/2127141322 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f679c0694f0 con 0x7f679c15f3a0 2024-04-26T23:20:07.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.635+0000 7f67927fc640 1 -- 172.21.15.47:0/2127141322 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6794007790 con 0x7f679c15f3a0 2024-04-26T23:20:07.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.635+0000 7f67927fc640 1 --2- 172.21.15.47:0/2127141322 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f678006c8f0 0x7f678006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:07.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.635+0000 7f67927fc640 1 -- 172.21.15.47:0/2127141322 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f67940ad650 con 0x7f679c15f3a0 2024-04-26T23:20:07.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.635+0000 7f67a1b14640 1 --2- 172.21.15.47:0/2127141322 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f678006c8f0 0x7f678006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:07.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.636+0000 7f67a1b14640 1 --2- 172.21.15.47:0/2127141322 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f678006c8f0 0x7f678006edb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6798000f60 tx=0x7f6798007a80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:07.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.640+0000 7f67927fc640 1 -- 172.21.15.47:0/2127141322 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f679407b800 con 0x7f679c15f3a0 2024-04-26T23:20:07.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.864+0000 7f67a3d78640 1 -- 172.21.15.47:0/2127141322 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f679c0b6860 con 0x7f679c15f3a0 2024-04-26T23:20:07.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.864+0000 7f67927fc640 1 -- 172.21.15.47:0/2127141322 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+5191 (secure 0 0 0) 0x7f679407b100 con 0x7f679c15f3a0 2024-04-26T23:20:07.866 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:20:07.867 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5080,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/1970124054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1970124054},{"type":"v1","addr":"172.21.15.203:6836","nonce":1970124054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5310,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/889573201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":889573201},{"type":"v1","addr":"172.21.15.47:6834","nonce":889573201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5316,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3898612760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":3898612760},{"type":"v1","addr":"172.21.15.47:6835","nonce":3898612760}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:19:44.091353+0000","modified":"2024-04-26T23:19:53.135935+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":14738},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14738":{"gid":14738,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/2526477051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":2526477051},{"type":"v1","addr":"172.21.15.203:6837","nonce":2526477051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":14738,"qdb_cluster":[14738]},"id":2}]} 2024-04-26T23:20:07.867 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 19 2024-04-26T23:20:07.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.866+0000 7f67a3d78640 1 -- 172.21.15.47:0/2127141322 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f678006c8f0 msgr2=0x7f678006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:07.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.866+0000 7f67a3d78640 1 --2- 172.21.15.47:0/2127141322 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f678006c8f0 0x7f678006edb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6798000f60 tx=0x7f6798007a80 comp rx=0 tx=0).stop 2024-04-26T23:20:07.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.866+0000 7f67a3d78640 1 -- 172.21.15.47:0/2127141322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f679c15f3a0 msgr2=0x7f679c0b8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:07.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.866+0000 7f67a3d78640 1 --2- 172.21.15.47:0/2127141322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f679c15f3a0 0x7f679c0b8590 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f679402ac50 tx=0x7f6794004a40 comp rx=0 tx=0).stop 2024-04-26T23:20:07.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.866+0000 7f67a3d78640 1 -- 172.21.15.47:0/2127141322 shutdown_connections 2024-04-26T23:20:07.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.866+0000 7f67a3d78640 1 --2- 172.21.15.47:0/2127141322 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f678006c8f0 0x7f678006edb0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:07.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.866+0000 7f67a3d78640 1 --2- 172.21.15.47:0/2127141322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f679c15f3a0 0x7f679c0b8590 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:07.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.866+0000 7f67a3d78640 1 --2- 172.21.15.47:0/2127141322 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f679c0d06f0 0x7f679c159780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:07.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.866+0000 7f67a3d78640 1 --2- 172.21.15.47:0/2127141322 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f679c0b2850 0x7f679c162040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:07.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.866+0000 7f67a3d78640 1 -- 172.21.15.47:0/2127141322 >> 172.21.15.47:0/2127141322 conn(0x7f679c0d4f90 msgr2=0x7f679c06e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:07.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.866+0000 7f67a3d78640 1 -- 172.21.15.47:0/2127141322 shutdown_connections 2024-04-26T23:20:07.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.866+0000 7f67a3d78640 1 -- 172.21.15.47:0/2127141322 wait complete. 2024-04-26T23:20:07.880 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:20:07.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.980+0000 7fa61b778640 1 Processor -- start 2024-04-26T23:20:07.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.980+0000 7fa61b778640 1 -- start start 2024-04-26T23:20:07.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.981+0000 7fa61b778640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6140d3bc0 0x7fa6140d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:07.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.981+0000 7fa61b778640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6140b7e50 con 0x7fa6140b5f20 2024-04-26T23:20:07.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.981+0000 7fa61b778640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6140d1800 con 0x7fa614162880 2024-04-26T23:20:07.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.981+0000 7fa61b778640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa6140d0580 con 0x7fa6140d3bc0 2024-04-26T23:20:07.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.981+0000 7fa619d15640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa6140b5f20 0x7fa6140b8300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:48960/0 (socket says 172.21.15.47:48960) 2024-04-26T23:20:07.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.981+0000 7fa619d15640 1 -- 172.21.15.47:0/347706074 learned_addr learned my addr 172.21.15.47:0/347706074 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:07.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.981+0000 7fa618d13640 1 --2- 172.21.15.47:0/347706074 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6140d3bc0 0x7fa6140d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:07.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.981+0000 7fa60bfff640 1 -- 172.21.15.47:0/347706074 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3611704135 0 0) 0x7fa6140b7e50 con 0x7fa6140b5f20 2024-04-26T23:20:07.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.981+0000 7fa60bfff640 1 -- 172.21.15.47:0/347706074 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa600003520 con 0x7fa6140b5f20 2024-04-26T23:20:07.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.981+0000 7fa60bfff640 1 -- 172.21.15.47:0/347706074 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2283509936 0 0) 0x7fa600003520 con 0x7fa6140b5f20 2024-04-26T23:20:07.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.981+0000 7fa60bfff640 1 -- 172.21.15.47:0/347706074 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6140d3bc0 msgr2=0x7fa6140d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:07.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.981+0000 7fa60bfff640 1 --2- 172.21.15.47:0/347706074 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6140d3bc0 0x7fa6140d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:07.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.981+0000 7fa60bfff640 1 -- 172.21.15.47:0/347706074 >> v1:172.21.15.203:6789/0 conn(0x7fa614162880 legacy=0x7fa6140d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:07.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.981+0000 7fa60bfff640 1 -- 172.21.15.47:0/347706074 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6140d07d0 con 0x7fa6140b5f20 2024-04-26T23:20:07.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.981+0000 7fa60bfff640 1 -- 172.21.15.47:0/347706074 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7fa60c0019f0 con 0x7fa6140b5f20 2024-04-26T23:20:07.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.981+0000 7fa60bfff640 1 -- 172.21.15.47:0/347706074 >> v1:172.21.15.47:6789/0 conn(0x7fa6140b5f20 legacy=0x7fa6140b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:07.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.981+0000 7fa60bfff640 1 --2- 172.21.15.47:0/347706074 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6000039e0 0x7fa600003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:07.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.981+0000 7fa60bfff640 1 --2- 172.21.15.47:0/347706074 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa600004390 0x7fa600004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:07.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.981+0000 7fa60bfff640 1 --2- 172.21.15.47:0/347706074 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6140d3bc0 0x7fa6000050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:07.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.981+0000 7fa60bfff640 1 -- 172.21.15.47:0/347706074 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa6140d0580 con 0x7fa6000039e0 2024-04-26T23:20:07.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.981+0000 7fa60bfff640 1 -- 172.21.15.47:0/347706074 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa60000b9c0 con 0x7fa600004390 2024-04-26T23:20:07.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.981+0000 7fa60bfff640 1 -- 172.21.15.47:0/347706074 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa60000bbc0 con 0x7fa6140d3bc0 2024-04-26T23:20:07.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.981+0000 7fa619514640 1 --2- 172.21.15.47:0/347706074 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6000039e0 0x7fa600003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:07.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.981+0000 7fa618d13640 1 --2- 172.21.15.47:0/347706074 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa600004390 0x7fa600004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:07.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.981+0000 7fa619d15640 1 --2- 172.21.15.47:0/347706074 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6140d3bc0 0x7fa6000050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:07.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.981+0000 7fa619514640 1 -- 172.21.15.47:0/347706074 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6140d3bc0 msgr2=0x7fa6000050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:07.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.981+0000 7fa619514640 1 --2- 172.21.15.47:0/347706074 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6140d3bc0 0x7fa6000050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:07.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.981+0000 7fa619514640 1 -- 172.21.15.47:0/347706074 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa600004390 msgr2=0x7fa600004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:07.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.981+0000 7fa619514640 1 --2- 172.21.15.47:0/347706074 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa600004390 0x7fa600004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:07.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.981+0000 7fa619514640 1 -- 172.21.15.47:0/347706074 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa600003520 con 0x7fa6000039e0 2024-04-26T23:20:07.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.981+0000 7fa619d15640 1 --2- 172.21.15.47:0/347706074 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6140d3bc0 0x7fa6000050c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:20:07.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.982+0000 7fa619514640 1 --2- 172.21.15.47:0/347706074 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6000039e0 0x7fa600003dc0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fa610001a30 tx=0x7fa600005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cdd372b2b2da4bf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:07.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.982+0000 7fa60bfff640 1 -- 172.21.15.47:0/347706074 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa610037070 con 0x7fa6000039e0 2024-04-26T23:20:07.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.982+0000 7fa60bfff640 1 -- 172.21.15.47:0/347706074 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa61002adf0 con 0x7fa6000039e0 2024-04-26T23:20:07.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.982+0000 7fa60bfff640 1 -- 172.21.15.47:0/347706074 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa610001600 con 0x7fa6000039e0 2024-04-26T23:20:07.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.982+0000 7fa61b778640 1 -- 172.21.15.47:0/347706074 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6000039e0 msgr2=0x7fa600003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:07.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.982+0000 7fa61b778640 1 --2- 172.21.15.47:0/347706074 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6000039e0 0x7fa600003dc0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fa610001a30 tx=0x7fa600005830 comp rx=0 tx=0).stop 2024-04-26T23:20:07.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.982+0000 7fa619d15640 1 -- 172.21.15.47:0/347706074 reap_dead start 2024-04-26T23:20:07.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.982+0000 7fa61b778640 1 -- 172.21.15.47:0/347706074 shutdown_connections 2024-04-26T23:20:07.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.982+0000 7fa61b778640 1 -- 172.21.15.47:0/347706074 >> 172.21.15.47:0/347706074 conn(0x7fa6140d4f60 msgr2=0x7fa6140d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:07.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.982+0000 7fa61b778640 1 -- 172.21.15.47:0/347706074 shutdown_connections 2024-04-26T23:20:07.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.982+0000 7fa61b778640 1 -- 172.21.15.47:0/347706074 wait complete. 2024-04-26T23:20:07.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.985+0000 7fa61b778640 1 Processor -- start 2024-04-26T23:20:07.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.985+0000 7fa61b778640 1 -- start start 2024-04-26T23:20:07.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.985+0000 7fa61b778640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6000039e0 0x7fa61415b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:07.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.985+0000 7fa61b778640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6140ba3d0 0x7fa614159e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:07.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.985+0000 7fa61b778640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa61415a3c0 0x7fa6140ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:07.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.985+0000 7fa61b778640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa614162c60 con 0x7fa61415a3c0 2024-04-26T23:20:07.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.985+0000 7fa61b778640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa6140d0d90 con 0x7fa6140ba3d0 2024-04-26T23:20:07.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.985+0000 7fa61b778640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa614162de0 con 0x7fa6000039e0 2024-04-26T23:20:07.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.985+0000 7fa619d15640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa61415a3c0 0x7fa6140ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:07.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.985+0000 7fa619d15640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa61415a3c0 0x7fa6140ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58056/0 (socket says 172.21.15.47:58056) 2024-04-26T23:20:07.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.985+0000 7fa619d15640 1 -- 172.21.15.47:0/2831102307 learned_addr learned my addr 172.21.15.47:0/2831102307 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:07.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.985+0000 7fa619514640 1 --2- 172.21.15.47:0/2831102307 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6000039e0 0x7fa61415b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:07.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.985+0000 7fa618d13640 1 --2- 172.21.15.47:0/2831102307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6140ba3d0 0x7fa614159e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:07.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.985+0000 7fa619d15640 1 -- 172.21.15.47:0/2831102307 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6000039e0 msgr2=0x7fa61415b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:07.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.985+0000 7fa619d15640 1 --2- 172.21.15.47:0/2831102307 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6000039e0 0x7fa61415b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:07.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.985+0000 7fa619d15640 1 -- 172.21.15.47:0/2831102307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6140ba3d0 msgr2=0x7fa614159e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:07.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.985+0000 7fa619d15640 1 --2- 172.21.15.47:0/2831102307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6140ba3d0 0x7fa614159e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:07.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.985+0000 7fa619d15640 1 -- 172.21.15.47:0/2831102307 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa61415a840 con 0x7fa61415a3c0 2024-04-26T23:20:07.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.985+0000 7fa619514640 1 --2- 172.21.15.47:0/2831102307 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6000039e0 0x7fa61415b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:07.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.985+0000 7fa618d13640 1 --2- 172.21.15.47:0/2831102307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6140ba3d0 0x7fa614159e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:07.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.985+0000 7fa619d15640 1 --2- 172.21.15.47:0/2831102307 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa61415a3c0 0x7fa6140ccff0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fa61408d460 tx=0x7fa60c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:07.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.985+0000 7fa609ffb640 1 -- 172.21.15.47:0/2831102307 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa60c00eda0 con 0x7fa61415a3c0 2024-04-26T23:20:07.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.985+0000 7fa609ffb640 1 -- 172.21.15.47:0/2831102307 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa60c018070 con 0x7fa61415a3c0 2024-04-26T23:20:07.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.985+0000 7fa609ffb640 1 -- 172.21.15.47:0/2831102307 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa60c013640 con 0x7fa61415a3c0 2024-04-26T23:20:07.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.985+0000 7fa61b778640 1 -- 172.21.15.47:0/2831102307 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa61415ab80 con 0x7fa61415a3c0 2024-04-26T23:20:07.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.985+0000 7fa61b778640 1 -- 172.21.15.47:0/2831102307 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa61415b3c0 con 0x7fa61415a3c0 2024-04-26T23:20:07.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.986+0000 7fa61b778640 1 -- 172.21.15.47:0/2831102307 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6140694f0 con 0x7fa61415a3c0 2024-04-26T23:20:07.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.987+0000 7fa609ffb640 1 -- 172.21.15.47:0/2831102307 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa60c0137e0 con 0x7fa61415a3c0 2024-04-26T23:20:07.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.987+0000 7fa609ffb640 1 --2- 172.21.15.47:0/2831102307 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa5f806c8f0 0x7fa5f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:07.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.987+0000 7fa609ffb640 1 -- 172.21.15.47:0/2831102307 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7fa60c08c220 con 0x7fa61415a3c0 2024-04-26T23:20:07.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.987+0000 7fa619514640 1 --2- 172.21.15.47:0/2831102307 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa5f806c8f0 0x7fa5f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:07.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.988+0000 7fa619514640 1 --2- 172.21.15.47:0/2831102307 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa5f806c8f0 0x7fa5f806edb0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa610001a30 tx=0x7fa610009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:07.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:07.991+0000 7fa609ffb640 1 -- 172.21.15.47:0/2831102307 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa61415ab80 con 0x7fa61415a3c0 2024-04-26T23:20:08.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.215+0000 7fa61b778640 1 -- 172.21.15.47:0/2831102307 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa61415b590 con 0x7fa61415a3c0 2024-04-26T23:20:08.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.216+0000 7fa609ffb640 1 -- 172.21.15.47:0/2831102307 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+5191 (secure 0 0 0) 0x7fa60c05a340 con 0x7fa61415a3c0 2024-04-26T23:20:08.218 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:20:08.218 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5080,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/1970124054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1970124054},{"type":"v1","addr":"172.21.15.203:6836","nonce":1970124054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5310,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/889573201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":889573201},{"type":"v1","addr":"172.21.15.47:6834","nonce":889573201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5316,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3898612760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":3898612760},{"type":"v1","addr":"172.21.15.47:6835","nonce":3898612760}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:19:44.091353+0000","modified":"2024-04-26T23:19:53.135935+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":14738},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14738":{"gid":14738,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/2526477051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":2526477051},{"type":"v1","addr":"172.21.15.203:6837","nonce":2526477051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":14738,"qdb_cluster":[14738]},"id":2}]} 2024-04-26T23:20:08.218 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 19 2024-04-26T23:20:08.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.218+0000 7fa61b778640 1 -- 172.21.15.47:0/2831102307 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa5f806c8f0 msgr2=0x7fa5f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:08.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.218+0000 7fa61b778640 1 --2- 172.21.15.47:0/2831102307 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa5f806c8f0 0x7fa5f806edb0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa610001a30 tx=0x7fa610009030 comp rx=0 tx=0).stop 2024-04-26T23:20:08.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.218+0000 7fa61b778640 1 -- 172.21.15.47:0/2831102307 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa61415a3c0 msgr2=0x7fa6140ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:08.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.218+0000 7fa61b778640 1 --2- 172.21.15.47:0/2831102307 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa61415a3c0 0x7fa6140ccff0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fa61408d460 tx=0x7fa60c00e530 comp rx=0 tx=0).stop 2024-04-26T23:20:08.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.218+0000 7fa61b778640 1 -- 172.21.15.47:0/2831102307 shutdown_connections 2024-04-26T23:20:08.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.218+0000 7fa61b778640 1 --2- 172.21.15.47:0/2831102307 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa5f806c8f0 0x7fa5f806edb0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:08.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.218+0000 7fa61b778640 1 --2- 172.21.15.47:0/2831102307 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa61415a3c0 0x7fa6140ccff0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:08.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.218+0000 7fa61b778640 1 --2- 172.21.15.47:0/2831102307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6140ba3d0 0x7fa614159e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:08.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.218+0000 7fa61b778640 1 --2- 172.21.15.47:0/2831102307 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6000039e0 0x7fa61415b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:08.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.218+0000 7fa61b778640 1 -- 172.21.15.47:0/2831102307 >> 172.21.15.47:0/2831102307 conn(0x7fa6140d4f60 msgr2=0x7fa6141605a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:08.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.218+0000 7fa61b778640 1 -- 172.21.15.47:0/2831102307 shutdown_connections 2024-04-26T23:20:08.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.218+0000 7fa61b778640 1 -- 172.21.15.47:0/2831102307 wait complete. 2024-04-26T23:20:08.231 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:20:08.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.331+0000 7fa353b66640 1 Processor -- start 2024-04-26T23:20:08.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.331+0000 7fa353b66640 1 -- start start 2024-04-26T23:20:08.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.331+0000 7fa353b66640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa34c06c950 0x7fa34c0ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:08.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.331+0000 7fa353b66640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa34c0cfd80 con 0x7fa34c0b0380 2024-04-26T23:20:08.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.331+0000 7fa353b66640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa34c0d0020 con 0x7fa34c0b10b0 2024-04-26T23:20:08.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.331+0000 7fa353b66640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa34c0af8e0 con 0x7fa34c06c950 2024-04-26T23:20:08.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.331+0000 7fa351902640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa34c0b0380 0x7fa34c06c240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:48970/0 (socket says 172.21.15.47:48970) 2024-04-26T23:20:08.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.331+0000 7fa351902640 1 -- 172.21.15.47:0/1794754145 learned_addr learned my addr 172.21.15.47:0/1794754145 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:08.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.331+0000 7fa352103640 1 --2- 172.21.15.47:0/1794754145 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa34c06c950 0x7fa34c0ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:08.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.331+0000 7fa350900640 1 -- 172.21.15.47:0/1794754145 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3346230437 0 0) 0x7fa34c0cfd80 con 0x7fa34c0b0380 2024-04-26T23:20:08.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.331+0000 7fa350900640 1 -- 172.21.15.47:0/1794754145 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa338003520 con 0x7fa34c0b0380 2024-04-26T23:20:08.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.331+0000 7fa350900640 1 -- 172.21.15.47:0/1794754145 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 95348588 0 0) 0x7fa34c0d0020 con 0x7fa34c0b10b0 2024-04-26T23:20:08.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.331+0000 7fa350900640 1 -- 172.21.15.47:0/1794754145 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa34c0cfd80 con 0x7fa34c0b10b0 2024-04-26T23:20:08.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.331+0000 7fa350900640 1 -- 172.21.15.47:0/1794754145 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2577699351 0 0) 0x7fa338003520 con 0x7fa34c0b0380 2024-04-26T23:20:08.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.331+0000 7fa350900640 1 -- 172.21.15.47:0/1794754145 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa34c06c950 msgr2=0x7fa34c0ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:08.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.331+0000 7fa350900640 1 --2- 172.21.15.47:0/1794754145 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa34c06c950 0x7fa34c0ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:08.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.331+0000 7fa350900640 1 -- 172.21.15.47:0/1794754145 >> v1:172.21.15.203:6789/0 conn(0x7fa34c0b10b0 legacy=0x7fa34c15e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:08.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.331+0000 7fa350900640 1 -- 172.21.15.47:0/1794754145 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa34c0d0300 con 0x7fa34c0b0380 2024-04-26T23:20:08.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.331+0000 7fa350900640 1 -- 172.21.15.47:0/1794754145 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7fa348001690 con 0x7fa34c0b0380 2024-04-26T23:20:08.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.331+0000 7fa352103640 1 --2- 172.21.15.47:0/1794754145 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa34c06c950 0x7fa34c0ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:20:08.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.331+0000 7fa350900640 1 -- 172.21.15.47:0/1794754145 >> v1:172.21.15.47:6789/0 conn(0x7fa34c0b0380 legacy=0x7fa34c06c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:08.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.331+0000 7fa350900640 1 --2- 172.21.15.47:0/1794754145 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa34c06c950 0x7fa338003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:08.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.331+0000 7fa350900640 1 --2- 172.21.15.47:0/1794754145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3380040b0 0x7fa338004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:08.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.331+0000 7fa350900640 1 --2- 172.21.15.47:0/1794754145 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa338004d40 0x7fa3380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:08.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.331+0000 7fa350900640 1 -- 172.21.15.47:0/1794754145 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa34c0af8e0 con 0x7fa3380040b0 2024-04-26T23:20:08.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.331+0000 7fa350900640 1 -- 172.21.15.47:0/1794754145 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa33800bb00 con 0x7fa338004d40 2024-04-26T23:20:08.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.331+0000 7fa350900640 1 -- 172.21.15.47:0/1794754145 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa33800bd00 con 0x7fa34c06c950 2024-04-26T23:20:08.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.332+0000 7fa351101640 1 --2- 172.21.15.47:0/1794754145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3380040b0 0x7fa338004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:08.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.332+0000 7fa351902640 1 --2- 172.21.15.47:0/1794754145 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa34c06c950 0x7fa338003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:08.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.332+0000 7fa352103640 1 --2- 172.21.15.47:0/1794754145 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa338004d40 0x7fa3380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:08.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.332+0000 7fa351101640 1 -- 172.21.15.47:0/1794754145 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa34c06c950 msgr2=0x7fa338003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:08.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.332+0000 7fa351101640 1 --2- 172.21.15.47:0/1794754145 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa34c06c950 0x7fa338003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:08.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.332+0000 7fa351101640 1 -- 172.21.15.47:0/1794754145 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa338004d40 msgr2=0x7fa3380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:08.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.332+0000 7fa351101640 1 --2- 172.21.15.47:0/1794754145 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa338004d40 0x7fa3380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:08.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.332+0000 7fa351101640 1 -- 172.21.15.47:0/1794754145 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa338003520 con 0x7fa3380040b0 2024-04-26T23:20:08.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.332+0000 7fa352103640 1 --2- 172.21.15.47:0/1794754145 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa338004d40 0x7fa3380051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:20:08.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.333+0000 7fa351101640 1 --2- 172.21.15.47:0/1794754145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3380040b0 0x7fa338004800 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fa33c0018d0 tx=0x7fa338005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9f574a3fba7030e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:08.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.333+0000 7fa350900640 1 -- 172.21.15.47:0/1794754145 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33c0014f0 con 0x7fa3380040b0 2024-04-26T23:20:08.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.333+0000 7fa350900640 1 -- 172.21.15.47:0/1794754145 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa33c0052d0 con 0x7fa3380040b0 2024-04-26T23:20:08.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.333+0000 7fa350900640 1 -- 172.21.15.47:0/1794754145 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33c02aca0 con 0x7fa3380040b0 2024-04-26T23:20:08.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.333+0000 7fa353b66640 1 -- 172.21.15.47:0/1794754145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3380040b0 msgr2=0x7fa338004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:08.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.333+0000 7fa353b66640 1 --2- 172.21.15.47:0/1794754145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3380040b0 0x7fa338004800 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fa33c0018d0 tx=0x7fa338005830 comp rx=0 tx=0).stop 2024-04-26T23:20:08.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.333+0000 7fa352103640 1 -- 172.21.15.47:0/1794754145 reap_dead start 2024-04-26T23:20:08.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.333+0000 7fa353b66640 1 -- 172.21.15.47:0/1794754145 shutdown_connections 2024-04-26T23:20:08.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.333+0000 7fa353b66640 1 -- 172.21.15.47:0/1794754145 >> 172.21.15.47:0/1794754145 conn(0x7fa34c0d1cf0 msgr2=0x7fa34c0d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:08.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.333+0000 7fa353b66640 1 -- 172.21.15.47:0/1794754145 shutdown_connections 2024-04-26T23:20:08.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.333+0000 7fa353b66640 1 -- 172.21.15.47:0/1794754145 wait complete. 2024-04-26T23:20:08.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.335+0000 7fa353b66640 1 Processor -- start 2024-04-26T23:20:08.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.335+0000 7fa353b66640 1 -- start start 2024-04-26T23:20:08.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.335+0000 7fa353b66640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3380040b0 0x7fa34c0b6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:08.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.335+0000 7fa353b66640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa34c0cb730 0x7fa34c151180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:08.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.335+0000 7fa353b66640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa34c0b4990 0x7fa34c0c7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:08.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.335+0000 7fa353b66640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa34c0b0760 con 0x7fa34c0cb730 2024-04-26T23:20:08.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.335+0000 7fa353b66640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa34c0d1980 con 0x7fa3380040b0 2024-04-26T23:20:08.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.335+0000 7fa353b66640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa34c0d15c0 con 0x7fa34c0b4990 2024-04-26T23:20:08.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.335+0000 7fa351101640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa34c0cb730 0x7fa34c151180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:08.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.335+0000 7fa351101640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa34c0cb730 0x7fa34c151180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58084/0 (socket says 172.21.15.47:58084) 2024-04-26T23:20:08.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.335+0000 7fa351101640 1 -- 172.21.15.47:0/2873370510 learned_addr learned my addr 172.21.15.47:0/2873370510 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:08.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.335+0000 7fa351902640 1 --2- 172.21.15.47:0/2873370510 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3380040b0 0x7fa34c0b6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:08.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.336+0000 7fa352103640 1 --2- 172.21.15.47:0/2873370510 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa34c0b4990 0x7fa34c0c7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:08.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.336+0000 7fa351101640 1 -- 172.21.15.47:0/2873370510 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa34c0b4990 msgr2=0x7fa34c0c7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:08.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.336+0000 7fa351101640 1 --2- 172.21.15.47:0/2873370510 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa34c0b4990 0x7fa34c0c7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:08.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.336+0000 7fa351101640 1 -- 172.21.15.47:0/2873370510 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3380040b0 msgr2=0x7fa34c0b6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:08.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.336+0000 7fa351101640 1 --2- 172.21.15.47:0/2873370510 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3380040b0 0x7fa34c0b6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:08.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.336+0000 7fa351101640 1 -- 172.21.15.47:0/2873370510 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa34c1516c0 con 0x7fa34c0cb730 2024-04-26T23:20:08.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.336+0000 7fa351101640 1 --2- 172.21.15.47:0/2873370510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa34c0cb730 0x7fa34c151180 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fa33c0019f0 tx=0x7fa33c0012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:08.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.336+0000 7fa3427fc640 1 -- 172.21.15.47:0/2873370510 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33c03f020 con 0x7fa34c0cb730 2024-04-26T23:20:08.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.336+0000 7fa3427fc640 1 -- 172.21.15.47:0/2873370510 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa33c006e50 con 0x7fa34c0cb730 2024-04-26T23:20:08.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.336+0000 7fa353b66640 1 -- 172.21.15.47:0/2873370510 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa34c0b51c0 con 0x7fa34c0cb730 2024-04-26T23:20:08.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.336+0000 7fa3427fc640 1 -- 172.21.15.47:0/2873370510 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33c037040 con 0x7fa34c0cb730 2024-04-26T23:20:08.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.336+0000 7fa353b66640 1 -- 172.21.15.47:0/2873370510 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa34c0b5780 con 0x7fa34c0cb730 2024-04-26T23:20:08.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.337+0000 7fa353b66640 1 -- 172.21.15.47:0/2873370510 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa34c0694f0 con 0x7fa34c0cb730 2024-04-26T23:20:08.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.338+0000 7fa3427fc640 1 -- 172.21.15.47:0/2873370510 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa33c042020 con 0x7fa34c0cb730 2024-04-26T23:20:08.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.338+0000 7fa3427fc640 1 --2- 172.21.15.47:0/2873370510 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa33006c8f0 0x7fa33006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:08.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.338+0000 7fa3427fc640 1 -- 172.21.15.47:0/2873370510 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7fa33c0abcf0 con 0x7fa34c0cb730 2024-04-26T23:20:08.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.338+0000 7fa351902640 1 --2- 172.21.15.47:0/2873370510 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa33006c8f0 0x7fa33006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:08.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.339+0000 7fa351902640 1 --2- 172.21.15.47:0/2873370510 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa33006c8f0 0x7fa33006edb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa348001220 tx=0x7fa348001e40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:08.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.344+0000 7fa3427fc640 1 -- 172.21.15.47:0/2873370510 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa33c0b0070 con 0x7fa34c0cb730 2024-04-26T23:20:08.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.529+0000 7fa353b66640 1 -- 172.21.15.47:0/2873370510 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa34c0ced10 con 0x7fa34c0cb730 2024-04-26T23:20:08.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.530+0000 7fa3427fc640 1 -- 172.21.15.47:0/2873370510 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+18718 (secure 0 0 0) 0x7fa33c049020 con 0x7fa34c0cb730 2024-04-26T23:20:08.531 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:20:08.532 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":41,"fsid":"399006d3-4db1-43e6-ba81-a79a276eabfd","created":"2024-04-26T23:18:06.731211+0000","modified":"2024-04-26T23:19:54.781247+0000","last_up_change":"2024-04-26T23:18:17.808736+0000","last_in_change":"2024-04-26T23:18:08.945859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:18:16.937492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:19:42.351747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-26T23:19:43.037763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2024-04-26T23:19:51.911275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ee091567-fea0-4a14-bf4e-0e16c20bb2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6819","nonce":326520779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6821","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6823","nonce":326520779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6829","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6831","nonce":326520779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6825","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6827","nonce":326520779}]},"public_addr":"172.21.15.47:6819/326520779","cluster_addr":"172.21.15.47:6823/326520779","heartbeat_back_addr":"172.21.15.47:6831/326520779","heartbeat_front_addr":"172.21.15.47:6827/326520779","state":["exists","up"]},{"osd":1,"uuid":"c14440a0-bba5-4931-a6db-b736c08e869b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6809","nonce":1912610931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6811","nonce":1912610931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6815","nonce":1912610931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6813","nonce":1912610931}]},"public_addr":"172.21.15.47:6809/1912610931","cluster_addr":"172.21.15.47:6811/1912610931","heartbeat_back_addr":"172.21.15.47:6815/1912610931","heartbeat_front_addr":"172.21.15.47:6813/1912610931","state":["exists","up"]},{"osd":2,"uuid":"3940f8de-c9b2-4cd3-a911-9c301729eb30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6801","nonce":2569869305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6803","nonce":2569869305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6807","nonce":2569869305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6805","nonce":2569869305}]},"public_addr":"172.21.15.47:6801/2569869305","cluster_addr":"172.21.15.47:6803/2569869305","heartbeat_back_addr":"172.21.15.47:6807/2569869305","heartbeat_front_addr":"172.21.15.47:6805/2569869305","state":["exists","up"]},{"osd":3,"uuid":"1286a67e-8958-426a-8c2e-bc85498c619c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6818","nonce":2791411683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6822","nonce":2791411683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6830","nonce":2791411683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6826","nonce":2791411683}]},"public_addr":"172.21.15.47:6818/2791411683","cluster_addr":"172.21.15.47:6822/2791411683","heartbeat_back_addr":"172.21.15.47:6830/2791411683","heartbeat_front_addr":"172.21.15.47:6826/2791411683","state":["exists","up"]},{"osd":4,"uuid":"d8ee2c20-e5b7-46c2-9999-b234070126cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6809","nonce":441313104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6811","nonce":441313104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6815","nonce":441313104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6813","nonce":441313104}]},"public_addr":"172.21.15.203:6809/441313104","cluster_addr":"172.21.15.203:6811/441313104","heartbeat_back_addr":"172.21.15.203:6815/441313104","heartbeat_front_addr":"172.21.15.203:6813/441313104","state":["exists","up"]},{"osd":5,"uuid":"b453a8f0-f324-4840-9ae0-d67bdadb9f8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6801","nonce":56838444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6803","nonce":56838444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6807","nonce":56838444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6805","nonce":56838444}]},"public_addr":"172.21.15.203:6801/56838444","cluster_addr":"172.21.15.203:6803/56838444","heartbeat_back_addr":"172.21.15.203:6807/56838444","heartbeat_front_addr":"172.21.15.203:6805/56838444","state":["exists","up"]},{"osd":6,"uuid":"a6d31951-2d4b-4e8d-a510-6283b01dc158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6825","nonce":2738517080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6827","nonce":2738517080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6831","nonce":2738517080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6829","nonce":2738517080}]},"public_addr":"172.21.15.203:6825/2738517080","cluster_addr":"172.21.15.203:6827/2738517080","heartbeat_back_addr":"172.21.15.203:6831/2738517080","heartbeat_front_addr":"172.21.15.203:6829/2738517080","state":["exists","up"]},{"osd":7,"uuid":"7119b8c0-fe96-402d-9e75-7b509d9d44d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6817","nonce":1315565058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6819","nonce":1315565058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6823","nonce":1315565058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6821","nonce":1315565058}]},"public_addr":"172.21.15.203:6817/1315565058","cluster_addr":"172.21.15.203:6819/1315565058","heartbeat_back_addr":"172.21.15.203:6823/1315565058","heartbeat_front_addr":"172.21.15.203:6821/1315565058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.121921+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.137494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.084814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.111593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.072430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.125066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.097867+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.057061+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:20:08.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.532+0000 7fa353b66640 1 -- 172.21.15.47:0/2873370510 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa33006c8f0 msgr2=0x7fa33006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:08.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.532+0000 7fa353b66640 1 --2- 172.21.15.47:0/2873370510 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa33006c8f0 0x7fa33006edb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa348001220 tx=0x7fa348001e40 comp rx=0 tx=0).stop 2024-04-26T23:20:08.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.532+0000 7fa353b66640 1 -- 172.21.15.47:0/2873370510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa34c0cb730 msgr2=0x7fa34c151180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:08.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.532+0000 7fa353b66640 1 --2- 172.21.15.47:0/2873370510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa34c0cb730 0x7fa34c151180 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fa33c0019f0 tx=0x7fa33c0012d0 comp rx=0 tx=0).stop 2024-04-26T23:20:08.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.532+0000 7fa353b66640 1 -- 172.21.15.47:0/2873370510 shutdown_connections 2024-04-26T23:20:08.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.532+0000 7fa353b66640 1 --2- 172.21.15.47:0/2873370510 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa33006c8f0 0x7fa33006edb0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:08.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.532+0000 7fa353b66640 1 --2- 172.21.15.47:0/2873370510 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa34c0b4990 0x7fa34c0c7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:08.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.532+0000 7fa353b66640 1 --2- 172.21.15.47:0/2873370510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa34c0cb730 0x7fa34c151180 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:08.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.532+0000 7fa353b66640 1 --2- 172.21.15.47:0/2873370510 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3380040b0 0x7fa34c0b6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:08.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.532+0000 7fa353b66640 1 -- 172.21.15.47:0/2873370510 >> 172.21.15.47:0/2873370510 conn(0x7fa34c0d1cf0 msgr2=0x7fa34c0b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:08.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.532+0000 7fa353b66640 1 -- 172.21.15.47:0/2873370510 shutdown_connections 2024-04-26T23:20:08.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.532+0000 7fa353b66640 1 -- 172.21.15.47:0/2873370510 wait complete. 2024-04-26T23:20:08.544 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T23:20:08.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.645+0000 7f01b0e83640 1 Processor -- start 2024-04-26T23:20:08.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.645+0000 7f01b0e83640 1 -- start start 2024-04-26T23:20:08.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.646+0000 7f01b0e83640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f01ac15f2d0 0x7f01ac1604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:08.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.646+0000 7f01b0e83640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f01ac072700 0x7f01ac06f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:08.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.646+0000 7f01b0e83640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01ac072f70 con 0x7f01ac06fcd0 2024-04-26T23:20:08.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.646+0000 7f01b0e83640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f01ac074010 con 0x7f01ac15f2d0 2024-04-26T23:20:08.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.646+0000 7f01b0e83640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f01ac070100 con 0x7f01ac072700 2024-04-26T23:20:08.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.646+0000 7f01aa59c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f01ac15f2d0 0x7f01ac1604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:08.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.646+0000 7f01aa59c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f01ac15f2d0 0x7f01ac1604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58098/0 (socket says 172.21.15.47:58098) 2024-04-26T23:20:08.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.646+0000 7f01aa59c640 1 -- 172.21.15.47:0/1843521731 learned_addr learned my addr 172.21.15.47:0/1843521731 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:08.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.646+0000 7f01a3fff640 1 --2- 172.21.15.47:0/1843521731 >> v2:172.21.15.203:3300/0 conn(0x7f01ac072700 0x7f01ac06f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:08.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.646+0000 7f01aa59c640 1 -- 172.21.15.47:0/1843521731 >> v2:172.21.15.203:3300/0 conn(0x7f01ac072700 msgr2=0x7f01ac06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:08.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.646+0000 7f01aa59c640 1 --2- 172.21.15.47:0/1843521731 >> v2:172.21.15.203:3300/0 conn(0x7f01ac072700 0x7f01ac06f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:08.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.646+0000 7f01aa59c640 1 -- 172.21.15.47:0/1843521731 >> v1:172.21.15.203:6789/0 conn(0x7f01ac06fcd0 legacy=0x7f01ac0cbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:08.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.646+0000 7f01aa59c640 1 -- 172.21.15.47:0/1843521731 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01ac073250 con 0x7f01ac15f2d0 2024-04-26T23:20:08.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.646+0000 7f01a3fff640 1 --2- 172.21.15.47:0/1843521731 >> v2:172.21.15.203:3300/0 conn(0x7f01ac072700 0x7f01ac06f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:08.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.647+0000 7f01aa59c640 1 --2- 172.21.15.47:0/1843521731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01ac15f2d0 0x7f01ac1604e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f01ac0717a0 tx=0x7f019c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de35c887c12d03bc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:08.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.647+0000 7f01a9d9b640 1 -- 172.21.15.47:0/1843521731 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019c037070 con 0x7f01ac15f2d0 2024-04-26T23:20:08.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.647+0000 7f01a9d9b640 1 -- 172.21.15.47:0/1843521731 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f019c032440 con 0x7f01ac15f2d0 2024-04-26T23:20:08.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.647+0000 7f01a9d9b640 1 -- 172.21.15.47:0/1843521731 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019c032810 con 0x7f01ac15f2d0 2024-04-26T23:20:08.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.647+0000 7f01b0e83640 1 -- 172.21.15.47:0/1843521731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01ac15f2d0 msgr2=0x7f01ac1604e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:08.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.647+0000 7f01b0e83640 1 --2- 172.21.15.47:0/1843521731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01ac15f2d0 0x7f01ac1604e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f01ac0717a0 tx=0x7f019c02a270 comp rx=0 tx=0).stop 2024-04-26T23:20:08.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.648+0000 7f01b0e83640 1 -- 172.21.15.47:0/1843521731 shutdown_connections 2024-04-26T23:20:08.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.648+0000 7f01b0e83640 1 --2- 172.21.15.47:0/1843521731 >> v2:172.21.15.203:3300/0 conn(0x7f01ac072700 0x7f01ac06f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:08.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.648+0000 7f01b0e83640 1 --2- 172.21.15.47:0/1843521731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01ac15f2d0 0x7f01ac1604e0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:08.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.648+0000 7f01b0e83640 1 -- 172.21.15.47:0/1843521731 >> 172.21.15.47:0/1843521731 conn(0x7f01ac0d0be0 msgr2=0x7f01ac0d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:08.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.648+0000 7f01b0e83640 1 -- 172.21.15.47:0/1843521731 shutdown_connections 2024-04-26T23:20:08.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.648+0000 7f01b0e83640 1 -- 172.21.15.47:0/1843521731 wait complete. 2024-04-26T23:20:08.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.649+0000 7f01b0e83640 1 Processor -- start 2024-04-26T23:20:08.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.649+0000 7f01b0e83640 1 -- start start 2024-04-26T23:20:08.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.650+0000 7f01b0e83640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01ac06fcd0 0x7f01ac1488f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:08.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.650+0000 7f01b0e83640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01ac072700 0x7f01ac0cd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:08.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.650+0000 7f01b0e83640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01ac15f2d0 0x7f01ac0c6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:08.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.650+0000 7f01b0e83640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac073af0 con 0x7f01ac072700 2024-04-26T23:20:08.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.650+0000 7f01b0e83640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac073810 con 0x7f01ac15f2d0 2024-04-26T23:20:08.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.650+0000 7f01b0e83640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f01ac15f6b0 con 0x7f01ac06fcd0 2024-04-26T23:20:08.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.650+0000 7f01a3fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01ac072700 0x7f01ac0cd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:08.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.650+0000 7f01a3fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01ac072700 0x7f01ac0cd6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58108/0 (socket says 172.21.15.47:58108) 2024-04-26T23:20:08.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.650+0000 7f01a3fff640 1 -- 172.21.15.47:0/2559200385 learned_addr learned my addr 172.21.15.47:0/2559200385 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:08.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.650+0000 7f01aa59c640 1 --2- 172.21.15.47:0/2559200385 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01ac06fcd0 0x7f01ac1488f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:08.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.650+0000 7f01aad9d640 1 --2- 172.21.15.47:0/2559200385 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01ac15f2d0 0x7f01ac0c6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:08.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.650+0000 7f01a3fff640 1 -- 172.21.15.47:0/2559200385 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01ac06fcd0 msgr2=0x7f01ac1488f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:08.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.650+0000 7f01a3fff640 1 --2- 172.21.15.47:0/2559200385 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01ac06fcd0 0x7f01ac1488f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:08.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.650+0000 7f01a3fff640 1 -- 172.21.15.47:0/2559200385 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01ac15f2d0 msgr2=0x7f01ac0c6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:08.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.650+0000 7f01a3fff640 1 --2- 172.21.15.47:0/2559200385 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01ac15f2d0 0x7f01ac0c6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:08.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.650+0000 7f01a3fff640 1 -- 172.21.15.47:0/2559200385 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01ac0c38a0 con 0x7f01ac072700 2024-04-26T23:20:08.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.650+0000 7f01a3fff640 1 --2- 172.21.15.47:0/2559200385 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01ac072700 0x7f01ac0cd6e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f019400c7f0 tx=0x7f019400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:08.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.650+0000 7f01a2ffd640 1 -- 172.21.15.47:0/2559200385 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0194015070 con 0x7f01ac072700 2024-04-26T23:20:08.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.650+0000 7f01a2ffd640 1 -- 172.21.15.47:0/2559200385 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0194007a30 con 0x7f01ac072700 2024-04-26T23:20:08.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.650+0000 7f01b0e83640 1 -- 172.21.15.47:0/2559200385 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01ac0c35c0 con 0x7f01ac072700 2024-04-26T23:20:08.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.650+0000 7f01a2ffd640 1 -- 172.21.15.47:0/2559200385 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0194007e00 con 0x7f01ac072700 2024-04-26T23:20:08.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.650+0000 7f01b0e83640 1 -- 172.21.15.47:0/2559200385 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01ac0c5b30 con 0x7f01ac072700 2024-04-26T23:20:08.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.651+0000 7f01b0e83640 1 -- 172.21.15.47:0/2559200385 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01ac0694f0 con 0x7f01ac072700 2024-04-26T23:20:08.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.652+0000 7f01a2ffd640 1 -- 172.21.15.47:0/2559200385 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0194018070 con 0x7f01ac072700 2024-04-26T23:20:08.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.652+0000 7f01a2ffd640 1 --2- 172.21.15.47:0/2559200385 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f018c06c8f0 0x7f018c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:08.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.652+0000 7f01a2ffd640 1 -- 172.21.15.47:0/2559200385 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6496+0+0 (secure 0 0 0) 0x7f019408bec0 con 0x7f01ac072700 2024-04-26T23:20:08.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.652+0000 7f01aa59c640 1 --2- 172.21.15.47:0/2559200385 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f018c06c8f0 0x7f018c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:08.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.653+0000 7f01aa59c640 1 --2- 172.21.15.47:0/2559200385 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f018c06c8f0 0x7f018c06edb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f019c034000 tx=0x7f019c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:08.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.657+0000 7f01a2ffd640 1 -- 172.21.15.47:0/2559200385 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0194059f60 con 0x7f01ac072700 2024-04-26T23:20:08.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.884+0000 7f01b0e83640 1 -- 172.21.15.47:0/2559200385 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f01ac06de80 con 0x7f01ac072700 2024-04-26T23:20:08.885 INFO:tasks.ceph.mon.a.smithi047.stderr:2024-04-26T23:20:08.884+0000 7fb226ccb640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T23:20:08.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.886+0000 7f01a2ffd640 1 -- 172.21.15.47:0/2559200385 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v20) ==== 161+0+0 (secure 0 0 0) 0x7f0194059860 con 0x7f01ac072700 2024-04-26T23:20:08.887 INFO:teuthology.orchestra.run.smithi047.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T23:20:08.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.889+0000 7f01b0e83640 1 -- 172.21.15.47:0/2559200385 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f018c06c8f0 msgr2=0x7f018c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:08.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.889+0000 7f01b0e83640 1 --2- 172.21.15.47:0/2559200385 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f018c06c8f0 0x7f018c06edb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f019c034000 tx=0x7f019c02e000 comp rx=0 tx=0).stop 2024-04-26T23:20:08.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.889+0000 7f01b0e83640 1 -- 172.21.15.47:0/2559200385 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01ac072700 msgr2=0x7f01ac0cd6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:08.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.889+0000 7f01b0e83640 1 --2- 172.21.15.47:0/2559200385 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01ac072700 0x7f01ac0cd6e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f019400c7f0 tx=0x7f019400cbf0 comp rx=0 tx=0).stop 2024-04-26T23:20:08.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.889+0000 7f01b0e83640 1 -- 172.21.15.47:0/2559200385 shutdown_connections 2024-04-26T23:20:08.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.889+0000 7f01b0e83640 1 --2- 172.21.15.47:0/2559200385 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f018c06c8f0 0x7f018c06edb0 secure :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f019c034000 tx=0x7f019c02e000 comp rx=0 tx=0).stop 2024-04-26T23:20:08.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.889+0000 7f01b0e83640 1 --2- 172.21.15.47:0/2559200385 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01ac15f2d0 0x7f01ac0c6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:08.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.889+0000 7f01b0e83640 1 --2- 172.21.15.47:0/2559200385 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01ac072700 0x7f01ac0cd6e0 secure :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f019400c7f0 tx=0x7f019400cbf0 comp rx=0 tx=0).stop 2024-04-26T23:20:08.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.889+0000 7f01b0e83640 1 --2- 172.21.15.47:0/2559200385 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01ac06fcd0 0x7f01ac1488f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:08.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.889+0000 7f01b0e83640 1 -- 172.21.15.47:0/2559200385 >> 172.21.15.47:0/2559200385 conn(0x7f01ac0d0be0 msgr2=0x7f01ac0b4c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:08.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.889+0000 7f01b0e83640 1 -- 172.21.15.47:0/2559200385 shutdown_connections 2024-04-26T23:20:08.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:08.889+0000 7f01b0e83640 1 -- 172.21.15.47:0/2559200385 wait complete. 2024-04-26T23:20:08.893 INFO:tasks.ceph.mds.b.smithi203.stderr: -4537> 2024-04-26T23:19:49.386+0000 7fd67e004640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T23:20:08.902 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T23:20:08.924 INFO:tasks.ceph.mds.b.smithi203.stdout:starting mds.b at 2024-04-26T23:20:09.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.004+0000 7f4e5a524640 1 Processor -- start 2024-04-26T23:20:09.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.005+0000 7f4e5a524640 1 -- start start 2024-04-26T23:20:09.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.005+0000 7f4e5a524640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e540d3bc0 0x7f4e540d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:09.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.005+0000 7f4e5a524640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4e540b5f20 0x7f4e540b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:09.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.005+0000 7f4e5a524640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e540b7e50 con 0x7f4e54162880 2024-04-26T23:20:09.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.005+0000 7f4e5a524640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f4e540d0580 con 0x7f4e540b5f20 2024-04-26T23:20:09.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.005+0000 7f4e5a524640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f4e540d1800 con 0x7f4e540d3bc0 2024-04-26T23:20:09.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.005+0000 7f4e58ac1640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4e540b5f20 0x7f4e540b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:09.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.005+0000 7f4e58ac1640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4e540b5f20 0x7f4e540b8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58118/0 (socket says 172.21.15.47:58118) 2024-04-26T23:20:09.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.005+0000 7f4e58ac1640 1 -- 172.21.15.47:0/1674159436 learned_addr learned my addr 172.21.15.47:0/1674159436 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:09.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.006+0000 7f4e537fe640 1 --2- 172.21.15.47:0/1674159436 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e540d3bc0 0x7f4e540d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:09.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.006+0000 7f4e58ac1640 1 -- 172.21.15.47:0/1674159436 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e540d3bc0 msgr2=0x7f4e540d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:09.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.006+0000 7f4e58ac1640 1 --2- 172.21.15.47:0/1674159436 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e540d3bc0 0x7f4e540d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:09.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.006+0000 7f4e58ac1640 1 -- 172.21.15.47:0/1674159436 >> v1:172.21.15.203:6789/0 conn(0x7f4e54162880 legacy=0x7f4e540d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:09.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.006+0000 7f4e58ac1640 1 -- 172.21.15.47:0/1674159436 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e540d07d0 con 0x7f4e540b5f20 2024-04-26T23:20:09.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.006+0000 7f4e58ac1640 1 --2- 172.21.15.47:0/1674159436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e540b5f20 0x7f4e540b8300 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c009e50 tx=0x7f4e4c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c47254814a53f1e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:09.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.007+0000 7f4e52ffd640 1 -- 172.21.15.47:0/1674159436 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c036070 con 0x7f4e540b5f20 2024-04-26T23:20:09.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.007+0000 7f4e52ffd640 1 -- 172.21.15.47:0/1674159436 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e4c031600 con 0x7f4e540b5f20 2024-04-26T23:20:09.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.007+0000 7f4e52ffd640 1 -- 172.21.15.47:0/1674159436 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c0319d0 con 0x7f4e540b5f20 2024-04-26T23:20:09.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.007+0000 7f4e5a524640 1 -- 172.21.15.47:0/1674159436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e540b5f20 msgr2=0x7f4e540b8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:09.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.007+0000 7f4e5a524640 1 --2- 172.21.15.47:0/1674159436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e540b5f20 0x7f4e540b8300 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c009e50 tx=0x7f4e4c02a710 comp rx=0 tx=0).stop 2024-04-26T23:20:09.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.007+0000 7f4e5a524640 1 -- 172.21.15.47:0/1674159436 shutdown_connections 2024-04-26T23:20:09.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.007+0000 7f4e5a524640 1 --2- 172.21.15.47:0/1674159436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e540b5f20 0x7f4e540b8300 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:09.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.007+0000 7f4e5a524640 1 --2- 172.21.15.47:0/1674159436 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e540d3bc0 0x7f4e540d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:09.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.007+0000 7f4e5a524640 1 -- 172.21.15.47:0/1674159436 >> 172.21.15.47:0/1674159436 conn(0x7f4e540d4f60 msgr2=0x7f4e540d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:09.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.007+0000 7f4e5a524640 1 -- 172.21.15.47:0/1674159436 shutdown_connections 2024-04-26T23:20:09.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.007+0000 7f4e5a524640 1 -- 172.21.15.47:0/1674159436 wait complete. 2024-04-26T23:20:09.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.008+0000 7f4e5a524640 1 Processor -- start 2024-04-26T23:20:09.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.009+0000 7f4e5a524640 1 -- start start 2024-04-26T23:20:09.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.009+0000 7f4e5a524640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e540b5f20 0x7f4e54153690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:09.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.009+0000 7f4e5a524640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e540d3bc0 0x7f4e540b24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:09.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.009+0000 7f4e5a524640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e54162880 0x7f4e540cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:09.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.009+0000 7f4e5a524640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4e54162c60 con 0x7f4e540b5f20 2024-04-26T23:20:09.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.009+0000 7f4e5a524640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4e540d0d90 con 0x7f4e540d3bc0 2024-04-26T23:20:09.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.009+0000 7f4e5a524640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f4e54162de0 con 0x7f4e54162880 2024-04-26T23:20:09.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.009+0000 7f4e53fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e540b5f20 0x7f4e54153690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:09.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.009+0000 7f4e53fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e540b5f20 0x7f4e54153690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58132/0 (socket says 172.21.15.47:58132) 2024-04-26T23:20:09.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.009+0000 7f4e53fff640 1 -- 172.21.15.47:0/429473721 learned_addr learned my addr 172.21.15.47:0/429473721 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:09.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.009+0000 7f4e537fe640 1 --2- 172.21.15.47:0/429473721 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e540d3bc0 0x7f4e540b24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:09.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.010+0000 7f4e58ac1640 1 --2- 172.21.15.47:0/429473721 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e54162880 0x7f4e540cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:09.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.010+0000 7f4e53fff640 1 -- 172.21.15.47:0/429473721 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e54162880 msgr2=0x7f4e540cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:09.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.010+0000 7f4e53fff640 1 --2- 172.21.15.47:0/429473721 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e54162880 0x7f4e540cdf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:09.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.010+0000 7f4e53fff640 1 -- 172.21.15.47:0/429473721 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e540d3bc0 msgr2=0x7f4e540b24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:09.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.010+0000 7f4e53fff640 1 --2- 172.21.15.47:0/429473721 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e540d3bc0 0x7f4e540b24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:09.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.010+0000 7f4e53fff640 1 -- 172.21.15.47:0/429473721 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e540ce980 con 0x7f4e540b5f20 2024-04-26T23:20:09.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.010+0000 7f4e53fff640 1 --2- 172.21.15.47:0/429473721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e540b5f20 0x7f4e54153690 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f4e4400c400 tx=0x7f4e4400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:09.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.010+0000 7f4e50ff9640 1 -- 172.21.15.47:0/429473721 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e440052d0 con 0x7f4e540b5f20 2024-04-26T23:20:09.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.010+0000 7f4e50ff9640 1 -- 172.21.15.47:0/429473721 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e44005490 con 0x7f4e540b5f20 2024-04-26T23:20:09.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.010+0000 7f4e5a524640 1 -- 172.21.15.47:0/429473721 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e540cec60 con 0x7f4e540b5f20 2024-04-26T23:20:09.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.010+0000 7f4e50ff9640 1 -- 172.21.15.47:0/429473721 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e44007260 con 0x7f4e540b5f20 2024-04-26T23:20:09.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.010+0000 7f4e5a524640 1 -- 172.21.15.47:0/429473721 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e540b5610 con 0x7f4e540b5f20 2024-04-26T23:20:09.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.011+0000 7f4e5a524640 1 -- 172.21.15.47:0/429473721 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e540764d0 con 0x7f4e540b5f20 2024-04-26T23:20:09.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.012+0000 7f4e50ff9640 1 -- 172.21.15.47:0/429473721 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4e44007400 con 0x7f4e540b5f20 2024-04-26T23:20:09.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.012+0000 7f4e50ff9640 1 --2- 172.21.15.47:0/429473721 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4e4006c8f0 0x7f4e4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:09.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.012+0000 7f4e50ff9640 1 -- 172.21.15.47:0/429473721 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6582+0+0 (secure 0 0 0) 0x7f4e4408b360 con 0x7f4e540b5f20 2024-04-26T23:20:09.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.012+0000 7f4e537fe640 1 --2- 172.21.15.47:0/429473721 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4e4006c8f0 0x7f4e4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:09.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.013+0000 7f4e537fe640 1 --2- 172.21.15.47:0/429473721 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4e4006c8f0 0x7f4e4006edb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4e540cdce0 tx=0x7f4e48008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:09.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.016+0000 7f4e50ff9640 1 -- 172.21.15.47:0/429473721 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4e540764d0 con 0x7f4e540b5f20 2024-04-26T23:20:09.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.246+0000 7f4e5a524640 1 -- 172.21.15.47:0/429473721 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f4e540694f0 con 0x7f4e540b5f20 2024-04-26T23:20:09.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.887+0000 7f4e50ff9640 1 -- 172.21.15.47:0/429473721 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v21) ==== 104+0+0 (secure 0 0 0) 0x7f4e44058dd0 con 0x7f4e540b5f20 2024-04-26T23:20:09.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.890+0000 7f4e5a524640 1 -- 172.21.15.47:0/429473721 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4e4006c8f0 msgr2=0x7f4e4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:09.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.890+0000 7f4e5a524640 1 --2- 172.21.15.47:0/429473721 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4e4006c8f0 0x7f4e4006edb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4e540cdce0 tx=0x7f4e48008040 comp rx=0 tx=0).stop 2024-04-26T23:20:09.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.890+0000 7f4e5a524640 1 -- 172.21.15.47:0/429473721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e540b5f20 msgr2=0x7f4e54153690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:09.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.890+0000 7f4e5a524640 1 --2- 172.21.15.47:0/429473721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e540b5f20 0x7f4e54153690 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f4e4400c400 tx=0x7f4e4400c8c0 comp rx=0 tx=0).stop 2024-04-26T23:20:09.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.890+0000 7f4e5a524640 1 -- 172.21.15.47:0/429473721 shutdown_connections 2024-04-26T23:20:09.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.890+0000 7f4e5a524640 1 --2- 172.21.15.47:0/429473721 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4e4006c8f0 0x7f4e4006edb0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:09.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.890+0000 7f4e5a524640 1 --2- 172.21.15.47:0/429473721 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e54162880 0x7f4e540cdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:09.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.890+0000 7f4e5a524640 1 --2- 172.21.15.47:0/429473721 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e540d3bc0 0x7f4e540b24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:09.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.890+0000 7f4e5a524640 1 --2- 172.21.15.47:0/429473721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e540b5f20 0x7f4e54153690 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:09.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.890+0000 7f4e5a524640 1 -- 172.21.15.47:0/429473721 >> 172.21.15.47:0/429473721 conn(0x7f4e540d4f60 msgr2=0x7f4e540d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:09.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.891+0000 7f4e5a524640 1 -- 172.21.15.47:0/429473721 shutdown_connections 2024-04-26T23:20:09.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:09.891+0000 7f4e5a524640 1 -- 172.21.15.47:0/429473721 wait complete. 2024-04-26T23:20:09.905 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T23:20:10.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.018+0000 7f39a606d640 1 Processor -- start 2024-04-26T23:20:10.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.018+0000 7f39a606d640 1 -- start start 2024-04-26T23:20:10.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.019+0000 7f39a606d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f39a015f460 0x7f39a00d3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:10.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.019+0000 7f39a606d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f39a00d2150 0x7f39a00d0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:10.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.019+0000 7f39a606d640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39a015f840 con 0x7f39a00d0760 2024-04-26T23:20:10.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.019+0000 7f39a606d640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f39a00d41c0 con 0x7f39a015f460 2024-04-26T23:20:10.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.019+0000 7f39a606d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f39a00d37f0 con 0x7f39a00d2150 2024-04-26T23:20:10.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.019+0000 7f399f7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f39a015f460 0x7f39a00d3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:10.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.019+0000 7f399f7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f39a015f460 0x7f39a00d3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46268/0 (socket says 172.21.15.47:46268) 2024-04-26T23:20:10.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.019+0000 7f399f7fe640 1 -- 172.21.15.47:0/296457631 learned_addr learned my addr 172.21.15.47:0/296457631 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:10.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.019+0000 7f399f7fe640 1 -- 172.21.15.47:0/296457631 >> v2:172.21.15.203:3300/0 conn(0x7f39a00d2150 msgr2=0x7f39a00d0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:10.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.019+0000 7f399f7fe640 1 --2- 172.21.15.47:0/296457631 >> v2:172.21.15.203:3300/0 conn(0x7f39a00d2150 0x7f39a00d0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:10.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.019+0000 7f399f7fe640 1 -- 172.21.15.47:0/296457631 >> v1:172.21.15.203:6789/0 conn(0x7f39a00d0760 legacy=0x7f39a00d1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:10.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.019+0000 7f399f7fe640 1 -- 172.21.15.47:0/296457631 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a00d2ec0 con 0x7f39a015f460 2024-04-26T23:20:10.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.020+0000 7f399f7fe640 1 --2- 172.21.15.47:0/296457631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39a015f460 0x7f39a00d3c80 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f39a008ef50 tx=0x7f399402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=872cbacef6884309 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:10.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.020+0000 7f399e7fc640 1 -- 172.21.15.47:0/296457631 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3994037070 con 0x7f39a015f460 2024-04-26T23:20:10.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.020+0000 7f399e7fc640 1 -- 172.21.15.47:0/296457631 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3994032440 con 0x7f39a015f460 2024-04-26T23:20:10.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.020+0000 7f399e7fc640 1 -- 172.21.15.47:0/296457631 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3994032810 con 0x7f39a015f460 2024-04-26T23:20:10.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.021+0000 7f39a606d640 1 -- 172.21.15.47:0/296457631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39a015f460 msgr2=0x7f39a00d3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:10.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.021+0000 7f39a606d640 1 --2- 172.21.15.47:0/296457631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39a015f460 0x7f39a00d3c80 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f39a008ef50 tx=0x7f399402a1a0 comp rx=0 tx=0).stop 2024-04-26T23:20:10.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.021+0000 7f39a606d640 1 -- 172.21.15.47:0/296457631 shutdown_connections 2024-04-26T23:20:10.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.021+0000 7f39a606d640 1 --2- 172.21.15.47:0/296457631 >> v2:172.21.15.203:3300/0 conn(0x7f39a00d2150 0x7f39a00d0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:10.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.021+0000 7f39a606d640 1 --2- 172.21.15.47:0/296457631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39a015f460 0x7f39a00d3c80 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:10.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.021+0000 7f39a606d640 1 -- 172.21.15.47:0/296457631 >> 172.21.15.47:0/296457631 conn(0x7f39a009e3f0 msgr2=0x7f39a00d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:10.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.021+0000 7f39a606d640 1 -- 172.21.15.47:0/296457631 shutdown_connections 2024-04-26T23:20:10.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.021+0000 7f39a606d640 1 -- 172.21.15.47:0/296457631 wait complete. 2024-04-26T23:20:10.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.022+0000 7f39a606d640 1 Processor -- start 2024-04-26T23:20:10.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.022+0000 7f39a606d640 1 -- start start 2024-04-26T23:20:10.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.023+0000 7f39a606d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39a00d0760 0x7f39a01f5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:10.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.023+0000 7f39a606d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39a00d2150 0x7f39a015a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:10.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.023+0000 7f39a606d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f39a015f460 0x7f39a01e8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:10.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.023+0000 7f39a606d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f39a00d5f30 con 0x7f39a00d2150 2024-04-26T23:20:10.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.023+0000 7f39a606d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f39a00d3480 con 0x7f39a00d0760 2024-04-26T23:20:10.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.023+0000 7f39a606d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f39a006ce20 con 0x7f39a015f460 2024-04-26T23:20:10.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.023+0000 7f399effd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39a00d2150 0x7f39a015a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:10.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.023+0000 7f399effd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39a00d2150 0x7f39a015a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46270/0 (socket says 172.21.15.47:46270) 2024-04-26T23:20:10.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.023+0000 7f399effd640 1 -- 172.21.15.47:0/2754622641 learned_addr learned my addr 172.21.15.47:0/2754622641 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:10.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.023+0000 7f399f7fe640 1 --2- 172.21.15.47:0/2754622641 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39a00d0760 0x7f39a01f5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:10.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.023+0000 7f399ffff640 1 --2- 172.21.15.47:0/2754622641 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f39a015f460 0x7f39a01e8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:10.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.023+0000 7f399effd640 1 -- 172.21.15.47:0/2754622641 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f39a015f460 msgr2=0x7f39a01e8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:10.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.023+0000 7f399effd640 1 --2- 172.21.15.47:0/2754622641 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f39a015f460 0x7f39a01e8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:10.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.023+0000 7f399effd640 1 -- 172.21.15.47:0/2754622641 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39a00d0760 msgr2=0x7f39a01f5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:10.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.023+0000 7f399effd640 1 --2- 172.21.15.47:0/2754622641 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39a00d0760 0x7f39a01f5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:10.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.023+0000 7f399effd640 1 -- 172.21.15.47:0/2754622641 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a01f90e0 con 0x7f39a00d2150 2024-04-26T23:20:10.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.023+0000 7f399f7fe640 1 --2- 172.21.15.47:0/2754622641 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39a00d0760 0x7f39a01f5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:20:10.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.023+0000 7f399ffff640 1 --2- 172.21.15.47:0/2754622641 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f39a015f460 0x7f39a01e8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:10.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.023+0000 7f399effd640 1 --2- 172.21.15.47:0/2754622641 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39a00d2150 0x7f39a015a880 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f399000c410 tx=0x7f399000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:10.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.024+0000 7f3987fff640 1 -- 172.21.15.47:0/2754622641 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3990017070 con 0x7f39a00d2150 2024-04-26T23:20:10.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.024+0000 7f3987fff640 1 -- 172.21.15.47:0/2754622641 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39900035f0 con 0x7f39a00d2150 2024-04-26T23:20:10.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.024+0000 7f39a606d640 1 -- 172.21.15.47:0/2754622641 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39a01f93c0 con 0x7f39a00d2150 2024-04-26T23:20:10.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.024+0000 7f3987fff640 1 -- 172.21.15.47:0/2754622641 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3990003a20 con 0x7f39a00d2150 2024-04-26T23:20:10.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.024+0000 7f39a606d640 1 -- 172.21.15.47:0/2754622641 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39a01f5670 con 0x7f39a00d2150 2024-04-26T23:20:10.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.025+0000 7f39a606d640 1 -- 172.21.15.47:0/2754622641 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39a0069600 con 0x7f39a00d2150 2024-04-26T23:20:10.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.025+0000 7f3987fff640 1 -- 172.21.15.47:0/2754622641 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f399001a070 con 0x7f39a00d2150 2024-04-26T23:20:10.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.025+0000 7f3987fff640 1 --2- 172.21.15.47:0/2754622641 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f398c06c8f0 0x7f398c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:10.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.026+0000 7f3987fff640 1 -- 172.21.15.47:0/2754622641 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6582+0+0 (secure 0 0 0) 0x7f399008b2d0 con 0x7f39a00d2150 2024-04-26T23:20:10.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.026+0000 7f399f7fe640 1 --2- 172.21.15.47:0/2754622641 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f398c06c8f0 0x7f398c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:10.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.026+0000 7f399f7fe640 1 --2- 172.21.15.47:0/2754622641 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f398c06c8f0 0x7f398c06edb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3994034000 tx=0x7f399402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:10.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.030+0000 7f3987fff640 1 -- 172.21.15.47:0/2754622641 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3990059420 con 0x7f39a00d2150 2024-04-26T23:20:10.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.216+0000 7f39a606d640 1 -- 172.21.15.47:0/2754622641 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f39a00d5af0 con 0x7f39a00d2150 2024-04-26T23:20:10.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.947+0000 7f3987fff640 1 -- 172.21.15.47:0/2754622641 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v43) ==== 181+0+0 (secure 0 0 0) 0x7f39900010c0 con 0x7f39a00d2150 2024-04-26T23:20:10.949 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' removed 2024-04-26T23:20:10.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.953+0000 7f39a606d640 1 -- 172.21.15.47:0/2754622641 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f398c06c8f0 msgr2=0x7f398c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:10.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.953+0000 7f39a606d640 1 --2- 172.21.15.47:0/2754622641 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f398c06c8f0 0x7f398c06edb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3994034000 tx=0x7f399402e000 comp rx=0 tx=0).stop 2024-04-26T23:20:10.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.953+0000 7f39a606d640 1 -- 172.21.15.47:0/2754622641 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39a00d2150 msgr2=0x7f39a015a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:10.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.953+0000 7f39a606d640 1 --2- 172.21.15.47:0/2754622641 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39a00d2150 0x7f39a015a880 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f399000c410 tx=0x7f399000c8d0 comp rx=0 tx=0).stop 2024-04-26T23:20:10.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.955+0000 7f39a606d640 1 -- 172.21.15.47:0/2754622641 shutdown_connections 2024-04-26T23:20:10.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.955+0000 7f39a606d640 1 --2- 172.21.15.47:0/2754622641 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f398c06c8f0 0x7f398c06edb0 secure :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3994034000 tx=0x7f399402e000 comp rx=0 tx=0).stop 2024-04-26T23:20:10.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.955+0000 7f39a606d640 1 --2- 172.21.15.47:0/2754622641 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f39a015f460 0x7f39a01e8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:10.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.955+0000 7f39a606d640 1 --2- 172.21.15.47:0/2754622641 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39a00d2150 0x7f39a015a880 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:10.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.955+0000 7f39a606d640 1 --2- 172.21.15.47:0/2754622641 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39a00d0760 0x7f39a01f5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:10.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.955+0000 7f39a606d640 1 -- 172.21.15.47:0/2754622641 >> 172.21.15.47:0/2754622641 conn(0x7f39a009e3f0 msgr2=0x7f39a00b4b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:10.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.956+0000 7f39a606d640 1 -- 172.21.15.47:0/2754622641 shutdown_connections 2024-04-26T23:20:10.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:10.956+0000 7f39a606d640 1 -- 172.21.15.47:0/2754622641 wait complete. 2024-04-26T23:20:10.973 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T23:20:11.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.084+0000 7fd1beec5640 1 Processor -- start 2024-04-26T23:20:11.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.084+0000 7fd1beec5640 1 -- start start 2024-04-26T23:20:11.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.085+0000 7fd1beec5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1b815f340 0x7fd1b80d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:11.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.085+0000 7fd1beec5640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd1b815fe30 0x7fd1b80cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:11.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.085+0000 7fd1beec5640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1b8163160 con 0x7fd1b81606c0 2024-04-26T23:20:11.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.085+0000 7fd1beec5640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fd1b8162f10 con 0x7fd1b815fe30 2024-04-26T23:20:11.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.085+0000 7fd1beec5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd1b8163d70 con 0x7fd1b815f340 2024-04-26T23:20:11.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.085+0000 7fd1bd462640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd1b815fe30 0x7fd1b80cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:11.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.085+0000 7fd1bd462640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd1b815fe30 0x7fd1b80cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46286/0 (socket says 172.21.15.47:46286) 2024-04-26T23:20:11.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.085+0000 7fd1bd462640 1 -- 172.21.15.47:0/1609983820 learned_addr learned my addr 172.21.15.47:0/1609983820 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:11.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.085+0000 7fd1bcc61640 1 --2- 172.21.15.47:0/1609983820 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1b815f340 0x7fd1b80d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:11.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.085+0000 7fd1bd462640 1 -- 172.21.15.47:0/1609983820 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1b815f340 msgr2=0x7fd1b80d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:11.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.085+0000 7fd1bd462640 1 --2- 172.21.15.47:0/1609983820 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1b815f340 0x7fd1b80d2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:11.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.085+0000 7fd1bd462640 1 -- 172.21.15.47:0/1609983820 >> v1:172.21.15.203:6789/0 conn(0x7fd1b81606c0 legacy=0x7fd1b815f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:11.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.085+0000 7fd1bd462640 1 -- 172.21.15.47:0/1609983820 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1b8163440 con 0x7fd1b815fe30 2024-04-26T23:20:11.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.086+0000 7fd1bd462640 1 --2- 172.21.15.47:0/1609983820 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1b815fe30 0x7fd1b80cf640 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0009e50 tx=0x7fd1b002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ab10ba3b0246191 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:11.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.086+0000 7fd1b77fe640 1 -- 172.21.15.47:0/1609983820 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b0036070 con 0x7fd1b815fe30 2024-04-26T23:20:11.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.086+0000 7fd1b77fe640 1 -- 172.21.15.47:0/1609983820 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1b0031600 con 0x7fd1b815fe30 2024-04-26T23:20:11.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.086+0000 7fd1b77fe640 1 -- 172.21.15.47:0/1609983820 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b00319d0 con 0x7fd1b815fe30 2024-04-26T23:20:11.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.087+0000 7fd1beec5640 1 -- 172.21.15.47:0/1609983820 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1b815fe30 msgr2=0x7fd1b80cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:11.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.087+0000 7fd1beec5640 1 --2- 172.21.15.47:0/1609983820 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1b815fe30 0x7fd1b80cf640 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0009e50 tx=0x7fd1b002a710 comp rx=0 tx=0).stop 2024-04-26T23:20:11.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.087+0000 7fd1beec5640 1 -- 172.21.15.47:0/1609983820 shutdown_connections 2024-04-26T23:20:11.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.087+0000 7fd1beec5640 1 --2- 172.21.15.47:0/1609983820 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1b815fe30 0x7fd1b80cf640 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:11.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.087+0000 7fd1beec5640 1 --2- 172.21.15.47:0/1609983820 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1b815f340 0x7fd1b80d2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:11.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.087+0000 7fd1beec5640 1 -- 172.21.15.47:0/1609983820 >> 172.21.15.47:0/1609983820 conn(0x7fd1b809ba50 msgr2=0x7fd1b809be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:11.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.087+0000 7fd1beec5640 1 -- 172.21.15.47:0/1609983820 shutdown_connections 2024-04-26T23:20:11.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.087+0000 7fd1beec5640 1 -- 172.21.15.47:0/1609983820 wait complete. 2024-04-26T23:20:11.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.088+0000 7fd1beec5640 1 Processor -- start 2024-04-26T23:20:11.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.088+0000 7fd1beec5640 1 -- start start 2024-04-26T23:20:11.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.089+0000 7fd1beec5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1b815f340 0x7fd1b8155c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:11.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.089+0000 7fd1beec5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1b815fe30 0x7fd1b806d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:11.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.089+0000 7fd1beec5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1b81606c0 0x7fd1b8157270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:11.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.089+0000 7fd1beec5640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b81602a0 con 0x7fd1b815f340 2024-04-26T23:20:11.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.089+0000 7fd1beec5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b8163a00 con 0x7fd1b815fe30 2024-04-26T23:20:11.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.089+0000 7fd1beec5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd1b80d2a90 con 0x7fd1b81606c0 2024-04-26T23:20:11.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.089+0000 7fd1bcc61640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1b815f340 0x7fd1b8155c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:11.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.089+0000 7fd1bcc61640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1b815f340 0x7fd1b8155c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46294/0 (socket says 172.21.15.47:46294) 2024-04-26T23:20:11.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.089+0000 7fd1bcc61640 1 -- 172.21.15.47:0/1135331089 learned_addr learned my addr 172.21.15.47:0/1135331089 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:11.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.089+0000 7fd1b7fff640 1 --2- 172.21.15.47:0/1135331089 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1b815fe30 0x7fd1b806d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:11.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.089+0000 7fd1bd462640 1 --2- 172.21.15.47:0/1135331089 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1b81606c0 0x7fd1b8157270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:11.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.089+0000 7fd1bcc61640 1 -- 172.21.15.47:0/1135331089 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1b81606c0 msgr2=0x7fd1b8157270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:11.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.089+0000 7fd1bcc61640 1 --2- 172.21.15.47:0/1135331089 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1b81606c0 0x7fd1b8157270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:11.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.089+0000 7fd1bcc61640 1 -- 172.21.15.47:0/1135331089 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1b815fe30 msgr2=0x7fd1b806d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:11.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.089+0000 7fd1bcc61640 1 --2- 172.21.15.47:0/1135331089 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1b815fe30 0x7fd1b806d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:11.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.089+0000 7fd1bcc61640 1 -- 172.21.15.47:0/1135331089 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1b806e070 con 0x7fd1b815f340 2024-04-26T23:20:11.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.089+0000 7fd1bd462640 1 --2- 172.21.15.47:0/1135331089 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1b81606c0 0x7fd1b8157270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:20:11.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.090+0000 7fd1bcc61640 1 --2- 172.21.15.47:0/1135331089 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1b815f340 0x7fd1b8155c60 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac00d800 tx=0x7fd1ac00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:11.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.090+0000 7fd1b57fa640 1 -- 172.21.15.47:0/1135331089 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac00b6c0 con 0x7fd1b815f340 2024-04-26T23:20:11.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.090+0000 7fd1b57fa640 1 -- 172.21.15.47:0/1135331089 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1ac019070 con 0x7fd1b815f340 2024-04-26T23:20:11.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.090+0000 7fd1b57fa640 1 -- 172.21.15.47:0/1135331089 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac00ba80 con 0x7fd1b815f340 2024-04-26T23:20:11.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.090+0000 7fd1beec5640 1 -- 172.21.15.47:0/1135331089 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1b806e350 con 0x7fd1b815f340 2024-04-26T23:20:11.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.090+0000 7fd1beec5640 1 -- 172.21.15.47:0/1135331089 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1b806e910 con 0x7fd1b815f340 2024-04-26T23:20:11.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.091+0000 7fd1beec5640 1 -- 172.21.15.47:0/1135331089 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1b8068e00 con 0x7fd1b815f340 2024-04-26T23:20:11.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.091+0000 7fd1b57fa640 1 -- 172.21.15.47:0/1135331089 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd1ac00bc20 con 0x7fd1b815f340 2024-04-26T23:20:11.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.092+0000 7fd1b57fa640 1 --2- 172.21.15.47:0/1135331089 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd19c06c8f0 0x7fd19c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:11.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.092+0000 7fd1b57fa640 1 -- 172.21.15.47:0/1135331089 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6144+0+0 (secure 0 0 0) 0x7fd1ac08c130 con 0x7fd1b815f340 2024-04-26T23:20:11.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.092+0000 7fd1b7fff640 1 --2- 172.21.15.47:0/1135331089 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd19c06c8f0 0x7fd19c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:11.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.093+0000 7fd1b7fff640 1 --2- 172.21.15.47:0/1135331089 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd19c06c8f0 0x7fd19c06edb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd1b815fd90 tx=0x7fd1a4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:11.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.096+0000 7fd1b57fa640 1 -- 172.21.15.47:0/1135331089 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd1ac05a3b0 con 0x7fd1b815f340 2024-04-26T23:20:11.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.282+0000 7fd1beec5640 1 -- 172.21.15.47:0/1135331089 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd1b80d1b30 con 0x7fd1b815f340 2024-04-26T23:20:11.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.953+0000 7fd1b57fa640 1 -- 172.21.15.47:0/1135331089 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v44) ==== 169+0+0 (secure 0 0 0) 0x7fd1ac059cb0 con 0x7fd1b815f340 2024-04-26T23:20:11.954 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' removed 2024-04-26T23:20:11.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.957+0000 7fd1beec5640 1 -- 172.21.15.47:0/1135331089 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd19c06c8f0 msgr2=0x7fd19c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:11.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.957+0000 7fd1beec5640 1 --2- 172.21.15.47:0/1135331089 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd19c06c8f0 0x7fd19c06edb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd1b815fd90 tx=0x7fd1a4009040 comp rx=0 tx=0).stop 2024-04-26T23:20:11.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.957+0000 7fd1beec5640 1 -- 172.21.15.47:0/1135331089 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1b815f340 msgr2=0x7fd1b8155c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:11.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.957+0000 7fd1beec5640 1 --2- 172.21.15.47:0/1135331089 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1b815f340 0x7fd1b8155c60 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac00d800 tx=0x7fd1ac00dcc0 comp rx=0 tx=0).stop 2024-04-26T23:20:11.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.957+0000 7fd1beec5640 1 -- 172.21.15.47:0/1135331089 shutdown_connections 2024-04-26T23:20:11.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.957+0000 7fd1beec5640 1 --2- 172.21.15.47:0/1135331089 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd19c06c8f0 0x7fd19c06edb0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:11.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.957+0000 7fd1beec5640 1 --2- 172.21.15.47:0/1135331089 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1b81606c0 0x7fd1b8157270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:11.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.957+0000 7fd1beec5640 1 --2- 172.21.15.47:0/1135331089 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1b815fe30 0x7fd1b806d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:11.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.957+0000 7fd1beec5640 1 --2- 172.21.15.47:0/1135331089 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1b815f340 0x7fd1b8155c60 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:11.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.957+0000 7fd1beec5640 1 -- 172.21.15.47:0/1135331089 >> 172.21.15.47:0/1135331089 conn(0x7fd1b809ba50 msgr2=0x7fd1b8160da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:11.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.958+0000 7fd1beec5640 1 -- 172.21.15.47:0/1135331089 shutdown_connections 2024-04-26T23:20:11.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:11.958+0000 7fd1beec5640 1 -- 172.21.15.47:0/1135331089 wait complete. 2024-04-26T23:20:11.976 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm data_new data_new --yes-i-really-really-mean-it 2024-04-26T23:20:12.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.085+0000 7f2b57d4e640 1 Processor -- start 2024-04-26T23:20:12.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.085+0000 7f2b57d4e640 1 -- start start 2024-04-26T23:20:12.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.086+0000 7f2b57d4e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2b5015f340 0x7f2b500d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:12.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.086+0000 7f2b57d4e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b501606c0 0x7f2b5015f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:12.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.086+0000 7f2b57d4e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b50163160 con 0x7f2b5015fc60 2024-04-26T23:20:12.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.086+0000 7f2b57d4e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f2b50162f10 con 0x7f2b5015f340 2024-04-26T23:20:12.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.086+0000 7f2b57d4e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f2b50163d70 con 0x7f2b501606c0 2024-04-26T23:20:12.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.087+0000 7f2b55aea640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2b5015f340 0x7f2b500d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:12.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.087+0000 7f2b552e9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b501606c0 0x7f2b5015f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:12.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.087+0000 7f2b55aea640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2b5015f340 0x7f2b500d2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.47:44860/0 (socket says 172.21.15.47:44860) 2024-04-26T23:20:12.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.087+0000 7f2b562eb640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f2b5015fc60 0x7f2b500d2e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.47:38852/0 (socket says 172.21.15.47:38852) 2024-04-26T23:20:12.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.087+0000 7f2b552e9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b501606c0 0x7f2b5015f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.47:36590/0 (socket says 172.21.15.47:36590) 2024-04-26T23:20:12.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.087+0000 7f2b55aea640 1 -- 172.21.15.47:0/774333517 learned_addr learned my addr 172.21.15.47:0/774333517 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:12.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.087+0000 7f2b552e9640 1 -- 172.21.15.47:0/774333517 >> v2:172.21.15.203:3300/0 conn(0x7f2b5015f340 msgr2=0x7f2b500d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:12.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.087+0000 7f2b552e9640 1 --2- 172.21.15.47:0/774333517 >> v2:172.21.15.203:3300/0 conn(0x7f2b5015f340 0x7f2b500d2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:12.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.087+0000 7f2b552e9640 1 -- 172.21.15.47:0/774333517 >> v1:172.21.15.203:6789/0 conn(0x7f2b5015fc60 legacy=0x7f2b500d2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:12.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.087+0000 7f2b552e9640 1 -- 172.21.15.47:0/774333517 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b50163440 con 0x7f2b501606c0 2024-04-26T23:20:12.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.087+0000 7f2b55aea640 1 --2- 172.21.15.47:0/774333517 >> v2:172.21.15.203:3300/0 conn(0x7f2b5015f340 0x7f2b500d2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:20:12.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.088+0000 7f2b552e9640 1 --2- 172.21.15.47:0/774333517 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b501606c0 0x7f2b5015f720 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2b440097a0 tx=0x7f2b4402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c560d4639aafaa36 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:12.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.088+0000 7f2b54ae8640 1 -- 172.21.15.47:0/774333517 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b44037070 con 0x7f2b501606c0 2024-04-26T23:20:12.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.088+0000 7f2b54ae8640 1 -- 172.21.15.47:0/774333517 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b44032440 con 0x7f2b501606c0 2024-04-26T23:20:12.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.088+0000 7f2b54ae8640 1 -- 172.21.15.47:0/774333517 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b44032810 con 0x7f2b501606c0 2024-04-26T23:20:12.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.088+0000 7f2b57d4e640 1 -- 172.21.15.47:0/774333517 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b501606c0 msgr2=0x7f2b5015f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:12.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.088+0000 7f2b57d4e640 1 --2- 172.21.15.47:0/774333517 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b501606c0 0x7f2b5015f720 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2b440097a0 tx=0x7f2b4402a180 comp rx=0 tx=0).stop 2024-04-26T23:20:12.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.089+0000 7f2b57d4e640 1 -- 172.21.15.47:0/774333517 shutdown_connections 2024-04-26T23:20:12.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.089+0000 7f2b57d4e640 1 --2- 172.21.15.47:0/774333517 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b501606c0 0x7f2b5015f720 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:12.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.089+0000 7f2b57d4e640 1 --2- 172.21.15.47:0/774333517 >> v2:172.21.15.203:3300/0 conn(0x7f2b5015f340 0x7f2b500d2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:12.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.089+0000 7f2b57d4e640 1 -- 172.21.15.47:0/774333517 >> 172.21.15.47:0/774333517 conn(0x7f2b5009ba50 msgr2=0x7f2b5009be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:12.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.089+0000 7f2b57d4e640 1 -- 172.21.15.47:0/774333517 shutdown_connections 2024-04-26T23:20:12.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.089+0000 7f2b57d4e640 1 -- 172.21.15.47:0/774333517 wait complete. 2024-04-26T23:20:12.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.090+0000 7f2b57d4e640 1 Processor -- start 2024-04-26T23:20:12.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.090+0000 7f2b57d4e640 1 -- start start 2024-04-26T23:20:12.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.090+0000 7f2b57d4e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b5015f340 0x7f2b5015aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:12.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.090+0000 7f2b57d4e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b5015fc60 0x7f2b5006e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:12.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.091+0000 7f2b57d4e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b501606c0 0x7f2b5015ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:12.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.091+0000 7f2b57d4e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2b500d2a90 con 0x7f2b5015f340 2024-04-26T23:20:12.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.091+0000 7f2b57d4e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2b50163a00 con 0x7f2b5015fc60 2024-04-26T23:20:12.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.091+0000 7f2b57d4e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f2b501600d0 con 0x7f2b501606c0 2024-04-26T23:20:12.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.091+0000 7f2b55aea640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b5015f340 0x7f2b5015aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:12.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.091+0000 7f2b55aea640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b5015f340 0x7f2b5015aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46310/0 (socket says 172.21.15.47:46310) 2024-04-26T23:20:12.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.091+0000 7f2b55aea640 1 -- 172.21.15.47:0/773237136 learned_addr learned my addr 172.21.15.47:0/773237136 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:12.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.091+0000 7f2b552e9640 1 --2- 172.21.15.47:0/773237136 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b5015fc60 0x7f2b5006e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:12.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.091+0000 7f2b562eb640 1 --2- 172.21.15.47:0/773237136 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b501606c0 0x7f2b5015ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:12.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.091+0000 7f2b55aea640 1 -- 172.21.15.47:0/773237136 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b501606c0 msgr2=0x7f2b5015ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:12.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.091+0000 7f2b55aea640 1 --2- 172.21.15.47:0/773237136 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b501606c0 0x7f2b5015ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:12.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.091+0000 7f2b55aea640 1 -- 172.21.15.47:0/773237136 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b5015fc60 msgr2=0x7f2b5006e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:12.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.091+0000 7f2b55aea640 1 --2- 172.21.15.47:0/773237136 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b5015fc60 0x7f2b5006e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:12.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.091+0000 7f2b55aea640 1 -- 172.21.15.47:0/773237136 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b5006d4f0 con 0x7f2b5015f340 2024-04-26T23:20:12.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.091+0000 7f2b552e9640 1 --2- 172.21.15.47:0/773237136 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b5015fc60 0x7f2b5006e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:12.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.091+0000 7f2b55aea640 1 --2- 172.21.15.47:0/773237136 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b5015f340 0x7f2b5015aad0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f2b4800e9d0 tx=0x7f2b4800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:12.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.091+0000 7f2b4e7fc640 1 -- 172.21.15.47:0/773237136 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b48017070 con 0x7f2b5015f340 2024-04-26T23:20:12.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.091+0000 7f2b4e7fc640 1 -- 172.21.15.47:0/773237136 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b4800ca60 con 0x7f2b5015f340 2024-04-26T23:20:12.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.091+0000 7f2b57d4e640 1 -- 172.21.15.47:0/773237136 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b5006d7d0 con 0x7f2b5015f340 2024-04-26T23:20:12.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.091+0000 7f2b4e7fc640 1 -- 172.21.15.47:0/773237136 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b4800ce10 con 0x7f2b5015f340 2024-04-26T23:20:12.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.091+0000 7f2b57d4e640 1 -- 172.21.15.47:0/773237136 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b5006dd90 con 0x7f2b5015f340 2024-04-26T23:20:12.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.092+0000 7f2b57d4e640 1 -- 172.21.15.47:0/773237136 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b50069600 con 0x7f2b5015f340 2024-04-26T23:20:12.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.093+0000 7f2b4e7fc640 1 -- 172.21.15.47:0/773237136 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2b4801a070 con 0x7f2b5015f340 2024-04-26T23:20:12.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.093+0000 7f2b4e7fc640 1 --2- 172.21.15.47:0/773237136 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f2b3c06c8f0 0x7f2b3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:12.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.093+0000 7f2b4e7fc640 1 -- 172.21.15.47:0/773237136 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5730+0+0 (secure 0 0 0) 0x7f2b4808bf00 con 0x7f2b5015f340 2024-04-26T23:20:12.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.094+0000 7f2b552e9640 1 --2- 172.21.15.47:0/773237136 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f2b3c06c8f0 0x7f2b3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:12.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.094+0000 7f2b552e9640 1 --2- 172.21.15.47:0/773237136 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f2b3c06c8f0 0x7f2b3c06edb0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2b44034000 tx=0x7f2b4402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:12.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.098+0000 7f2b4e7fc640 1 -- 172.21.15.47:0/773237136 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2b4805a280 con 0x7f2b5015f340 2024-04-26T23:20:12.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.287+0000 7f2b57d4e640 1 -- 172.21.15.47:0/773237136 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "data_new", "pool2": "data_new", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2b50068e00 con 0x7f2b5015f340 2024-04-26T23:20:12.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.958+0000 7f2b4e7fc640 1 -- 172.21.15.47:0/773237136 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "data_new", "pool2": "data_new", "yes_i_really_really_mean_it": true}]=0 pool 'data_new' removed v45) ==== 160+0+0 (secure 0 0 0) 0x7f2b48059b80 con 0x7f2b5015f340 2024-04-26T23:20:12.959 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'data_new' removed 2024-04-26T23:20:12.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.961+0000 7f2b57d4e640 1 -- 172.21.15.47:0/773237136 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f2b3c06c8f0 msgr2=0x7f2b3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:12.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.961+0000 7f2b57d4e640 1 --2- 172.21.15.47:0/773237136 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f2b3c06c8f0 0x7f2b3c06edb0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2b44034000 tx=0x7f2b4402e000 comp rx=0 tx=0).stop 2024-04-26T23:20:12.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.961+0000 7f2b57d4e640 1 -- 172.21.15.47:0/773237136 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b5015f340 msgr2=0x7f2b5015aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:12.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.961+0000 7f2b57d4e640 1 --2- 172.21.15.47:0/773237136 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b5015f340 0x7f2b5015aad0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f2b4800e9d0 tx=0x7f2b4800ee90 comp rx=0 tx=0).stop 2024-04-26T23:20:12.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.961+0000 7f2b57d4e640 1 -- 172.21.15.47:0/773237136 shutdown_connections 2024-04-26T23:20:12.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.961+0000 7f2b57d4e640 1 --2- 172.21.15.47:0/773237136 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f2b3c06c8f0 0x7f2b3c06edb0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:12.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.961+0000 7f2b57d4e640 1 --2- 172.21.15.47:0/773237136 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b501606c0 0x7f2b5015ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:12.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.961+0000 7f2b57d4e640 1 --2- 172.21.15.47:0/773237136 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b5015fc60 0x7f2b5006e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:12.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.961+0000 7f2b57d4e640 1 --2- 172.21.15.47:0/773237136 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b5015f340 0x7f2b5015aad0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:12.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.961+0000 7f2b57d4e640 1 -- 172.21.15.47:0/773237136 >> 172.21.15.47:0/773237136 conn(0x7f2b5009ba50 msgr2=0x7f2b50157a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:12.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.962+0000 7f2b57d4e640 1 -- 172.21.15.47:0/773237136 shutdown_connections 2024-04-26T23:20:12.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:12.962+0000 7f2b57d4e640 1 -- 172.21.15.47:0/773237136 wait complete. 2024-04-26T23:20:12.982 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-26T23:20:13.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.091+0000 7f32ae683640 1 Processor -- start 2024-04-26T23:20:13.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.091+0000 7f32ae683640 1 -- start start 2024-04-26T23:20:13.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.091+0000 7f32ae683640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f32a80d3bc0 0x7f32a80d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:13.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.091+0000 7f32ae683640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f32a80b5f20 0x7f32a80b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:13.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.091+0000 7f32ae683640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32a80b7e50 con 0x7f32a8162880 2024-04-26T23:20:13.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.091+0000 7f32ae683640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f32a80d0580 con 0x7f32a80d3bc0 2024-04-26T23:20:13.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.091+0000 7f32ae683640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f32a80d1800 con 0x7f32a80b5f20 2024-04-26T23:20:13.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.092+0000 7f32a7fff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f32a8162880 0x7f32a80d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35322/0 (socket says 172.21.15.47:35322) 2024-04-26T23:20:13.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.092+0000 7f32a7fff640 1 -- 172.21.15.47:0/2492362841 learned_addr learned my addr 172.21.15.47:0/2492362841 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:13.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.092+0000 7f32a6ffd640 1 -- 172.21.15.47:0/2492362841 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2559126618 0 0) 0x7f32a80b7e50 con 0x7f32a8162880 2024-04-26T23:20:13.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.092+0000 7f32a77fe640 1 --2- 172.21.15.47:0/2492362841 >> v2:172.21.15.203:3300/0 conn(0x7f32a80d3bc0 0x7f32a80d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:13.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.092+0000 7f32a6ffd640 1 -- 172.21.15.47:0/2492362841 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3290003520 con 0x7f32a8162880 2024-04-26T23:20:13.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.092+0000 7f32acc20640 1 --2- 172.21.15.47:0/2492362841 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f32a80b5f20 0x7f32a80b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:13.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.092+0000 7f32a6ffd640 1 -- 172.21.15.47:0/2492362841 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3197295357 0 0) 0x7f3290003520 con 0x7f32a8162880 2024-04-26T23:20:13.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.092+0000 7f32a6ffd640 1 -- 172.21.15.47:0/2492362841 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f32a80b5f20 msgr2=0x7f32a80b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:13.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.092+0000 7f32a6ffd640 1 --2- 172.21.15.47:0/2492362841 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f32a80b5f20 0x7f32a80b8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:13.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.092+0000 7f32a6ffd640 1 -- 172.21.15.47:0/2492362841 >> v2:172.21.15.203:3300/0 conn(0x7f32a80d3bc0 msgr2=0x7f32a80d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:13.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.092+0000 7f32a6ffd640 1 --2- 172.21.15.47:0/2492362841 >> v2:172.21.15.203:3300/0 conn(0x7f32a80d3bc0 0x7f32a80d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:13.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.092+0000 7f32a6ffd640 1 -- 172.21.15.47:0/2492362841 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32a80d07d0 con 0x7f32a8162880 2024-04-26T23:20:13.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.092+0000 7f32a6ffd640 1 -- 172.21.15.47:0/2492362841 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7f329c001670 con 0x7f32a8162880 2024-04-26T23:20:13.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.092+0000 7f32a6ffd640 1 -- 172.21.15.47:0/2492362841 >> v1:172.21.15.47:6789/0 conn(0x7f32a8162880 legacy=0x7f32a80d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:13.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.092+0000 7f32a6ffd640 1 --2- 172.21.15.47:0/2492362841 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f32a80b5f20 0x7f32900039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:13.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.092+0000 7f32a6ffd640 1 --2- 172.21.15.47:0/2492362841 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3290003f30 0x7f3290004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:13.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.092+0000 7f32a6ffd640 1 --2- 172.21.15.47:0/2492362841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3290004d40 0x7f32900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:13.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.092+0000 7f32a6ffd640 1 -- 172.21.15.47:0/2492362841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f32a80d0580 con 0x7f3290004d40 2024-04-26T23:20:13.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.092+0000 7f32a6ffd640 1 -- 172.21.15.47:0/2492362841 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f32a80d1800 con 0x7f3290003f30 2024-04-26T23:20:13.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.092+0000 7f32a6ffd640 1 -- 172.21.15.47:0/2492362841 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f329000b9c0 con 0x7f32a80b5f20 2024-04-26T23:20:13.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.092+0000 7f32acc20640 1 --2- 172.21.15.47:0/2492362841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3290004d40 0x7f32900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:13.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.092+0000 7f32a7fff640 1 --2- 172.21.15.47:0/2492362841 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f32a80b5f20 0x7f32900039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:13.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.092+0000 7f32a77fe640 1 --2- 172.21.15.47:0/2492362841 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3290003f30 0x7f3290004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:13.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.093+0000 7f32acc20640 1 -- 172.21.15.47:0/2492362841 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f32a80b5f20 msgr2=0x7f32900039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:13.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.093+0000 7f32acc20640 1 --2- 172.21.15.47:0/2492362841 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f32a80b5f20 0x7f32900039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:13.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.093+0000 7f32acc20640 1 -- 172.21.15.47:0/2492362841 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3290003f30 msgr2=0x7f3290004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:13.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.093+0000 7f32acc20640 1 --2- 172.21.15.47:0/2492362841 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3290003f30 0x7f3290004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:13.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.093+0000 7f32acc20640 1 -- 172.21.15.47:0/2492362841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3290003520 con 0x7f3290004d40 2024-04-26T23:20:13.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.093+0000 7f32acc20640 1 --2- 172.21.15.47:0/2492362841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3290004d40 0x7f32900051c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f32a0009fc0 tx=0x7f3290005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e01a96c062faad6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:13.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.094+0000 7f32a6ffd640 1 -- 172.21.15.47:0/2492362841 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a0009c10 con 0x7f3290004d40 2024-04-26T23:20:13.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.094+0000 7f32a6ffd640 1 -- 172.21.15.47:0/2492362841 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32a0036070 con 0x7f3290004d40 2024-04-26T23:20:13.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.094+0000 7f32a6ffd640 1 -- 172.21.15.47:0/2492362841 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a0031670 con 0x7f3290004d40 2024-04-26T23:20:13.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.094+0000 7f32ae683640 1 -- 172.21.15.47:0/2492362841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3290004d40 msgr2=0x7f32900051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:13.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.094+0000 7f32ae683640 1 --2- 172.21.15.47:0/2492362841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3290004d40 0x7f32900051c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f32a0009fc0 tx=0x7f3290005830 comp rx=0 tx=0).stop 2024-04-26T23:20:13.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.094+0000 7f32acc20640 1 -- 172.21.15.47:0/2492362841 reap_dead start 2024-04-26T23:20:13.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.094+0000 7f32ae683640 1 -- 172.21.15.47:0/2492362841 shutdown_connections 2024-04-26T23:20:13.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.094+0000 7f32ae683640 1 -- 172.21.15.47:0/2492362841 >> 172.21.15.47:0/2492362841 conn(0x7f32a80d4f60 msgr2=0x7f32a80d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:13.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.094+0000 7f32ae683640 1 -- 172.21.15.47:0/2492362841 shutdown_connections 2024-04-26T23:20:13.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.094+0000 7f32ae683640 1 -- 172.21.15.47:0/2492362841 wait complete. 2024-04-26T23:20:13.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.095+0000 7f32ae683640 1 Processor -- start 2024-04-26T23:20:13.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.096+0000 7f32ae683640 1 -- start start 2024-04-26T23:20:13.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.096+0000 7f32ae683640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32a80b4670 0x7f32a81f5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:13.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.096+0000 7f32ae683640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f32a81f80c0 0x7f32a806d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:13.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.096+0000 7f32ae683640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32a81f8540 0x7f32a806d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:13.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.096+0000 7f32ae683640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f32a8162c60 con 0x7f32a80b4670 2024-04-26T23:20:13.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.096+0000 7f32ae683640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f32a80d0d90 con 0x7f32a81f8540 2024-04-26T23:20:13.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.096+0000 7f32ae683640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f32a8162de0 con 0x7f32a81f80c0 2024-04-26T23:20:13.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.096+0000 7f32a7fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32a80b4670 0x7f32a81f5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:13.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.096+0000 7f32a7fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32a80b4670 0x7f32a81f5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46328/0 (socket says 172.21.15.47:46328) 2024-04-26T23:20:13.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.096+0000 7f32a7fff640 1 -- 172.21.15.47:0/1219504338 learned_addr learned my addr 172.21.15.47:0/1219504338 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:13.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.097+0000 7f32acc20640 1 --2- 172.21.15.47:0/1219504338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32a81f8540 0x7f32a806d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:13.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.097+0000 7f32a77fe640 1 --2- 172.21.15.47:0/1219504338 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f32a81f80c0 0x7f32a806d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:13.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.097+0000 7f32a7fff640 1 -- 172.21.15.47:0/1219504338 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f32a81f80c0 msgr2=0x7f32a806d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:13.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.097+0000 7f32a7fff640 1 --2- 172.21.15.47:0/1219504338 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f32a81f80c0 0x7f32a806d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:13.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.097+0000 7f32a7fff640 1 -- 172.21.15.47:0/1219504338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32a81f8540 msgr2=0x7f32a806d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:13.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.097+0000 7f32a7fff640 1 --2- 172.21.15.47:0/1219504338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32a81f8540 0x7f32a806d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:13.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.097+0000 7f32a7fff640 1 -- 172.21.15.47:0/1219504338 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32a81f8a80 con 0x7f32a80b4670 2024-04-26T23:20:13.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.097+0000 7f32a77fe640 1 --2- 172.21.15.47:0/1219504338 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f32a81f80c0 0x7f32a806d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:20:13.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.097+0000 7f32a7fff640 1 --2- 172.21.15.47:0/1219504338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32a80b4670 0x7f32a81f5470 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f329c0017e0 tx=0x7f329c0077a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:13.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.097+0000 7f32a4ff9640 1 -- 172.21.15.47:0/1219504338 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c00f9c0 con 0x7f32a80b4670 2024-04-26T23:20:13.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.097+0000 7f32a4ff9640 1 -- 172.21.15.47:0/1219504338 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f329c018070 con 0x7f32a80b4670 2024-04-26T23:20:13.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.097+0000 7f32ae683640 1 -- 172.21.15.47:0/1219504338 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32a81f8d60 con 0x7f32a80b4670 2024-04-26T23:20:13.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.097+0000 7f32a4ff9640 1 -- 172.21.15.47:0/1219504338 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c00fdc0 con 0x7f32a80b4670 2024-04-26T23:20:13.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.097+0000 7f32ae683640 1 -- 172.21.15.47:0/1219504338 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32a81f59b0 con 0x7f32a80b4670 2024-04-26T23:20:13.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.098+0000 7f32ae683640 1 -- 172.21.15.47:0/1219504338 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32a81f4aa0 con 0x7f32a80b4670 2024-04-26T23:20:13.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.099+0000 7f32a4ff9640 1 -- 172.21.15.47:0/1219504338 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f329c01d460 con 0x7f32a80b4670 2024-04-26T23:20:13.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.099+0000 7f32a4ff9640 1 --2- 172.21.15.47:0/1219504338 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f329406c920 0x7f329406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:13.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.099+0000 7f32a4ff9640 1 -- 172.21.15.47:0/1219504338 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f329c08ba90 con 0x7f32a80b4670 2024-04-26T23:20:13.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.100+0000 7f32a77fe640 1 --2- 172.21.15.47:0/1219504338 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f329406c920 0x7f329406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:13.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.100+0000 7f32a77fe640 1 --2- 172.21.15.47:0/1219504338 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f329406c920 0x7f329406ede0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f32a80b5c60 tx=0x7f3298008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:13.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.104+0000 7f32a4ff9640 1 -- 172.21.15.47:0/1219504338 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f329c05a0c0 con 0x7f32a80b4670 2024-04-26T23:20:13.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.290+0000 7f32ae683640 1 -- 172.21.15.47:0/1219504338 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f32a80694f0 con 0x7f32a80b4670 2024-04-26T23:20:13.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.293+0000 7f32a4ff9640 1 -- 172.21.15.47:0/1219504338 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f329c0599c0 con 0x7f32a80b4670 2024-04-26T23:20:13.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.295+0000 7f32ae683640 1 -- 172.21.15.47:0/1219504338 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f329406c920 msgr2=0x7f329406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:13.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.295+0000 7f32ae683640 1 --2- 172.21.15.47:0/1219504338 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f329406c920 0x7f329406ede0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f32a80b5c60 tx=0x7f3298008040 comp rx=0 tx=0).stop 2024-04-26T23:20:13.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.295+0000 7f32ae683640 1 -- 172.21.15.47:0/1219504338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32a80b4670 msgr2=0x7f32a81f5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:13.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.295+0000 7f32ae683640 1 --2- 172.21.15.47:0/1219504338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32a80b4670 0x7f32a81f5470 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f329c0017e0 tx=0x7f329c0077a0 comp rx=0 tx=0).stop 2024-04-26T23:20:13.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.295+0000 7f32ae683640 1 -- 172.21.15.47:0/1219504338 shutdown_connections 2024-04-26T23:20:13.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.295+0000 7f32ae683640 1 --2- 172.21.15.47:0/1219504338 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f329406c920 0x7f329406ede0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:13.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.295+0000 7f32ae683640 1 --2- 172.21.15.47:0/1219504338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32a81f8540 0x7f32a806d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:13.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.295+0000 7f32ae683640 1 --2- 172.21.15.47:0/1219504338 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f32a81f80c0 0x7f32a806d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:13.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.295+0000 7f32ae683640 1 --2- 172.21.15.47:0/1219504338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32a80b4670 0x7f32a81f5470 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:13.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.295+0000 7f32ae683640 1 -- 172.21.15.47:0/1219504338 >> 172.21.15.47:0/1219504338 conn(0x7f32a80d4f60 msgr2=0x7f32a815f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:13.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.295+0000 7f32ae683640 1 -- 172.21.15.47:0/1219504338 shutdown_connections 2024-04-26T23:20:13.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.295+0000 7f32ae683640 1 -- 172.21.15.47:0/1219504338 wait complete. 2024-04-26T23:20:13.307 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification' 2024-04-26T23:20:13.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.411+0000 7fbcc14f8640 1 Processor -- start 2024-04-26T23:20:13.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.411+0000 7fbcc14f8640 1 -- start start 2024-04-26T23:20:13.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.412+0000 7fbcc14f8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbcbc162880 0x7fbcbc0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:13.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.412+0000 7fbcc14f8640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcbc0b7e50 con 0x7fbcbc0d3bc0 2024-04-26T23:20:13.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.412+0000 7fbcc14f8640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcbc0d1800 con 0x7fbcbc0b5f20 2024-04-26T23:20:13.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.412+0000 7fbcc14f8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fbcbc0d0580 con 0x7fbcbc162880 2024-04-26T23:20:13.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.412+0000 7fbcbb7fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fbcbc0d3bc0 0x7fbcbc0b8300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35336/0 (socket says 172.21.15.47:35336) 2024-04-26T23:20:13.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.412+0000 7fbcbb7fe640 1 -- 172.21.15.47:0/2379108842 learned_addr learned my addr 172.21.15.47:0/2379108842 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:13.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.412+0000 7fbcb9ffb640 1 -- 172.21.15.47:0/2379108842 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2754946210 0 0) 0x7fbcbc0b7e50 con 0x7fbcbc0d3bc0 2024-04-26T23:20:13.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.412+0000 7fbcbaffd640 1 --2- 172.21.15.47:0/2379108842 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbcbc162880 0x7fbcbc0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:13.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.412+0000 7fbcb9ffb640 1 -- 172.21.15.47:0/2379108842 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbca8003520 con 0x7fbcbc0d3bc0 2024-04-26T23:20:13.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.412+0000 7fbcb9ffb640 1 -- 172.21.15.47:0/2379108842 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3293340270 0 0) 0x7fbca8003520 con 0x7fbcbc0d3bc0 2024-04-26T23:20:13.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.412+0000 7fbcb9ffb640 1 -- 172.21.15.47:0/2379108842 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbcbc162880 msgr2=0x7fbcbc0d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:13.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.412+0000 7fbcb9ffb640 1 --2- 172.21.15.47:0/2379108842 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbcbc162880 0x7fbcbc0d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:13.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.412+0000 7fbcb9ffb640 1 -- 172.21.15.47:0/2379108842 >> v1:172.21.15.203:6789/0 conn(0x7fbcbc0b5f20 legacy=0x7fbcbc0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:13.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.412+0000 7fbcb9ffb640 1 -- 172.21.15.47:0/2379108842 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcbc0d07d0 con 0x7fbcbc0d3bc0 2024-04-26T23:20:13.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.412+0000 7fbcb9ffb640 1 -- 172.21.15.47:0/2379108842 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7fbcb40019f0 con 0x7fbcbc0d3bc0 2024-04-26T23:20:13.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.413+0000 7fbcb9ffb640 1 -- 172.21.15.47:0/2379108842 >> v1:172.21.15.47:6789/0 conn(0x7fbcbc0d3bc0 legacy=0x7fbcbc0b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:13.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.413+0000 7fbcb9ffb640 1 --2- 172.21.15.47:0/2379108842 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbca80039e0 0x7fbca8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:13.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.413+0000 7fbcb9ffb640 1 --2- 172.21.15.47:0/2379108842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbca8004390 0x7fbca8004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:13.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.413+0000 7fbcb9ffb640 1 --2- 172.21.15.47:0/2379108842 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbcbc162880 0x7fbca80050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:13.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.413+0000 7fbcb9ffb640 1 -- 172.21.15.47:0/2379108842 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fbcbc0d0580 con 0x7fbca8004390 2024-04-26T23:20:13.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.413+0000 7fbcb9ffb640 1 -- 172.21.15.47:0/2379108842 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbca800b9c0 con 0x7fbca80039e0 2024-04-26T23:20:13.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.413+0000 7fbcb9ffb640 1 -- 172.21.15.47:0/2379108842 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fbca800bbc0 con 0x7fbcbc162880 2024-04-26T23:20:13.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.413+0000 7fbcba7fc640 1 --2- 172.21.15.47:0/2379108842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbca8004390 0x7fbca8004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:13.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.413+0000 7fbcbaffd640 1 --2- 172.21.15.47:0/2379108842 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbca80039e0 0x7fbca8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:13.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.413+0000 7fbcbb7fe640 1 --2- 172.21.15.47:0/2379108842 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbcbc162880 0x7fbca80050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:13.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.413+0000 7fbcba7fc640 1 -- 172.21.15.47:0/2379108842 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbcbc162880 msgr2=0x7fbca80050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:13.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.413+0000 7fbcba7fc640 1 --2- 172.21.15.47:0/2379108842 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbcbc162880 0x7fbca80050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:13.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.413+0000 7fbcba7fc640 1 -- 172.21.15.47:0/2379108842 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbca80039e0 msgr2=0x7fbca8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:13.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.413+0000 7fbcba7fc640 1 --2- 172.21.15.47:0/2379108842 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbca80039e0 0x7fbca8003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:13.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.413+0000 7fbcba7fc640 1 -- 172.21.15.47:0/2379108842 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbca8003520 con 0x7fbca8004390 2024-04-26T23:20:13.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.414+0000 7fbcba7fc640 1 --2- 172.21.15.47:0/2379108842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbca8004390 0x7fbca8004860 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fbca4001a30 tx=0x7fbca8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a75ea89cd1a78b9b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:13.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.414+0000 7fbcb9ffb640 1 -- 172.21.15.47:0/2379108842 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca4037070 con 0x7fbca8004390 2024-04-26T23:20:13.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.414+0000 7fbcb9ffb640 1 -- 172.21.15.47:0/2379108842 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbca402ade0 con 0x7fbca8004390 2024-04-26T23:20:13.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.414+0000 7fbcb9ffb640 1 -- 172.21.15.47:0/2379108842 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca4001600 con 0x7fbca8004390 2024-04-26T23:20:13.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.414+0000 7fbcc14f8640 1 -- 172.21.15.47:0/2379108842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbca8004390 msgr2=0x7fbca8004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:13.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.414+0000 7fbcc14f8640 1 --2- 172.21.15.47:0/2379108842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbca8004390 0x7fbca8004860 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fbca4001a30 tx=0x7fbca8005830 comp rx=0 tx=0).stop 2024-04-26T23:20:13.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.414+0000 7fbcbb7fe640 1 -- 172.21.15.47:0/2379108842 reap_dead start 2024-04-26T23:20:13.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.414+0000 7fbcc14f8640 1 -- 172.21.15.47:0/2379108842 shutdown_connections 2024-04-26T23:20:13.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.414+0000 7fbcc14f8640 1 -- 172.21.15.47:0/2379108842 >> 172.21.15.47:0/2379108842 conn(0x7fbcbc0d4f60 msgr2=0x7fbcbc0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:13.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.415+0000 7fbcc14f8640 1 -- 172.21.15.47:0/2379108842 shutdown_connections 2024-04-26T23:20:13.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.415+0000 7fbcc14f8640 1 -- 172.21.15.47:0/2379108842 wait complete. 2024-04-26T23:20:13.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.416+0000 7fbcc14f8640 1 Processor -- start 2024-04-26T23:20:13.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.416+0000 7fbcc14f8640 1 -- start start 2024-04-26T23:20:13.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.417+0000 7fbcc14f8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbca8004390 0x7fbcbc155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:13.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.417+0000 7fbcc14f8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcbc1f8170 0x7fbcbc161370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:13.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.417+0000 7fbcc14f8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbcbc1f85f0 0x7fbcbc1618b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:13.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.417+0000 7fbcc14f8640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fbcbc162c60 con 0x7fbca8004390 2024-04-26T23:20:13.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.417+0000 7fbcc14f8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbcbc0d0d90 con 0x7fbcbc1f8170 2024-04-26T23:20:13.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.417+0000 7fbcc14f8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fbcbc162de0 con 0x7fbcbc1f85f0 2024-04-26T23:20:13.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.417+0000 7fbcbaffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbca8004390 0x7fbcbc155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:13.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.417+0000 7fbcbaffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbca8004390 0x7fbcbc155590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46354/0 (socket says 172.21.15.47:46354) 2024-04-26T23:20:13.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.417+0000 7fbcbaffd640 1 -- 172.21.15.47:0/3929135810 learned_addr learned my addr 172.21.15.47:0/3929135810 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:13.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.417+0000 7fbcba7fc640 1 --2- 172.21.15.47:0/3929135810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcbc1f8170 0x7fbcbc161370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:13.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.417+0000 7fbcbb7fe640 1 --2- 172.21.15.47:0/3929135810 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbcbc1f85f0 0x7fbcbc1618b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:13.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.417+0000 7fbcbaffd640 1 -- 172.21.15.47:0/3929135810 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbcbc1f85f0 msgr2=0x7fbcbc1618b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:13.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.417+0000 7fbcbaffd640 1 --2- 172.21.15.47:0/3929135810 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbcbc1f85f0 0x7fbcbc1618b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:13.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.417+0000 7fbcbaffd640 1 -- 172.21.15.47:0/3929135810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcbc1f8170 msgr2=0x7fbcbc161370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:13.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.417+0000 7fbcbaffd640 1 --2- 172.21.15.47:0/3929135810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcbc1f8170 0x7fbcbc161370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:13.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.417+0000 7fbcbaffd640 1 -- 172.21.15.47:0/3929135810 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcbc1f8b50 con 0x7fbca8004390 2024-04-26T23:20:13.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.417+0000 7fbcbb7fe640 1 --2- 172.21.15.47:0/3929135810 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbcbc1f85f0 0x7fbcbc1618b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:20:13.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.417+0000 7fbcbaffd640 1 --2- 172.21.15.47:0/3929135810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbca8004390 0x7fbcbc155590 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fbcac011940 tx=0x7fbcac011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:13.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.417+0000 7fbcb37fe640 1 -- 172.21.15.47:0/3929135810 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcac00faf0 con 0x7fbca8004390 2024-04-26T23:20:13.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.417+0000 7fbcb37fe640 1 -- 172.21.15.47:0/3929135810 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcac00fc90 con 0x7fbca8004390 2024-04-26T23:20:13.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.417+0000 7fbcc14f8640 1 -- 172.21.15.47:0/3929135810 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcbc1f8e30 con 0x7fbca8004390 2024-04-26T23:20:13.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.417+0000 7fbcb37fe640 1 -- 172.21.15.47:0/3929135810 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcac014440 con 0x7fbca8004390 2024-04-26T23:20:13.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.417+0000 7fbcc14f8640 1 -- 172.21.15.47:0/3929135810 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcbc155ad0 con 0x7fbca8004390 2024-04-26T23:20:13.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.418+0000 7fbcc14f8640 1 -- 172.21.15.47:0/3929135810 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcbc0764d0 con 0x7fbca8004390 2024-04-26T23:20:13.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.419+0000 7fbcb37fe640 1 -- 172.21.15.47:0/3929135810 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbcac012040 con 0x7fbca8004390 2024-04-26T23:20:13.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.419+0000 7fbcb37fe640 1 --2- 172.21.15.47:0/3929135810 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fbc9c06c920 0x7fbc9c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:13.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.419+0000 7fbcb37fe640 1 -- 172.21.15.47:0/3929135810 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7fbcac08bff0 con 0x7fbca8004390 2024-04-26T23:20:13.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.420+0000 7fbcba7fc640 1 --2- 172.21.15.47:0/3929135810 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fbc9c06c920 0x7fbc9c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:13.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.420+0000 7fbcba7fc640 1 --2- 172.21.15.47:0/3929135810 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fbc9c06c920 0x7fbc9c06ede0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbca4001a30 tx=0x7fbca4009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:13.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.424+0000 7fbcb37fe640 1 -- 172.21.15.47:0/3929135810 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbcbc0764d0 con 0x7fbca8004390 2024-04-26T23:20:13.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.680+0000 7fbcc14f8640 1 -- 172.21.15.47:0/3929135810 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]} v 0) -- 0x7fbcbc0694f0 con 0x7fbca8004390 2024-04-26T23:20:13.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.959+0000 7fbcb37fe640 1 -- 172.21.15.47:0/3929135810 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]}]=0 v122) ==== 147+0+0 (secure 0 0 0) 0x7fbcac019030 con 0x7fbca8004390 2024-04-26T23:20:13.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.962+0000 7fbcc14f8640 1 -- 172.21.15.47:0/3929135810 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fbc9c06c920 msgr2=0x7fbc9c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:13.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.962+0000 7fbcc14f8640 1 --2- 172.21.15.47:0/3929135810 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fbc9c06c920 0x7fbc9c06ede0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbca4001a30 tx=0x7fbca4009030 comp rx=0 tx=0).stop 2024-04-26T23:20:13.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.962+0000 7fbcc14f8640 1 -- 172.21.15.47:0/3929135810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbca8004390 msgr2=0x7fbcbc155590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:13.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.962+0000 7fbcc14f8640 1 --2- 172.21.15.47:0/3929135810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbca8004390 0x7fbcbc155590 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fbcac011940 tx=0x7fbcac011e00 comp rx=0 tx=0).stop 2024-04-26T23:20:13.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.962+0000 7fbcc14f8640 1 -- 172.21.15.47:0/3929135810 shutdown_connections 2024-04-26T23:20:13.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.962+0000 7fbcc14f8640 1 --2- 172.21.15.47:0/3929135810 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fbc9c06c920 0x7fbc9c06ede0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:13.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.962+0000 7fbcc14f8640 1 --2- 172.21.15.47:0/3929135810 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbcbc1f85f0 0x7fbcbc1618b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:13.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.962+0000 7fbcc14f8640 1 --2- 172.21.15.47:0/3929135810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcbc1f8170 0x7fbcbc161370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:13.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.962+0000 7fbcc14f8640 1 --2- 172.21.15.47:0/3929135810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbca8004390 0x7fbcbc155590 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:13.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.962+0000 7fbcc14f8640 1 -- 172.21.15.47:0/3929135810 >> 172.21.15.47:0/3929135810 conn(0x7fbcbc0d4f60 msgr2=0x7fbcbc0cb6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:13.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.962+0000 7fbcc14f8640 1 -- 172.21.15.47:0/3929135810 shutdown_connections 2024-04-26T23:20:13.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:13.962+0000 7fbcc14f8640 1 -- 172.21.15.47:0/3929135810 wait complete. 2024-04-26T23:20:13.975 INFO:tasks.cephfs_test_runner:That a client who does not have the capability for setting ... ok 2024-04-26T23:20:13.975 INFO:tasks.cephfs_test_runner:Starting test: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2024-04-26T23:20:13.975 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:20:14.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.077+0000 7f912319d640 1 Processor -- start 2024-04-26T23:20:14.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.077+0000 7f912319d640 1 -- start start 2024-04-26T23:20:14.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.078+0000 7f912319d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f911c0b2340 0x7f911c0cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:14.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.078+0000 7f912319d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f911c0cd930 0x7f911c0d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:14.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.078+0000 7f912319d640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f911c0d10d0 con 0x7f911c162810 2024-04-26T23:20:14.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.078+0000 7f912319d640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f911c0d0750 con 0x7f911c0b2340 2024-04-26T23:20:14.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.078+0000 7f912319d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f911c0d16a0 con 0x7f911c0cd930 2024-04-26T23:20:14.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.078+0000 7f9113fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f911c0b2340 0x7f911c0cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:14.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.078+0000 7f9120f39640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f911c162810 0x7f911c070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35338/0 (socket says 172.21.15.47:35338) 2024-04-26T23:20:14.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.078+0000 7f9120f39640 1 -- 172.21.15.47:0/1883310495 learned_addr learned my addr 172.21.15.47:0/1883310495 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:14.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.078+0000 7f912173a640 1 --2- 172.21.15.47:0/1883310495 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f911c0cd930 0x7f911c0d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:14.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.078+0000 7f9113fff640 1 -- 172.21.15.47:0/1883310495 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f911c0cd930 msgr2=0x7f911c0d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:14.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.078+0000 7f9113fff640 1 --2- 172.21.15.47:0/1883310495 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f911c0cd930 0x7f911c0d1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:14.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.078+0000 7f9113fff640 1 -- 172.21.15.47:0/1883310495 >> v1:172.21.15.47:6789/0 conn(0x7f911c162810 legacy=0x7f911c070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:14.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.078+0000 7f9113fff640 1 -- 172.21.15.47:0/1883310495 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f911c0d1370 con 0x7f911c0b2340 2024-04-26T23:20:14.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.078+0000 7f91137fe640 1 -- 172.21.15.47:0/1883310495 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1732353463 0 0) 0x7f911c0d10d0 con 0x7f911c162810 2024-04-26T23:20:14.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.079+0000 7f9113fff640 1 --2- 172.21.15.47:0/1883310495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f911c0b2340 0x7f911c0cd3f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f911c0d1820 tx=0x7f910c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3a87df5072b7ce2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:14.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.079+0000 7f91137fe640 1 -- 172.21.15.47:0/1883310495 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910c037070 con 0x7f911c0b2340 2024-04-26T23:20:14.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.079+0000 7f91137fe640 1 -- 172.21.15.47:0/1883310495 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f910c032440 con 0x7f911c0b2340 2024-04-26T23:20:14.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.079+0000 7f91137fe640 1 -- 172.21.15.47:0/1883310495 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910c032810 con 0x7f911c0b2340 2024-04-26T23:20:14.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.080+0000 7f912319d640 1 -- 172.21.15.47:0/1883310495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f911c0b2340 msgr2=0x7f911c0cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:14.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.080+0000 7f912319d640 1 --2- 172.21.15.47:0/1883310495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f911c0b2340 0x7f911c0cd3f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f911c0d1820 tx=0x7f910c02a180 comp rx=0 tx=0).stop 2024-04-26T23:20:14.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.080+0000 7f912319d640 1 -- 172.21.15.47:0/1883310495 shutdown_connections 2024-04-26T23:20:14.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.080+0000 7f912319d640 1 --2- 172.21.15.47:0/1883310495 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f911c0cd930 0x7f911c0d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:14.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.080+0000 7f912319d640 1 --2- 172.21.15.47:0/1883310495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f911c0b2340 0x7f911c0cd3f0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:14.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.080+0000 7f912319d640 1 -- 172.21.15.47:0/1883310495 >> 172.21.15.47:0/1883310495 conn(0x7f911c0d2da0 msgr2=0x7f911c0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:14.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.080+0000 7f912319d640 1 -- 172.21.15.47:0/1883310495 shutdown_connections 2024-04-26T23:20:14.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.080+0000 7f912319d640 1 -- 172.21.15.47:0/1883310495 wait complete. 2024-04-26T23:20:14.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.081+0000 7f912319d640 1 Processor -- start 2024-04-26T23:20:14.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.081+0000 7f912319d640 1 -- start start 2024-04-26T23:20:14.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.082+0000 7f912319d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f911c0b2340 0x7f911c14d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:14.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.082+0000 7f912319d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f911c0cd930 0x7f911c0cbb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:14.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.082+0000 7f912319d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f911c162810 0x7f911c0cc080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:14.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.082+0000 7f912319d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f911c0d2070 con 0x7f911c162810 2024-04-26T23:20:14.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.082+0000 7f912319d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f911c0d0470 con 0x7f911c0b2340 2024-04-26T23:20:14.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.082+0000 7f912319d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f911c0d21f0 con 0x7f911c0cd930 2024-04-26T23:20:14.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.082+0000 7f912173a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f911c162810 0x7f911c0cc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:14.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.082+0000 7f912173a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f911c162810 0x7f911c0cc080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46370/0 (socket says 172.21.15.47:46370) 2024-04-26T23:20:14.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.082+0000 7f912173a640 1 -- 172.21.15.47:0/2219816667 learned_addr learned my addr 172.21.15.47:0/2219816667 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:14.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.082+0000 7f912173a640 1 -- 172.21.15.47:0/2219816667 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f911c0cd930 msgr2=0x7f911c0cbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:14.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.082+0000 7f912173a640 1 --2- 172.21.15.47:0/2219816667 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f911c0cd930 0x7f911c0cbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:14.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.082+0000 7f912173a640 1 -- 172.21.15.47:0/2219816667 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f911c0b2340 msgr2=0x7f911c14d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:14.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.082+0000 7f9120f39640 1 --2- 172.21.15.47:0/2219816667 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f911c0b2340 0x7f911c14d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:14.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.082+0000 7f912173a640 1 --2- 172.21.15.47:0/2219816667 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f911c0b2340 0x7f911c14d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:14.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.082+0000 7f912173a640 1 -- 172.21.15.47:0/2219816667 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f911c0c7750 con 0x7f911c162810 2024-04-26T23:20:14.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.082+0000 7f9120f39640 1 --2- 172.21.15.47:0/2219816667 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f911c0b2340 0x7f911c14d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:14.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.082+0000 7f912173a640 1 --2- 172.21.15.47:0/2219816667 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f911c162810 0x7f911c0cc080 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f911400cd70 tx=0x7f911400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:14.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.083+0000 7f91117fa640 1 -- 172.21.15.47:0/2219816667 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9114003b90 con 0x7f911c162810 2024-04-26T23:20:14.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.083+0000 7f91117fa640 1 -- 172.21.15.47:0/2219816667 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9114003d30 con 0x7f911c162810 2024-04-26T23:20:14.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.083+0000 7f912319d640 1 -- 172.21.15.47:0/2219816667 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f911c0c7a30 con 0x7f911c162810 2024-04-26T23:20:14.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.083+0000 7f91117fa640 1 -- 172.21.15.47:0/2219816667 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9114004240 con 0x7f911c162810 2024-04-26T23:20:14.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.083+0000 7f912319d640 1 -- 172.21.15.47:0/2219816667 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f911c0c7ff0 con 0x7f911c162810 2024-04-26T23:20:14.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.084+0000 7f912319d640 1 -- 172.21.15.47:0/2219816667 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f911c0694f0 con 0x7f911c162810 2024-04-26T23:20:14.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.084+0000 7f91117fa640 1 -- 172.21.15.47:0/2219816667 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f91140043e0 con 0x7f911c162810 2024-04-26T23:20:14.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.084+0000 7f91117fa640 1 --2- 172.21.15.47:0/2219816667 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f910806c920 0x7f910806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:14.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.085+0000 7f91117fa640 1 -- 172.21.15.47:0/2219816667 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f911408c4b0 con 0x7f911c162810 2024-04-26T23:20:14.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.085+0000 7f9120f39640 1 --2- 172.21.15.47:0/2219816667 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f910806c920 0x7f910806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:14.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.085+0000 7f9120f39640 1 --2- 172.21.15.47:0/2219816667 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f910806c920 0x7f910806ede0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9118000f60 tx=0x7f9118009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:14.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.089+0000 7f91117fa640 1 -- 172.21.15.47:0/2219816667 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f911405aa50 con 0x7f911c162810 2024-04-26T23:20:14.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.276+0000 7f912319d640 1 -- 172.21.15.47:0/2219816667 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f911c0b2bc0 con 0x7f911c162810 2024-04-26T23:20:14.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.276+0000 7f91117fa640 1 -- 172.21.15.47:0/2219816667 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+13259 (secure 0 0 0) 0x7f911405a350 con 0x7f911c162810 2024-04-26T23:20:14.277 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:20:14.278 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":45,"fsid":"399006d3-4db1-43e6-ba81-a79a276eabfd","created":"2024-04-26T23:18:06.731211+0000","modified":"2024-04-26T23:20:12.954084+0000","last_up_change":"2024-04-26T23:18:17.808736+0000","last_in_change":"2024-04-26T23:18:08.945859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:18:16.937492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ee091567-fea0-4a14-bf4e-0e16c20bb2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6819","nonce":326520779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6821","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6823","nonce":326520779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6829","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6831","nonce":326520779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6825","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6827","nonce":326520779}]},"public_addr":"172.21.15.47:6819/326520779","cluster_addr":"172.21.15.47:6823/326520779","heartbeat_back_addr":"172.21.15.47:6831/326520779","heartbeat_front_addr":"172.21.15.47:6827/326520779","state":["exists","up"]},{"osd":1,"uuid":"c14440a0-bba5-4931-a6db-b736c08e869b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6809","nonce":1912610931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6811","nonce":1912610931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6815","nonce":1912610931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6813","nonce":1912610931}]},"public_addr":"172.21.15.47:6809/1912610931","cluster_addr":"172.21.15.47:6811/1912610931","heartbeat_back_addr":"172.21.15.47:6815/1912610931","heartbeat_front_addr":"172.21.15.47:6813/1912610931","state":["exists","up"]},{"osd":2,"uuid":"3940f8de-c9b2-4cd3-a911-9c301729eb30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6801","nonce":2569869305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6803","nonce":2569869305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6807","nonce":2569869305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6805","nonce":2569869305}]},"public_addr":"172.21.15.47:6801/2569869305","cluster_addr":"172.21.15.47:6803/2569869305","heartbeat_back_addr":"172.21.15.47:6807/2569869305","heartbeat_front_addr":"172.21.15.47:6805/2569869305","state":["exists","up"]},{"osd":3,"uuid":"1286a67e-8958-426a-8c2e-bc85498c619c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6818","nonce":2791411683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6822","nonce":2791411683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6830","nonce":2791411683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6826","nonce":2791411683}]},"public_addr":"172.21.15.47:6818/2791411683","cluster_addr":"172.21.15.47:6822/2791411683","heartbeat_back_addr":"172.21.15.47:6830/2791411683","heartbeat_front_addr":"172.21.15.47:6826/2791411683","state":["exists","up"]},{"osd":4,"uuid":"d8ee2c20-e5b7-46c2-9999-b234070126cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6809","nonce":441313104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6811","nonce":441313104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6815","nonce":441313104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6813","nonce":441313104}]},"public_addr":"172.21.15.203:6809/441313104","cluster_addr":"172.21.15.203:6811/441313104","heartbeat_back_addr":"172.21.15.203:6815/441313104","heartbeat_front_addr":"172.21.15.203:6813/441313104","state":["exists","up"]},{"osd":5,"uuid":"b453a8f0-f324-4840-9ae0-d67bdadb9f8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6801","nonce":56838444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6803","nonce":56838444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6807","nonce":56838444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6805","nonce":56838444}]},"public_addr":"172.21.15.203:6801/56838444","cluster_addr":"172.21.15.203:6803/56838444","heartbeat_back_addr":"172.21.15.203:6807/56838444","heartbeat_front_addr":"172.21.15.203:6805/56838444","state":["exists","up"]},{"osd":6,"uuid":"a6d31951-2d4b-4e8d-a510-6283b01dc158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6825","nonce":2738517080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6827","nonce":2738517080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6831","nonce":2738517080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6829","nonce":2738517080}]},"public_addr":"172.21.15.203:6825/2738517080","cluster_addr":"172.21.15.203:6827/2738517080","heartbeat_back_addr":"172.21.15.203:6831/2738517080","heartbeat_front_addr":"172.21.15.203:6829/2738517080","state":["exists","up"]},{"osd":7,"uuid":"7119b8c0-fe96-402d-9e75-7b509d9d44d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6817","nonce":1315565058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6819","nonce":1315565058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6823","nonce":1315565058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6821","nonce":1315565058}]},"public_addr":"172.21.15.203:6817/1315565058","cluster_addr":"172.21.15.203:6819/1315565058","heartbeat_back_addr":"172.21.15.203:6823/1315565058","heartbeat_front_addr":"172.21.15.203:6821/1315565058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.121921+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.137494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.084814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.111593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.072430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.125066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.097867+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.057061+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:6837/2526477051":"2024-04-27T23:20:08.885245+0000","172.21.15.203:6835/2526477051":"2024-04-27T23:20:08.885245+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:20:14.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.278+0000 7f912319d640 1 -- 172.21.15.47:0/2219816667 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f910806c920 msgr2=0x7f910806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:14.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.279+0000 7f912319d640 1 --2- 172.21.15.47:0/2219816667 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f910806c920 0x7f910806ede0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9118000f60 tx=0x7f9118009040 comp rx=0 tx=0).stop 2024-04-26T23:20:14.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.279+0000 7f912319d640 1 -- 172.21.15.47:0/2219816667 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f911c162810 msgr2=0x7f911c0cc080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:14.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.279+0000 7f912319d640 1 --2- 172.21.15.47:0/2219816667 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f911c162810 0x7f911c0cc080 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f911400cd70 tx=0x7f911400a660 comp rx=0 tx=0).stop 2024-04-26T23:20:14.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.279+0000 7f912319d640 1 -- 172.21.15.47:0/2219816667 shutdown_connections 2024-04-26T23:20:14.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.279+0000 7f912319d640 1 --2- 172.21.15.47:0/2219816667 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f910806c920 0x7f910806ede0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:14.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.279+0000 7f912319d640 1 --2- 172.21.15.47:0/2219816667 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f911c162810 0x7f911c0cc080 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:14.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.279+0000 7f912319d640 1 --2- 172.21.15.47:0/2219816667 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f911c0cd930 0x7f911c0cbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:14.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.279+0000 7f912319d640 1 --2- 172.21.15.47:0/2219816667 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f911c0b2340 0x7f911c14d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:14.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.279+0000 7f912319d640 1 -- 172.21.15.47:0/2219816667 >> 172.21.15.47:0/2219816667 conn(0x7f911c0d2da0 msgr2=0x7f911c0b2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:14.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.279+0000 7f912319d640 1 -- 172.21.15.47:0/2219816667 shutdown_connections 2024-04-26T23:20:14.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.279+0000 7f912319d640 1 -- 172.21.15.47:0/2219816667 wait complete. 2024-04-26T23:20:14.291 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:18:16.937492+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:20:14.291 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:20:14.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.393+0000 7fd37f82a640 1 Processor -- start 2024-04-26T23:20:14.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.394+0000 7fd37f82a640 1 -- start start 2024-04-26T23:20:14.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.394+0000 7fd37f82a640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd3780d1a00 0x7fd3780d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:14.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.394+0000 7fd37f82a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd3780b40d0 0x7fd3780b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:14.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.394+0000 7fd37f82a640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3780d0a10 con 0x7fd3780d2320 2024-04-26T23:20:14.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.394+0000 7fd37f82a640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fd3780d0cf0 con 0x7fd3780d1a00 2024-04-26T23:20:14.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.394+0000 7fd37f82a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd3780d0fd0 con 0x7fd3780b40d0 2024-04-26T23:20:14.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.394+0000 7fd37d5c6640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd3780d1a00 0x7fd3780d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:14.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.394+0000 7fd37cdc5640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd3780d2320 0x7fd3781625f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35340/0 (socket says 172.21.15.47:35340) 2024-04-26T23:20:14.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.394+0000 7fd37d5c6640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd3780d1a00 0x7fd3780d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46374/0 (socket says 172.21.15.47:46374) 2024-04-26T23:20:14.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.395+0000 7fd37cdc5640 1 -- 172.21.15.47:0/3177834836 learned_addr learned my addr 172.21.15.47:0/3177834836 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:14.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.395+0000 7fd37ddc7640 1 --2- 172.21.15.47:0/3177834836 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd3780b40d0 0x7fd3780b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:14.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.395+0000 7fd37d5c6640 1 -- 172.21.15.47:0/3177834836 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd3780b40d0 msgr2=0x7fd3780b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:14.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.395+0000 7fd37d5c6640 1 --2- 172.21.15.47:0/3177834836 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd3780b40d0 0x7fd3780b4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:14.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.395+0000 7fd37d5c6640 1 -- 172.21.15.47:0/3177834836 >> v1:172.21.15.47:6789/0 conn(0x7fd3780d2320 legacy=0x7fd3781625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:14.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.395+0000 7fd377fff640 1 -- 172.21.15.47:0/3177834836 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 953407393 0 0) 0x7fd3780d0a10 con 0x7fd3780d2320 2024-04-26T23:20:14.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.395+0000 7fd37d5c6640 1 -- 172.21.15.47:0/3177834836 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3780d40a0 con 0x7fd3780d1a00 2024-04-26T23:20:14.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.396+0000 7fd37d5c6640 1 --2- 172.21.15.47:0/3177834836 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3780d1a00 0x7fd3780d1de0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd3700097a0 tx=0x7fd37002a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e78bf4a60ca969a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:14.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.396+0000 7fd377fff640 1 -- 172.21.15.47:0/3177834836 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd370037070 con 0x7fd3780d1a00 2024-04-26T23:20:14.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.396+0000 7fd377fff640 1 -- 172.21.15.47:0/3177834836 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd37002adb0 con 0x7fd3780d1a00 2024-04-26T23:20:14.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.396+0000 7fd377fff640 1 -- 172.21.15.47:0/3177834836 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd370032660 con 0x7fd3780d1a00 2024-04-26T23:20:14.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.396+0000 7fd37f82a640 1 -- 172.21.15.47:0/3177834836 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3780d1a00 msgr2=0x7fd3780d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:14.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.396+0000 7fd37f82a640 1 --2- 172.21.15.47:0/3177834836 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3780d1a00 0x7fd3780d1de0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd3700097a0 tx=0x7fd37002a060 comp rx=0 tx=0).stop 2024-04-26T23:20:14.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.396+0000 7fd37f82a640 1 -- 172.21.15.47:0/3177834836 shutdown_connections 2024-04-26T23:20:14.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.396+0000 7fd37f82a640 1 --2- 172.21.15.47:0/3177834836 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd3780b40d0 0x7fd3780b4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:14.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.396+0000 7fd37f82a640 1 --2- 172.21.15.47:0/3177834836 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3780d1a00 0x7fd3780d1de0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:14.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.396+0000 7fd37f82a640 1 -- 172.21.15.47:0/3177834836 >> 172.21.15.47:0/3177834836 conn(0x7fd3780d4fd0 msgr2=0x7fd3780d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:14.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.396+0000 7fd37f82a640 1 -- 172.21.15.47:0/3177834836 shutdown_connections 2024-04-26T23:20:14.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.396+0000 7fd37f82a640 1 -- 172.21.15.47:0/3177834836 wait complete. 2024-04-26T23:20:14.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.397+0000 7fd37f82a640 1 Processor -- start 2024-04-26T23:20:14.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.398+0000 7fd37f82a640 1 -- start start 2024-04-26T23:20:14.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.398+0000 7fd37f82a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd3780b40d0 0x7fd3780cc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:14.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.398+0000 7fd37f82a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3780d1a00 0x7fd3780ceb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:14.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.398+0000 7fd37f82a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd3780d2320 0x7fd3780b3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:14.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.398+0000 7fd37f82a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd3780b5040 con 0x7fd3780d1a00 2024-04-26T23:20:14.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.398+0000 7fd37f82a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd378162f50 con 0x7fd3780b40d0 2024-04-26T23:20:14.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.398+0000 7fd37f82a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd3780b51c0 con 0x7fd3780d2320 2024-04-26T23:20:14.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.398+0000 7fd37cdc5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3780d1a00 0x7fd3780ceb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:14.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.398+0000 7fd37cdc5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3780d1a00 0x7fd3780ceb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46376/0 (socket says 172.21.15.47:46376) 2024-04-26T23:20:14.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.398+0000 7fd37cdc5640 1 -- 172.21.15.47:0/249020683 learned_addr learned my addr 172.21.15.47:0/249020683 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:14.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.399+0000 7fd37d5c6640 1 --2- 172.21.15.47:0/249020683 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd3780b40d0 0x7fd3780cc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:14.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.399+0000 7fd37ddc7640 1 --2- 172.21.15.47:0/249020683 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd3780d2320 0x7fd3780b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:14.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.399+0000 7fd37cdc5640 1 -- 172.21.15.47:0/249020683 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd3780d2320 msgr2=0x7fd3780b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:14.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.399+0000 7fd37cdc5640 1 --2- 172.21.15.47:0/249020683 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd3780d2320 0x7fd3780b3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:14.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.399+0000 7fd37cdc5640 1 -- 172.21.15.47:0/249020683 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd3780b40d0 msgr2=0x7fd3780cc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:14.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.399+0000 7fd37cdc5640 1 --2- 172.21.15.47:0/249020683 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd3780b40d0 0x7fd3780cc0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:14.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.399+0000 7fd37cdc5640 1 -- 172.21.15.47:0/249020683 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3781f95d0 con 0x7fd3780d1a00 2024-04-26T23:20:14.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.399+0000 7fd37cdc5640 1 --2- 172.21.15.47:0/249020683 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3780d1a00 0x7fd3780ceb30 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fd36c00c430 tx=0x7fd36c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:14.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.399+0000 7fd375ffb640 1 -- 172.21.15.47:0/249020683 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd36c0074e0 con 0x7fd3780d1a00 2024-04-26T23:20:14.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.399+0000 7fd375ffb640 1 -- 172.21.15.47:0/249020683 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd36c0047e0 con 0x7fd3780d1a00 2024-04-26T23:20:14.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.399+0000 7fd37f82a640 1 -- 172.21.15.47:0/249020683 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3780ca470 con 0x7fd3780d1a00 2024-04-26T23:20:14.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.399+0000 7fd375ffb640 1 -- 172.21.15.47:0/249020683 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd36c004bd0 con 0x7fd3780d1a00 2024-04-26T23:20:14.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.399+0000 7fd37f82a640 1 -- 172.21.15.47:0/249020683 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3780caa30 con 0x7fd3780d1a00 2024-04-26T23:20:14.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.400+0000 7fd37f82a640 1 -- 172.21.15.47:0/249020683 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3780cac30 con 0x7fd3780d1a00 2024-04-26T23:20:14.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.401+0000 7fd375ffb640 1 -- 172.21.15.47:0/249020683 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd36c011070 con 0x7fd3780d1a00 2024-04-26T23:20:14.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.401+0000 7fd375ffb640 1 --2- 172.21.15.47:0/249020683 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd36406c8f0 0x7fd36406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:14.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.401+0000 7fd375ffb640 1 -- 172.21.15.47:0/249020683 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7fd36c08b410 con 0x7fd3780d1a00 2024-04-26T23:20:14.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.402+0000 7fd37d5c6640 1 --2- 172.21.15.47:0/249020683 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd36406c8f0 0x7fd36406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:14.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.402+0000 7fd37d5c6640 1 --2- 172.21.15.47:0/249020683 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd36406c8f0 0x7fd36406edb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd370034000 tx=0x7fd37002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:14.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.406+0000 7fd375ffb640 1 -- 172.21.15.47:0/249020683 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd3780cac30 con 0x7fd3780d1a00 2024-04-26T23:20:14.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.588+0000 7fd37f82a640 1 -- 172.21.15.47:0/249020683 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd3780694f0 con 0x7fd3780d1a00 2024-04-26T23:20:14.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.589+0000 7fd375ffb640 1 -- 172.21.15.47:0/249020683 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) ==== 93+0+10 (secure 0 0 0) 0x7fd36c0593f0 con 0x7fd3780d1a00 2024-04-26T23:20:14.590 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-26T23:20:14.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.591+0000 7fd37f82a640 1 -- 172.21.15.47:0/249020683 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd36406c8f0 msgr2=0x7fd36406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:14.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.591+0000 7fd37f82a640 1 --2- 172.21.15.47:0/249020683 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd36406c8f0 0x7fd36406edb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd370034000 tx=0x7fd37002e000 comp rx=0 tx=0).stop 2024-04-26T23:20:14.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.591+0000 7fd37f82a640 1 -- 172.21.15.47:0/249020683 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3780d1a00 msgr2=0x7fd3780ceb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:14.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.591+0000 7fd37f82a640 1 --2- 172.21.15.47:0/249020683 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3780d1a00 0x7fd3780ceb30 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fd36c00c430 tx=0x7fd36c00c8f0 comp rx=0 tx=0).stop 2024-04-26T23:20:14.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.591+0000 7fd37f82a640 1 -- 172.21.15.47:0/249020683 shutdown_connections 2024-04-26T23:20:14.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.591+0000 7fd37f82a640 1 --2- 172.21.15.47:0/249020683 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd36406c8f0 0x7fd36406edb0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:14.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.591+0000 7fd37f82a640 1 --2- 172.21.15.47:0/249020683 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd3780d2320 0x7fd3780b3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:14.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.591+0000 7fd37f82a640 1 --2- 172.21.15.47:0/249020683 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3780d1a00 0x7fd3780ceb30 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:14.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.591+0000 7fd37f82a640 1 --2- 172.21.15.47:0/249020683 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd3780b40d0 0x7fd3780cc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:14.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.591+0000 7fd37f82a640 1 -- 172.21.15.47:0/249020683 >> 172.21.15.47:0/249020683 conn(0x7fd3780d4fd0 msgr2=0x7fd37806dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:14.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.591+0000 7fd37f82a640 1 -- 172.21.15.47:0/249020683 shutdown_connections 2024-04-26T23:20:14.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.591+0000 7fd37f82a640 1 -- 172.21.15.47:0/249020683 wait complete. 2024-04-26T23:20:14.603 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm' 2024-04-26T23:20:14.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.703+0000 7f25c2c47640 1 Processor -- start 2024-04-26T23:20:14.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.703+0000 7f25c2c47640 1 -- start start 2024-04-26T23:20:14.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.703+0000 7f25c2c47640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f25bc0b2340 0x7f25bc0cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:14.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.703+0000 7f25c2c47640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25bc0d10d0 con 0x7f25bc162810 2024-04-26T23:20:14.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.703+0000 7f25c2c47640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25bc0d1370 con 0x7f25bc0cd930 2024-04-26T23:20:14.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.703+0000 7f25c2c47640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f25bc0d0750 con 0x7f25bc0b2340 2024-04-26T23:20:14.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.704+0000 7f25c09e3640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f25bc162810 0x7f25bc070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35354/0 (socket says 172.21.15.47:35354) 2024-04-26T23:20:14.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.704+0000 7f25c09e3640 1 -- 172.21.15.47:0/3762785976 learned_addr learned my addr 172.21.15.47:0/3762785976 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:14.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.704+0000 7f25bbfff640 1 --2- 172.21.15.47:0/3762785976 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f25bc0b2340 0x7f25bc0cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:14.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.704+0000 7f25bb7fe640 1 -- 172.21.15.47:0/3762785976 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1786148687 0 0) 0x7f25bc0d10d0 con 0x7f25bc162810 2024-04-26T23:20:14.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.704+0000 7f25bb7fe640 1 -- 172.21.15.47:0/3762785976 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f25a4003520 con 0x7f25bc162810 2024-04-26T23:20:14.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.704+0000 7f25bb7fe640 1 -- 172.21.15.47:0/3762785976 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3481433942 0 0) 0x7f25a4003520 con 0x7f25bc162810 2024-04-26T23:20:14.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.704+0000 7f25bb7fe640 1 -- 172.21.15.47:0/3762785976 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f25bc0b2340 msgr2=0x7f25bc0cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:14.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.704+0000 7f25bb7fe640 1 --2- 172.21.15.47:0/3762785976 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f25bc0b2340 0x7f25bc0cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:14.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.704+0000 7f25bb7fe640 1 -- 172.21.15.47:0/3762785976 >> v1:172.21.15.203:6789/0 conn(0x7f25bc0cd930 legacy=0x7f25bc0d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:14.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.704+0000 7f25bb7fe640 1 -- 172.21.15.47:0/3762785976 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25bc0d01b0 con 0x7f25bc162810 2024-04-26T23:20:14.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.704+0000 7f25bb7fe640 1 -- 172.21.15.47:0/3762785976 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7f25ac001670 con 0x7f25bc162810 2024-04-26T23:20:14.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.704+0000 7f25bb7fe640 1 -- 172.21.15.47:0/3762785976 >> v1:172.21.15.47:6789/0 conn(0x7f25bc162810 legacy=0x7f25bc070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:14.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.704+0000 7f25bb7fe640 1 --2- 172.21.15.47:0/3762785976 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25a40039e0 0x7f25a4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:14.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.704+0000 7f25bb7fe640 1 --2- 172.21.15.47:0/3762785976 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25a40043b0 0x7f25a4004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:14.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.704+0000 7f25bb7fe640 1 --2- 172.21.15.47:0/3762785976 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f25bc0b2340 0x7f25a40050e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:14.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.704+0000 7f25bb7fe640 1 -- 172.21.15.47:0/3762785976 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f25bc0d0750 con 0x7f25a40043b0 2024-04-26T23:20:14.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.704+0000 7f25bb7fe640 1 -- 172.21.15.47:0/3762785976 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f25a400b9f0 con 0x7f25a40039e0 2024-04-26T23:20:14.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.704+0000 7f25bb7fe640 1 -- 172.21.15.47:0/3762785976 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f25a400bbf0 con 0x7f25bc0b2340 2024-04-26T23:20:14.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.704+0000 7f25bbfff640 1 --2- 172.21.15.47:0/3762785976 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25a40043b0 0x7f25a4004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:14.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.704+0000 7f25c09e3640 1 --2- 172.21.15.47:0/3762785976 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25a40039e0 0x7f25a4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:14.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.704+0000 7f25c11e4640 1 --2- 172.21.15.47:0/3762785976 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f25bc0b2340 0x7f25a40050e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:14.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.705+0000 7f25bbfff640 1 -- 172.21.15.47:0/3762785976 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f25bc0b2340 msgr2=0x7f25a40050e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:14.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.705+0000 7f25bbfff640 1 --2- 172.21.15.47:0/3762785976 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f25bc0b2340 0x7f25a40050e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:14.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.705+0000 7f25bbfff640 1 -- 172.21.15.47:0/3762785976 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25a40039e0 msgr2=0x7f25a4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:14.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.705+0000 7f25bbfff640 1 --2- 172.21.15.47:0/3762785976 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25a40039e0 0x7f25a4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:14.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.705+0000 7f25bbfff640 1 -- 172.21.15.47:0/3762785976 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25a4003520 con 0x7f25a40043b0 2024-04-26T23:20:14.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.705+0000 7f25c11e4640 1 --2- 172.21.15.47:0/3762785976 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f25bc0b2340 0x7f25a40050e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:20:14.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.705+0000 7f25bbfff640 1 --2- 172.21.15.47:0/3762785976 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25a40043b0 0x7f25a4004880 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f25b0009ac0 tx=0x7f25a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e7ced53bd910465 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:14.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.705+0000 7f25bb7fe640 1 -- 172.21.15.47:0/3762785976 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b0036070 con 0x7f25a40043b0 2024-04-26T23:20:14.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.705+0000 7f25bb7fe640 1 -- 172.21.15.47:0/3762785976 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25b0031400 con 0x7f25a40043b0 2024-04-26T23:20:14.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.705+0000 7f25bb7fe640 1 -- 172.21.15.47:0/3762785976 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b0009690 con 0x7f25a40043b0 2024-04-26T23:20:14.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.706+0000 7f25c2c47640 1 -- 172.21.15.47:0/3762785976 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25a40043b0 msgr2=0x7f25a4004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:14.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.706+0000 7f25c2c47640 1 --2- 172.21.15.47:0/3762785976 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25a40043b0 0x7f25a4004880 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f25b0009ac0 tx=0x7f25a4005830 comp rx=0 tx=0).stop 2024-04-26T23:20:14.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.706+0000 7f25c11e4640 1 -- 172.21.15.47:0/3762785976 reap_dead start 2024-04-26T23:20:14.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.706+0000 7f25c2c47640 1 -- 172.21.15.47:0/3762785976 shutdown_connections 2024-04-26T23:20:14.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.706+0000 7f25c2c47640 1 -- 172.21.15.47:0/3762785976 >> 172.21.15.47:0/3762785976 conn(0x7f25bc0d2da0 msgr2=0x7f25bc0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:14.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.706+0000 7f25c2c47640 1 -- 172.21.15.47:0/3762785976 shutdown_connections 2024-04-26T23:20:14.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.706+0000 7f25c2c47640 1 -- 172.21.15.47:0/3762785976 wait complete. 2024-04-26T23:20:14.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.707+0000 7f25c2c47640 1 Processor -- start 2024-04-26T23:20:14.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.707+0000 7f25c2c47640 1 -- start start 2024-04-26T23:20:14.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.708+0000 7f25c2c47640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25a40043b0 0x7f25bc152230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:14.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.708+0000 7f25c2c47640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f25bc0c83d0 0x7f25bc0c8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:14.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.708+0000 7f25c2c47640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25bc15d320 0x7f25bc15d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:14.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.708+0000 7f25c2c47640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f25bc0d2240 con 0x7f25bc15d320 2024-04-26T23:20:14.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.708+0000 7f25c2c47640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f25bc0d16a0 con 0x7f25a40043b0 2024-04-26T23:20:14.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.708+0000 7f25c2c47640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f25bc0d23c0 con 0x7f25bc0c83d0 2024-04-26T23:20:14.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.708+0000 7f25c11e4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25bc15d320 0x7f25bc15d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:14.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.708+0000 7f25c11e4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25bc15d320 0x7f25bc15d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46406/0 (socket says 172.21.15.47:46406) 2024-04-26T23:20:14.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.708+0000 7f25c11e4640 1 -- 172.21.15.47:0/1332899787 learned_addr learned my addr 172.21.15.47:0/1332899787 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:14.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.708+0000 7f25c09e3640 1 --2- 172.21.15.47:0/1332899787 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25a40043b0 0x7f25bc152230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:14.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.709+0000 7f25bbfff640 1 --2- 172.21.15.47:0/1332899787 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f25bc0c83d0 0x7f25bc0c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:14.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.709+0000 7f25c11e4640 1 -- 172.21.15.47:0/1332899787 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f25bc0c83d0 msgr2=0x7f25bc0c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:14.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.709+0000 7f25c11e4640 1 --2- 172.21.15.47:0/1332899787 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f25bc0c83d0 0x7f25bc0c8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:14.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.709+0000 7f25c11e4640 1 -- 172.21.15.47:0/1332899787 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25a40043b0 msgr2=0x7f25bc152230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:14.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.709+0000 7f25c11e4640 1 --2- 172.21.15.47:0/1332899787 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25a40043b0 0x7f25bc152230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:14.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.709+0000 7f25c11e4640 1 -- 172.21.15.47:0/1332899787 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25bc152770 con 0x7f25bc15d320 2024-04-26T23:20:14.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.709+0000 7f25c09e3640 1 --2- 172.21.15.47:0/1332899787 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25a40043b0 0x7f25bc152230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:14.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.709+0000 7f25c11e4640 1 --2- 172.21.15.47:0/1332899787 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25bc15d320 0x7f25bc15d7a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f25bc0cdd10 tx=0x7f25b400ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:14.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.709+0000 7f25b97fa640 1 -- 172.21.15.47:0/1332899787 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b4019070 con 0x7f25bc15d320 2024-04-26T23:20:14.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.709+0000 7f25b97fa640 1 -- 172.21.15.47:0/1332899787 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25b4006b50 con 0x7f25bc15d320 2024-04-26T23:20:14.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.709+0000 7f25c2c47640 1 -- 172.21.15.47:0/1332899787 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25bc1f51e0 con 0x7f25bc15d320 2024-04-26T23:20:14.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.709+0000 7f25b97fa640 1 -- 172.21.15.47:0/1332899787 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b4003970 con 0x7f25bc15d320 2024-04-26T23:20:14.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.709+0000 7f25c2c47640 1 -- 172.21.15.47:0/1332899787 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25bc1f5780 con 0x7f25bc15d320 2024-04-26T23:20:14.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.710+0000 7f25c2c47640 1 -- 172.21.15.47:0/1332899787 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25bc15dce0 con 0x7f25bc15d320 2024-04-26T23:20:14.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.710+0000 7f25b97fa640 1 -- 172.21.15.47:0/1332899787 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f25b401c070 con 0x7f25bc15d320 2024-04-26T23:20:14.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.711+0000 7f25b97fa640 1 --2- 172.21.15.47:0/1332899787 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f25a806c8f0 0x7f25a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:14.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.711+0000 7f25b97fa640 1 -- 172.21.15.47:0/1332899787 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f25b408ae10 con 0x7f25bc15d320 2024-04-26T23:20:14.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.711+0000 7f25c09e3640 1 --2- 172.21.15.47:0/1332899787 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f25a806c8f0 0x7f25a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:14.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.712+0000 7f25c09e3640 1 --2- 172.21.15.47:0/1332899787 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f25a806c8f0 0x7f25a806edb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f25ac000c00 tx=0x7f25ac009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:14.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.715+0000 7f25b97fa640 1 -- 172.21.15.47:0/1332899787 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f25bc152770 con 0x7f25bc15d320 2024-04-26T23:20:14.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:14.972+0000 7f25c2c47640 1 -- 172.21.15.47:0/1332899787 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]} v 0) -- 0x7f25bc0694f0 con 0x7f25bc15d320 2024-04-26T23:20:15.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:15.965+0000 7f25b97fa640 1 -- 172.21.15.47:0/1332899787 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]}]=0 v124) ==== 137+0+0 (secure 0 0 0) 0x7f25b4059320 con 0x7f25bc15d320 2024-04-26T23:20:15.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:15.968+0000 7f25c2c47640 1 -- 172.21.15.47:0/1332899787 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f25a806c8f0 msgr2=0x7f25a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:15.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:15.968+0000 7f25c2c47640 1 --2- 172.21.15.47:0/1332899787 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f25a806c8f0 0x7f25a806edb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f25ac000c00 tx=0x7f25ac009040 comp rx=0 tx=0).stop 2024-04-26T23:20:15.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:15.968+0000 7f25c2c47640 1 -- 172.21.15.47:0/1332899787 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25bc15d320 msgr2=0x7f25bc15d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:15.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:15.968+0000 7f25c2c47640 1 --2- 172.21.15.47:0/1332899787 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25bc15d320 0x7f25bc15d7a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f25bc0cdd10 tx=0x7f25b400ebe0 comp rx=0 tx=0).stop 2024-04-26T23:20:15.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:15.968+0000 7f25c2c47640 1 -- 172.21.15.47:0/1332899787 shutdown_connections 2024-04-26T23:20:15.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:15.968+0000 7f25c2c47640 1 --2- 172.21.15.47:0/1332899787 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f25a806c8f0 0x7f25a806edb0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:15.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:15.968+0000 7f25c2c47640 1 --2- 172.21.15.47:0/1332899787 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25bc15d320 0x7f25bc15d7a0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:15.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:15.968+0000 7f25c2c47640 1 --2- 172.21.15.47:0/1332899787 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f25bc0c83d0 0x7f25bc0c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:15.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:15.968+0000 7f25c2c47640 1 --2- 172.21.15.47:0/1332899787 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25a40043b0 0x7f25bc152230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:15.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:15.968+0000 7f25c2c47640 1 -- 172.21.15.47:0/1332899787 >> 172.21.15.47:0/1332899787 conn(0x7f25bc0d2da0 msgr2=0x7f25bc0b38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:15.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:15.968+0000 7f25c2c47640 1 -- 172.21.15.47:0/1332899787 shutdown_connections 2024-04-26T23:20:15.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:15.968+0000 7f25c2c47640 1 -- 172.21.15.47:0/1332899787 wait complete. 2024-04-26T23:20:15.983 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T23:20:16.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.094+0000 7f145b12b640 1 Processor -- start 2024-04-26T23:20:16.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.095+0000 7f145b12b640 1 -- start start 2024-04-26T23:20:16.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.095+0000 7f145b12b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f14540b40d0 0x7f14541625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:16.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.095+0000 7f145b12b640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14540d0a10 con 0x7f1454162b30 2024-04-26T23:20:16.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.095+0000 7f145b12b640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14540d0cf0 con 0x7f14540d1a00 2024-04-26T23:20:16.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.095+0000 7f145b12b640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f14540d0fd0 con 0x7f14540b40d0 2024-04-26T23:20:16.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.095+0000 7f1453fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f14540b40d0 0x7f14541625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:16.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.095+0000 7f14596c8640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1454162b30 0x7f14540b4b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35370/0 (socket says 172.21.15.47:35370) 2024-04-26T23:20:16.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.096+0000 7f14596c8640 1 -- 172.21.15.47:0/1689253958 learned_addr learned my addr 172.21.15.47:0/1689253958 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:16.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.096+0000 7f1453fff640 1 -- 172.21.15.47:0/1689253958 >> v1:172.21.15.203:6789/0 conn(0x7f14540d1a00 legacy=0x7f14540d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:16.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.096+0000 7f1453fff640 1 -- 172.21.15.47:0/1689253958 >> v1:172.21.15.47:6789/0 conn(0x7f1454162b30 legacy=0x7f14540b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:16.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.096+0000 7f14537fe640 1 -- 172.21.15.47:0/1689253958 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1312743327 0 0) 0x7f14540d0a10 con 0x7f1454162b30 2024-04-26T23:20:16.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.096+0000 7f1453fff640 1 -- 172.21.15.47:0/1689253958 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1454162f80 con 0x7f14540b40d0 2024-04-26T23:20:16.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.097+0000 7f1453fff640 1 --2- 172.21.15.47:0/1689253958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14540b40d0 0x7f14541625f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f14440097a0 tx=0x7f144402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1caedb29c4b23737 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:16.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.097+0000 7f14537fe640 1 -- 172.21.15.47:0/1689253958 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1444037070 con 0x7f14540b40d0 2024-04-26T23:20:16.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.097+0000 7f14537fe640 1 -- 172.21.15.47:0/1689253958 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1444032440 con 0x7f14540b40d0 2024-04-26T23:20:16.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.097+0000 7f14537fe640 1 -- 172.21.15.47:0/1689253958 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1444032810 con 0x7f14540b40d0 2024-04-26T23:20:16.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.097+0000 7f145b12b640 1 -- 172.21.15.47:0/1689253958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14540b40d0 msgr2=0x7f14541625f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:16.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.097+0000 7f145b12b640 1 --2- 172.21.15.47:0/1689253958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14540b40d0 0x7f14541625f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f14440097a0 tx=0x7f144402a1a0 comp rx=0 tx=0).stop 2024-04-26T23:20:16.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.097+0000 7f145b12b640 1 -- 172.21.15.47:0/1689253958 shutdown_connections 2024-04-26T23:20:16.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.097+0000 7f145b12b640 1 --2- 172.21.15.47:0/1689253958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14540b40d0 0x7f14541625f0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:16.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.097+0000 7f145b12b640 1 -- 172.21.15.47:0/1689253958 >> 172.21.15.47:0/1689253958 conn(0x7f14540d4fd0 msgr2=0x7f14540d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:16.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.097+0000 7f145b12b640 1 -- 172.21.15.47:0/1689253958 shutdown_connections 2024-04-26T23:20:16.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.097+0000 7f145b12b640 1 -- 172.21.15.47:0/1689253958 wait complete. 2024-04-26T23:20:16.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.098+0000 7f145b12b640 1 Processor -- start 2024-04-26T23:20:16.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.099+0000 7f145b12b640 1 -- start start 2024-04-26T23:20:16.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.099+0000 7f145b12b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14540b40d0 0x7f14540cea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:16.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.099+0000 7f145b12b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14540d1a00 0x7f145414f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:16.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.099+0000 7f145b12b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1454162b30 0x7f14540b9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:16.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.099+0000 7f145b12b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f14540d2670 con 0x7f14540b40d0 2024-04-26T23:20:16.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.099+0000 7f145b12b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f14540d24f0 con 0x7f1454162b30 2024-04-26T23:20:16.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.099+0000 7f145b12b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f14540b5210 con 0x7f14540d1a00 2024-04-26T23:20:16.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.100+0000 7f1458ec7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14540b40d0 0x7f14540cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:16.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.100+0000 7f1458ec7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14540b40d0 0x7f14540cea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46428/0 (socket says 172.21.15.47:46428) 2024-04-26T23:20:16.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.100+0000 7f1458ec7640 1 -- 172.21.15.47:0/914179629 learned_addr learned my addr 172.21.15.47:0/914179629 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:16.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.100+0000 7f14596c8640 1 --2- 172.21.15.47:0/914179629 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1454162b30 0x7f14540b9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:16.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.100+0000 7f1453fff640 1 --2- 172.21.15.47:0/914179629 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14540d1a00 0x7f145414f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:16.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.100+0000 7f1458ec7640 1 -- 172.21.15.47:0/914179629 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14540d1a00 msgr2=0x7f145414f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:16.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.100+0000 7f1458ec7640 1 --2- 172.21.15.47:0/914179629 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14540d1a00 0x7f145414f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:16.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.100+0000 7f1458ec7640 1 -- 172.21.15.47:0/914179629 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1454162b30 msgr2=0x7f14540b9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:16.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.100+0000 7f1458ec7640 1 --2- 172.21.15.47:0/914179629 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1454162b30 0x7f14540b9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:16.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.100+0000 7f1458ec7640 1 -- 172.21.15.47:0/914179629 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14540cef70 con 0x7f14540b40d0 2024-04-26T23:20:16.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.100+0000 7f1458ec7640 1 --2- 172.21.15.47:0/914179629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14540b40d0 0x7f14540cea30 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f144c00c870 tx=0x7f144c00cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:16.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.100+0000 7f14517fa640 1 -- 172.21.15.47:0/914179629 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144c003b70 con 0x7f14540b40d0 2024-04-26T23:20:16.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.100+0000 7f14517fa640 1 -- 172.21.15.47:0/914179629 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f144c003d30 con 0x7f14540b40d0 2024-04-26T23:20:16.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.100+0000 7f145b12b640 1 -- 172.21.15.47:0/914179629 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14540b8c40 con 0x7f14540b40d0 2024-04-26T23:20:16.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.100+0000 7f14517fa640 1 -- 172.21.15.47:0/914179629 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144c00aa90 con 0x7f14540b40d0 2024-04-26T23:20:16.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.100+0000 7f145b12b640 1 -- 172.21.15.47:0/914179629 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14540b9200 con 0x7f14540b40d0 2024-04-26T23:20:16.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.101+0000 7f145b12b640 1 -- 172.21.15.47:0/914179629 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14540694f0 con 0x7f14540b40d0 2024-04-26T23:20:16.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.102+0000 7f14517fa640 1 -- 172.21.15.47:0/914179629 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f144c00ac30 con 0x7f14540b40d0 2024-04-26T23:20:16.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.102+0000 7f14517fa640 1 --2- 172.21.15.47:0/914179629 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f143406c8f0 0x7f143406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:16.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.102+0000 7f14517fa640 1 -- 172.21.15.47:0/914179629 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f144c08b910 con 0x7f14540b40d0 2024-04-26T23:20:16.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.102+0000 7f1453fff640 1 --2- 172.21.15.47:0/914179629 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f143406c8f0 0x7f143406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:16.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.103+0000 7f1453fff640 1 --2- 172.21.15.47:0/914179629 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f143406c8f0 0x7f143406edb0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1444034000 tx=0x7f144402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:16.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.106+0000 7f14517fa640 1 -- 172.21.15.47:0/914179629 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f144c059f40 con 0x7f14540b40d0 2024-04-26T23:20:16.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.291+0000 7f145b12b640 1 -- 172.21.15.47:0/914179629 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f14540b6780 con 0x7f14540b40d0 2024-04-26T23:20:16.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.294+0000 7f14517fa640 1 -- 172.21.15.47:0/914179629 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7f144c059840 con 0x7f14540b40d0 2024-04-26T23:20:16.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.296+0000 7f145b12b640 1 -- 172.21.15.47:0/914179629 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f143406c8f0 msgr2=0x7f143406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:16.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.296+0000 7f145b12b640 1 --2- 172.21.15.47:0/914179629 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f143406c8f0 0x7f143406edb0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1444034000 tx=0x7f144402e000 comp rx=0 tx=0).stop 2024-04-26T23:20:16.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.296+0000 7f145b12b640 1 -- 172.21.15.47:0/914179629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14540b40d0 msgr2=0x7f14540cea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:16.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.296+0000 7f145b12b640 1 --2- 172.21.15.47:0/914179629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14540b40d0 0x7f14540cea30 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f144c00c870 tx=0x7f144c00cd30 comp rx=0 tx=0).stop 2024-04-26T23:20:16.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.296+0000 7f145b12b640 1 -- 172.21.15.47:0/914179629 shutdown_connections 2024-04-26T23:20:16.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.296+0000 7f145b12b640 1 --2- 172.21.15.47:0/914179629 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f143406c8f0 0x7f143406edb0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:16.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.296+0000 7f145b12b640 1 --2- 172.21.15.47:0/914179629 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1454162b30 0x7f14540b9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:16.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.296+0000 7f145b12b640 1 --2- 172.21.15.47:0/914179629 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14540d1a00 0x7f145414f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:16.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.296+0000 7f145b12b640 1 --2- 172.21.15.47:0/914179629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14540b40d0 0x7f14540cea30 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:16.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.296+0000 7f145b12b640 1 -- 172.21.15.47:0/914179629 >> 172.21.15.47:0/914179629 conn(0x7f14540d4fd0 msgr2=0x7f14540b4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:16.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.296+0000 7f145b12b640 1 -- 172.21.15.47:0/914179629 shutdown_connections 2024-04-26T23:20:16.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.296+0000 7f145b12b640 1 -- 172.21.15.47:0/914179629 wait complete. 2024-04-26T23:20:16.308 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T23:20:16.308 DEBUG:teuthology.orchestra.run.smithi047:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:20:16.355 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:20:16.403 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T23:20:16.403 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:20:16.403 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:20:16.419 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=361 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T23:20:16.419 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-26T23:20:16.419 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:20:16.437 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056088,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:20:16.437 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:20:16.437 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:20:16.475 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=361 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T23:20:16.476 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-26T23:20:16.476 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:20:16.494 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056088,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:20:16.494 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-26T23:20:16.494 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:20:16.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.615+0000 7faeebd59640 1 Processor -- start 2024-04-26T23:20:16.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.615+0000 7faeebd59640 1 -- start start 2024-04-26T23:20:16.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.616+0000 7faeebd59640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7faee40cb7e0 0x7faee4162e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:16.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.616+0000 7faeebd59640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faee406f5a0 con 0x7faee41633c0 2024-04-26T23:20:16.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.616+0000 7faeebd59640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faee406f880 con 0x7faee415f290 2024-04-26T23:20:16.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.616+0000 7faeebd59640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7faee40cbc50 con 0x7faee40cb7e0 2024-04-26T23:20:16.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.616+0000 7faee92f4640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7faee40cb7e0 0x7faee4162e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:16.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.616+0000 7faeea2f6640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7faee41633c0 0x7faee40726c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35386/0 (socket says 172.21.15.47:35386) 2024-04-26T23:20:16.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.616+0000 7faeea2f6640 1 -- 172.21.15.47:0/213306380 learned_addr learned my addr 172.21.15.47:0/213306380 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:16.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.616+0000 7faee92f4640 1 -- 172.21.15.47:0/213306380 >> v1:172.21.15.203:6789/0 conn(0x7faee415f290 legacy=0x7faee40cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:16.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.617+0000 7faee92f4640 1 -- 172.21.15.47:0/213306380 >> v1:172.21.15.47:6789/0 conn(0x7faee41633c0 legacy=0x7faee40726c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:16.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.617+0000 7faee92f4640 1 -- 172.21.15.47:0/213306380 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faee406fb60 con 0x7faee40cb7e0 2024-04-26T23:20:16.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.617+0000 7faee8af3640 1 -- 172.21.15.47:0/213306380 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3615156803 0 0) 0x7faee406f5a0 con 0x7faee41633c0 2024-04-26T23:20:16.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.617+0000 7faee92f4640 1 --2- 172.21.15.47:0/213306380 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faee40cb7e0 0x7faee4162e80 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7faed40097a0 tx=0x7faed402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1398dfbde4af3cad server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:16.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.617+0000 7faee8af3640 1 -- 172.21.15.47:0/213306380 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed4037070 con 0x7faee40cb7e0 2024-04-26T23:20:16.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.617+0000 7faee8af3640 1 -- 172.21.15.47:0/213306380 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faed4032440 con 0x7faee40cb7e0 2024-04-26T23:20:16.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.617+0000 7faee8af3640 1 -- 172.21.15.47:0/213306380 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed4032810 con 0x7faee40cb7e0 2024-04-26T23:20:16.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.618+0000 7faeebd59640 1 -- 172.21.15.47:0/213306380 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faee40cb7e0 msgr2=0x7faee4162e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:16.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.618+0000 7faeebd59640 1 --2- 172.21.15.47:0/213306380 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faee40cb7e0 0x7faee4162e80 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7faed40097a0 tx=0x7faed402a270 comp rx=0 tx=0).stop 2024-04-26T23:20:16.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.618+0000 7faeebd59640 1 -- 172.21.15.47:0/213306380 shutdown_connections 2024-04-26T23:20:16.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.618+0000 7faeebd59640 1 --2- 172.21.15.47:0/213306380 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faee40cb7e0 0x7faee4162e80 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:16.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.618+0000 7faeebd59640 1 -- 172.21.15.47:0/213306380 >> 172.21.15.47:0/213306380 conn(0x7faee40d0fd0 msgr2=0x7faee40d13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:16.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.618+0000 7faeebd59640 1 -- 172.21.15.47:0/213306380 shutdown_connections 2024-04-26T23:20:16.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.618+0000 7faeebd59640 1 -- 172.21.15.47:0/213306380 wait complete. 2024-04-26T23:20:16.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.619+0000 7faeebd59640 1 Processor -- start 2024-04-26T23:20:16.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.619+0000 7faeebd59640 1 -- start start 2024-04-26T23:20:16.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.620+0000 7faeebd59640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faee40cb7e0 0x7faee409ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:16.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.620+0000 7faeebd59640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faee415f290 0x7faee415a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:16.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.620+0000 7faeebd59640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faee41633c0 0x7faee40c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:16.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.620+0000 7faeebd59640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7faee4072dd0 con 0x7faee415f290 2024-04-26T23:20:16.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.620+0000 7faeebd59640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7faee406f350 con 0x7faee40cb7e0 2024-04-26T23:20:16.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.620+0000 7faeebd59640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7faee4072f50 con 0x7faee41633c0 2024-04-26T23:20:16.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.620+0000 7faee92f4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faee415f290 0x7faee415a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:16.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.620+0000 7faee92f4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faee415f290 0x7faee415a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46442/0 (socket says 172.21.15.47:46442) 2024-04-26T23:20:16.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.620+0000 7faee92f4640 1 -- 172.21.15.47:0/1019937810 learned_addr learned my addr 172.21.15.47:0/1019937810 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:16.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.620+0000 7faee9af5640 1 --2- 172.21.15.47:0/1019937810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faee40cb7e0 0x7faee409ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:16.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.620+0000 7faeea2f6640 1 --2- 172.21.15.47:0/1019937810 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faee41633c0 0x7faee40c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:16.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.620+0000 7faee92f4640 1 -- 172.21.15.47:0/1019937810 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faee41633c0 msgr2=0x7faee40c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:16.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.620+0000 7faee92f4640 1 --2- 172.21.15.47:0/1019937810 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faee41633c0 0x7faee40c8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:16.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.620+0000 7faee92f4640 1 -- 172.21.15.47:0/1019937810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faee40cb7e0 msgr2=0x7faee409ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:16.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.620+0000 7faee92f4640 1 --2- 172.21.15.47:0/1019937810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faee40cb7e0 0x7faee409ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:16.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.620+0000 7faee92f4640 1 -- 172.21.15.47:0/1019937810 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faee40c97a0 con 0x7faee415f290 2024-04-26T23:20:16.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.620+0000 7faeea2f6640 1 --2- 172.21.15.47:0/1019937810 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faee41633c0 0x7faee40c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:20:16.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.621+0000 7faee92f4640 1 --2- 172.21.15.47:0/1019937810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faee415f290 0x7faee415a9e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7faed40097a0 tx=0x7faed40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:16.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.621+0000 7faeda7fc640 1 -- 172.21.15.47:0/1019937810 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed4041070 con 0x7faee415f290 2024-04-26T23:20:16.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.621+0000 7faeda7fc640 1 -- 172.21.15.47:0/1019937810 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faed403c870 con 0x7faee415f290 2024-04-26T23:20:16.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.621+0000 7faeebd59640 1 -- 172.21.15.47:0/1019937810 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faee40c9a80 con 0x7faee415f290 2024-04-26T23:20:16.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.621+0000 7faeda7fc640 1 -- 172.21.15.47:0/1019937810 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed4037040 con 0x7faee415f290 2024-04-26T23:20:16.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.621+0000 7faeebd59640 1 -- 172.21.15.47:0/1019937810 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faee40b5510 con 0x7faee415f290 2024-04-26T23:20:16.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.622+0000 7faeebd59640 1 -- 172.21.15.47:0/1019937810 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faee40694f0 con 0x7faee415f290 2024-04-26T23:20:16.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.622+0000 7faeda7fc640 1 -- 172.21.15.47:0/1019937810 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faed4044070 con 0x7faee415f290 2024-04-26T23:20:16.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.623+0000 7faeda7fc640 1 --2- 172.21.15.47:0/1019937810 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7faec806c8f0 0x7faec806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:16.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.623+0000 7faeda7fc640 1 -- 172.21.15.47:0/1019937810 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7faed40ac430 con 0x7faee415f290 2024-04-26T23:20:16.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.623+0000 7faee9af5640 1 --2- 172.21.15.47:0/1019937810 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7faec806c8f0 0x7faec806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:16.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.623+0000 7faee9af5640 1 --2- 172.21.15.47:0/1019937810 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7faec806c8f0 0x7faec806edb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7faee0004f60 tx=0x7faee0007e30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:16.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.627+0000 7faeda7fc640 1 -- 172.21.15.47:0/1019937810 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faed407a410 con 0x7faee415f290 2024-04-26T23:20:16.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.851+0000 7faeebd59640 1 -- 172.21.15.47:0/1019937810 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faee406ff80 con 0x7faee415f290 2024-04-26T23:20:16.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.851+0000 7faeda7fc640 1 -- 172.21.15.47:0/1019937810 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+3873 (secure 0 0 0) 0x7faed407e710 con 0x7faee415f290 2024-04-26T23:20:16.852 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:20:16.853 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":21,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5080,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/1970124054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1970124054},{"type":"v1","addr":"172.21.15.203:6836","nonce":1970124054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5310,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/889573201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":889573201},{"type":"v1","addr":"172.21.15.47:6834","nonce":889573201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5316,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3898612760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":3898612760},{"type":"v1","addr":"172.21.15.47:6835","nonce":3898612760}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":14945,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/3371473441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":3371473441},{"type":"v1","addr":"172.21.15.203:6837","nonce":3371473441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[]} 2024-04-26T23:20:16.853 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 21 2024-04-26T23:20:16.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.853+0000 7faeebd59640 1 -- 172.21.15.47:0/1019937810 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7faec806c8f0 msgr2=0x7faec806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:16.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.853+0000 7faeebd59640 1 --2- 172.21.15.47:0/1019937810 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7faec806c8f0 0x7faec806edb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7faee0004f60 tx=0x7faee0007e30 comp rx=0 tx=0).stop 2024-04-26T23:20:16.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.853+0000 7faeebd59640 1 -- 172.21.15.47:0/1019937810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faee415f290 msgr2=0x7faee415a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:16.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.853+0000 7faeebd59640 1 --2- 172.21.15.47:0/1019937810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faee415f290 0x7faee415a9e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7faed40097a0 tx=0x7faed40046c0 comp rx=0 tx=0).stop 2024-04-26T23:20:16.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.853+0000 7faeebd59640 1 -- 172.21.15.47:0/1019937810 shutdown_connections 2024-04-26T23:20:16.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.853+0000 7faeebd59640 1 --2- 172.21.15.47:0/1019937810 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7faec806c8f0 0x7faec806edb0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:16.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.853+0000 7faeebd59640 1 --2- 172.21.15.47:0/1019937810 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faee41633c0 0x7faee40c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:16.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.853+0000 7faeebd59640 1 --2- 172.21.15.47:0/1019937810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faee415f290 0x7faee415a9e0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:16.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.853+0000 7faeebd59640 1 --2- 172.21.15.47:0/1019937810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faee40cb7e0 0x7faee409ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:16.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.853+0000 7faeebd59640 1 -- 172.21.15.47:0/1019937810 >> 172.21.15.47:0/1019937810 conn(0x7faee40d0fd0 msgr2=0x7faee4073310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:16.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.853+0000 7faeebd59640 1 -- 172.21.15.47:0/1019937810 shutdown_connections 2024-04-26T23:20:16.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:16.853+0000 7faeebd59640 1 -- 172.21.15.47:0/1019937810 wait complete. 2024-04-26T23:20:16.866 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T23:20:16.866 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-26T23:20:16.866 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T23:20:16.867 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:20:16.867 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T23:20:16.868 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-26T23:20:16.868 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T23:20:16.868 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:20:16.868 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T23:20:16.868 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-26T23:20:16.869 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T23:20:16.869 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:20:16.869 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T23:20:16.869 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-26T23:20:16.869 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T23:20:16.869 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:20:16.876 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T23:20:16.877 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T23:20:16.878 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T23:20:16.879 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T23:20:16.882 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T23:20:16.882 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T23:20:16.883 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T23:20:16.883 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T23:20:16.920 INFO:tasks.ceph.mds.a:Started 2024-04-26T23:20:16.920 DEBUG:teuthology.parallel:result is None 2024-04-26T23:20:16.921 INFO:tasks.ceph.mds.c:Started 2024-04-26T23:20:16.922 DEBUG:teuthology.parallel:result is None 2024-04-26T23:20:16.924 INFO:tasks.ceph.mds.b:Started 2024-04-26T23:20:16.925 DEBUG:teuthology.parallel:result is None 2024-04-26T23:20:16.926 INFO:tasks.ceph.mds.d:Started 2024-04-26T23:20:16.927 DEBUG:teuthology.parallel:result is None 2024-04-26T23:20:16.927 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-26T23:20:16.996 INFO:tasks.ceph.mds.c.smithi047.stdout:starting mds.c at 2024-04-26T23:20:16.997 INFO:tasks.ceph.mds.a.smithi047.stdout:starting mds.a at 2024-04-26T23:20:17.006 INFO:tasks.ceph.mds.b.smithi203.stdout:starting mds.b at 2024-04-26T23:20:17.007 INFO:tasks.ceph.mds.d.smithi203.stdout:starting mds.d at 2024-04-26T23:20:17.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.033+0000 7f64fb30a640 1 Processor -- start 2024-04-26T23:20:17.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.034+0000 7f64fb30a640 1 -- start start 2024-04-26T23:20:17.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.034+0000 7f64fb30a640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f64f40d0070 0x7f64f40d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:17.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.034+0000 7f64fb30a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f64f406d560 0x7f64f406db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:17.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.034+0000 7f64fb30a640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f64f406e0a0 0x7f64f4162f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:17.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.034+0000 7f64fb30a640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f64f4162d00 con 0x7f64f40d0070 2024-04-26T23:20:17.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.034+0000 7f64fb30a640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f64f40d2d10 con 0x7f64f406e0a0 2024-04-26T23:20:17.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.034+0000 7f64fb30a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f64f40d2420 con 0x7f64f406d560 2024-04-26T23:20:17.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.035+0000 7f64f90a6640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f64f40d0070 0x7f64f40d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:17.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.035+0000 7f64f90a6640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f64f40d0070 0x7f64f40d1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46494/0 (socket says 172.21.15.47:46494) 2024-04-26T23:20:17.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.035+0000 7f64f90a6640 1 -- 172.21.15.47:0/1439129449 learned_addr learned my addr 172.21.15.47:0/1439129449 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:17.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.035+0000 7f64f88a5640 1 --2- 172.21.15.47:0/1439129449 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f64f406d560 0x7f64f406db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:17.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.035+0000 7f64f98a7640 1 --2- 172.21.15.47:0/1439129449 >> v2:172.21.15.203:3300/0 conn(0x7f64f406e0a0 0x7f64f4162f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:17.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.035+0000 7f64f90a6640 1 -- 172.21.15.47:0/1439129449 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f64f406d560 msgr2=0x7f64f406db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:17.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.035+0000 7f64f90a6640 1 --2- 172.21.15.47:0/1439129449 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f64f406d560 0x7f64f406db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:17.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.035+0000 7f64f90a6640 1 -- 172.21.15.47:0/1439129449 >> v2:172.21.15.203:3300/0 conn(0x7f64f406e0a0 msgr2=0x7f64f4162f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:17.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.035+0000 7f64f90a6640 1 --2- 172.21.15.47:0/1439129449 >> v2:172.21.15.203:3300/0 conn(0x7f64f406e0a0 0x7f64f4162f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:17.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.035+0000 7f64f90a6640 1 -- 172.21.15.47:0/1439129449 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64f40cd6b0 con 0x7f64f40d0070 2024-04-26T23:20:17.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.035+0000 7f64f88a5640 1 --2- 172.21.15.47:0/1439129449 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f64f406d560 0x7f64f406db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:17.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.036+0000 7f64f90a6640 1 --2- 172.21.15.47:0/1439129449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64f40d0070 0x7f64f40d1d40 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f64f4073370 tx=0x7f64e402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=495a20a552c33088 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:17.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.036+0000 7f64ebfff640 1 -- 172.21.15.47:0/1439129449 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e4037070 con 0x7f64f40d0070 2024-04-26T23:20:17.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.036+0000 7f64ebfff640 1 -- 172.21.15.47:0/1439129449 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64e4032440 con 0x7f64f40d0070 2024-04-26T23:20:17.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.036+0000 7f64ebfff640 1 -- 172.21.15.47:0/1439129449 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e4032810 con 0x7f64f40d0070 2024-04-26T23:20:17.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.036+0000 7f64fb30a640 1 -- 172.21.15.47:0/1439129449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64f40d0070 msgr2=0x7f64f40d1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:17.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.036+0000 7f64fb30a640 1 --2- 172.21.15.47:0/1439129449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64f40d0070 0x7f64f40d1d40 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f64f4073370 tx=0x7f64e402a270 comp rx=0 tx=0).stop 2024-04-26T23:20:17.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.036+0000 7f64fb30a640 1 -- 172.21.15.47:0/1439129449 shutdown_connections 2024-04-26T23:20:17.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.036+0000 7f64fb30a640 1 --2- 172.21.15.47:0/1439129449 >> v2:172.21.15.203:3300/0 conn(0x7f64f406e0a0 0x7f64f4162f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:17.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.036+0000 7f64fb30a640 1 --2- 172.21.15.47:0/1439129449 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f64f406d560 0x7f64f406db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:17.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.036+0000 7f64fb30a640 1 --2- 172.21.15.47:0/1439129449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64f40d0070 0x7f64f40d1d40 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:17.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.036+0000 7f64fb30a640 1 -- 172.21.15.47:0/1439129449 >> 172.21.15.47:0/1439129449 conn(0x7f64f40d2ec0 msgr2=0x7f64f40d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:17.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.036+0000 7f64fb30a640 1 -- 172.21.15.47:0/1439129449 shutdown_connections 2024-04-26T23:20:17.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.036+0000 7f64fb30a640 1 -- 172.21.15.47:0/1439129449 wait complete. 2024-04-26T23:20:17.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.038+0000 7f64fb30a640 1 Processor -- start 2024-04-26T23:20:17.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.038+0000 7f64fb30a640 1 -- start start 2024-04-26T23:20:17.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.039+0000 7f64fb30a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64f406d560 0x7f64f41f4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:17.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.039+0000 7f64fb30a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64f406e0a0 0x7f64f40c7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:17.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.039+0000 7f64fb30a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f64f40d0070 0x7f64f40c6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:17.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.039+0000 7f64fb30a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f64f4163450 con 0x7f64f406e0a0 2024-04-26T23:20:17.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.039+0000 7f64fb30a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f64f40cdc70 con 0x7f64f406d560 2024-04-26T23:20:17.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.039+0000 7f64f88a5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64f406e0a0 0x7f64f40c7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:17.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.039+0000 7f64fb30a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f64f41635d0 con 0x7f64f40d0070 2024-04-26T23:20:17.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.039+0000 7f64f88a5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64f406e0a0 0x7f64f40c7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46496/0 (socket says 172.21.15.47:46496) 2024-04-26T23:20:17.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.039+0000 7f64f88a5640 1 -- 172.21.15.47:0/3035534984 learned_addr learned my addr 172.21.15.47:0/3035534984 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:17.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.039+0000 7f64f90a6640 1 --2- 172.21.15.47:0/3035534984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64f406d560 0x7f64f41f4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:17.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.039+0000 7f64f98a7640 1 --2- 172.21.15.47:0/3035534984 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f64f40d0070 0x7f64f40c6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:17.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.039+0000 7f64f88a5640 1 -- 172.21.15.47:0/3035534984 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f64f40d0070 msgr2=0x7f64f40c6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:17.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.039+0000 7f64f88a5640 1 --2- 172.21.15.47:0/3035534984 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f64f40d0070 0x7f64f40c6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:17.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.039+0000 7f64f88a5640 1 -- 172.21.15.47:0/3035534984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64f406d560 msgr2=0x7f64f41f4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:17.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.039+0000 7f64f88a5640 1 --2- 172.21.15.47:0/3035534984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64f406d560 0x7f64f41f4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:17.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.039+0000 7f64f88a5640 1 -- 172.21.15.47:0/3035534984 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64f40c72e0 con 0x7f64f406e0a0 2024-04-26T23:20:17.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.039+0000 7f64f88a5640 1 --2- 172.21.15.47:0/3035534984 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64f406e0a0 0x7f64f40c7dd0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f64f000c7f0 tx=0x7f64f000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:17.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.039+0000 7f64e9ffb640 1 -- 172.21.15.47:0/3035534984 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f0015070 con 0x7f64f406e0a0 2024-04-26T23:20:17.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.040+0000 7f64e9ffb640 1 -- 172.21.15.47:0/3035534984 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64f0007a30 con 0x7f64f406e0a0 2024-04-26T23:20:17.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.040+0000 7f64fb30a640 1 -- 172.21.15.47:0/3035534984 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64f40c75c0 con 0x7f64f406e0a0 2024-04-26T23:20:17.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.040+0000 7f64e9ffb640 1 -- 172.21.15.47:0/3035534984 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f0007e00 con 0x7f64f406e0a0 2024-04-26T23:20:17.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.040+0000 7f64fb30a640 1 -- 172.21.15.47:0/3035534984 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64f409fad0 con 0x7f64f406e0a0 2024-04-26T23:20:17.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.041+0000 7f64fb30a640 1 -- 172.21.15.47:0/3035534984 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64f409fcd0 con 0x7f64f406e0a0 2024-04-26T23:20:17.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.041+0000 7f64e9ffb640 1 -- 172.21.15.47:0/3035534984 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f64f0018070 con 0x7f64f406e0a0 2024-04-26T23:20:17.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.042+0000 7f64e9ffb640 1 --2- 172.21.15.47:0/3035534984 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f64d806c920 0x7f64d806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:17.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.042+0000 7f64e9ffb640 1 -- 172.21.15.47:0/3035534984 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f64f008b9d0 con 0x7f64f406e0a0 2024-04-26T23:20:17.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.042+0000 7f64f90a6640 1 --2- 172.21.15.47:0/3035534984 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f64d806c920 0x7f64d806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:17.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.042+0000 7f64f90a6640 1 --2- 172.21.15.47:0/3035534984 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f64d806c920 0x7f64d806ede0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f64e4034000 tx=0x7f64e402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:17.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.046+0000 7f64e9ffb640 1 -- 172.21.15.47:0/3035534984 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f64f409fcd0 con 0x7f64f406e0a0 2024-04-26T23:20:17.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:17.246+0000 7f64fb30a640 1 -- 172.21.15.47:0/3035534984 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f64f40d0470 con 0x7f64f406e0a0 2024-04-26T23:20:18.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.322+0000 7f64e9ffb640 1 -- 172.21.15.47:0/3035534984 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v50) ==== 97+0+0 (secure 0 0 0) 0x7f64f0059f60 con 0x7f64f406e0a0 2024-04-26T23:20:18.323 INFO:teuthology.orchestra.run.smithi047.stderr: removed all blocklist entries 2024-04-26T23:20:18.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.326+0000 7f64fb30a640 1 -- 172.21.15.47:0/3035534984 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f64d806c920 msgr2=0x7f64d806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:18.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.326+0000 7f64fb30a640 1 --2- 172.21.15.47:0/3035534984 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f64d806c920 0x7f64d806ede0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f64e4034000 tx=0x7f64e402e000 comp rx=0 tx=0).stop 2024-04-26T23:20:18.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.326+0000 7f64fb30a640 1 -- 172.21.15.47:0/3035534984 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64f406e0a0 msgr2=0x7f64f40c7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:18.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.326+0000 7f64fb30a640 1 --2- 172.21.15.47:0/3035534984 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64f406e0a0 0x7f64f40c7dd0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f64f000c7f0 tx=0x7f64f000cbf0 comp rx=0 tx=0).stop 2024-04-26T23:20:18.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.326+0000 7f64fb30a640 1 -- 172.21.15.47:0/3035534984 shutdown_connections 2024-04-26T23:20:18.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.326+0000 7f64fb30a640 1 --2- 172.21.15.47:0/3035534984 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f64d806c920 0x7f64d806ede0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:18.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.326+0000 7f64fb30a640 1 --2- 172.21.15.47:0/3035534984 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f64f40d0070 0x7f64f40c6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:18.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.326+0000 7f64fb30a640 1 --2- 172.21.15.47:0/3035534984 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64f406e0a0 0x7f64f40c7dd0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:18.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.326+0000 7f64fb30a640 1 --2- 172.21.15.47:0/3035534984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64f406d560 0x7f64f41f4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:18.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.326+0000 7f64fb30a640 1 -- 172.21.15.47:0/3035534984 >> 172.21.15.47:0/3035534984 conn(0x7f64f40d2ec0 msgr2=0x7f64f4071540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:18.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.326+0000 7f64fb30a640 1 -- 172.21.15.47:0/3035534984 shutdown_connections 2024-04-26T23:20:18.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.326+0000 7f64fb30a640 1 -- 172.21.15.47:0/3035534984 wait complete. 2024-04-26T23:20:18.340 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-26T23:20:18.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.446+0000 7f53cf9c9640 1 Processor -- start 2024-04-26T23:20:18.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.446+0000 7f53cf9c9640 1 -- start start 2024-04-26T23:20:18.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.447+0000 7f53cf9c9640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f53c80b3860 0x7f53c816a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:18.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.447+0000 7f53cf9c9640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53c80d0ba0 con 0x7f53c80d06f0 2024-04-26T23:20:18.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.447+0000 7f53cf9c9640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53c80d2590 con 0x7f53c815f3a0 2024-04-26T23:20:18.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.447+0000 7f53cf9c9640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f53c80d0e20 con 0x7f53c80b3860 2024-04-26T23:20:18.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.447+0000 7f53ccf64640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f53c80d06f0 0x7f53c80d1e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35402/0 (socket says 172.21.15.47:35402) 2024-04-26T23:20:18.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.447+0000 7f53ccf64640 1 -- 172.21.15.47:0/3442578133 learned_addr learned my addr 172.21.15.47:0/3442578133 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:18.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.447+0000 7f53cdf66640 1 --2- 172.21.15.47:0/3442578133 >> v2:172.21.15.47:3300/0 conn(0x7f53c80b3860 0x7f53c816a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:18.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.448+0000 7f53cdf66640 1 -- 172.21.15.47:0/3442578133 >> v1:172.21.15.203:6789/0 conn(0x7f53c815f3a0 legacy=0x7f53c80d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:18.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.448+0000 7f53cdf66640 1 -- 172.21.15.47:0/3442578133 >> v1:172.21.15.47:6789/0 conn(0x7f53c80d06f0 legacy=0x7f53c80d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:18.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.448+0000 7f53c7fff640 1 -- 172.21.15.47:0/3442578133 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2174887658 0 0) 0x7f53bc001110 con 0x7f53c80d06f0 2024-04-26T23:20:18.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.448+0000 7f53cdf66640 1 -- 172.21.15.47:0/3442578133 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53c80b1d70 con 0x7f53c80b3860 2024-04-26T23:20:18.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.448+0000 7f53cdf66640 1 --2- 172.21.15.47:0/3442578133 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53c80b3860 0x7f53c816a000 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f53b8009d90 tx=0x7f53b802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dee662d25333202 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:18.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.449+0000 7f53c7fff640 1 -- 172.21.15.47:0/3442578133 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b8031600 con 0x7f53c80b3860 2024-04-26T23:20:18.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.449+0000 7f53c7fff640 1 -- 172.21.15.47:0/3442578133 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53b80317a0 con 0x7f53c80b3860 2024-04-26T23:20:18.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.449+0000 7f53c7fff640 1 -- 172.21.15.47:0/3442578133 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b8031b70 con 0x7f53c80b3860 2024-04-26T23:20:18.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.449+0000 7f53cf9c9640 1 -- 172.21.15.47:0/3442578133 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53c80b3860 msgr2=0x7f53c816a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:18.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.449+0000 7f53cf9c9640 1 --2- 172.21.15.47:0/3442578133 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53c80b3860 0x7f53c816a000 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f53b8009d90 tx=0x7f53b802a610 comp rx=0 tx=0).stop 2024-04-26T23:20:18.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.449+0000 7f53cf9c9640 1 -- 172.21.15.47:0/3442578133 shutdown_connections 2024-04-26T23:20:18.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.449+0000 7f53cf9c9640 1 --2- 172.21.15.47:0/3442578133 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53c80b3860 0x7f53c816a000 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:18.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.449+0000 7f53cf9c9640 1 -- 172.21.15.47:0/3442578133 >> 172.21.15.47:0/3442578133 conn(0x7f53c80d4f90 msgr2=0x7f53c80d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:18.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.449+0000 7f53cf9c9640 1 -- 172.21.15.47:0/3442578133 shutdown_connections 2024-04-26T23:20:18.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.449+0000 7f53cf9c9640 1 -- 172.21.15.47:0/3442578133 wait complete. 2024-04-26T23:20:18.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.450+0000 7f53cf9c9640 1 Processor -- start 2024-04-26T23:20:18.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.451+0000 7f53cf9c9640 1 -- start start 2024-04-26T23:20:18.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.451+0000 7f53cf9c9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53c80b3860 0x7f53c815b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:18.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.451+0000 7f53cf9c9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53c80d06f0 0x7f53c8153480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:18.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.451+0000 7f53cf9c9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53c815f3a0 0x7f53c80caad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:18.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.451+0000 7f53cf9c9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f53c815f780 con 0x7f53c80b3860 2024-04-26T23:20:18.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.451+0000 7f53cf9c9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f53c80b2330 con 0x7f53c80d06f0 2024-04-26T23:20:18.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.451+0000 7f53cf9c9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f53c815f900 con 0x7f53c815f3a0 2024-04-26T23:20:18.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.451+0000 7f53cd765640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53c80b3860 0x7f53c815b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:18.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.451+0000 7f53cd765640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53c80b3860 0x7f53c815b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46504/0 (socket says 172.21.15.47:46504) 2024-04-26T23:20:18.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.451+0000 7f53cd765640 1 -- 172.21.15.47:0/1741582893 learned_addr learned my addr 172.21.15.47:0/1741582893 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:18.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.451+0000 7f53cdf66640 1 --2- 172.21.15.47:0/1741582893 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53c815f3a0 0x7f53c80caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:18.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.451+0000 7f53ccf64640 1 --2- 172.21.15.47:0/1741582893 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53c80d06f0 0x7f53c8153480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:18.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.452+0000 7f53cd765640 1 -- 172.21.15.47:0/1741582893 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53c815f3a0 msgr2=0x7f53c80caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:18.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.452+0000 7f53cd765640 1 --2- 172.21.15.47:0/1741582893 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53c815f3a0 0x7f53c80caad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:18.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.452+0000 7f53cd765640 1 -- 172.21.15.47:0/1741582893 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53c80d06f0 msgr2=0x7f53c8153480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:18.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.452+0000 7f53cd765640 1 --2- 172.21.15.47:0/1741582893 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53c80d06f0 0x7f53c8153480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:18.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.452+0000 7f53cd765640 1 -- 172.21.15.47:0/1741582893 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53c80b9ac0 con 0x7f53c80b3860 2024-04-26T23:20:18.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.452+0000 7f53cdf66640 1 --2- 172.21.15.47:0/1741582893 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53c815f3a0 0x7f53c80caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:20:18.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.452+0000 7f53ccf64640 1 --2- 172.21.15.47:0/1741582893 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53c80d06f0 0x7f53c8153480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:20:18.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.452+0000 7f53cd765640 1 --2- 172.21.15.47:0/1741582893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53c80b3860 0x7f53c815b4b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f53c000b3f0 tx=0x7f53c000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:18.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.452+0000 7f53c5ffb640 1 -- 172.21.15.47:0/1741582893 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c0015020 con 0x7f53c80b3860 2024-04-26T23:20:18.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.452+0000 7f53c5ffb640 1 -- 172.21.15.47:0/1741582893 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53c0011830 con 0x7f53c80b3860 2024-04-26T23:20:18.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.452+0000 7f53cf9c9640 1 -- 172.21.15.47:0/1741582893 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53c80b9da0 con 0x7f53c80b3860 2024-04-26T23:20:18.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.452+0000 7f53c5ffb640 1 -- 172.21.15.47:0/1741582893 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c0011c20 con 0x7f53c80b3860 2024-04-26T23:20:18.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.452+0000 7f53cf9c9640 1 -- 172.21.15.47:0/1741582893 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53c80ba360 con 0x7f53c80b3860 2024-04-26T23:20:18.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.453+0000 7f53cf9c9640 1 -- 172.21.15.47:0/1741582893 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53c80694f0 con 0x7f53c80b3860 2024-04-26T23:20:18.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.454+0000 7f53c5ffb640 1 -- 172.21.15.47:0/1741582893 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f53c0018020 con 0x7f53c80b3860 2024-04-26T23:20:18.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.454+0000 7f53c5ffb640 1 --2- 172.21.15.47:0/1741582893 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f53b406c8f0 0x7f53b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:18.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.454+0000 7f53c5ffb640 1 -- 172.21.15.47:0/1741582893 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5249+0+0 (secure 0 0 0) 0x7f53c008b560 con 0x7f53c80b3860 2024-04-26T23:20:18.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.455+0000 7f53ccf64640 1 --2- 172.21.15.47:0/1741582893 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f53b406c8f0 0x7f53b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:18.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.455+0000 7f53ccf64640 1 --2- 172.21.15.47:0/1741582893 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f53b406c8f0 0x7f53b406edb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f53c815a700 tx=0x7f53bc007a10 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:18.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.458+0000 7f53c5ffb640 1 -- 172.21.15.47:0/1741582893 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f53c001f030 con 0x7f53c80b3860 2024-04-26T23:20:18.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.713+0000 7f53cf9c9640 1 -- 172.21.15.47:0/1741582893 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f53c80b6860 con 0x7f53c80b3860 2024-04-26T23:20:18.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.713+0000 7f53c5ffb640 1 -- 172.21.15.47:0/1741582893 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) ==== 80+0+5298 (secure 0 0 0) 0x7f53c00172c0 con 0x7f53c80b3860 2024-04-26T23:20:18.714 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:20:18.714 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-26T23:20:18.715 INFO:teuthology.orchestra.run.smithi047.stdout: "auth_dump": [ 2024-04-26T23:20:18.715 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-26T23:20:18.715 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.a", 2024-04-26T23:20:18.715 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQAKNixmrb+7BhAA9Tl5qHRAAITpDe0o1RpB8A==", 2024-04-26T23:20:18.715 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-26T23:20:18.715 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-04-26T23:20:18.715 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2024-04-26T23:20:18.715 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2024-04-26T23:20:18.715 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-26T23:20:18.715 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-26T23:20:18.715 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-26T23:20:18.715 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-26T23:20:18.716 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.b", 2024-04-26T23:20:18.716 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQAKNixmZYgrEhAA4XwxVuQT3PVAfsalOSSpRg==", 2024-04-26T23:20:18.716 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-26T23:20:18.716 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-04-26T23:20:18.716 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2024-04-26T23:20:18.716 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2024-04-26T23:20:18.716 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-26T23:20:18.716 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-26T23:20:18.716 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-26T23:20:18.716 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-26T23:20:18.716 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.c", 2024-04-26T23:20:18.716 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQAKNixmy+yoDRAAeSQ/dzhhLyysouyco6YrRw==", 2024-04-26T23:20:18.716 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-26T23:20:18.717 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-04-26T23:20:18.717 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2024-04-26T23:20:18.717 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2024-04-26T23:20:18.717 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-26T23:20:18.717 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-26T23:20:18.717 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-26T23:20:18.717 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-26T23:20:18.717 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.d", 2024-04-26T23:20:18.717 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQAKNixmr1gJGRAA2M7//N7u9lk7edXxKxKdJQ==", 2024-04-26T23:20:18.717 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-26T23:20:18.717 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-04-26T23:20:18.717 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2024-04-26T23:20:18.717 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2024-04-26T23:20:18.718 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-26T23:20:18.718 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-26T23:20:18.718 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-26T23:20:18.718 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-26T23:20:18.718 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.0", 2024-04-26T23:20:18.718 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQALNixm3dBSOxAA0heGY4osjvaNAB25znAAZw==", 2024-04-26T23:20:18.718 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-26T23:20:18.718 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-04-26T23:20:18.718 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-04-26T23:20:18.718 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-26T23:20:18.718 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-26T23:20:18.718 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-26T23:20:18.719 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-26T23:20:18.719 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.1", 2024-04-26T23:20:18.719 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQAONixmH0ZoORAAaNuvYE66+ypC0bIeWxVaPw==", 2024-04-26T23:20:18.719 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-26T23:20:18.719 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-04-26T23:20:18.719 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-04-26T23:20:18.719 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-26T23:20:18.719 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-26T23:20:18.719 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-26T23:20:18.719 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-26T23:20:18.719 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.2", 2024-04-26T23:20:18.719 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQARNixmQCOoNxAAl7vmPd2PBbBHjl7VJobIeg==", 2024-04-26T23:20:18.719 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-26T23:20:18.720 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-04-26T23:20:18.720 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-04-26T23:20:18.720 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-26T23:20:18.720 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-26T23:20:18.720 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-26T23:20:18.720 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-26T23:20:18.720 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.3", 2024-04-26T23:20:18.720 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQAUNixmMtdiNhAA+xeZUbroEshbAQO7BQm9kg==", 2024-04-26T23:20:18.720 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-26T23:20:18.720 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-04-26T23:20:18.720 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-04-26T23:20:18.720 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-26T23:20:18.721 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-26T23:20:18.721 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-26T23:20:18.721 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-26T23:20:18.721 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.4", 2024-04-26T23:20:18.721 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQAaNixmiYETOhAAkz/BfU0a8SnawltF0pBU/A==", 2024-04-26T23:20:18.721 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-26T23:20:18.721 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-04-26T23:20:18.721 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-04-26T23:20:18.721 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-26T23:20:18.721 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-26T23:20:18.721 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-26T23:20:18.721 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-26T23:20:18.721 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.5", 2024-04-26T23:20:18.722 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQAdNixmKCoPORAAp3FfD88sxG9IlXK1HqpiPA==", 2024-04-26T23:20:18.722 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-26T23:20:18.722 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-04-26T23:20:18.722 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-04-26T23:20:18.722 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-26T23:20:18.722 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-26T23:20:18.722 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-26T23:20:18.722 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-26T23:20:18.722 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.6", 2024-04-26T23:20:18.722 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQAgNixm/ipONxAAzgJ2wJAHFbHlv09nqA3soQ==", 2024-04-26T23:20:18.722 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-26T23:20:18.722 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-04-26T23:20:18.723 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-04-26T23:20:18.723 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-26T23:20:18.723 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-26T23:20:18.723 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-26T23:20:18.723 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-26T23:20:18.723 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.7", 2024-04-26T23:20:18.723 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQAjNixmXDWkNRAAJYh2B7nH7ViaGPii7Yk69A==", 2024-04-26T23:20:18.723 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-26T23:20:18.723 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-04-26T23:20:18.723 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-04-26T23:20:18.723 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-26T23:20:18.723 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-26T23:20:18.723 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-26T23:20:18.724 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-26T23:20:18.724 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.0", 2024-04-26T23:20:18.724 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQAKNixmA5QFHRAAT89WPyzDs0acPluyvU4svw==", 2024-04-26T23:20:18.724 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-26T23:20:18.724 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-04-26T23:20:18.724 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow r", 2024-04-26T23:20:18.724 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rw pool=cephfs_data" 2024-04-26T23:20:18.724 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-26T23:20:18.724 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-26T23:20:18.724 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-26T23:20:18.724 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.1", 2024-04-26T23:20:18.724 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQAKNixmtRW8JRAAqh+Q2ZhqvdhTxty04GtU2g==", 2024-04-26T23:20:18.724 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-26T23:20:18.725 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-04-26T23:20:18.725 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow r", 2024-04-26T23:20:18.725 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow rw", 2024-04-26T23:20:18.725 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rwx" 2024-04-26T23:20:18.725 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-26T23:20:18.725 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-26T23:20:18.725 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-26T23:20:18.725 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.2", 2024-04-26T23:20:18.725 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQAKNixmTb2VIhAAhoBSsrzlPIsljZt1jZVscQ==", 2024-04-26T23:20:18.725 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-26T23:20:18.725 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-04-26T23:20:18.725 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow r", 2024-04-26T23:20:18.725 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow rw", 2024-04-26T23:20:18.726 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rwx" 2024-04-26T23:20:18.726 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-26T23:20:18.726 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-26T23:20:18.726 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-26T23:20:18.726 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.3", 2024-04-26T23:20:18.726 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQAKNixmDE8nKxAA9zbeR9CqdL0p8gcuy3k/kQ==", 2024-04-26T23:20:18.726 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-26T23:20:18.726 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-04-26T23:20:18.726 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow r", 2024-04-26T23:20:18.726 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow rw", 2024-04-26T23:20:18.726 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rwx" 2024-04-26T23:20:18.726 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-26T23:20:18.726 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-26T23:20:18.727 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-26T23:20:18.727 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.admin", 2024-04-26T23:20:18.727 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQAJNixmtExhJhAAOUYKNUWtJEGRp9Shql0u8g==", 2024-04-26T23:20:18.727 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-26T23:20:18.727 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2024-04-26T23:20:18.727 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2024-04-26T23:20:18.727 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2024-04-26T23:20:18.727 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-26T23:20:18.727 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-26T23:20:18.727 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-26T23:20:18.727 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-26T23:20:18.727 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.x", 2024-04-26T23:20:18.728 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQAKNixmf2OaAxAAv3TUVplM33vJKhqsWyV84w==", 2024-04-26T23:20:18.728 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-26T23:20:18.728 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2024-04-26T23:20:18.728 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2024-04-26T23:20:18.728 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-26T23:20:18.728 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-26T23:20:18.728 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-26T23:20:18.728 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-26T23:20:18.728 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.y", 2024-04-26T23:20:18.728 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQAJNixmQ2HSOhAACgAm3LAXwI4PB/P1+oRdyw==", 2024-04-26T23:20:18.728 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-26T23:20:18.728 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2024-04-26T23:20:18.728 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2024-04-26T23:20:18.729 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-26T23:20:18.729 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-26T23:20:18.729 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-26T23:20:18.729 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-26T23:20:18.729 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-26T23:20:18.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.715+0000 7f53cf9c9640 1 -- 172.21.15.47:0/1741582893 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f53b406c8f0 msgr2=0x7f53b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:18.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.715+0000 7f53cf9c9640 1 --2- 172.21.15.47:0/1741582893 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f53b406c8f0 0x7f53b406edb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f53c815a700 tx=0x7f53bc007a10 comp rx=0 tx=0).stop 2024-04-26T23:20:18.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.715+0000 7f53cf9c9640 1 -- 172.21.15.47:0/1741582893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53c80b3860 msgr2=0x7f53c815b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:18.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.715+0000 7f53cf9c9640 1 --2- 172.21.15.47:0/1741582893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53c80b3860 0x7f53c815b4b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f53c000b3f0 tx=0x7f53c000b8b0 comp rx=0 tx=0).stop 2024-04-26T23:20:18.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.715+0000 7f53cf9c9640 1 -- 172.21.15.47:0/1741582893 shutdown_connections 2024-04-26T23:20:18.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.715+0000 7f53cf9c9640 1 --2- 172.21.15.47:0/1741582893 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f53b406c8f0 0x7f53b406edb0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:18.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.715+0000 7f53cf9c9640 1 --2- 172.21.15.47:0/1741582893 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53c815f3a0 0x7f53c80caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:18.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.715+0000 7f53cf9c9640 1 --2- 172.21.15.47:0/1741582893 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53c80d06f0 0x7f53c8153480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:18.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.715+0000 7f53cf9c9640 1 --2- 172.21.15.47:0/1741582893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53c80b3860 0x7f53c815b4b0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:18.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.715+0000 7f53cf9c9640 1 -- 172.21.15.47:0/1741582893 >> 172.21.15.47:0/1741582893 conn(0x7f53c80d4f90 msgr2=0x7f53c806dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:18.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.715+0000 7f53cf9c9640 1 -- 172.21.15.47:0/1741582893 shutdown_connections 2024-04-26T23:20:18.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.715+0000 7f53cf9c9640 1 -- 172.21.15.47:0/1741582893 wait complete. 2024-04-26T23:20:18.732 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:20:18.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.832+0000 7feb3b5b4640 1 Processor -- start 2024-04-26T23:20:18.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.832+0000 7feb3b5b4640 1 -- start start 2024-04-26T23:20:18.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.833+0000 7feb3b5b4640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7feb34162810 0x7feb34070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:18.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.833+0000 7feb3b5b4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb340b2340 0x7feb340cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:18.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.833+0000 7feb3b5b4640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7feb340cd930 0x7feb340d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:18.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.833+0000 7feb3b5b4640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7feb340d0750 con 0x7feb34162810 2024-04-26T23:20:18.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.833+0000 7feb3b5b4640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7feb340d16a0 con 0x7feb340cd930 2024-04-26T23:20:18.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.833+0000 7feb3b5b4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7feb340d10d0 con 0x7feb340b2340 2024-04-26T23:20:18.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.833+0000 7feb39350640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7feb34162810 0x7feb34070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:18.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.833+0000 7feb39350640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7feb34162810 0x7feb34070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46516/0 (socket says 172.21.15.47:46516) 2024-04-26T23:20:18.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.833+0000 7feb39350640 1 -- 172.21.15.47:0/295076516 learned_addr learned my addr 172.21.15.47:0/295076516 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:18.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.833+0000 7feb38b4f640 1 --2- 172.21.15.47:0/295076516 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb340b2340 0x7feb340cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:18.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.833+0000 7feb39b51640 1 --2- 172.21.15.47:0/295076516 >> v2:172.21.15.203:3300/0 conn(0x7feb340cd930 0x7feb340d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:18.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.833+0000 7feb39350640 1 -- 172.21.15.47:0/295076516 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb340b2340 msgr2=0x7feb340cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:18.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.833+0000 7feb39350640 1 --2- 172.21.15.47:0/295076516 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb340b2340 0x7feb340cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:18.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.833+0000 7feb39350640 1 -- 172.21.15.47:0/295076516 >> v2:172.21.15.203:3300/0 conn(0x7feb340cd930 msgr2=0x7feb340d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:18.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.833+0000 7feb39350640 1 --2- 172.21.15.47:0/295076516 >> v2:172.21.15.203:3300/0 conn(0x7feb340cd930 0x7feb340d1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:18.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.833+0000 7feb39350640 1 -- 172.21.15.47:0/295076516 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb340d1370 con 0x7feb34162810 2024-04-26T23:20:18.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.833+0000 7feb38b4f640 1 --2- 172.21.15.47:0/295076516 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb340b2340 0x7feb340cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:18.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.834+0000 7feb39350640 1 --2- 172.21.15.47:0/295076516 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb34162810 0x7feb34070350 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7feb340b2250 tx=0x7feb3002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=815cfb67f98b833 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:18.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.834+0000 7feb2bfff640 1 -- 172.21.15.47:0/295076516 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb30037070 con 0x7feb34162810 2024-04-26T23:20:18.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.834+0000 7feb2bfff640 1 -- 172.21.15.47:0/295076516 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb30032440 con 0x7feb34162810 2024-04-26T23:20:18.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.834+0000 7feb2bfff640 1 -- 172.21.15.47:0/295076516 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb30032810 con 0x7feb34162810 2024-04-26T23:20:18.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.835+0000 7feb3b5b4640 1 -- 172.21.15.47:0/295076516 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb34162810 msgr2=0x7feb34070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:18.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.835+0000 7feb3b5b4640 1 --2- 172.21.15.47:0/295076516 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb34162810 0x7feb34070350 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7feb340b2250 tx=0x7feb3002a1e0 comp rx=0 tx=0).stop 2024-04-26T23:20:18.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.835+0000 7feb3b5b4640 1 -- 172.21.15.47:0/295076516 shutdown_connections 2024-04-26T23:20:18.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.835+0000 7feb3b5b4640 1 --2- 172.21.15.47:0/295076516 >> v2:172.21.15.203:3300/0 conn(0x7feb340cd930 0x7feb340d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:18.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.835+0000 7feb3b5b4640 1 --2- 172.21.15.47:0/295076516 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb340b2340 0x7feb340cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:18.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.835+0000 7feb3b5b4640 1 --2- 172.21.15.47:0/295076516 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb34162810 0x7feb34070350 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:18.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.835+0000 7feb3b5b4640 1 -- 172.21.15.47:0/295076516 >> 172.21.15.47:0/295076516 conn(0x7feb340d2da0 msgr2=0x7feb340d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:18.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.835+0000 7feb3b5b4640 1 -- 172.21.15.47:0/295076516 shutdown_connections 2024-04-26T23:20:18.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.835+0000 7feb3b5b4640 1 -- 172.21.15.47:0/295076516 wait complete. 2024-04-26T23:20:18.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.836+0000 7feb3b5b4640 1 Processor -- start 2024-04-26T23:20:18.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.836+0000 7feb3b5b4640 1 -- start start 2024-04-26T23:20:18.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.837+0000 7feb3b5b4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb340b2340 0x7feb34158320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:18.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.837+0000 7feb3b5b4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb340cd930 0x7feb340c9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:18.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.837+0000 7feb3b5b4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb34162810 0x7feb3415d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:18.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.837+0000 7feb3b5b4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7feb34070890 con 0x7feb34162810 2024-04-26T23:20:18.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.837+0000 7feb3b5b4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7feb340d0470 con 0x7feb340cd930 2024-04-26T23:20:18.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.837+0000 7feb3b5b4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7feb340d2070 con 0x7feb340b2340 2024-04-26T23:20:18.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.837+0000 7feb39b51640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb34162810 0x7feb3415d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:18.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.837+0000 7feb39b51640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb34162810 0x7feb3415d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46518/0 (socket says 172.21.15.47:46518) 2024-04-26T23:20:18.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.837+0000 7feb39b51640 1 -- 172.21.15.47:0/785750036 learned_addr learned my addr 172.21.15.47:0/785750036 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:18.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.837+0000 7feb38b4f640 1 --2- 172.21.15.47:0/785750036 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb340cd930 0x7feb340c9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:18.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.837+0000 7feb39350640 1 --2- 172.21.15.47:0/785750036 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb340b2340 0x7feb34158320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:18.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.837+0000 7feb39b51640 1 -- 172.21.15.47:0/785750036 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb340b2340 msgr2=0x7feb34158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:18.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.837+0000 7feb39b51640 1 --2- 172.21.15.47:0/785750036 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb340b2340 0x7feb34158320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:18.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.837+0000 7feb39b51640 1 -- 172.21.15.47:0/785750036 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb340cd930 msgr2=0x7feb340c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:18.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.837+0000 7feb39b51640 1 --2- 172.21.15.47:0/785750036 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb340cd930 0x7feb340c9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:18.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.837+0000 7feb39b51640 1 -- 172.21.15.47:0/785750036 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb340c9780 con 0x7feb34162810 2024-04-26T23:20:18.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.837+0000 7feb38b4f640 1 --2- 172.21.15.47:0/785750036 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb340cd930 0x7feb340c9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:20:18.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.837+0000 7feb39350640 1 --2- 172.21.15.47:0/785750036 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb340b2340 0x7feb34158320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:20:18.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.837+0000 7feb39b51640 1 --2- 172.21.15.47:0/785750036 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb34162810 0x7feb3415d7d0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7feb2c00cd20 tx=0x7feb2c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:18.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.838+0000 7feb29ffb640 1 -- 172.21.15.47:0/785750036 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2c003b90 con 0x7feb34162810 2024-04-26T23:20:18.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.838+0000 7feb29ffb640 1 -- 172.21.15.47:0/785750036 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb2c003d30 con 0x7feb34162810 2024-04-26T23:20:18.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.838+0000 7feb3b5b4640 1 -- 172.21.15.47:0/785750036 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb341e9340 con 0x7feb34162810 2024-04-26T23:20:18.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.838+0000 7feb29ffb640 1 -- 172.21.15.47:0/785750036 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2c004240 con 0x7feb34162810 2024-04-26T23:20:18.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.838+0000 7feb3b5b4640 1 -- 172.21.15.47:0/785750036 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb341e9900 con 0x7feb34162810 2024-04-26T23:20:18.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.839+0000 7feb3b5b4640 1 -- 172.21.15.47:0/785750036 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb340d2430 con 0x7feb34162810 2024-04-26T23:20:18.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.839+0000 7feb29ffb640 1 -- 172.21.15.47:0/785750036 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feb2c0043e0 con 0x7feb34162810 2024-04-26T23:20:18.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.840+0000 7feb29ffb640 1 --2- 172.21.15.47:0/785750036 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7feb1806c8f0 0x7feb1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:18.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.840+0000 7feb29ffb640 1 -- 172.21.15.47:0/785750036 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5249+0+0 (secure 0 0 0) 0x7feb2c08c3d0 con 0x7feb34162810 2024-04-26T23:20:18.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.840+0000 7feb39350640 1 --2- 172.21.15.47:0/785750036 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7feb1806c8f0 0x7feb1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:18.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.840+0000 7feb39350640 1 --2- 172.21.15.47:0/785750036 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7feb1806c8f0 0x7feb1806edb0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7feb30034000 tx=0x7feb3002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:18.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:18.844+0000 7feb29ffb640 1 -- 172.21.15.47:0/785750036 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7feb2c01d340 con 0x7feb34162810 2024-04-26T23:20:19.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.031+0000 7feb3b5b4640 1 -- 172.21.15.47:0/785750036 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feb34075300 con 0x7feb34162810 2024-04-26T23:20:19.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.032+0000 7feb29ffb640 1 -- 172.21.15.47:0/785750036 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) ==== 74+0+13128 (secure 0 0 0) 0x7feb2c05a9c0 con 0x7feb34162810 2024-04-26T23:20:19.033 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:20:19.033 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":50,"fsid":"399006d3-4db1-43e6-ba81-a79a276eabfd","created":"2024-04-26T23:18:06.731211+0000","modified":"2024-04-26T23:20:18.318441+0000","last_up_change":"2024-04-26T23:18:17.808736+0000","last_in_change":"2024-04-26T23:18:08.945859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:18:16.937492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ee091567-fea0-4a14-bf4e-0e16c20bb2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6819","nonce":326520779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6821","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6823","nonce":326520779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6829","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6831","nonce":326520779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6825","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6827","nonce":326520779}]},"public_addr":"172.21.15.47:6819/326520779","cluster_addr":"172.21.15.47:6823/326520779","heartbeat_back_addr":"172.21.15.47:6831/326520779","heartbeat_front_addr":"172.21.15.47:6827/326520779","state":["exists","up"]},{"osd":1,"uuid":"c14440a0-bba5-4931-a6db-b736c08e869b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6809","nonce":1912610931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6811","nonce":1912610931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6815","nonce":1912610931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6813","nonce":1912610931}]},"public_addr":"172.21.15.47:6809/1912610931","cluster_addr":"172.21.15.47:6811/1912610931","heartbeat_back_addr":"172.21.15.47:6815/1912610931","heartbeat_front_addr":"172.21.15.47:6813/1912610931","state":["exists","up"]},{"osd":2,"uuid":"3940f8de-c9b2-4cd3-a911-9c301729eb30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6801","nonce":2569869305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6803","nonce":2569869305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6807","nonce":2569869305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6805","nonce":2569869305}]},"public_addr":"172.21.15.47:6801/2569869305","cluster_addr":"172.21.15.47:6803/2569869305","heartbeat_back_addr":"172.21.15.47:6807/2569869305","heartbeat_front_addr":"172.21.15.47:6805/2569869305","state":["exists","up"]},{"osd":3,"uuid":"1286a67e-8958-426a-8c2e-bc85498c619c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6818","nonce":2791411683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6822","nonce":2791411683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6830","nonce":2791411683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6826","nonce":2791411683}]},"public_addr":"172.21.15.47:6818/2791411683","cluster_addr":"172.21.15.47:6822/2791411683","heartbeat_back_addr":"172.21.15.47:6830/2791411683","heartbeat_front_addr":"172.21.15.47:6826/2791411683","state":["exists","up"]},{"osd":4,"uuid":"d8ee2c20-e5b7-46c2-9999-b234070126cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6809","nonce":441313104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6811","nonce":441313104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6815","nonce":441313104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6813","nonce":441313104}]},"public_addr":"172.21.15.203:6809/441313104","cluster_addr":"172.21.15.203:6811/441313104","heartbeat_back_addr":"172.21.15.203:6815/441313104","heartbeat_front_addr":"172.21.15.203:6813/441313104","state":["exists","up"]},{"osd":5,"uuid":"b453a8f0-f324-4840-9ae0-d67bdadb9f8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6801","nonce":56838444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6803","nonce":56838444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6807","nonce":56838444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6805","nonce":56838444}]},"public_addr":"172.21.15.203:6801/56838444","cluster_addr":"172.21.15.203:6803/56838444","heartbeat_back_addr":"172.21.15.203:6807/56838444","heartbeat_front_addr":"172.21.15.203:6805/56838444","state":["exists","up"]},{"osd":6,"uuid":"a6d31951-2d4b-4e8d-a510-6283b01dc158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6825","nonce":2738517080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6827","nonce":2738517080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6831","nonce":2738517080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6829","nonce":2738517080}]},"public_addr":"172.21.15.203:6825/2738517080","cluster_addr":"172.21.15.203:6827/2738517080","heartbeat_back_addr":"172.21.15.203:6831/2738517080","heartbeat_front_addr":"172.21.15.203:6829/2738517080","state":["exists","up"]},{"osd":7,"uuid":"7119b8c0-fe96-402d-9e75-7b509d9d44d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6817","nonce":1315565058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6819","nonce":1315565058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6823","nonce":1315565058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6821","nonce":1315565058}]},"public_addr":"172.21.15.203:6817/1315565058","cluster_addr":"172.21.15.203:6819/1315565058","heartbeat_back_addr":"172.21.15.203:6823/1315565058","heartbeat_front_addr":"172.21.15.203:6821/1315565058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.121921+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.137494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.084814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.111593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.072430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.125066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.097867+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.057061+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:20:19.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.033+0000 7feb3b5b4640 1 -- 172.21.15.47:0/785750036 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7feb1806c8f0 msgr2=0x7feb1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:19.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.033+0000 7feb3b5b4640 1 --2- 172.21.15.47:0/785750036 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7feb1806c8f0 0x7feb1806edb0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7feb30034000 tx=0x7feb3002e000 comp rx=0 tx=0).stop 2024-04-26T23:20:19.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.033+0000 7feb3b5b4640 1 -- 172.21.15.47:0/785750036 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb34162810 msgr2=0x7feb3415d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:19.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.033+0000 7feb3b5b4640 1 --2- 172.21.15.47:0/785750036 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb34162810 0x7feb3415d7d0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7feb2c00cd20 tx=0x7feb2c00a660 comp rx=0 tx=0).stop 2024-04-26T23:20:19.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.033+0000 7feb3b5b4640 1 -- 172.21.15.47:0/785750036 shutdown_connections 2024-04-26T23:20:19.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.033+0000 7feb3b5b4640 1 --2- 172.21.15.47:0/785750036 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7feb1806c8f0 0x7feb1806edb0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:19.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.033+0000 7feb3b5b4640 1 --2- 172.21.15.47:0/785750036 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb34162810 0x7feb3415d7d0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:19.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.033+0000 7feb3b5b4640 1 --2- 172.21.15.47:0/785750036 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb340cd930 0x7feb340c9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:19.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.033+0000 7feb3b5b4640 1 --2- 172.21.15.47:0/785750036 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb340b2340 0x7feb34158320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:19.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.033+0000 7feb3b5b4640 1 -- 172.21.15.47:0/785750036 >> 172.21.15.47:0/785750036 conn(0x7feb340d2da0 msgr2=0x7feb340b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:19.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.034+0000 7feb3b5b4640 1 -- 172.21.15.47:0/785750036 shutdown_connections 2024-04-26T23:20:19.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.034+0000 7feb3b5b4640 1 -- 172.21.15.47:0/785750036 wait complete. 2024-04-26T23:20:19.045 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:18:16.937492+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:20:19.046 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:20:19.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.148+0000 7f9dca987640 1 Processor -- start 2024-04-26T23:20:19.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.148+0000 7f9dca987640 1 -- start start 2024-04-26T23:20:19.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.149+0000 7f9dca987640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9dc40cfde0 0x7f9dc40d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:19.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.149+0000 7f9dca987640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9dc4070350 0x7f9dc4070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:19.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.149+0000 7f9dca987640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9dc4071090 0x7f9dc415e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:19.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.149+0000 7f9dca987640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f9dc4071500 con 0x7f9dc4070350 2024-04-26T23:20:19.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.149+0000 7f9dca987640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f9dc40afab0 con 0x7f9dc4071090 2024-04-26T23:20:19.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.149+0000 7f9dca987640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f9dc406dd20 con 0x7f9dc40cfde0 2024-04-26T23:20:19.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.149+0000 7f9dc37fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9dc4070350 0x7f9dc4070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:19.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.149+0000 7f9dc37fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9dc4070350 0x7f9dc4070b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46532/0 (socket says 172.21.15.47:46532) 2024-04-26T23:20:19.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.149+0000 7f9dc37fe640 1 -- 172.21.15.47:0/2880764971 learned_addr learned my addr 172.21.15.47:0/2880764971 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:19.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.149+0000 7f9dc3fff640 1 --2- 172.21.15.47:0/2880764971 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9dc40cfde0 0x7f9dc40d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:19.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.149+0000 7f9dc8f24640 1 --2- 172.21.15.47:0/2880764971 >> v2:172.21.15.203:3300/0 conn(0x7f9dc4071090 0x7f9dc415e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:19.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.149+0000 7f9dc37fe640 1 -- 172.21.15.47:0/2880764971 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9dc40cfde0 msgr2=0x7f9dc40d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:19.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.149+0000 7f9dc37fe640 1 --2- 172.21.15.47:0/2880764971 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9dc40cfde0 0x7f9dc40d1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:19.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.149+0000 7f9dc37fe640 1 -- 172.21.15.47:0/2880764971 >> v2:172.21.15.203:3300/0 conn(0x7f9dc4071090 msgr2=0x7f9dc415e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:19.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.149+0000 7f9dc37fe640 1 --2- 172.21.15.47:0/2880764971 >> v2:172.21.15.203:3300/0 conn(0x7f9dc4071090 0x7f9dc415e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:19.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.149+0000 7f9dc37fe640 1 -- 172.21.15.47:0/2880764971 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dc40d2390 con 0x7f9dc4070350 2024-04-26T23:20:19.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.149+0000 7f9dc8f24640 1 --2- 172.21.15.47:0/2880764971 >> v2:172.21.15.203:3300/0 conn(0x7f9dc4071090 0x7f9dc415e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:19.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.150+0000 7f9dc37fe640 1 --2- 172.21.15.47:0/2880764971 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9dc4070350 0x7f9dc4070b50 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f9dc40732e0 tx=0x7f9db402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc25d982e1424ac3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:19.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.150+0000 7f9dc2ffd640 1 -- 172.21.15.47:0/2880764971 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db4037070 con 0x7f9dc4070350 2024-04-26T23:20:19.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.150+0000 7f9dc2ffd640 1 -- 172.21.15.47:0/2880764971 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9db4032440 con 0x7f9dc4070350 2024-04-26T23:20:19.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.150+0000 7f9dc2ffd640 1 -- 172.21.15.47:0/2880764971 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db4032810 con 0x7f9dc4070350 2024-04-26T23:20:19.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.151+0000 7f9dca987640 1 -- 172.21.15.47:0/2880764971 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9dc4070350 msgr2=0x7f9dc4070b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:19.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.151+0000 7f9dca987640 1 --2- 172.21.15.47:0/2880764971 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9dc4070350 0x7f9dc4070b50 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f9dc40732e0 tx=0x7f9db402a140 comp rx=0 tx=0).stop 2024-04-26T23:20:19.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.151+0000 7f9dca987640 1 -- 172.21.15.47:0/2880764971 shutdown_connections 2024-04-26T23:20:19.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.151+0000 7f9dca987640 1 --2- 172.21.15.47:0/2880764971 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9dc4070350 0x7f9dc4070b50 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:19.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.151+0000 7f9dca987640 1 --2- 172.21.15.47:0/2880764971 >> v2:172.21.15.203:3300/0 conn(0x7f9dc4071090 0x7f9dc415e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:19.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.151+0000 7f9dca987640 1 --2- 172.21.15.47:0/2880764971 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9dc40cfde0 0x7f9dc40d1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:19.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.151+0000 7f9dca987640 1 -- 172.21.15.47:0/2880764971 >> 172.21.15.47:0/2880764971 conn(0x7f9dc40d2d80 msgr2=0x7f9dc40d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:19.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.151+0000 7f9dca987640 1 -- 172.21.15.47:0/2880764971 shutdown_connections 2024-04-26T23:20:19.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.151+0000 7f9dca987640 1 -- 172.21.15.47:0/2880764971 wait complete. 2024-04-26T23:20:19.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.152+0000 7f9dca987640 1 Processor -- start 2024-04-26T23:20:19.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.152+0000 7f9dca987640 1 -- start start 2024-04-26T23:20:19.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.153+0000 7f9dca987640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9dc4070350 0x7f9dc4158f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:19.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.153+0000 7f9dca987640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9dc4071090 0x7f9dc415a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:19.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.153+0000 7f9dca987640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9dc40cfde0 0x7f9dc41f4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.153+0000 7f9dca987640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc406d9b0 con 0x7f9dc4071090 2024-04-26T23:20:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.153+0000 7f9dca987640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc406d6d0 con 0x7f9dc40cfde0 2024-04-26T23:20:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.153+0000 7f9dca987640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f9dc40d20a0 con 0x7f9dc4070350 2024-04-26T23:20:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.153+0000 7f9dc37fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9dc4071090 0x7f9dc415a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.153+0000 7f9dc37fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9dc4071090 0x7f9dc415a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46548/0 (socket says 172.21.15.47:46548) 2024-04-26T23:20:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.153+0000 7f9dc37fe640 1 -- 172.21.15.47:0/1603183986 learned_addr learned my addr 172.21.15.47:0/1603183986 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.153+0000 7f9dc3fff640 1 --2- 172.21.15.47:0/1603183986 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9dc4070350 0x7f9dc4158f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.153+0000 7f9dc8f24640 1 --2- 172.21.15.47:0/1603183986 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9dc40cfde0 0x7f9dc41f4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.153+0000 7f9dc37fe640 1 -- 172.21.15.47:0/1603183986 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9dc4070350 msgr2=0x7f9dc4158f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.153+0000 7f9dc37fe640 1 --2- 172.21.15.47:0/1603183986 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9dc4070350 0x7f9dc4158f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.153+0000 7f9dc37fe640 1 -- 172.21.15.47:0/1603183986 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9dc40cfde0 msgr2=0x7f9dc41f4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:19.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.153+0000 7f9dc37fe640 1 --2- 172.21.15.47:0/1603183986 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9dc40cfde0 0x7f9dc41f4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:19.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.154+0000 7f9dc37fe640 1 -- 172.21.15.47:0/1603183986 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dc415ae20 con 0x7f9dc4071090 2024-04-26T23:20:19.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.154+0000 7f9dc37fe640 1 --2- 172.21.15.47:0/1603183986 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9dc4071090 0x7f9dc415a560 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f9db402a180 tx=0x7f9db40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:19.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.154+0000 7f9dc0ff9640 1 -- 172.21.15.47:0/1603183986 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db4041020 con 0x7f9dc4071090 2024-04-26T23:20:19.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.154+0000 7f9dc0ff9640 1 -- 172.21.15.47:0/1603183986 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9db40306c0 con 0x7f9dc4071090 2024-04-26T23:20:19.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.154+0000 7f9dca987640 1 -- 172.21.15.47:0/1603183986 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dc415b100 con 0x7f9dc4071090 2024-04-26T23:20:19.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.154+0000 7f9dc0ff9640 1 -- 172.21.15.47:0/1603183986 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db4037040 con 0x7f9dc4071090 2024-04-26T23:20:19.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.154+0000 7f9dca987640 1 -- 172.21.15.47:0/1603183986 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dc415b6c0 con 0x7f9dc4071090 2024-04-26T23:20:19.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.155+0000 7f9dca987640 1 -- 172.21.15.47:0/1603183986 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dc40694f0 con 0x7f9dc4071090 2024-04-26T23:20:19.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.155+0000 7f9dc0ff9640 1 -- 172.21.15.47:0/1603183986 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9db4044020 con 0x7f9dc4071090 2024-04-26T23:20:19.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.156+0000 7f9dc0ff9640 1 --2- 172.21.15.47:0/1603183986 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9da406c8f0 0x7f9da406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:19.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.156+0000 7f9dc0ff9640 1 -- 172.21.15.47:0/1603183986 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5249+0+0 (secure 0 0 0) 0x7f9db40ac190 con 0x7f9dc4071090 2024-04-26T23:20:19.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.156+0000 7f9dc3fff640 1 --2- 172.21.15.47:0/1603183986 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9da406c8f0 0x7f9da406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:19.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.156+0000 7f9dc3fff640 1 --2- 172.21.15.47:0/1603183986 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9da406c8f0 0x7f9da406edb0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9dac006fd0 tx=0x7f9dac008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:19.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.160+0000 7f9dc0ff9640 1 -- 172.21.15.47:0/1603183986 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9db404b030 con 0x7f9dc4071090 2024-04-26T23:20:19.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.345+0000 7f9dca987640 1 -- 172.21.15.47:0/1603183986 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9dc40b2390 con 0x7f9dc4071090 2024-04-26T23:20:19.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.346+0000 7f9dc0ff9640 1 -- 172.21.15.47:0/1603183986 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) ==== 93+0+10 (secure 0 0 0) 0x7f9db407a780 con 0x7f9dc4071090 2024-04-26T23:20:19.347 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-26T23:20:19.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.348+0000 7f9dca987640 1 -- 172.21.15.47:0/1603183986 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9da406c8f0 msgr2=0x7f9da406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:19.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.348+0000 7f9dca987640 1 --2- 172.21.15.47:0/1603183986 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9da406c8f0 0x7f9da406edb0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9dac006fd0 tx=0x7f9dac008040 comp rx=0 tx=0).stop 2024-04-26T23:20:19.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.348+0000 7f9dca987640 1 -- 172.21.15.47:0/1603183986 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9dc4071090 msgr2=0x7f9dc415a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:19.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.348+0000 7f9dca987640 1 --2- 172.21.15.47:0/1603183986 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9dc4071090 0x7f9dc415a560 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f9db402a180 tx=0x7f9db40046c0 comp rx=0 tx=0).stop 2024-04-26T23:20:19.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.348+0000 7f9dca987640 1 -- 172.21.15.47:0/1603183986 shutdown_connections 2024-04-26T23:20:19.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.348+0000 7f9dca987640 1 --2- 172.21.15.47:0/1603183986 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9da406c8f0 0x7f9da406edb0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:19.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.348+0000 7f9dca987640 1 --2- 172.21.15.47:0/1603183986 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9dc40cfde0 0x7f9dc41f4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:19.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.348+0000 7f9dca987640 1 --2- 172.21.15.47:0/1603183986 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9dc4071090 0x7f9dc415a560 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:19.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.348+0000 7f9dca987640 1 --2- 172.21.15.47:0/1603183986 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9dc4070350 0x7f9dc4158f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:19.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.348+0000 7f9dca987640 1 -- 172.21.15.47:0/1603183986 >> 172.21.15.47:0/1603183986 conn(0x7f9dc40d2d80 msgr2=0x7f9dc40afc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:19.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.348+0000 7f9dca987640 1 -- 172.21.15.47:0/1603183986 shutdown_connections 2024-04-26T23:20:19.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.348+0000 7f9dca987640 1 -- 172.21.15.47:0/1603183986 wait complete. 2024-04-26T23:20:19.360 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T23:20:19.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.460+0000 7f5e7be07640 1 Processor -- start 2024-04-26T23:20:19.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.461+0000 7f5e7be07640 1 -- start start 2024-04-26T23:20:19.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.461+0000 7f5e7be07640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5e74162810 0x7f5e74070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:19.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.461+0000 7f5e7be07640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e740d1b30 0x7f5e740d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:19.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.462+0000 7f5e7be07640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e740d10d0 con 0x7f5e740b2340 2024-04-26T23:20:19.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.462+0000 7f5e7be07640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f5e740d0750 con 0x7f5e74162810 2024-04-26T23:20:19.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.462+0000 7f5e7be07640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f5e740d16a0 con 0x7f5e740d1b30 2024-04-26T23:20:19.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.462+0000 7f5e79ba3640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5e74162810 0x7f5e74070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:19.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.462+0000 7f5e79ba3640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5e74162810 0x7f5e74070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46564/0 (socket says 172.21.15.47:46564) 2024-04-26T23:20:19.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.462+0000 7f5e793a2640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f5e740b2340 0x7f5e740cd3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35410/0 (socket says 172.21.15.47:35410) 2024-04-26T23:20:19.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.462+0000 7f5e79ba3640 1 -- 172.21.15.47:0/3806645243 learned_addr learned my addr 172.21.15.47:0/3806645243 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:19.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.462+0000 7f5e7a3a4640 1 --2- 172.21.15.47:0/3806645243 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e740d1b30 0x7f5e740d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:19.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.462+0000 7f5e79ba3640 1 -- 172.21.15.47:0/3806645243 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e740d1b30 msgr2=0x7f5e740d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:19.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.462+0000 7f5e79ba3640 1 --2- 172.21.15.47:0/3806645243 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e740d1b30 0x7f5e740d1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:19.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.462+0000 7f5e79ba3640 1 -- 172.21.15.47:0/3806645243 >> v1:172.21.15.47:6789/0 conn(0x7f5e740b2340 legacy=0x7f5e740cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:19.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.462+0000 7f5e79ba3640 1 -- 172.21.15.47:0/3806645243 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e740d1370 con 0x7f5e74162810 2024-04-26T23:20:19.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.462+0000 7f5e78ba1640 1 -- 172.21.15.47:0/3806645243 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3162368645 0 0) 0x7f5e740d10d0 con 0x7f5e740b2340 2024-04-26T23:20:19.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.462+0000 7f5e7a3a4640 1 --2- 172.21.15.47:0/3806645243 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e740d1b30 0x7f5e740d1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:20:19.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.463+0000 7f5e79ba3640 1 --2- 172.21.15.47:0/3806645243 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e74162810 0x7f5e74070350 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f5e70009870 tx=0x7f5e7002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcaad2533304f97a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:19.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.463+0000 7f5e78ba1640 1 -- 172.21.15.47:0/3806645243 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e70037070 con 0x7f5e74162810 2024-04-26T23:20:19.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.463+0000 7f5e78ba1640 1 -- 172.21.15.47:0/3806645243 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e70032440 con 0x7f5e74162810 2024-04-26T23:20:19.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.463+0000 7f5e78ba1640 1 -- 172.21.15.47:0/3806645243 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e70032810 con 0x7f5e74162810 2024-04-26T23:20:19.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.463+0000 7f5e7be07640 1 -- 172.21.15.47:0/3806645243 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e74162810 msgr2=0x7f5e74070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:19.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.463+0000 7f5e7be07640 1 --2- 172.21.15.47:0/3806645243 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e74162810 0x7f5e74070350 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f5e70009870 tx=0x7f5e7002a270 comp rx=0 tx=0).stop 2024-04-26T23:20:19.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.463+0000 7f5e7be07640 1 -- 172.21.15.47:0/3806645243 shutdown_connections 2024-04-26T23:20:19.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.463+0000 7f5e7be07640 1 --2- 172.21.15.47:0/3806645243 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e740d1b30 0x7f5e740d1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:19.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.463+0000 7f5e7be07640 1 --2- 172.21.15.47:0/3806645243 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e74162810 0x7f5e74070350 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:19.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.463+0000 7f5e7be07640 1 -- 172.21.15.47:0/3806645243 >> 172.21.15.47:0/3806645243 conn(0x7f5e740d2da0 msgr2=0x7f5e740d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:19.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.463+0000 7f5e7be07640 1 -- 172.21.15.47:0/3806645243 shutdown_connections 2024-04-26T23:20:19.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.464+0000 7f5e7be07640 1 -- 172.21.15.47:0/3806645243 wait complete. 2024-04-26T23:20:19.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.465+0000 7f5e7be07640 1 Processor -- start 2024-04-26T23:20:19.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.465+0000 7f5e7be07640 1 -- start start 2024-04-26T23:20:19.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.465+0000 7f5e7be07640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e740b2340 0x7f5e74161340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:19.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.465+0000 7f5e7be07640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e740d1b30 0x7f5e7415ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:19.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.465+0000 7f5e7be07640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e74162810 0x7f5e740cf410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:19.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.465+0000 7f5e7be07640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5e74070890 con 0x7f5e740d1b30 2024-04-26T23:20:19.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.465+0000 7f5e7be07640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5e740d0470 con 0x7f5e740b2340 2024-04-26T23:20:19.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.465+0000 7f5e7be07640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f5e740cdb00 con 0x7f5e74162810 2024-04-26T23:20:19.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.465+0000 7f5e793a2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e740d1b30 0x7f5e7415ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:19.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.466+0000 7f5e793a2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e740d1b30 0x7f5e7415ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46580/0 (socket says 172.21.15.47:46580) 2024-04-26T23:20:19.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.466+0000 7f5e793a2640 1 -- 172.21.15.47:0/16207083 learned_addr learned my addr 172.21.15.47:0/16207083 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:19.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.466+0000 7f5e7a3a4640 1 --2- 172.21.15.47:0/16207083 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e74162810 0x7f5e740cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:19.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.466+0000 7f5e79ba3640 1 --2- 172.21.15.47:0/16207083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e740b2340 0x7f5e74161340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:19.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.466+0000 7f5e793a2640 1 -- 172.21.15.47:0/16207083 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e74162810 msgr2=0x7f5e740cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:19.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.466+0000 7f5e793a2640 1 --2- 172.21.15.47:0/16207083 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e74162810 0x7f5e740cf410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:19.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.466+0000 7f5e793a2640 1 -- 172.21.15.47:0/16207083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e740b2340 msgr2=0x7f5e74161340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:19.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.466+0000 7f5e793a2640 1 --2- 172.21.15.47:0/16207083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e740b2340 0x7f5e74161340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:19.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.466+0000 7f5e793a2640 1 -- 172.21.15.47:0/16207083 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e740cac60 con 0x7f5e740d1b30 2024-04-26T23:20:19.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.466+0000 7f5e793a2640 1 --2- 172.21.15.47:0/16207083 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e740d1b30 0x7f5e7415ec80 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f5e6400c4d0 tx=0x7f5e6400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:19.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.466+0000 7f5e6a7fc640 1 -- 172.21.15.47:0/16207083 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e64018070 con 0x7f5e740d1b30 2024-04-26T23:20:19.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.466+0000 7f5e6a7fc640 1 -- 172.21.15.47:0/16207083 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e64007500 con 0x7f5e740d1b30 2024-04-26T23:20:19.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.466+0000 7f5e7be07640 1 -- 172.21.15.47:0/16207083 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e740caf40 con 0x7f5e740d1b30 2024-04-26T23:20:19.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.466+0000 7f5e6a7fc640 1 -- 172.21.15.47:0/16207083 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e64004a20 con 0x7f5e740d1b30 2024-04-26T23:20:19.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.466+0000 7f5e7be07640 1 -- 172.21.15.47:0/16207083 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e740cb500 con 0x7f5e740d1b30 2024-04-26T23:20:19.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.467+0000 7f5e7be07640 1 -- 172.21.15.47:0/16207083 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e740694f0 con 0x7f5e740d1b30 2024-04-26T23:20:19.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.468+0000 7f5e6a7fc640 1 -- 172.21.15.47:0/16207083 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5e6401b070 con 0x7f5e740d1b30 2024-04-26T23:20:19.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.468+0000 7f5e6a7fc640 1 --2- 172.21.15.47:0/16207083 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f5e5806c920 0x7f5e5806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:19.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.468+0000 7f5e6a7fc640 1 -- 172.21.15.47:0/16207083 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5249+0+0 (secure 0 0 0) 0x7f5e6408acd0 con 0x7f5e740d1b30 2024-04-26T23:20:19.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.468+0000 7f5e79ba3640 1 --2- 172.21.15.47:0/16207083 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f5e5806c920 0x7f5e5806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:19.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.469+0000 7f5e79ba3640 1 --2- 172.21.15.47:0/16207083 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f5e5806c920 0x7f5e5806ede0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5e70034000 tx=0x7f5e7002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:19.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.472+0000 7f5e6a7fc640 1 -- 172.21.15.47:0/16207083 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5e640592c0 con 0x7f5e740d1b30 2024-04-26T23:20:19.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.658+0000 7f5e7be07640 1 -- 172.21.15.47:0/16207083 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f5e740b18b0 con 0x7f5e740d1b30 2024-04-26T23:20:19.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.658+0000 7f5e6a7fc640 1 -- 172.21.15.47:0/16207083 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v50) ==== 84+0+65 (secure 0 0 0) 0x7f5e64058bc0 con 0x7f5e740d1b30 2024-04-26T23:20:19.659 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:20:19.660 INFO:teuthology.orchestra.run.smithi047.stdout:[ 2024-04-26T23:20:19.660 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-26T23:20:19.660 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 1, 2024-04-26T23:20:19.660 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": ".mgr" 2024-04-26T23:20:19.660 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-26T23:20:19.660 INFO:teuthology.orchestra.run.smithi047.stdout:] 2024-04-26T23:20:19.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.660+0000 7f5e7be07640 1 -- 172.21.15.47:0/16207083 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f5e5806c920 msgr2=0x7f5e5806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:19.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.660+0000 7f5e7be07640 1 --2- 172.21.15.47:0/16207083 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f5e5806c920 0x7f5e5806ede0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5e70034000 tx=0x7f5e7002e000 comp rx=0 tx=0).stop 2024-04-26T23:20:19.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.660+0000 7f5e7be07640 1 -- 172.21.15.47:0/16207083 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e740d1b30 msgr2=0x7f5e7415ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:19.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.660+0000 7f5e7be07640 1 --2- 172.21.15.47:0/16207083 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e740d1b30 0x7f5e7415ec80 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f5e6400c4d0 tx=0x7f5e6400c990 comp rx=0 tx=0).stop 2024-04-26T23:20:19.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.660+0000 7f5e7be07640 1 -- 172.21.15.47:0/16207083 shutdown_connections 2024-04-26T23:20:19.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.660+0000 7f5e7be07640 1 --2- 172.21.15.47:0/16207083 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f5e5806c920 0x7f5e5806ede0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:19.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.660+0000 7f5e7be07640 1 --2- 172.21.15.47:0/16207083 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e74162810 0x7f5e740cf410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:19.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.660+0000 7f5e7be07640 1 --2- 172.21.15.47:0/16207083 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e740d1b30 0x7f5e7415ec80 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:19.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.660+0000 7f5e7be07640 1 --2- 172.21.15.47:0/16207083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e740b2340 0x7f5e74161340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:19.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.660+0000 7f5e7be07640 1 -- 172.21.15.47:0/16207083 >> 172.21.15.47:0/16207083 conn(0x7f5e740d2da0 msgr2=0x7f5e740d20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:19.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.660+0000 7f5e7be07640 1 -- 172.21.15.47:0/16207083 shutdown_connections 2024-04-26T23:20:19.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.660+0000 7f5e7be07640 1 -- 172.21.15.47:0/16207083 wait complete. 2024-04-26T23:20:19.672 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T23:20:19.672 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T23:20:19.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.776+0000 7f9f916fc640 1 Processor -- start 2024-04-26T23:20:19.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.776+0000 7f9f916fc640 1 -- start start 2024-04-26T23:20:19.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.777+0000 7f9f916fc640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9f8c163820 0x7f9f8c162c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:19.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.777+0000 7f9f916fc640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9f8c163150 0x7f9f8c0b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:19.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.777+0000 7f9f916fc640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f8c156550 con 0x7f9f8c156170 2024-04-26T23:20:19.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.777+0000 7f9f916fc640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f9f8c1635a0 con 0x7f9f8c163150 2024-04-26T23:20:19.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.777+0000 7f9f916fc640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f9f8c0b48d0 con 0x7f9f8c163820 2024-04-26T23:20:19.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.777+0000 7f9f8affd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9f8c156170 0x7f9f8c070170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35726/0 (socket says 172.21.15.47:35726) 2024-04-26T23:20:19.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.777+0000 7f9f8affd640 1 -- 172.21.15.47:0/1428246287 learned_addr learned my addr 172.21.15.47:0/1428246287 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:19.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.777+0000 7f9f8b7fe640 1 --2- 172.21.15.47:0/1428246287 >> v2:172.21.15.47:3300/0 conn(0x7f9f8c163150 0x7f9f8c0b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:19.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.777+0000 7f9f89ffb640 1 -- 172.21.15.47:0/1428246287 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3662724068 0 0) 0x7f9f8c156550 con 0x7f9f8c156170 2024-04-26T23:20:19.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.777+0000 7f9f89ffb640 1 -- 172.21.15.47:0/1428246287 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f74003520 con 0x7f9f8c156170 2024-04-26T23:20:19.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.777+0000 7f9f8a7fc640 1 --2- 172.21.15.47:0/1428246287 >> v2:172.21.15.203:3300/0 conn(0x7f9f8c163820 0x7f9f8c162c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:19.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.777+0000 7f9f8b7fe640 1 -- 172.21.15.47:0/1428246287 >> v2:172.21.15.203:3300/0 conn(0x7f9f8c163820 msgr2=0x7f9f8c162c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:19.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.777+0000 7f9f8b7fe640 1 --2- 172.21.15.47:0/1428246287 >> v2:172.21.15.203:3300/0 conn(0x7f9f8c163820 0x7f9f8c162c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:19.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.777+0000 7f9f8b7fe640 1 -- 172.21.15.47:0/1428246287 >> v1:172.21.15.47:6789/0 conn(0x7f9f8c156170 legacy=0x7f9f8c070170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:19.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.777+0000 7f9f8b7fe640 1 -- 172.21.15.47:0/1428246287 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f8c070a50 con 0x7f9f8c163150 2024-04-26T23:20:19.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.777+0000 7f9f8a7fc640 1 --2- 172.21.15.47:0/1428246287 >> v2:172.21.15.203:3300/0 conn(0x7f9f8c163820 0x7f9f8c162c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:19.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.778+0000 7f9f8b7fe640 1 --2- 172.21.15.47:0/1428246287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f8c163150 0x7f9f8c0b4d60 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c0b6650 tx=0x7f9f8402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64745d4b9562499c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:19.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.778+0000 7f9f89ffb640 1 -- 172.21.15.47:0/1428246287 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f84007740 con 0x7f9f8c163150 2024-04-26T23:20:19.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.778+0000 7f9f89ffb640 1 -- 172.21.15.47:0/1428246287 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f840078e0 con 0x7f9f8c163150 2024-04-26T23:20:19.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.778+0000 7f9f89ffb640 1 -- 172.21.15.47:0/1428246287 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f84007cb0 con 0x7f9f8c163150 2024-04-26T23:20:19.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.778+0000 7f9f916fc640 1 -- 172.21.15.47:0/1428246287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f8c163150 msgr2=0x7f9f8c0b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:19.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.778+0000 7f9f916fc640 1 --2- 172.21.15.47:0/1428246287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f8c163150 0x7f9f8c0b4d60 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c0b6650 tx=0x7f9f8402a680 comp rx=0 tx=0).stop 2024-04-26T23:20:19.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.779+0000 7f9f916fc640 1 -- 172.21.15.47:0/1428246287 shutdown_connections 2024-04-26T23:20:19.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.779+0000 7f9f916fc640 1 --2- 172.21.15.47:0/1428246287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f8c163150 0x7f9f8c0b4d60 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:19.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.779+0000 7f9f916fc640 1 --2- 172.21.15.47:0/1428246287 >> v2:172.21.15.203:3300/0 conn(0x7f9f8c163820 0x7f9f8c162c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:19.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.779+0000 7f9f916fc640 1 -- 172.21.15.47:0/1428246287 >> 172.21.15.47:0/1428246287 conn(0x7f9f8c0b3360 msgr2=0x7f9f8c0b3770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:19.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.779+0000 7f9f916fc640 1 -- 172.21.15.47:0/1428246287 shutdown_connections 2024-04-26T23:20:19.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.779+0000 7f9f916fc640 1 -- 172.21.15.47:0/1428246287 wait complete. 2024-04-26T23:20:19.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.780+0000 7f9f916fc640 1 Processor -- start 2024-04-26T23:20:19.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.780+0000 7f9f916fc640 1 -- start start 2024-04-26T23:20:19.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.781+0000 7f9f916fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f8c156170 0x7f9f8c13fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:19.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.781+0000 7f9f916fc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f8c163150 0x7f9f8c14db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:19.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.781+0000 7f9f916fc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9f8c163820 0x7f9f8c14b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:19.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.781+0000 7f9f916fc640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8c0b52a0 con 0x7f9f8c156170 2024-04-26T23:20:19.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.781+0000 7f9f916fc640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8c071010 con 0x7f9f8c163150 2024-04-26T23:20:19.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.781+0000 7f9f916fc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f9f8c0b5420 con 0x7f9f8c163820 2024-04-26T23:20:19.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.781+0000 7f9f8affd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f8c156170 0x7f9f8c13fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:19.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.781+0000 7f9f8affd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f8c156170 0x7f9f8c13fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40638/0 (socket says 172.21.15.47:40638) 2024-04-26T23:20:19.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.781+0000 7f9f8affd640 1 -- 172.21.15.47:0/609929149 learned_addr learned my addr 172.21.15.47:0/609929149 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:19.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.781+0000 7f9f8b7fe640 1 --2- 172.21.15.47:0/609929149 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9f8c163820 0x7f9f8c14b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:19.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.781+0000 7f9f8affd640 1 -- 172.21.15.47:0/609929149 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9f8c163820 msgr2=0x7f9f8c14b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:19.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.781+0000 7f9f8a7fc640 1 --2- 172.21.15.47:0/609929149 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f8c163150 0x7f9f8c14db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:19.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.781+0000 7f9f8affd640 1 --2- 172.21.15.47:0/609929149 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9f8c163820 0x7f9f8c14b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:19.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.781+0000 7f9f8affd640 1 -- 172.21.15.47:0/609929149 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f8c163150 msgr2=0x7f9f8c14db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:19.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.781+0000 7f9f8affd640 1 --2- 172.21.15.47:0/609929149 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f8c163150 0x7f9f8c14db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:19.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.781+0000 7f9f8affd640 1 -- 172.21.15.47:0/609929149 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f8c1588a0 con 0x7f9f8c156170 2024-04-26T23:20:19.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.781+0000 7f9f8b7fe640 1 --2- 172.21.15.47:0/609929149 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9f8c163820 0x7f9f8c14b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:19.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.781+0000 7f9f8a7fc640 1 --2- 172.21.15.47:0/609929149 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f8c163150 0x7f9f8c14db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:19.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.781+0000 7f9f8affd640 1 --2- 172.21.15.47:0/609929149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f8c156170 0x7f9f8c13fbe0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c00c430 tx=0x7f9f7c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:19.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.781+0000 7f9f737fe640 1 -- 172.21.15.47:0/609929149 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c017070 con 0x7f9f8c156170 2024-04-26T23:20:19.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.781+0000 7f9f737fe640 1 -- 172.21.15.47:0/609929149 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f7c005ce0 con 0x7f9f8c156170 2024-04-26T23:20:19.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.781+0000 7f9f737fe640 1 -- 172.21.15.47:0/609929149 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c0072f0 con 0x7f9f8c156170 2024-04-26T23:20:19.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.781+0000 7f9f916fc640 1 -- 172.21.15.47:0/609929149 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f8c158b40 con 0x7f9f8c156170 2024-04-26T23:20:19.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.781+0000 7f9f916fc640 1 -- 172.21.15.47:0/609929149 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f8c159100 con 0x7f9f8c156170 2024-04-26T23:20:19.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.782+0000 7f9f916fc640 1 -- 172.21.15.47:0/609929149 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f8c069600 con 0x7f9f8c156170 2024-04-26T23:20:19.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.783+0000 7f9f737fe640 1 -- 172.21.15.47:0/609929149 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9f7c01a070 con 0x7f9f8c156170 2024-04-26T23:20:19.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.783+0000 7f9f737fe640 1 --2- 172.21.15.47:0/609929149 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9f7806c920 0x7f9f7806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:19.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.783+0000 7f9f737fe640 1 -- 172.21.15.47:0/609929149 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5249+0+0 (secure 0 0 0) 0x7f9f7c08ad70 con 0x7f9f8c156170 2024-04-26T23:20:19.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.784+0000 7f9f8a7fc640 1 --2- 172.21.15.47:0/609929149 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9f7806c920 0x7f9f7806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:19.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.784+0000 7f9f8a7fc640 1 --2- 172.21.15.47:0/609929149 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9f7806c920 0x7f9f7806ede0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9f80006fd0 tx=0x7f9f80008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:19.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.788+0000 7f9f737fe640 1 -- 172.21.15.47:0/609929149 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9f7c0593f0 con 0x7f9f8c156170 2024-04-26T23:20:19.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:19.971+0000 7f9f916fc640 1 -- 172.21.15.47:0/609929149 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f9f8c0cd740 con 0x7f9f8c156170 2024-04-26T23:20:20.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.325+0000 7f9f737fe640 1 -- 172.21.15.47:0/609929149 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v51) ==== 138+0+0 (secure 0 0 0) 0x7f9f7c058cf0 con 0x7f9f8c156170 2024-04-26T23:20:20.326 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' created 2024-04-26T23:20:20.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.329+0000 7f9f916fc640 1 -- 172.21.15.47:0/609929149 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9f7806c920 msgr2=0x7f9f7806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:20.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.329+0000 7f9f916fc640 1 --2- 172.21.15.47:0/609929149 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9f7806c920 0x7f9f7806ede0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9f80006fd0 tx=0x7f9f80008040 comp rx=0 tx=0).stop 2024-04-26T23:20:20.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.329+0000 7f9f916fc640 1 -- 172.21.15.47:0/609929149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f8c156170 msgr2=0x7f9f8c13fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:20.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.329+0000 7f9f916fc640 1 --2- 172.21.15.47:0/609929149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f8c156170 0x7f9f8c13fbe0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c00c430 tx=0x7f9f7c00c8f0 comp rx=0 tx=0).stop 2024-04-26T23:20:20.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.329+0000 7f9f916fc640 1 -- 172.21.15.47:0/609929149 shutdown_connections 2024-04-26T23:20:20.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.329+0000 7f9f916fc640 1 --2- 172.21.15.47:0/609929149 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9f7806c920 0x7f9f7806ede0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:20.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.329+0000 7f9f916fc640 1 --2- 172.21.15.47:0/609929149 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9f8c163820 0x7f9f8c14b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:20.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.329+0000 7f9f916fc640 1 --2- 172.21.15.47:0/609929149 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f8c163150 0x7f9f8c14db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:20.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.329+0000 7f9f916fc640 1 --2- 172.21.15.47:0/609929149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f8c156170 0x7f9f8c13fbe0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:20.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.329+0000 7f9f916fc640 1 -- 172.21.15.47:0/609929149 >> 172.21.15.47:0/609929149 conn(0x7f9f8c0b3360 msgr2=0x7f9f8c0aeb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:20.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.330+0000 7f9f916fc640 1 -- 172.21.15.47:0/609929149 shutdown_connections 2024-04-26T23:20:20.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.330+0000 7f9f916fc640 1 -- 172.21.15.47:0/609929149 wait complete. 2024-04-26T23:20:20.345 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T23:20:20.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.454+0000 7f5b9d76a640 1 Processor -- start 2024-04-26T23:20:20.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.454+0000 7f5b9d76a640 1 -- start start 2024-04-26T23:20:20.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.455+0000 7f5b9d76a640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5b98070930 0x7f5b98160ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:20.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.455+0000 7f5b9d76a640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b980b0990 con 0x7f5b9806c950 2024-04-26T23:20:20.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.455+0000 7f5b9d76a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b980b0c70 con 0x7f5b9815d100 2024-04-26T23:20:20.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.455+0000 7f5b9d76a640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f5b980b0f50 con 0x7f5b98070930 2024-04-26T23:20:20.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.455+0000 7f5b967fc640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f5b9806c950 0x7f5b98073160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35728/0 (socket says 172.21.15.47:35728) 2024-04-26T23:20:20.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.455+0000 7f5b967fc640 1 -- 172.21.15.47:0/4151239092 learned_addr learned my addr 172.21.15.47:0/4151239092 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:20.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.456+0000 7f5b977fe640 1 --2- 172.21.15.47:0/4151239092 >> v2:172.21.15.203:3300/0 conn(0x7f5b98070930 0x7f5b98160ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:20.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.456+0000 7f5b95ffb640 1 -- 172.21.15.47:0/4151239092 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3945988348 0 0) 0x7f5b980b0990 con 0x7f5b9806c950 2024-04-26T23:20:20.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.456+0000 7f5b95ffb640 1 -- 172.21.15.47:0/4151239092 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b84003520 con 0x7f5b9806c950 2024-04-26T23:20:20.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.456+0000 7f5b95ffb640 1 -- 172.21.15.47:0/4151239092 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1681876659 0 0) 0x7f5b84003520 con 0x7f5b9806c950 2024-04-26T23:20:20.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.456+0000 7f5b95ffb640 1 -- 172.21.15.47:0/4151239092 >> v2:172.21.15.203:3300/0 conn(0x7f5b98070930 msgr2=0x7f5b98160ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:20.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.456+0000 7f5b95ffb640 1 --2- 172.21.15.47:0/4151239092 >> v2:172.21.15.203:3300/0 conn(0x7f5b98070930 0x7f5b98160ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:20.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.456+0000 7f5b95ffb640 1 -- 172.21.15.47:0/4151239092 >> v1:172.21.15.203:6789/0 conn(0x7f5b9815d100 legacy=0x7f5b9806c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:20.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.456+0000 7f5b95ffb640 1 -- 172.21.15.47:0/4151239092 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b980b1230 con 0x7f5b9806c950 2024-04-26T23:20:20.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.456+0000 7f5b95ffb640 1 -- 172.21.15.47:0/4151239092 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7f5b80001670 con 0x7f5b9806c950 2024-04-26T23:20:20.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.456+0000 7f5b95ffb640 1 -- 172.21.15.47:0/4151239092 >> v1:172.21.15.47:6789/0 conn(0x7f5b9806c950 legacy=0x7f5b98073160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:20.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.456+0000 7f5b95ffb640 1 --2- 172.21.15.47:0/4151239092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b840039b0 0x7f5b84003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:20.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.456+0000 7f5b95ffb640 1 --2- 172.21.15.47:0/4151239092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b84004800 0x7f5b84004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:20.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.456+0000 7f5b95ffb640 1 --2- 172.21.15.47:0/4151239092 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b840051c0 0x7f5b8400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:20.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.456+0000 7f5b95ffb640 1 -- 172.21.15.47:0/4151239092 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5b980b0f50 con 0x7f5b84004800 2024-04-26T23:20:20.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.456+0000 7f5b95ffb640 1 -- 172.21.15.47:0/4151239092 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8400c060 con 0x7f5b840039b0 2024-04-26T23:20:20.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.456+0000 7f5b95ffb640 1 -- 172.21.15.47:0/4151239092 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f5b8400c260 con 0x7f5b840051c0 2024-04-26T23:20:20.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.456+0000 7f5b967fc640 1 --2- 172.21.15.47:0/4151239092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b84004800 0x7f5b84004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:20.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.456+0000 7f5b96ffd640 1 --2- 172.21.15.47:0/4151239092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b840039b0 0x7f5b84003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:20.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.456+0000 7f5b977fe640 1 --2- 172.21.15.47:0/4151239092 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b840051c0 0x7f5b8400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:20.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.456+0000 7f5b967fc640 1 -- 172.21.15.47:0/4151239092 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b840051c0 msgr2=0x7f5b8400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:20.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.456+0000 7f5b967fc640 1 --2- 172.21.15.47:0/4151239092 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b840051c0 0x7f5b8400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:20.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.456+0000 7f5b967fc640 1 -- 172.21.15.47:0/4151239092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b840039b0 msgr2=0x7f5b84003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:20.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.456+0000 7f5b967fc640 1 --2- 172.21.15.47:0/4151239092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b840039b0 0x7f5b84003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:20.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.456+0000 7f5b967fc640 1 -- 172.21.15.47:0/4151239092 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b84003520 con 0x7f5b84004800 2024-04-26T23:20:20.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.456+0000 7f5b977fe640 1 -- 172.21.15.47:0/4151239092 reap_dead start 2024-04-26T23:20:20.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.457+0000 7f5b967fc640 1 --2- 172.21.15.47:0/4151239092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b84004800 0x7f5b84004c80 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f5b8000c210 tx=0x7f5b84005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab0b19641f363f8b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:20.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.457+0000 7f5b95ffb640 1 -- 172.21.15.47:0/4151239092 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b80007d30 con 0x7f5b84004800 2024-04-26T23:20:20.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.457+0000 7f5b95ffb640 1 -- 172.21.15.47:0/4151239092 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b80005ad0 con 0x7f5b84004800 2024-04-26T23:20:20.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.457+0000 7f5b95ffb640 1 -- 172.21.15.47:0/4151239092 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b80007070 con 0x7f5b84004800 2024-04-26T23:20:20.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.458+0000 7f5b9d76a640 1 -- 172.21.15.47:0/4151239092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b84004800 msgr2=0x7f5b84004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:20.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.458+0000 7f5b9d76a640 1 --2- 172.21.15.47:0/4151239092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b84004800 0x7f5b84004c80 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f5b8000c210 tx=0x7f5b84005830 comp rx=0 tx=0).stop 2024-04-26T23:20:20.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.458+0000 7f5b9d76a640 1 -- 172.21.15.47:0/4151239092 shutdown_connections 2024-04-26T23:20:20.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.458+0000 7f5b9d76a640 1 --2- 172.21.15.47:0/4151239092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b84004800 0x7f5b84004c80 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:20.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.458+0000 7f5b9d76a640 1 -- 172.21.15.47:0/4151239092 >> 172.21.15.47:0/4151239092 conn(0x7f5b980b39b0 msgr2=0x7f5b980b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:20.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.458+0000 7f5b9d76a640 1 -- 172.21.15.47:0/4151239092 shutdown_connections 2024-04-26T23:20:20.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.458+0000 7f5b9d76a640 1 -- 172.21.15.47:0/4151239092 wait complete. 2024-04-26T23:20:20.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.459+0000 7f5b9d76a640 1 Processor -- start 2024-04-26T23:20:20.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.459+0000 7f5b9d76a640 1 -- start start 2024-04-26T23:20:20.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.460+0000 7f5b9d76a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b84004800 0x7f5b981f5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:20.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.460+0000 7f5b9d76a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b981568f0 0x7f5b981f55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:20.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.460+0000 7f5b9d76a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b9815f360 0x7f5b9815f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:20.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.460+0000 7f5b9d76a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5b98160d20 con 0x7f5b84004800 2024-04-26T23:20:20.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.460+0000 7f5b9d76a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5b98160ba0 con 0x7f5b981568f0 2024-04-26T23:20:20.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.460+0000 7f5b9d76a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f5b98160ea0 con 0x7f5b9815f360 2024-04-26T23:20:20.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.460+0000 7f5b96ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b84004800 0x7f5b981f5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:20.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.460+0000 7f5b96ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b84004800 0x7f5b981f5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40658/0 (socket says 172.21.15.47:40658) 2024-04-26T23:20:20.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.460+0000 7f5b96ffd640 1 -- 172.21.15.47:0/756904705 learned_addr learned my addr 172.21.15.47:0/756904705 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:20.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.460+0000 7f5b967fc640 1 --2- 172.21.15.47:0/756904705 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b981568f0 0x7f5b981f55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:20.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.460+0000 7f5b977fe640 1 --2- 172.21.15.47:0/756904705 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b9815f360 0x7f5b9815f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:20.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.460+0000 7f5b96ffd640 1 -- 172.21.15.47:0/756904705 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b9815f360 msgr2=0x7f5b9815f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:20.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.460+0000 7f5b96ffd640 1 --2- 172.21.15.47:0/756904705 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b9815f360 0x7f5b9815f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:20.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.460+0000 7f5b96ffd640 1 -- 172.21.15.47:0/756904705 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b981568f0 msgr2=0x7f5b981f55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:20.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.460+0000 7f5b96ffd640 1 --2- 172.21.15.47:0/756904705 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b981568f0 0x7f5b981f55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:20.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.460+0000 7f5b96ffd640 1 -- 172.21.15.47:0/756904705 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b981f5c10 con 0x7f5b84004800 2024-04-26T23:20:20.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.460+0000 7f5b977fe640 1 --2- 172.21.15.47:0/756904705 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b9815f360 0x7f5b9815f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:20:20.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.461+0000 7f5b96ffd640 1 --2- 172.21.15.47:0/756904705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b84004800 0x7f5b981f5080 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f5b8800f990 tx=0x7f5b8800fe50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:20.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.461+0000 7f5b8f7fe640 1 -- 172.21.15.47:0/756904705 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b88003b80 con 0x7f5b84004800 2024-04-26T23:20:20.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.461+0000 7f5b8f7fe640 1 -- 172.21.15.47:0/756904705 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b88018070 con 0x7f5b84004800 2024-04-26T23:20:20.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.461+0000 7f5b8f7fe640 1 -- 172.21.15.47:0/756904705 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8800d7d0 con 0x7f5b84004800 2024-04-26T23:20:20.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.461+0000 7f5b9d76a640 1 -- 172.21.15.47:0/756904705 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b981f5ef0 con 0x7f5b84004800 2024-04-26T23:20:20.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.461+0000 7f5b9d76a640 1 -- 172.21.15.47:0/756904705 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b9809d920 con 0x7f5b84004800 2024-04-26T23:20:20.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.462+0000 7f5b9d76a640 1 -- 172.21.15.47:0/756904705 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b9815fd20 con 0x7f5b84004800 2024-04-26T23:20:20.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.462+0000 7f5b8f7fe640 1 -- 172.21.15.47:0/756904705 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5b8800d970 con 0x7f5b84004800 2024-04-26T23:20:20.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.463+0000 7f5b8f7fe640 1 --2- 172.21.15.47:0/756904705 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f5b7806c920 0x7f5b7806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:20.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.463+0000 7f5b8f7fe640 1 -- 172.21.15.47:0/756904705 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5619+0+0 (secure 0 0 0) 0x7f5b8808ba90 con 0x7f5b84004800 2024-04-26T23:20:20.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.463+0000 7f5b967fc640 1 --2- 172.21.15.47:0/756904705 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f5b7806c920 0x7f5b7806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:20.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.464+0000 7f5b967fc640 1 --2- 172.21.15.47:0/756904705 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f5b7806c920 0x7f5b7806ede0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5b80032000 tx=0x7f5b8002d000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:20.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.467+0000 7f5b8f7fe640 1 -- 172.21.15.47:0/756904705 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5b88059fa0 con 0x7f5b84004800 2024-04-26T23:20:20.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:20.653+0000 7f5b9d76a640 1 -- 172.21.15.47:0/756904705 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f5b980694f0 con 0x7f5b84004800 2024-04-26T23:20:21.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.330+0000 7f5b8f7fe640 1 -- 172.21.15.47:0/756904705 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v52) ==== 170+0+0 (secure 0 0 0) 0x7f5b880598a0 con 0x7f5b84004800 2024-04-26T23:20:21.332 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' created 2024-04-26T23:20:21.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.335+0000 7f5b9d76a640 1 -- 172.21.15.47:0/756904705 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f5b7806c920 msgr2=0x7f5b7806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:21.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.335+0000 7f5b9d76a640 1 --2- 172.21.15.47:0/756904705 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f5b7806c920 0x7f5b7806ede0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5b80032000 tx=0x7f5b8002d000 comp rx=0 tx=0).stop 2024-04-26T23:20:21.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.335+0000 7f5b9d76a640 1 -- 172.21.15.47:0/756904705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b84004800 msgr2=0x7f5b981f5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:21.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.335+0000 7f5b9d76a640 1 --2- 172.21.15.47:0/756904705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b84004800 0x7f5b981f5080 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f5b8800f990 tx=0x7f5b8800fe50 comp rx=0 tx=0).stop 2024-04-26T23:20:21.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.335+0000 7f5b9d76a640 1 -- 172.21.15.47:0/756904705 shutdown_connections 2024-04-26T23:20:21.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.335+0000 7f5b9d76a640 1 --2- 172.21.15.47:0/756904705 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f5b7806c920 0x7f5b7806ede0 secure :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5b80032000 tx=0x7f5b8002d000 comp rx=0 tx=0).stop 2024-04-26T23:20:21.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.335+0000 7f5b9d76a640 1 --2- 172.21.15.47:0/756904705 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b9815f360 0x7f5b9815f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:21.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.335+0000 7f5b9d76a640 1 --2- 172.21.15.47:0/756904705 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b981568f0 0x7f5b981f55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:21.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.335+0000 7f5b9d76a640 1 --2- 172.21.15.47:0/756904705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b84004800 0x7f5b981f5080 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:21.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.335+0000 7f5b9d76a640 1 -- 172.21.15.47:0/756904705 >> 172.21.15.47:0/756904705 conn(0x7f5b980b39b0 msgr2=0x7f5b980aed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:21.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.337+0000 7f5b9d76a640 1 -- 172.21.15.47:0/756904705 shutdown_connections 2024-04-26T23:20:21.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.337+0000 7f5b9d76a640 1 -- 172.21.15.47:0/756904705 wait complete. 2024-04-26T23:20:21.364 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T23:20:21.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.473+0000 7fd963fff640 1 Processor -- start 2024-04-26T23:20:21.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.473+0000 7fd963fff640 1 -- start start 2024-04-26T23:20:21.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.473+0000 7fd963fff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd9640b5120 0x7fd9640b5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:21.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.473+0000 7fd963fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd9640b5500 0x7fd9640b6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:21.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.473+0000 7fd963fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9640b59b0 0x7fd9640b6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:21.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.473+0000 7fd963fff640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fd9640d03b0 con 0x7fd9640b5500 2024-04-26T23:20:21.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.474+0000 7fd963fff640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fd9640d3dc0 con 0x7fd9640b5120 2024-04-26T23:20:21.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.474+0000 7fd963fff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd9640b75b0 con 0x7fd9640b59b0 2024-04-26T23:20:21.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.474+0000 7fd9627fc640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd9640b5500 0x7fd9640b6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:21.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.474+0000 7fd9627fc640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd9640b5500 0x7fd9640b6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40668/0 (socket says 172.21.15.47:40668) 2024-04-26T23:20:21.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.474+0000 7fd9627fc640 1 -- 172.21.15.47:0/4012457031 learned_addr learned my addr 172.21.15.47:0/4012457031 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:21.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.474+0000 7fd962ffd640 1 --2- 172.21.15.47:0/4012457031 >> v2:172.21.15.203:3300/0 conn(0x7fd9640b5120 0x7fd9640b5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:21.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.474+0000 7fd9637fe640 1 --2- 172.21.15.47:0/4012457031 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9640b59b0 0x7fd9640b6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:21.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.474+0000 7fd9627fc640 1 -- 172.21.15.47:0/4012457031 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9640b59b0 msgr2=0x7fd9640b6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:21.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.474+0000 7fd9627fc640 1 --2- 172.21.15.47:0/4012457031 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9640b59b0 0x7fd9640b6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:21.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.474+0000 7fd9627fc640 1 -- 172.21.15.47:0/4012457031 >> v2:172.21.15.203:3300/0 conn(0x7fd9640b5120 msgr2=0x7fd9640b5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:21.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.474+0000 7fd9627fc640 1 --2- 172.21.15.47:0/4012457031 >> v2:172.21.15.203:3300/0 conn(0x7fd9640b5120 0x7fd9640b5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:21.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.474+0000 7fd9627fc640 1 -- 172.21.15.47:0/4012457031 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9640d0600 con 0x7fd9640b5500 2024-04-26T23:20:21.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.475+0000 7fd9627fc640 1 --2- 172.21.15.47:0/4012457031 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9640b5500 0x7fd9640b6830 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd94c009870 tx=0x7fd94c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1f9737e9837e1d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:21.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.475+0000 7fd961ffb640 1 -- 172.21.15.47:0/4012457031 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94c037070 con 0x7fd9640b5500 2024-04-26T23:20:21.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.475+0000 7fd961ffb640 1 -- 172.21.15.47:0/4012457031 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd94c032440 con 0x7fd9640b5500 2024-04-26T23:20:21.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.475+0000 7fd961ffb640 1 -- 172.21.15.47:0/4012457031 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94c032810 con 0x7fd9640b5500 2024-04-26T23:20:21.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.475+0000 7fd963fff640 1 -- 172.21.15.47:0/4012457031 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9640b5500 msgr2=0x7fd9640b6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:21.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.475+0000 7fd963fff640 1 --2- 172.21.15.47:0/4012457031 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9640b5500 0x7fd9640b6830 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd94c009870 tx=0x7fd94c02a180 comp rx=0 tx=0).stop 2024-04-26T23:20:21.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.475+0000 7fd963fff640 1 -- 172.21.15.47:0/4012457031 shutdown_connections 2024-04-26T23:20:21.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.475+0000 7fd963fff640 1 --2- 172.21.15.47:0/4012457031 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9640b5500 0x7fd9640b6830 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:21.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.475+0000 7fd963fff640 1 --2- 172.21.15.47:0/4012457031 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9640b59b0 0x7fd9640b6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:21.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.475+0000 7fd963fff640 1 --2- 172.21.15.47:0/4012457031 >> v2:172.21.15.203:3300/0 conn(0x7fd9640b5120 0x7fd9640b5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:21.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.475+0000 7fd963fff640 1 -- 172.21.15.47:0/4012457031 >> 172.21.15.47:0/4012457031 conn(0x7fd9640d4db0 msgr2=0x7fd9640d51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:21.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.475+0000 7fd963fff640 1 -- 172.21.15.47:0/4012457031 shutdown_connections 2024-04-26T23:20:21.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.475+0000 7fd963fff640 1 -- 172.21.15.47:0/4012457031 wait complete. 2024-04-26T23:20:21.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.477+0000 7fd963fff640 1 Processor -- start 2024-04-26T23:20:21.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.477+0000 7fd963fff640 1 -- start start 2024-04-26T23:20:21.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.478+0000 7fd963fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9640b5120 0x7fd9641589a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:21.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.478+0000 7fd963fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9640b5500 0x7fd96415f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:21.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.478+0000 7fd963fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9640b59b0 0x7fd96415f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:21.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.478+0000 7fd963fff640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd9640b72b0 con 0x7fd9640b5120 2024-04-26T23:20:21.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.478+0000 7fd963fff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd9640d0bc0 con 0x7fd9640b59b0 2024-04-26T23:20:21.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.478+0000 7fd963fff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd9640d2b00 con 0x7fd9640b5500 2024-04-26T23:20:21.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.478+0000 7fd962ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9640b5120 0x7fd9641589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:21.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.478+0000 7fd962ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9640b5120 0x7fd9641589a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40674/0 (socket says 172.21.15.47:40674) 2024-04-26T23:20:21.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.478+0000 7fd962ffd640 1 -- 172.21.15.47:0/3607901676 learned_addr learned my addr 172.21.15.47:0/3607901676 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:21.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.478+0000 7fd9627fc640 1 --2- 172.21.15.47:0/3607901676 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9640b5500 0x7fd96415f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:21.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.478+0000 7fd9637fe640 1 --2- 172.21.15.47:0/3607901676 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9640b59b0 0x7fd96415f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:21.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.478+0000 7fd962ffd640 1 -- 172.21.15.47:0/3607901676 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9640b5500 msgr2=0x7fd96415f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:21.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.478+0000 7fd962ffd640 1 --2- 172.21.15.47:0/3607901676 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9640b5500 0x7fd96415f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:21.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.478+0000 7fd962ffd640 1 -- 172.21.15.47:0/3607901676 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9640b59b0 msgr2=0x7fd96415f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:21.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.478+0000 7fd962ffd640 1 --2- 172.21.15.47:0/3607901676 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9640b59b0 0x7fd96415f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:21.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.478+0000 7fd962ffd640 1 -- 172.21.15.47:0/3607901676 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9641f4c90 con 0x7fd9640b5120 2024-04-26T23:20:21.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.478+0000 7fd962ffd640 1 --2- 172.21.15.47:0/3607901676 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9640b5120 0x7fd9641589a0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd954007e60 tx=0x7fd95400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:21.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.479+0000 7fd95bfff640 1 -- 172.21.15.47:0/3607901676 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd954014620 con 0x7fd9640b5120 2024-04-26T23:20:21.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.479+0000 7fd95bfff640 1 -- 172.21.15.47:0/3607901676 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd954019070 con 0x7fd9640b5120 2024-04-26T23:20:21.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.479+0000 7fd95bfff640 1 -- 172.21.15.47:0/3607901676 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd954014a00 con 0x7fd9640b5120 2024-04-26T23:20:21.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.479+0000 7fd963fff640 1 -- 172.21.15.47:0/3607901676 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9641f4f70 con 0x7fd9640b5120 2024-04-26T23:20:21.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.479+0000 7fd963fff640 1 -- 172.21.15.47:0/3607901676 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9641f5530 con 0x7fd9640b5120 2024-04-26T23:20:21.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.480+0000 7fd963fff640 1 -- 172.21.15.47:0/3607901676 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd964069370 con 0x7fd9640b5120 2024-04-26T23:20:21.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.480+0000 7fd95bfff640 1 -- 172.21.15.47:0/3607901676 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd954014ba0 con 0x7fd9640b5120 2024-04-26T23:20:21.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.481+0000 7fd95bfff640 1 --2- 172.21.15.47:0/3607901676 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd94406c8f0 0x7fd94406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:21.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.481+0000 7fd95bfff640 1 -- 172.21.15.47:0/3607901676 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6001+0+0 (secure 0 0 0) 0x7fd95408c3b0 con 0x7fd9640b5120 2024-04-26T23:20:21.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.481+0000 7fd9627fc640 1 --2- 172.21.15.47:0/3607901676 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd94406c8f0 0x7fd94406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:21.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.482+0000 7fd9627fc640 1 --2- 172.21.15.47:0/3607901676 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd94406c8f0 0x7fd94406edb0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd94c034000 tx=0x7fd94c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:21.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.485+0000 7fd95bfff640 1 -- 172.21.15.47:0/3607901676 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd95405a060 con 0x7fd9640b5120 2024-04-26T23:20:21.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:21.711+0000 7fd963fff640 1 -- 172.21.15.47:0/3607901676 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fd9640751f0 con 0x7fd9640b5120 2024-04-26T23:20:22.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.332+0000 7fd95bfff640 1 -- 172.21.15.47:0/3607901676 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v23) ==== 172+0+0 (secure 0 0 0) 0x7fd95405e360 con 0x7fd9640b5120 2024-04-26T23:20:22.333 INFO:teuthology.orchestra.run.smithi047.stderr:new fs with metadata pool 8 and data pool 9 2024-04-26T23:20:22.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.338+0000 7fd963fff640 1 -- 172.21.15.47:0/3607901676 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd94406c8f0 msgr2=0x7fd94406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:22.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.338+0000 7fd963fff640 1 --2- 172.21.15.47:0/3607901676 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd94406c8f0 0x7fd94406edb0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd94c034000 tx=0x7fd94c02e000 comp rx=0 tx=0).stop 2024-04-26T23:20:22.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.338+0000 7fd963fff640 1 -- 172.21.15.47:0/3607901676 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9640b5120 msgr2=0x7fd9641589a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:22.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.338+0000 7fd963fff640 1 --2- 172.21.15.47:0/3607901676 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9640b5120 0x7fd9641589a0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd954007e60 tx=0x7fd95400e4b0 comp rx=0 tx=0).stop 2024-04-26T23:20:22.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.338+0000 7fd963fff640 1 -- 172.21.15.47:0/3607901676 shutdown_connections 2024-04-26T23:20:22.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.338+0000 7fd963fff640 1 --2- 172.21.15.47:0/3607901676 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd94406c8f0 0x7fd94406edb0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:22.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.338+0000 7fd963fff640 1 --2- 172.21.15.47:0/3607901676 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9640b59b0 0x7fd96415f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:22.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.338+0000 7fd963fff640 1 --2- 172.21.15.47:0/3607901676 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9640b5500 0x7fd96415f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:22.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.338+0000 7fd963fff640 1 --2- 172.21.15.47:0/3607901676 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9640b5120 0x7fd9641589a0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:22.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.338+0000 7fd963fff640 1 -- 172.21.15.47:0/3607901676 >> 172.21.15.47:0/3607901676 conn(0x7fd9640d4db0 msgr2=0x7fd9640d2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:22.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.348+0000 7fd963fff640 1 -- 172.21.15.47:0/3607901676 shutdown_connections 2024-04-26T23:20:22.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.349+0000 7fd963fff640 1 -- 172.21.15.47:0/3607901676 wait complete. 2024-04-26T23:20:22.374 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:20:22.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.478+0000 7ff820d58640 1 Processor -- start 2024-04-26T23:20:22.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.478+0000 7ff820d58640 1 -- start start 2024-04-26T23:20:22.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.479+0000 7ff820d58640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff81c162810 0x7ff81c070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:22.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.479+0000 7ff820d58640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff81c0d1b30 0x7ff81c0d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:22.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.479+0000 7ff820d58640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff81c0d10d0 con 0x7ff81c0b2340 2024-04-26T23:20:22.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.479+0000 7ff820d58640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7ff81c0d0750 con 0x7ff81c0d1b30 2024-04-26T23:20:22.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.479+0000 7ff820d58640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7ff81c0d16a0 con 0x7ff81c162810 2024-04-26T23:20:22.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.479+0000 7ff813fff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff81c0b2340 0x7ff81c0cd3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35738/0 (socket says 172.21.15.47:35738) 2024-04-26T23:20:22.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.479+0000 7ff813fff640 1 -- 172.21.15.47:0/2828111552 learned_addr learned my addr 172.21.15.47:0/2828111552 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:22.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.479+0000 7ff81a59c640 1 --2- 172.21.15.47:0/2828111552 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff81c162810 0x7ff81c070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:22.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.479+0000 7ff819d9b640 1 -- 172.21.15.47:0/2828111552 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3078148975 0 0) 0x7ff81c0d10d0 con 0x7ff81c0b2340 2024-04-26T23:20:22.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.479+0000 7ff81ad9d640 1 --2- 172.21.15.47:0/2828111552 >> v2:172.21.15.203:3300/0 conn(0x7ff81c0d1b30 0x7ff81c0d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:22.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.479+0000 7ff819d9b640 1 -- 172.21.15.47:0/2828111552 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff808003520 con 0x7ff81c0b2340 2024-04-26T23:20:22.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.479+0000 7ff819d9b640 1 -- 172.21.15.47:0/2828111552 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3307941859 0 0) 0x7ff808003520 con 0x7ff81c0b2340 2024-04-26T23:20:22.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.479+0000 7ff819d9b640 1 -- 172.21.15.47:0/2828111552 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff81c162810 msgr2=0x7ff81c070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:22.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.479+0000 7ff819d9b640 1 --2- 172.21.15.47:0/2828111552 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff81c162810 0x7ff81c070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:22.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.479+0000 7ff819d9b640 1 -- 172.21.15.47:0/2828111552 >> v2:172.21.15.203:3300/0 conn(0x7ff81c0d1b30 msgr2=0x7ff81c0d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:22.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.479+0000 7ff819d9b640 1 --2- 172.21.15.47:0/2828111552 >> v2:172.21.15.203:3300/0 conn(0x7ff81c0d1b30 0x7ff81c0d1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:22.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.479+0000 7ff819d9b640 1 -- 172.21.15.47:0/2828111552 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff81c0d1370 con 0x7ff81c0b2340 2024-04-26T23:20:22.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.479+0000 7ff819d9b640 1 -- 172.21.15.47:0/2828111552 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7ff804001670 con 0x7ff81c0b2340 2024-04-26T23:20:22.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.480+0000 7ff819d9b640 1 -- 172.21.15.47:0/2828111552 >> v1:172.21.15.47:6789/0 conn(0x7ff81c0b2340 legacy=0x7ff81c0cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:22.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.480+0000 7ff819d9b640 1 --2- 172.21.15.47:0/2828111552 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8080039b0 0x7ff808003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:22.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.480+0000 7ff819d9b640 1 --2- 172.21.15.47:0/2828111552 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff81c162810 0x7ff808004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:22.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.480+0000 7ff819d9b640 1 --2- 172.21.15.47:0/2828111552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff808004d40 0x7ff8080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:22.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.480+0000 7ff819d9b640 1 -- 172.21.15.47:0/2828111552 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c0d0750 con 0x7ff808004d40 2024-04-26T23:20:22.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.480+0000 7ff819d9b640 1 -- 172.21.15.47:0/2828111552 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c0d16a0 con 0x7ff8080039b0 2024-04-26T23:20:22.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.480+0000 7ff819d9b640 1 -- 172.21.15.47:0/2828111552 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7ff80800b9c0 con 0x7ff81c162810 2024-04-26T23:20:22.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.480+0000 7ff81ad9d640 1 --2- 172.21.15.47:0/2828111552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff808004d40 0x7ff8080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:22.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.480+0000 7ff81a59c640 1 --2- 172.21.15.47:0/2828111552 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8080039b0 0x7ff808003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:22.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.480+0000 7ff813fff640 1 --2- 172.21.15.47:0/2828111552 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff81c162810 0x7ff808004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:22.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.480+0000 7ff81ad9d640 1 -- 172.21.15.47:0/2828111552 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff81c162810 msgr2=0x7ff808004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:22.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.480+0000 7ff81ad9d640 1 --2- 172.21.15.47:0/2828111552 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff81c162810 0x7ff808004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:22.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.480+0000 7ff81ad9d640 1 -- 172.21.15.47:0/2828111552 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8080039b0 msgr2=0x7ff808003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:22.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.480+0000 7ff81ad9d640 1 --2- 172.21.15.47:0/2828111552 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8080039b0 0x7ff808003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:22.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.480+0000 7ff81ad9d640 1 -- 172.21.15.47:0/2828111552 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff808003520 con 0x7ff808004d40 2024-04-26T23:20:22.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.481+0000 7ff81ad9d640 1 --2- 172.21.15.47:0/2828111552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff808004d40 0x7ff8080051c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7ff814009fc0 tx=0x7ff808005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de3f290d90477c82 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:22.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.481+0000 7ff819d9b640 1 -- 172.21.15.47:0/2828111552 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff814009c10 con 0x7ff808004d40 2024-04-26T23:20:22.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.481+0000 7ff819d9b640 1 -- 172.21.15.47:0/2828111552 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff814036070 con 0x7ff808004d40 2024-04-26T23:20:22.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.481+0000 7ff819d9b640 1 -- 172.21.15.47:0/2828111552 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81402d070 con 0x7ff808004d40 2024-04-26T23:20:22.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.481+0000 7ff820d58640 1 -- 172.21.15.47:0/2828111552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff808004d40 msgr2=0x7ff8080051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:22.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.481+0000 7ff820d58640 1 --2- 172.21.15.47:0/2828111552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff808004d40 0x7ff8080051c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7ff814009fc0 tx=0x7ff808005830 comp rx=0 tx=0).stop 2024-04-26T23:20:22.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.481+0000 7ff81ad9d640 1 -- 172.21.15.47:0/2828111552 reap_dead start 2024-04-26T23:20:22.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.482+0000 7ff820d58640 1 -- 172.21.15.47:0/2828111552 shutdown_connections 2024-04-26T23:20:22.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.482+0000 7ff820d58640 1 -- 172.21.15.47:0/2828111552 >> 172.21.15.47:0/2828111552 conn(0x7ff81c0d2da0 msgr2=0x7ff81c0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:22.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.482+0000 7ff820d58640 1 -- 172.21.15.47:0/2828111552 shutdown_connections 2024-04-26T23:20:22.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.482+0000 7ff820d58640 1 -- 172.21.15.47:0/2828111552 wait complete. 2024-04-26T23:20:22.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.483+0000 7ff820d58640 1 Processor -- start 2024-04-26T23:20:22.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.483+0000 7ff820d58640 1 -- start start 2024-04-26T23:20:22.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.484+0000 7ff820d58640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff808004d40 0x7ff81c0cf280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:22.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.484+0000 7ff820d58640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff81c1f8190 0x7ff81c148940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:22.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.484+0000 7ff820d58640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff81c148e80 0x7ff81c0ca240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:22.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.484+0000 7ff820d58640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c0cdb00 con 0x7ff808004d40 2024-04-26T23:20:22.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.484+0000 7ff820d58640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c0d0470 con 0x7ff81c1f8190 2024-04-26T23:20:22.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.484+0000 7ff820d58640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7ff81c070350 con 0x7ff81c148e80 2024-04-26T23:20:22.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.484+0000 7ff81a59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff808004d40 0x7ff81c0cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:22.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.484+0000 7ff81a59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff808004d40 0x7ff81c0cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40692/0 (socket says 172.21.15.47:40692) 2024-04-26T23:20:22.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.484+0000 7ff81a59c640 1 -- 172.21.15.47:0/2021307866 learned_addr learned my addr 172.21.15.47:0/2021307866 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:22.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.484+0000 7ff813fff640 1 --2- 172.21.15.47:0/2021307866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff81c1f8190 0x7ff81c148940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:22.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.484+0000 7ff81ad9d640 1 --2- 172.21.15.47:0/2021307866 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff81c148e80 0x7ff81c0ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:22.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.484+0000 7ff81a59c640 1 -- 172.21.15.47:0/2021307866 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff81c148e80 msgr2=0x7ff81c0ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:22.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.484+0000 7ff81a59c640 1 --2- 172.21.15.47:0/2021307866 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff81c148e80 0x7ff81c0ca240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:22.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.484+0000 7ff81a59c640 1 -- 172.21.15.47:0/2021307866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff81c1f8190 msgr2=0x7ff81c148940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:22.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.484+0000 7ff81a59c640 1 --2- 172.21.15.47:0/2021307866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff81c1f8190 0x7ff81c148940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:22.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.484+0000 7ff81a59c640 1 -- 172.21.15.47:0/2021307866 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff81c0ccdd0 con 0x7ff808004d40 2024-04-26T23:20:22.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.484+0000 7ff81a59c640 1 --2- 172.21.15.47:0/2021307866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff808004d40 0x7ff81c0cf280 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7ff80c0118e0 tx=0x7ff80c011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:22.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.484+0000 7ff812ffd640 1 -- 172.21.15.47:0/2021307866 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff80c00f9d0 con 0x7ff808004d40 2024-04-26T23:20:22.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.484+0000 7ff812ffd640 1 -- 172.21.15.47:0/2021307866 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff80c00fb70 con 0x7ff808004d40 2024-04-26T23:20:22.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.484+0000 7ff820d58640 1 -- 172.21.15.47:0/2021307866 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff81c1497c0 con 0x7ff808004d40 2024-04-26T23:20:22.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.484+0000 7ff812ffd640 1 -- 172.21.15.47:0/2021307866 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff80c014440 con 0x7ff808004d40 2024-04-26T23:20:22.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.485+0000 7ff820d58640 1 -- 172.21.15.47:0/2021307866 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff81c0ce7a0 con 0x7ff808004d40 2024-04-26T23:20:22.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.486+0000 7ff820d58640 1 -- 172.21.15.47:0/2021307866 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff81c0694f0 con 0x7ff808004d40 2024-04-26T23:20:22.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.486+0000 7ff812ffd640 1 -- 172.21.15.47:0/2021307866 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff80c012040 con 0x7ff808004d40 2024-04-26T23:20:22.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.486+0000 7ff812ffd640 1 --2- 172.21.15.47:0/2021307866 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7ff7fc06c920 0x7ff7fc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:22.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.486+0000 7ff812ffd640 1 -- 172.21.15.47:0/2021307866 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7ff80c08c250 con 0x7ff808004d40 2024-04-26T23:20:22.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.486+0000 7ff813fff640 1 --2- 172.21.15.47:0/2021307866 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7ff7fc06c920 0x7ff7fc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:22.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.487+0000 7ff813fff640 1 --2- 172.21.15.47:0/2021307866 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7ff7fc06c920 0x7ff7fc06ede0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff81c0c9dd0 tx=0x7ff804009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:22.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.491+0000 7ff812ffd640 1 -- 172.21.15.47:0/2021307866 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff80c019030 con 0x7ff808004d40 2024-04-26T23:20:22.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.676+0000 7ff820d58640 1 -- 172.21.15.47:0/2021307866 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff81c0b3730 con 0x7ff808004d40 2024-04-26T23:20:22.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.677+0000 7ff812ffd640 1 -- 172.21.15.47:0/2021307866 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) ==== 74+0+16915 (secure 0 0 0) 0x7ff80c05a4f0 con 0x7ff808004d40 2024-04-26T23:20:22.679 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:20:22.679 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":53,"fsid":"399006d3-4db1-43e6-ba81-a79a276eabfd","created":"2024-04-26T23:18:06.731211+0000","modified":"2024-04-26T23:20:21.712640+0000","last_up_change":"2024-04-26T23:18:17.808736+0000","last_in_change":"2024-04-26T23:18:08.945859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:18:16.937492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:20:19.975691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-26T23:20:20.657778+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ee091567-fea0-4a14-bf4e-0e16c20bb2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6819","nonce":326520779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6821","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6823","nonce":326520779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6829","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6831","nonce":326520779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6825","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6827","nonce":326520779}]},"public_addr":"172.21.15.47:6819/326520779","cluster_addr":"172.21.15.47:6823/326520779","heartbeat_back_addr":"172.21.15.47:6831/326520779","heartbeat_front_addr":"172.21.15.47:6827/326520779","state":["exists","up"]},{"osd":1,"uuid":"c14440a0-bba5-4931-a6db-b736c08e869b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6809","nonce":1912610931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6811","nonce":1912610931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6815","nonce":1912610931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6813","nonce":1912610931}]},"public_addr":"172.21.15.47:6809/1912610931","cluster_addr":"172.21.15.47:6811/1912610931","heartbeat_back_addr":"172.21.15.47:6815/1912610931","heartbeat_front_addr":"172.21.15.47:6813/1912610931","state":["exists","up"]},{"osd":2,"uuid":"3940f8de-c9b2-4cd3-a911-9c301729eb30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6801","nonce":2569869305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6803","nonce":2569869305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6807","nonce":2569869305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6805","nonce":2569869305}]},"public_addr":"172.21.15.47:6801/2569869305","cluster_addr":"172.21.15.47:6803/2569869305","heartbeat_back_addr":"172.21.15.47:6807/2569869305","heartbeat_front_addr":"172.21.15.47:6805/2569869305","state":["exists","up"]},{"osd":3,"uuid":"1286a67e-8958-426a-8c2e-bc85498c619c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6818","nonce":2791411683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6822","nonce":2791411683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6830","nonce":2791411683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6826","nonce":2791411683}]},"public_addr":"172.21.15.47:6818/2791411683","cluster_addr":"172.21.15.47:6822/2791411683","heartbeat_back_addr":"172.21.15.47:6830/2791411683","heartbeat_front_addr":"172.21.15.47:6826/2791411683","state":["exists","up"]},{"osd":4,"uuid":"d8ee2c20-e5b7-46c2-9999-b234070126cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6809","nonce":441313104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6811","nonce":441313104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6815","nonce":441313104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6813","nonce":441313104}]},"public_addr":"172.21.15.203:6809/441313104","cluster_addr":"172.21.15.203:6811/441313104","heartbeat_back_addr":"172.21.15.203:6815/441313104","heartbeat_front_addr":"172.21.15.203:6813/441313104","state":["exists","up"]},{"osd":5,"uuid":"b453a8f0-f324-4840-9ae0-d67bdadb9f8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6801","nonce":56838444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6803","nonce":56838444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6807","nonce":56838444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6805","nonce":56838444}]},"public_addr":"172.21.15.203:6801/56838444","cluster_addr":"172.21.15.203:6803/56838444","heartbeat_back_addr":"172.21.15.203:6807/56838444","heartbeat_front_addr":"172.21.15.203:6805/56838444","state":["exists","up"]},{"osd":6,"uuid":"a6d31951-2d4b-4e8d-a510-6283b01dc158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6825","nonce":2738517080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6827","nonce":2738517080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6831","nonce":2738517080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6829","nonce":2738517080}]},"public_addr":"172.21.15.203:6825/2738517080","cluster_addr":"172.21.15.203:6827/2738517080","heartbeat_back_addr":"172.21.15.203:6831/2738517080","heartbeat_front_addr":"172.21.15.203:6829/2738517080","state":["exists","up"]},{"osd":7,"uuid":"7119b8c0-fe96-402d-9e75-7b509d9d44d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6817","nonce":1315565058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6819","nonce":1315565058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6823","nonce":1315565058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6821","nonce":1315565058}]},"public_addr":"172.21.15.203:6817/1315565058","cluster_addr":"172.21.15.203:6819/1315565058","heartbeat_back_addr":"172.21.15.203:6823/1315565058","heartbeat_front_addr":"172.21.15.203:6821/1315565058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.121921+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.137494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.084814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.111593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.072430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.125066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.097867+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.057061+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:20:22.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.679+0000 7ff820d58640 1 -- 172.21.15.47:0/2021307866 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7ff7fc06c920 msgr2=0x7ff7fc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:22.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.679+0000 7ff820d58640 1 --2- 172.21.15.47:0/2021307866 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7ff7fc06c920 0x7ff7fc06ede0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff81c0c9dd0 tx=0x7ff804009040 comp rx=0 tx=0).stop 2024-04-26T23:20:22.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.679+0000 7ff820d58640 1 -- 172.21.15.47:0/2021307866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff808004d40 msgr2=0x7ff81c0cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:22.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.679+0000 7ff820d58640 1 --2- 172.21.15.47:0/2021307866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff808004d40 0x7ff81c0cf280 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7ff80c0118e0 tx=0x7ff80c011da0 comp rx=0 tx=0).stop 2024-04-26T23:20:22.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.679+0000 7ff820d58640 1 -- 172.21.15.47:0/2021307866 shutdown_connections 2024-04-26T23:20:22.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.679+0000 7ff820d58640 1 --2- 172.21.15.47:0/2021307866 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7ff7fc06c920 0x7ff7fc06ede0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:22.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.679+0000 7ff820d58640 1 --2- 172.21.15.47:0/2021307866 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff81c148e80 0x7ff81c0ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:22.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.679+0000 7ff820d58640 1 --2- 172.21.15.47:0/2021307866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff81c1f8190 0x7ff81c148940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:22.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.679+0000 7ff820d58640 1 --2- 172.21.15.47:0/2021307866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff808004d40 0x7ff81c0cf280 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:22.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.679+0000 7ff820d58640 1 -- 172.21.15.47:0/2021307866 >> 172.21.15.47:0/2021307866 conn(0x7ff81c0d2da0 msgr2=0x7ff81c0b3c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:22.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.679+0000 7ff820d58640 1 -- 172.21.15.47:0/2021307866 shutdown_connections 2024-04-26T23:20:22.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.679+0000 7ff820d58640 1 -- 172.21.15.47:0/2021307866 wait complete. 2024-04-26T23:20:22.691 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:20:22.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.791+0000 7fd47ce37640 1 Processor -- start 2024-04-26T23:20:22.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.791+0000 7fd47ce37640 1 -- start start 2024-04-26T23:20:22.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.792+0000 7fd47ce37640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd478071e60 0x7fd47806f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:22.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.792+0000 7fd47ce37640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd4780cc480 0x7fd478166ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:22.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.792+0000 7fd47ce37640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd47815d7c0 con 0x7fd47806fe20 2024-04-26T23:20:22.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.792+0000 7fd47ce37640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fd47815daa0 con 0x7fd4780cc480 2024-04-26T23:20:22.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.792+0000 7fd47ce37640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd47815dd80 con 0x7fd478071e60 2024-04-26T23:20:22.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.792+0000 7fd47659c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd478071e60 0x7fd47806f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:22.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.792+0000 7fd47659c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd478071e60 0x7fd47806f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.47:51626/0 (socket says 172.21.15.47:51626) 2024-04-26T23:20:22.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.792+0000 7fd476d9d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd4780cc480 0x7fd478166ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:22.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.792+0000 7fd47659c640 1 -- 172.21.15.47:0/1645905433 learned_addr learned my addr 172.21.15.47:0/1645905433 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:22.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.793+0000 7fd476d9d640 1 -- 172.21.15.47:0/1645905433 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd478071e60 msgr2=0x7fd47806f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:22.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.793+0000 7fd476d9d640 1 --2- 172.21.15.47:0/1645905433 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd478071e60 0x7fd47806f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:22.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.793+0000 7fd476d9d640 1 -- 172.21.15.47:0/1645905433 >> v1:172.21.15.203:6789/0 conn(0x7fd47806fe20 legacy=0x7fd478072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:22.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.793+0000 7fd47559a640 1 -- 172.21.15.47:0/1645905433 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 969997451 0 0) 0x7fd47815d7c0 con 0x7fd47806fe20 2024-04-26T23:20:22.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.793+0000 7fd476d9d640 1 -- 172.21.15.47:0/1645905433 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd47815e060 con 0x7fd4780cc480 2024-04-26T23:20:22.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.794+0000 7fd476d9d640 1 --2- 172.21.15.47:0/1645905433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4780cc480 0x7fd478166ad0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd470009d90 tx=0x7fd47002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3698f3f38d0cda30 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:22.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.794+0000 7fd47559a640 1 -- 172.21.15.47:0/1645905433 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd470007740 con 0x7fd4780cc480 2024-04-26T23:20:22.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.794+0000 7fd47559a640 1 -- 172.21.15.47:0/1645905433 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4700078e0 con 0x7fd4780cc480 2024-04-26T23:20:22.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.794+0000 7fd47559a640 1 -- 172.21.15.47:0/1645905433 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd470007cb0 con 0x7fd4780cc480 2024-04-26T23:20:22.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.794+0000 7fd47ce37640 1 -- 172.21.15.47:0/1645905433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4780cc480 msgr2=0x7fd478166ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:22.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.794+0000 7fd47ce37640 1 --2- 172.21.15.47:0/1645905433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4780cc480 0x7fd478166ad0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd470009d90 tx=0x7fd47002a660 comp rx=0 tx=0).stop 2024-04-26T23:20:22.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.794+0000 7fd47ce37640 1 -- 172.21.15.47:0/1645905433 shutdown_connections 2024-04-26T23:20:22.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.794+0000 7fd47ce37640 1 --2- 172.21.15.47:0/1645905433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4780cc480 0x7fd478166ad0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:22.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.794+0000 7fd47ce37640 1 --2- 172.21.15.47:0/1645905433 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd478071e60 0x7fd47806f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:22.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.794+0000 7fd47ce37640 1 -- 172.21.15.47:0/1645905433 >> 172.21.15.47:0/1645905433 conn(0x7fd4780d0be0 msgr2=0x7fd4780d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:22.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.794+0000 7fd47ce37640 1 -- 172.21.15.47:0/1645905433 shutdown_connections 2024-04-26T23:20:22.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.794+0000 7fd47ce37640 1 -- 172.21.15.47:0/1645905433 wait complete. 2024-04-26T23:20:22.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.796+0000 7fd47ce37640 1 Processor -- start 2024-04-26T23:20:22.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.796+0000 7fd47ce37640 1 -- start start 2024-04-26T23:20:22.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.796+0000 7fd47ce37640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd47806fe20 0x7fd4781f01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:22.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.796+0000 7fd47ce37640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd478071e60 0x7fd4780b4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:22.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.796+0000 7fd47ce37640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4780cc480 0x7fd47815b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:22.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.797+0000 7fd47ce37640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd47815d570 con 0x7fd478071e60 2024-04-26T23:20:22.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.797+0000 7fd47ce37640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd47815d3f0 con 0x7fd4780cc480 2024-04-26T23:20:22.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.797+0000 7fd47ce37640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd478072950 con 0x7fd47806fe20 2024-04-26T23:20:22.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.797+0000 7fd475d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd478071e60 0x7fd4780b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:22.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.797+0000 7fd475d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd478071e60 0x7fd4780b4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40700/0 (socket says 172.21.15.47:40700) 2024-04-26T23:20:22.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.797+0000 7fd475d9b640 1 -- 172.21.15.47:0/1107147448 learned_addr learned my addr 172.21.15.47:0/1107147448 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:22.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.797+0000 7fd47659c640 1 --2- 172.21.15.47:0/1107147448 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd47806fe20 0x7fd4781f01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:22.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.797+0000 7fd476d9d640 1 --2- 172.21.15.47:0/1107147448 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4780cc480 0x7fd47815b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:22.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.797+0000 7fd475d9b640 1 -- 172.21.15.47:0/1107147448 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd47806fe20 msgr2=0x7fd4781f01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:22.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.797+0000 7fd475d9b640 1 --2- 172.21.15.47:0/1107147448 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd47806fe20 0x7fd4781f01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:22.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.797+0000 7fd475d9b640 1 -- 172.21.15.47:0/1107147448 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4780cc480 msgr2=0x7fd47815b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:22.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.797+0000 7fd475d9b640 1 --2- 172.21.15.47:0/1107147448 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4780cc480 0x7fd47815b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:22.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.797+0000 7fd475d9b640 1 -- 172.21.15.47:0/1107147448 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4781f3730 con 0x7fd478071e60 2024-04-26T23:20:22.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.797+0000 7fd47659c640 1 --2- 172.21.15.47:0/1107147448 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd47806fe20 0x7fd4781f01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:20:22.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.797+0000 7fd475d9b640 1 --2- 172.21.15.47:0/1107147448 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd478071e60 0x7fd4780b4360 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fd46c00c4b0 tx=0x7fd46c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:22.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.797+0000 7fd466ffd640 1 -- 172.21.15.47:0/1107147448 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd46c017070 con 0x7fd478071e60 2024-04-26T23:20:22.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.797+0000 7fd466ffd640 1 -- 172.21.15.47:0/1107147448 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd46c005480 con 0x7fd478071e60 2024-04-26T23:20:22.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.797+0000 7fd466ffd640 1 -- 172.21.15.47:0/1107147448 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd46c003830 con 0x7fd478071e60 2024-04-26T23:20:22.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.797+0000 7fd47ce37640 1 -- 172.21.15.47:0/1107147448 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4781f3a10 con 0x7fd478071e60 2024-04-26T23:20:22.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.797+0000 7fd47ce37640 1 -- 172.21.15.47:0/1107147448 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4780b3b60 con 0x7fd478071e60 2024-04-26T23:20:22.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.798+0000 7fd47ce37640 1 -- 172.21.15.47:0/1107147448 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4780694f0 con 0x7fd478071e60 2024-04-26T23:20:22.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.799+0000 7fd466ffd640 1 -- 172.21.15.47:0/1107147448 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd46c01a070 con 0x7fd478071e60 2024-04-26T23:20:22.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.799+0000 7fd466ffd640 1 --2- 172.21.15.47:0/1107147448 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd45806c8f0 0x7fd45806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:22.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.799+0000 7fd466ffd640 1 -- 172.21.15.47:0/1107147448 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7fd46c08b080 con 0x7fd478071e60 2024-04-26T23:20:22.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.800+0000 7fd47659c640 1 --2- 172.21.15.47:0/1107147448 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd45806c8f0 0x7fd45806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:22.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.800+0000 7fd47659c640 1 --2- 172.21.15.47:0/1107147448 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd45806c8f0 0x7fd45806edb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd468009c20 tx=0x7fd468008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:22.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.804+0000 7fd466ffd640 1 -- 172.21.15.47:0/1107147448 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd46c059320 con 0x7fd478071e60 2024-04-26T23:20:22.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.992+0000 7fd47ce37640 1 -- 172.21.15.47:0/1107147448 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd4780c7220 con 0x7fd478071e60 2024-04-26T23:20:22.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.993+0000 7fd466ffd640 1 -- 172.21.15.47:0/1107147448 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) ==== 74+0+16915 (secure 0 0 0) 0x7fd46c058c20 con 0x7fd478071e60 2024-04-26T23:20:22.994 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:20:22.994 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":53,"fsid":"399006d3-4db1-43e6-ba81-a79a276eabfd","created":"2024-04-26T23:18:06.731211+0000","modified":"2024-04-26T23:20:21.712640+0000","last_up_change":"2024-04-26T23:18:17.808736+0000","last_in_change":"2024-04-26T23:18:08.945859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:18:16.937492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:20:19.975691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-26T23:20:20.657778+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ee091567-fea0-4a14-bf4e-0e16c20bb2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6819","nonce":326520779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6821","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6823","nonce":326520779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6829","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6831","nonce":326520779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6825","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6827","nonce":326520779}]},"public_addr":"172.21.15.47:6819/326520779","cluster_addr":"172.21.15.47:6823/326520779","heartbeat_back_addr":"172.21.15.47:6831/326520779","heartbeat_front_addr":"172.21.15.47:6827/326520779","state":["exists","up"]},{"osd":1,"uuid":"c14440a0-bba5-4931-a6db-b736c08e869b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6809","nonce":1912610931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6811","nonce":1912610931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6815","nonce":1912610931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6813","nonce":1912610931}]},"public_addr":"172.21.15.47:6809/1912610931","cluster_addr":"172.21.15.47:6811/1912610931","heartbeat_back_addr":"172.21.15.47:6815/1912610931","heartbeat_front_addr":"172.21.15.47:6813/1912610931","state":["exists","up"]},{"osd":2,"uuid":"3940f8de-c9b2-4cd3-a911-9c301729eb30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6801","nonce":2569869305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6803","nonce":2569869305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6807","nonce":2569869305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6805","nonce":2569869305}]},"public_addr":"172.21.15.47:6801/2569869305","cluster_addr":"172.21.15.47:6803/2569869305","heartbeat_back_addr":"172.21.15.47:6807/2569869305","heartbeat_front_addr":"172.21.15.47:6805/2569869305","state":["exists","up"]},{"osd":3,"uuid":"1286a67e-8958-426a-8c2e-bc85498c619c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6818","nonce":2791411683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6822","nonce":2791411683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6830","nonce":2791411683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6826","nonce":2791411683}]},"public_addr":"172.21.15.47:6818/2791411683","cluster_addr":"172.21.15.47:6822/2791411683","heartbeat_back_addr":"172.21.15.47:6830/2791411683","heartbeat_front_addr":"172.21.15.47:6826/2791411683","state":["exists","up"]},{"osd":4,"uuid":"d8ee2c20-e5b7-46c2-9999-b234070126cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6809","nonce":441313104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6811","nonce":441313104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6815","nonce":441313104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6813","nonce":441313104}]},"public_addr":"172.21.15.203:6809/441313104","cluster_addr":"172.21.15.203:6811/441313104","heartbeat_back_addr":"172.21.15.203:6815/441313104","heartbeat_front_addr":"172.21.15.203:6813/441313104","state":["exists","up"]},{"osd":5,"uuid":"b453a8f0-f324-4840-9ae0-d67bdadb9f8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6801","nonce":56838444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6803","nonce":56838444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6807","nonce":56838444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6805","nonce":56838444}]},"public_addr":"172.21.15.203:6801/56838444","cluster_addr":"172.21.15.203:6803/56838444","heartbeat_back_addr":"172.21.15.203:6807/56838444","heartbeat_front_addr":"172.21.15.203:6805/56838444","state":["exists","up"]},{"osd":6,"uuid":"a6d31951-2d4b-4e8d-a510-6283b01dc158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6825","nonce":2738517080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6827","nonce":2738517080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6831","nonce":2738517080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6829","nonce":2738517080}]},"public_addr":"172.21.15.203:6825/2738517080","cluster_addr":"172.21.15.203:6827/2738517080","heartbeat_back_addr":"172.21.15.203:6831/2738517080","heartbeat_front_addr":"172.21.15.203:6829/2738517080","state":["exists","up"]},{"osd":7,"uuid":"7119b8c0-fe96-402d-9e75-7b509d9d44d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6817","nonce":1315565058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6819","nonce":1315565058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6823","nonce":1315565058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6821","nonce":1315565058}]},"public_addr":"172.21.15.203:6817/1315565058","cluster_addr":"172.21.15.203:6819/1315565058","heartbeat_back_addr":"172.21.15.203:6823/1315565058","heartbeat_front_addr":"172.21.15.203:6821/1315565058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.121921+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.137494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.084814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.111593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.072430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.125066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.097867+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.057061+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:20:22.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.995+0000 7fd47ce37640 1 -- 172.21.15.47:0/1107147448 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd45806c8f0 msgr2=0x7fd45806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:22.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.995+0000 7fd47ce37640 1 --2- 172.21.15.47:0/1107147448 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd45806c8f0 0x7fd45806edb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd468009c20 tx=0x7fd468008040 comp rx=0 tx=0).stop 2024-04-26T23:20:22.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.995+0000 7fd47ce37640 1 -- 172.21.15.47:0/1107147448 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd478071e60 msgr2=0x7fd4780b4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:22.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.995+0000 7fd47ce37640 1 --2- 172.21.15.47:0/1107147448 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd478071e60 0x7fd4780b4360 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fd46c00c4b0 tx=0x7fd46c00c970 comp rx=0 tx=0).stop 2024-04-26T23:20:22.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.995+0000 7fd47ce37640 1 -- 172.21.15.47:0/1107147448 shutdown_connections 2024-04-26T23:20:22.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.995+0000 7fd47ce37640 1 --2- 172.21.15.47:0/1107147448 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd45806c8f0 0x7fd45806edb0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:22.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.995+0000 7fd47ce37640 1 --2- 172.21.15.47:0/1107147448 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4780cc480 0x7fd47815b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:22.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.995+0000 7fd47ce37640 1 --2- 172.21.15.47:0/1107147448 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd478071e60 0x7fd4780b4360 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:22.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.995+0000 7fd47ce37640 1 --2- 172.21.15.47:0/1107147448 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd47806fe20 0x7fd4781f01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:22.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.995+0000 7fd47ce37640 1 -- 172.21.15.47:0/1107147448 >> 172.21.15.47:0/1107147448 conn(0x7fd4780d0be0 msgr2=0x7fd478159b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:22.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.995+0000 7fd47ce37640 1 -- 172.21.15.47:0/1107147448 shutdown_connections 2024-04-26T23:20:22.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:22.995+0000 7fd47ce37640 1 -- 172.21.15.47:0/1107147448 wait complete. 2024-04-26T23:20:23.007 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T23:20:23.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.111+0000 7fc0da722640 1 Processor -- start 2024-04-26T23:20:23.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.111+0000 7fc0da722640 1 -- start start 2024-04-26T23:20:23.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.111+0000 7fc0da722640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0d415f170 0x7fc0d40d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:23.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.111+0000 7fc0da722640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc0d40d3bc0 0x7fc0d40d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:23.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.111+0000 7fc0da722640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0d40d1040 con 0x7fc0d415f550 2024-04-26T23:20:23.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.111+0000 7fc0da722640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fc0d40c9430 con 0x7fc0d40d3bc0 2024-04-26T23:20:23.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.111+0000 7fc0da722640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc0d40c9680 con 0x7fc0d415f170 2024-04-26T23:20:23.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.112+0000 7fc0d8cbf640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc0d40d3bc0 0x7fc0d40d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:23.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.112+0000 7fc0d8cbf640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc0d40d3bc0 0x7fc0d40d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40710/0 (socket says 172.21.15.47:40710) 2024-04-26T23:20:23.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.112+0000 7fc0d37fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc0d415f550 0x7fc0d40d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35750/0 (socket says 172.21.15.47:35750) 2024-04-26T23:20:23.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.112+0000 7fc0d8cbf640 1 -- 172.21.15.47:0/3036336918 learned_addr learned my addr 172.21.15.47:0/3036336918 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:23.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.112+0000 7fc0d3fff640 1 --2- 172.21.15.47:0/3036336918 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0d415f170 0x7fc0d40d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:23.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.112+0000 7fc0d8cbf640 1 -- 172.21.15.47:0/3036336918 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0d415f170 msgr2=0x7fc0d40d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:23.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.112+0000 7fc0d8cbf640 1 --2- 172.21.15.47:0/3036336918 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0d415f170 0x7fc0d40d5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:23.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.112+0000 7fc0d8cbf640 1 -- 172.21.15.47:0/3036336918 >> v1:172.21.15.47:6789/0 conn(0x7fc0d415f550 legacy=0x7fc0d40d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:23.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.112+0000 7fc0d8cbf640 1 -- 172.21.15.47:0/3036336918 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0d40c9960 con 0x7fc0d40d3bc0 2024-04-26T23:20:23.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.112+0000 7fc0d2ffd640 1 -- 172.21.15.47:0/3036336918 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3122135427 0 0) 0x7fc0c8001110 con 0x7fc0d415f550 2024-04-26T23:20:23.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.113+0000 7fc0d8cbf640 1 --2- 172.21.15.47:0/3036336918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0d40d3bc0 0x7fc0d40d2cb0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc009e50 tx=0x7fc0cc02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69e5dc673c3f752 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:23.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.113+0000 7fc0d2ffd640 1 -- 172.21.15.47:0/3036336918 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0cc036070 con 0x7fc0d40d3bc0 2024-04-26T23:20:23.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.113+0000 7fc0d2ffd640 1 -- 172.21.15.47:0/3036336918 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0cc031600 con 0x7fc0d40d3bc0 2024-04-26T23:20:23.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.113+0000 7fc0d2ffd640 1 -- 172.21.15.47:0/3036336918 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0cc0319d0 con 0x7fc0d40d3bc0 2024-04-26T23:20:23.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.113+0000 7fc0da722640 1 -- 172.21.15.47:0/3036336918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0d40d3bc0 msgr2=0x7fc0d40d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:23.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.113+0000 7fc0da722640 1 --2- 172.21.15.47:0/3036336918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0d40d3bc0 0x7fc0d40d2cb0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc009e50 tx=0x7fc0cc02a710 comp rx=0 tx=0).stop 2024-04-26T23:20:23.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.113+0000 7fc0da722640 1 -- 172.21.15.47:0/3036336918 shutdown_connections 2024-04-26T23:20:23.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.113+0000 7fc0da722640 1 --2- 172.21.15.47:0/3036336918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0d40d3bc0 0x7fc0d40d2cb0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:23.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.113+0000 7fc0da722640 1 --2- 172.21.15.47:0/3036336918 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0d415f170 0x7fc0d40d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:23.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.113+0000 7fc0da722640 1 -- 172.21.15.47:0/3036336918 >> 172.21.15.47:0/3036336918 conn(0x7fc0d40d4f60 msgr2=0x7fc0d40d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:23.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.113+0000 7fc0da722640 1 -- 172.21.15.47:0/3036336918 shutdown_connections 2024-04-26T23:20:23.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.113+0000 7fc0da722640 1 -- 172.21.15.47:0/3036336918 wait complete. 2024-04-26T23:20:23.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.115+0000 7fc0da722640 1 Processor -- start 2024-04-26T23:20:23.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.115+0000 7fc0da722640 1 -- start start 2024-04-26T23:20:23.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.115+0000 7fc0da722640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0d40d3bc0 0x7fc0d40ce830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:23.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.115+0000 7fc0da722640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0d415f170 0x7fc0d415b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:23.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.115+0000 7fc0da722640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0d4158020 0x7fc0d4158460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:23.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.115+0000 7fc0da722640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d40ca5c0 con 0x7fc0d415f170 2024-04-26T23:20:23.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.115+0000 7fc0da722640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d40ca440 con 0x7fc0d40d3bc0 2024-04-26T23:20:23.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.115+0000 7fc0da722640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc0d40ca740 con 0x7fc0d4158020 2024-04-26T23:20:23.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.116+0000 7fc0d37fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0d415f170 0x7fc0d415b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:23.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.116+0000 7fc0d37fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0d415f170 0x7fc0d415b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40712/0 (socket says 172.21.15.47:40712) 2024-04-26T23:20:23.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.116+0000 7fc0d37fe640 1 -- 172.21.15.47:0/2042790005 learned_addr learned my addr 172.21.15.47:0/2042790005 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:23.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.116+0000 7fc0d8cbf640 1 --2- 172.21.15.47:0/2042790005 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0d4158020 0x7fc0d4158460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:23.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.116+0000 7fc0d37fe640 1 -- 172.21.15.47:0/2042790005 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0d4158020 msgr2=0x7fc0d4158460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:23.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.116+0000 7fc0d37fe640 1 --2- 172.21.15.47:0/2042790005 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0d4158020 0x7fc0d4158460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:23.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.116+0000 7fc0d37fe640 1 -- 172.21.15.47:0/2042790005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0d40d3bc0 msgr2=0x7fc0d40ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:23.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.116+0000 7fc0d3fff640 1 --2- 172.21.15.47:0/2042790005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0d40d3bc0 0x7fc0d40ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:23.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.116+0000 7fc0d37fe640 1 --2- 172.21.15.47:0/2042790005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0d40d3bc0 0x7fc0d40ce830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:23.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.116+0000 7fc0d37fe640 1 -- 172.21.15.47:0/2042790005 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0d41589a0 con 0x7fc0d415f170 2024-04-26T23:20:23.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.116+0000 7fc0d37fe640 1 --2- 172.21.15.47:0/2042790005 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0d415f170 0x7fc0d415b240 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fc0c800c430 tx=0x7fc0c800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:23.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.116+0000 7fc0d0ff9640 1 -- 172.21.15.47:0/2042790005 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c80074e0 con 0x7fc0d415f170 2024-04-26T23:20:23.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.116+0000 7fc0d0ff9640 1 -- 172.21.15.47:0/2042790005 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0c80047e0 con 0x7fc0d415f170 2024-04-26T23:20:23.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.116+0000 7fc0da722640 1 -- 172.21.15.47:0/2042790005 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0d40cce20 con 0x7fc0d415f170 2024-04-26T23:20:23.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.116+0000 7fc0d0ff9640 1 -- 172.21.15.47:0/2042790005 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c8004bd0 con 0x7fc0d415f170 2024-04-26T23:20:23.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.116+0000 7fc0da722640 1 -- 172.21.15.47:0/2042790005 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0d40cd3e0 con 0x7fc0d415f170 2024-04-26T23:20:23.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.117+0000 7fc0da722640 1 -- 172.21.15.47:0/2042790005 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0d4158f00 con 0x7fc0d415f170 2024-04-26T23:20:23.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.118+0000 7fc0d0ff9640 1 -- 172.21.15.47:0/2042790005 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc0c8011070 con 0x7fc0d415f170 2024-04-26T23:20:23.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.118+0000 7fc0d0ff9640 1 --2- 172.21.15.47:0/2042790005 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fc0c006c8f0 0x7fc0c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:23.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.118+0000 7fc0d0ff9640 1 -- 172.21.15.47:0/2042790005 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7fc0c808b910 con 0x7fc0d415f170 2024-04-26T23:20:23.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.118+0000 7fc0d3fff640 1 --2- 172.21.15.47:0/2042790005 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fc0c006c8f0 0x7fc0c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:23.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.119+0000 7fc0d3fff640 1 --2- 172.21.15.47:0/2042790005 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fc0c006c8f0 0x7fc0c006edb0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4009a10 tx=0x7fc0c4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:23.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.123+0000 7fc0d0ff9640 1 -- 172.21.15.47:0/2042790005 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc0d40d1040 con 0x7fc0d415f170 2024-04-26T23:20:23.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:23.350+0000 7fc0da722640 1 -- 172.21.15.47:0/2042790005 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fc0d40694f0 con 0x7fc0d415f170 2024-04-26T23:20:24.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.355+0000 7fc0d0ff9640 1 -- 172.21.15.47:0/2042790005 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v26) ==== 118+0+0 (secure 0 0 0) 0x7fc0c800e280 con 0x7fc0d415f170 2024-04-26T23:20:24.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.358+0000 7fc0da722640 1 -- 172.21.15.47:0/2042790005 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fc0c006c8f0 msgr2=0x7fc0c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:24.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.358+0000 7fc0da722640 1 --2- 172.21.15.47:0/2042790005 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fc0c006c8f0 0x7fc0c006edb0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4009a10 tx=0x7fc0c4008040 comp rx=0 tx=0).stop 2024-04-26T23:20:24.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.358+0000 7fc0da722640 1 -- 172.21.15.47:0/2042790005 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0d415f170 msgr2=0x7fc0d415b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:24.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.358+0000 7fc0da722640 1 --2- 172.21.15.47:0/2042790005 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0d415f170 0x7fc0d415b240 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fc0c800c430 tx=0x7fc0c800c8f0 comp rx=0 tx=0).stop 2024-04-26T23:20:24.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.358+0000 7fc0da722640 1 -- 172.21.15.47:0/2042790005 shutdown_connections 2024-04-26T23:20:24.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.358+0000 7fc0da722640 1 --2- 172.21.15.47:0/2042790005 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fc0c006c8f0 0x7fc0c006edb0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:24.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.358+0000 7fc0da722640 1 --2- 172.21.15.47:0/2042790005 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0d4158020 0x7fc0d4158460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:24.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.358+0000 7fc0da722640 1 --2- 172.21.15.47:0/2042790005 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0d415f170 0x7fc0d415b240 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:24.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.358+0000 7fc0da722640 1 --2- 172.21.15.47:0/2042790005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0d40d3bc0 0x7fc0d40ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:24.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.358+0000 7fc0da722640 1 -- 172.21.15.47:0/2042790005 >> 172.21.15.47:0/2042790005 conn(0x7fc0d40d4f60 msgr2=0x7fc0d406d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:24.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.358+0000 7fc0da722640 1 -- 172.21.15.47:0/2042790005 shutdown_connections 2024-04-26T23:20:24.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.358+0000 7fc0da722640 1 -- 172.21.15.47:0/2042790005 wait complete. 2024-04-26T23:20:24.373 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-26T23:20:24.373 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T23:20:24.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.482+0000 7ff8fe971640 1 Processor -- start 2024-04-26T23:20:24.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.482+0000 7ff8fe971640 1 -- start start 2024-04-26T23:20:24.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.483+0000 7ff8fe971640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff8f8070350 0x7ff8f8070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:24.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.483+0000 7ff8fe971640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8f8071090 0x7ff8f815e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:24.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.483+0000 7ff8fe971640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8f80d2390 con 0x7ff8f80cfde0 2024-04-26T23:20:24.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.483+0000 7ff8fe971640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7ff8f8071500 con 0x7ff8f8070350 2024-04-26T23:20:24.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.483+0000 7ff8fe971640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7ff8f80afab0 con 0x7ff8f8071090 2024-04-26T23:20:24.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.483+0000 7ff8f7fff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff8f80cfde0 0x7ff8f80d1b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35758/0 (socket says 172.21.15.47:35758) 2024-04-26T23:20:24.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.483+0000 7ff8f7fff640 1 -- 172.21.15.47:0/3802577958 learned_addr learned my addr 172.21.15.47:0/3802577958 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:24.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.483+0000 7ff8f77fe640 1 --2- 172.21.15.47:0/3802577958 >> v2:172.21.15.203:3300/0 conn(0x7ff8f8070350 0x7ff8f8070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:24.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.483+0000 7ff8fcf0e640 1 --2- 172.21.15.47:0/3802577958 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8f8071090 0x7ff8f815e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:24.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.483+0000 7ff8f6ffd640 1 -- 172.21.15.47:0/3802577958 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 559070056 0 0) 0x7ff8f80d2390 con 0x7ff8f80cfde0 2024-04-26T23:20:24.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.483+0000 7ff8f6ffd640 1 -- 172.21.15.47:0/3802577958 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff8e4003520 con 0x7ff8f80cfde0 2024-04-26T23:20:24.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.484+0000 7ff8f6ffd640 1 -- 172.21.15.47:0/3802577958 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4057355888 0 0) 0x7ff8e4003520 con 0x7ff8f80cfde0 2024-04-26T23:20:24.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.484+0000 7ff8f6ffd640 1 -- 172.21.15.47:0/3802577958 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8f8071090 msgr2=0x7ff8f815e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:24.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.484+0000 7ff8f6ffd640 1 --2- 172.21.15.47:0/3802577958 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8f8071090 0x7ff8f815e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:24.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.484+0000 7ff8f6ffd640 1 -- 172.21.15.47:0/3802577958 >> v2:172.21.15.203:3300/0 conn(0x7ff8f8070350 msgr2=0x7ff8f8070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:24.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.484+0000 7ff8f6ffd640 1 --2- 172.21.15.47:0/3802577958 >> v2:172.21.15.203:3300/0 conn(0x7ff8f8070350 0x7ff8f8070b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:24.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.484+0000 7ff8f6ffd640 1 -- 172.21.15.47:0/3802577958 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f806d3f0 con 0x7ff8f80cfde0 2024-04-26T23:20:24.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.484+0000 7ff8f6ffd640 1 -- 172.21.15.47:0/3802577958 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7ff8e0001670 con 0x7ff8f80cfde0 2024-04-26T23:20:24.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.484+0000 7ff8f6ffd640 1 -- 172.21.15.47:0/3802577958 >> v1:172.21.15.47:6789/0 conn(0x7ff8f80cfde0 legacy=0x7ff8f80d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:24.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.484+0000 7ff8f6ffd640 1 --2- 172.21.15.47:0/3802577958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8e40039b0 0x7ff8e4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:24.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.484+0000 7ff8f6ffd640 1 --2- 172.21.15.47:0/3802577958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8e4004800 0x7ff8e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:24.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.484+0000 7ff8f6ffd640 1 --2- 172.21.15.47:0/3802577958 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8f8071090 0x7ff8e40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:24.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.484+0000 7ff8f6ffd640 1 -- 172.21.15.47:0/3802577958 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f8071500 con 0x7ff8e40039b0 2024-04-26T23:20:24.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.484+0000 7ff8f6ffd640 1 -- 172.21.15.47:0/3802577958 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e400b9c0 con 0x7ff8e4004800 2024-04-26T23:20:24.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.484+0000 7ff8f6ffd640 1 -- 172.21.15.47:0/3802577958 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7ff8e400bb40 con 0x7ff8f8071090 2024-04-26T23:20:24.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.484+0000 7ff8f77fe640 1 --2- 172.21.15.47:0/3802577958 >> v2:172.21.15.203:3300/0 conn(0x7ff8f8070350 0x7ff8f8070b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:20:24.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.484+0000 7ff8f7fff640 1 --2- 172.21.15.47:0/3802577958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8e40039b0 0x7ff8e4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:24.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.484+0000 7ff8fcf0e640 1 --2- 172.21.15.47:0/3802577958 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8f8071090 0x7ff8e40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:24.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.484+0000 7ff8f77fe640 1 --2- 172.21.15.47:0/3802577958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8e4004800 0x7ff8e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:24.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.484+0000 7ff8f7fff640 1 -- 172.21.15.47:0/3802577958 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8f8071090 msgr2=0x7ff8e40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:24.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.484+0000 7ff8f7fff640 1 --2- 172.21.15.47:0/3802577958 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8f8071090 0x7ff8e40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:24.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.484+0000 7ff8f7fff640 1 -- 172.21.15.47:0/3802577958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8e4004800 msgr2=0x7ff8e4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:24.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.484+0000 7ff8f7fff640 1 --2- 172.21.15.47:0/3802577958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8e4004800 0x7ff8e4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:24.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.484+0000 7ff8f7fff640 1 -- 172.21.15.47:0/3802577958 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8e4003520 con 0x7ff8e40039b0 2024-04-26T23:20:24.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.485+0000 7ff8f7fff640 1 --2- 172.21.15.47:0/3802577958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8e40039b0 0x7ff8e4003dd0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7ff8e000c240 tx=0x7ff8e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1e9928893838c5a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:24.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.485+0000 7ff8f6ffd640 1 -- 172.21.15.47:0/3802577958 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e0007d50 con 0x7ff8e40039b0 2024-04-26T23:20:24.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.485+0000 7ff8f6ffd640 1 -- 172.21.15.47:0/3802577958 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8e002e070 con 0x7ff8e40039b0 2024-04-26T23:20:24.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.485+0000 7ff8f6ffd640 1 -- 172.21.15.47:0/3802577958 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e0005d30 con 0x7ff8e40039b0 2024-04-26T23:20:24.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.486+0000 7ff8fe971640 1 -- 172.21.15.47:0/3802577958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8e40039b0 msgr2=0x7ff8e4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:24.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.486+0000 7ff8fe971640 1 --2- 172.21.15.47:0/3802577958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8e40039b0 0x7ff8e4003dd0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7ff8e000c240 tx=0x7ff8e4005830 comp rx=0 tx=0).stop 2024-04-26T23:20:24.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.486+0000 7ff8fcf0e640 1 -- 172.21.15.47:0/3802577958 reap_dead start 2024-04-26T23:20:24.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.486+0000 7ff8fe971640 1 -- 172.21.15.47:0/3802577958 shutdown_connections 2024-04-26T23:20:24.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.486+0000 7ff8fe971640 1 -- 172.21.15.47:0/3802577958 >> 172.21.15.47:0/3802577958 conn(0x7ff8f80d2d80 msgr2=0x7ff8f80d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:24.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.486+0000 7ff8fe971640 1 -- 172.21.15.47:0/3802577958 shutdown_connections 2024-04-26T23:20:24.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.486+0000 7ff8fe971640 1 -- 172.21.15.47:0/3802577958 wait complete. 2024-04-26T23:20:24.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.487+0000 7ff8fe971640 1 Processor -- start 2024-04-26T23:20:24.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.487+0000 7ff8fe971640 1 -- start start 2024-04-26T23:20:24.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.488+0000 7ff8fe971640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8e40039b0 0x7ff8f809d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:24.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.488+0000 7ff8fe971640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8f8161800 0x7ff8f80b3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:24.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.488+0000 7ff8fe971640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8f8161c60 0x7ff8f80b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:24.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.488+0000 7ff8fe971640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f806d9b0 con 0x7ff8e40039b0 2024-04-26T23:20:24.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.488+0000 7ff8fe971640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f806dd20 con 0x7ff8f8161c60 2024-04-26T23:20:24.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.488+0000 7ff8fe971640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7ff8f815e9f0 con 0x7ff8f8161800 2024-04-26T23:20:24.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.488+0000 7ff8f7fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8e40039b0 0x7ff8f809d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:24.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.488+0000 7ff8f7fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8e40039b0 0x7ff8f809d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40730/0 (socket says 172.21.15.47:40730) 2024-04-26T23:20:24.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.488+0000 7ff8f7fff640 1 -- 172.21.15.47:0/2370607715 learned_addr learned my addr 172.21.15.47:0/2370607715 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:24.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.488+0000 7ff8f77fe640 1 --2- 172.21.15.47:0/2370607715 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8f8161800 0x7ff8f80b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:24.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.488+0000 7ff8fcf0e640 1 --2- 172.21.15.47:0/2370607715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8f8161c60 0x7ff8f80b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:24.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.488+0000 7ff8f7fff640 1 -- 172.21.15.47:0/2370607715 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8f8161800 msgr2=0x7ff8f80b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:24.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.488+0000 7ff8f7fff640 1 --2- 172.21.15.47:0/2370607715 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8f8161800 0x7ff8f80b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:24.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.488+0000 7ff8f7fff640 1 -- 172.21.15.47:0/2370607715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8f8161c60 msgr2=0x7ff8f80b3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:24.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.488+0000 7ff8f7fff640 1 --2- 172.21.15.47:0/2370607715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8f8161c60 0x7ff8f80b3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:24.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.488+0000 7ff8f7fff640 1 -- 172.21.15.47:0/2370607715 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f814e1f0 con 0x7ff8e40039b0 2024-04-26T23:20:24.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.488+0000 7ff8fcf0e640 1 --2- 172.21.15.47:0/2370607715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8f8161c60 0x7ff8f80b3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:20:24.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.489+0000 7ff8f7fff640 1 --2- 172.21.15.47:0/2370607715 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8e40039b0 0x7ff8f809d550 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7ff8e000c240 tx=0x7ff8e0001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:24.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.489+0000 7ff8f4ff9640 1 -- 172.21.15.47:0/2370607715 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e0041070 con 0x7ff8e40039b0 2024-04-26T23:20:24.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.489+0000 7ff8f4ff9640 1 -- 172.21.15.47:0/2370607715 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8e0007280 con 0x7ff8e40039b0 2024-04-26T23:20:24.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.489+0000 7ff8fe971640 1 -- 172.21.15.47:0/2370607715 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8f814e4d0 con 0x7ff8e40039b0 2024-04-26T23:20:24.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.489+0000 7ff8f4ff9640 1 -- 172.21.15.47:0/2370607715 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e0007d50 con 0x7ff8e40039b0 2024-04-26T23:20:24.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.489+0000 7ff8fe971640 1 -- 172.21.15.47:0/2370607715 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8f814ea90 con 0x7ff8e40039b0 2024-04-26T23:20:24.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.490+0000 7ff8fe971640 1 -- 172.21.15.47:0/2370607715 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8f80694f0 con 0x7ff8e40039b0 2024-04-26T23:20:24.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.490+0000 7ff8f4ff9640 1 -- 172.21.15.47:0/2370607715 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff8e0044070 con 0x7ff8e40039b0 2024-04-26T23:20:24.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.491+0000 7ff8f4ff9640 1 --2- 172.21.15.47:0/2370607715 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7ff8d806c8f0 0x7ff8d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:24.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.491+0000 7ff8f4ff9640 1 -- 172.21.15.47:0/2370607715 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7ff8e00abf30 con 0x7ff8e40039b0 2024-04-26T23:20:24.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.491+0000 7ff8f77fe640 1 --2- 172.21.15.47:0/2370607715 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7ff8d806c8f0 0x7ff8d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:24.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.491+0000 7ff8f77fe640 1 --2- 172.21.15.47:0/2370607715 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7ff8d806c8f0 0x7ff8d806edb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff8f80d04a0 tx=0x7ff8e8009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:24.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.495+0000 7ff8f4ff9640 1 -- 172.21.15.47:0/2370607715 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff8e004b070 con 0x7ff8e40039b0 2024-04-26T23:20:24.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:24.724+0000 7ff8fe971640 1 -- 172.21.15.47:0/2370607715 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7ff8f80b22c0 con 0x7ff8e40039b0 2024-04-26T23:20:25.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.360+0000 7ff8f4ff9640 1 -- 172.21.15.47:0/2370607715 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v27) ==== 122+0+0 (secure 0 0 0) 0x7ff8e007a260 con 0x7ff8e40039b0 2024-04-26T23:20:25.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.364+0000 7ff8fe971640 1 -- 172.21.15.47:0/2370607715 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7ff8d806c8f0 msgr2=0x7ff8d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:25.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.364+0000 7ff8fe971640 1 --2- 172.21.15.47:0/2370607715 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7ff8d806c8f0 0x7ff8d806edb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff8f80d04a0 tx=0x7ff8e8009190 comp rx=0 tx=0).stop 2024-04-26T23:20:25.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.364+0000 7ff8fe971640 1 -- 172.21.15.47:0/2370607715 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8e40039b0 msgr2=0x7ff8f809d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:25.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.364+0000 7ff8fe971640 1 --2- 172.21.15.47:0/2370607715 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8e40039b0 0x7ff8f809d550 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7ff8e000c240 tx=0x7ff8e0001a40 comp rx=0 tx=0).stop 2024-04-26T23:20:25.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.364+0000 7ff8fe971640 1 -- 172.21.15.47:0/2370607715 shutdown_connections 2024-04-26T23:20:25.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.364+0000 7ff8fe971640 1 --2- 172.21.15.47:0/2370607715 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7ff8d806c8f0 0x7ff8d806edb0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:25.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.364+0000 7ff8fe971640 1 --2- 172.21.15.47:0/2370607715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8f8161c60 0x7ff8f80b3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:25.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.364+0000 7ff8fe971640 1 --2- 172.21.15.47:0/2370607715 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8f8161800 0x7ff8f80b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:25.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.364+0000 7ff8fe971640 1 --2- 172.21.15.47:0/2370607715 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8e40039b0 0x7ff8f809d550 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:25.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.364+0000 7ff8fe971640 1 -- 172.21.15.47:0/2370607715 >> 172.21.15.47:0/2370607715 conn(0x7ff8f80d2d80 msgr2=0x7ff8f80c88c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:25.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.364+0000 7ff8fe971640 1 -- 172.21.15.47:0/2370607715 shutdown_connections 2024-04-26T23:20:25.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.365+0000 7ff8fe971640 1 -- 172.21.15.47:0/2370607715 wait complete. 2024-04-26T23:20:25.379 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T23:20:25.379 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:20:25.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.486+0000 7f09871e4640 1 Processor -- start 2024-04-26T23:20:25.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.486+0000 7f09871e4640 1 -- start start 2024-04-26T23:20:25.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.487+0000 7f09871e4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0980162810 0x7f0980070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:25.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.487+0000 7f09871e4640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09800d10d0 con 0x7f09800d1b30 2024-04-26T23:20:25.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.487+0000 7f09871e4640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09800d1370 con 0x7f09800b2340 2024-04-26T23:20:25.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.487+0000 7f09871e4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f09800d0750 con 0x7f0980162810 2024-04-26T23:20:25.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.487+0000 7f0985781640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f09800d1b30 0x7f0980070b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35774/0 (socket says 172.21.15.47:35774) 2024-04-26T23:20:25.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.487+0000 7f0985781640 1 -- 172.21.15.47:0/2577105502 learned_addr learned my addr 172.21.15.47:0/2577105502 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:25.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.487+0000 7f0984f80640 1 --2- 172.21.15.47:0/2577105502 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0980162810 0x7f0980070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:25.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.487+0000 7f097f7fe640 1 -- 172.21.15.47:0/2577105502 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3844415481 0 0) 0x7f09800d10d0 con 0x7f09800d1b30 2024-04-26T23:20:25.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.487+0000 7f097f7fe640 1 -- 172.21.15.47:0/2577105502 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0970003520 con 0x7f09800d1b30 2024-04-26T23:20:25.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.487+0000 7f097f7fe640 1 -- 172.21.15.47:0/2577105502 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3554025272 0 0) 0x7f09800d1370 con 0x7f09800b2340 2024-04-26T23:20:25.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.487+0000 7f097f7fe640 1 -- 172.21.15.47:0/2577105502 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f09800d10d0 con 0x7f09800b2340 2024-04-26T23:20:25.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.488+0000 7f097f7fe640 1 -- 172.21.15.47:0/2577105502 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3097659582 0 0) 0x7f0970003520 con 0x7f09800d1b30 2024-04-26T23:20:25.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.488+0000 7f097f7fe640 1 -- 172.21.15.47:0/2577105502 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0980162810 msgr2=0x7f0980070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:25.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.488+0000 7f097f7fe640 1 --2- 172.21.15.47:0/2577105502 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0980162810 0x7f0980070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:25.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.488+0000 7f097f7fe640 1 -- 172.21.15.47:0/2577105502 >> v1:172.21.15.203:6789/0 conn(0x7f09800b2340 legacy=0x7f09800cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:25.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.488+0000 7f097f7fe640 1 -- 172.21.15.47:0/2577105502 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09800d01b0 con 0x7f09800d1b30 2024-04-26T23:20:25.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.488+0000 7f097f7fe640 1 -- 172.21.15.47:0/2577105502 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7f09780019f0 con 0x7f09800d1b30 2024-04-26T23:20:25.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.488+0000 7f0984f80640 1 --2- 172.21.15.47:0/2577105502 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0980162810 0x7f0980070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:20:25.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.488+0000 7f097f7fe640 1 -- 172.21.15.47:0/2577105502 >> v1:172.21.15.47:6789/0 conn(0x7f09800d1b30 legacy=0x7f0980070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:25.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.488+0000 7f097f7fe640 1 --2- 172.21.15.47:0/2577105502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0970003b70 0x7f0970003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:25.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.488+0000 7f097f7fe640 1 --2- 172.21.15.47:0/2577105502 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0980162810 0x7f0970004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:25.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.488+0000 7f097f7fe640 1 --2- 172.21.15.47:0/2577105502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0970004d40 0x7f09700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:25.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.488+0000 7f097f7fe640 1 -- 172.21.15.47:0/2577105502 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f09800d0750 con 0x7f0970004d40 2024-04-26T23:20:25.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.488+0000 7f097f7fe640 1 -- 172.21.15.47:0/2577105502 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f097000ba70 con 0x7f0970003b70 2024-04-26T23:20:25.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.488+0000 7f097f7fe640 1 -- 172.21.15.47:0/2577105502 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f097000bbf0 con 0x7f0980162810 2024-04-26T23:20:25.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.488+0000 7f0985781640 1 --2- 172.21.15.47:0/2577105502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0970004d40 0x7f09700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:25.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.488+0000 7f097ffff640 1 --2- 172.21.15.47:0/2577105502 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0980162810 0x7f0970004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:25.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.488+0000 7f0984f80640 1 --2- 172.21.15.47:0/2577105502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0970003b70 0x7f0970003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:25.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.488+0000 7f0985781640 1 -- 172.21.15.47:0/2577105502 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0980162810 msgr2=0x7f0970004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:25.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.488+0000 7f0985781640 1 --2- 172.21.15.47:0/2577105502 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0980162810 0x7f0970004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:25.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.488+0000 7f0985781640 1 -- 172.21.15.47:0/2577105502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0970003b70 msgr2=0x7f0970003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:25.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.488+0000 7f0985781640 1 --2- 172.21.15.47:0/2577105502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0970003b70 0x7f0970003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:25.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.488+0000 7f0985781640 1 -- 172.21.15.47:0/2577105502 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0970003520 con 0x7f0970004d40 2024-04-26T23:20:25.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.489+0000 7f0985781640 1 --2- 172.21.15.47:0/2577105502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0970004d40 0x7f09700051c0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f097800a840 tx=0x7f0970005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3eb412d245e47df server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:25.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.489+0000 7f097f7fe640 1 -- 172.21.15.47:0/2577105502 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f097800a430 con 0x7f0970004d40 2024-04-26T23:20:25.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.489+0000 7f097f7fe640 1 -- 172.21.15.47:0/2577105502 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f097802f070 con 0x7f0970004d40 2024-04-26T23:20:25.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.489+0000 7f097f7fe640 1 -- 172.21.15.47:0/2577105502 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f097802bc50 con 0x7f0970004d40 2024-04-26T23:20:25.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.489+0000 7f09871e4640 1 -- 172.21.15.47:0/2577105502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0970004d40 msgr2=0x7f09700051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:25.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.489+0000 7f09871e4640 1 --2- 172.21.15.47:0/2577105502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0970004d40 0x7f09700051c0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f097800a840 tx=0x7f0970005830 comp rx=0 tx=0).stop 2024-04-26T23:20:25.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.489+0000 7f0985781640 1 -- 172.21.15.47:0/2577105502 reap_dead start 2024-04-26T23:20:25.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.490+0000 7f09871e4640 1 -- 172.21.15.47:0/2577105502 shutdown_connections 2024-04-26T23:20:25.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.490+0000 7f09871e4640 1 -- 172.21.15.47:0/2577105502 >> 172.21.15.47:0/2577105502 conn(0x7f09800d2da0 msgr2=0x7f09800d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:25.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.490+0000 7f09871e4640 1 -- 172.21.15.47:0/2577105502 shutdown_connections 2024-04-26T23:20:25.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.490+0000 7f09871e4640 1 -- 172.21.15.47:0/2577105502 wait complete. 2024-04-26T23:20:25.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.491+0000 7f09871e4640 1 Processor -- start 2024-04-26T23:20:25.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.491+0000 7f09871e4640 1 -- start start 2024-04-26T23:20:25.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.492+0000 7f09871e4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0970004d40 0x7f09800c68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:25.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.492+0000 7f09871e4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09800c9400 0x7f09800b0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:25.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.492+0000 7f09871e4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09800c9860 0x7f09800b1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:25.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.492+0000 7f09871e4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f09800cdb00 con 0x7f09800c9400 2024-04-26T23:20:25.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.492+0000 7f09871e4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f09800d16a0 con 0x7f0970004d40 2024-04-26T23:20:25.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.492+0000 7f09871e4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f0980071260 con 0x7f09800c9860 2024-04-26T23:20:25.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.492+0000 7f097ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09800c9400 0x7f09800b0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:25.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.492+0000 7f097ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09800c9400 0x7f09800b0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40758/0 (socket says 172.21.15.47:40758) 2024-04-26T23:20:25.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.492+0000 7f097ffff640 1 -- 172.21.15.47:0/1889926469 learned_addr learned my addr 172.21.15.47:0/1889926469 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:25.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.492+0000 7f0984f80640 1 --2- 172.21.15.47:0/1889926469 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0970004d40 0x7f09800c68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:25.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.492+0000 7f0985781640 1 --2- 172.21.15.47:0/1889926469 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09800c9860 0x7f09800b1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:25.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.492+0000 7f097ffff640 1 -- 172.21.15.47:0/1889926469 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09800c9860 msgr2=0x7f09800b1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:25.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.492+0000 7f097ffff640 1 --2- 172.21.15.47:0/1889926469 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09800c9860 0x7f09800b1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:25.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.492+0000 7f097ffff640 1 -- 172.21.15.47:0/1889926469 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0970004d40 msgr2=0x7f09800c68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:25.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.492+0000 7f097ffff640 1 --2- 172.21.15.47:0/1889926469 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0970004d40 0x7f09800c68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:25.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.492+0000 7f097ffff640 1 -- 172.21.15.47:0/1889926469 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09800c6e10 con 0x7f09800c9400 2024-04-26T23:20:25.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.492+0000 7f0984f80640 1 --2- 172.21.15.47:0/1889926469 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0970004d40 0x7f09800c68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:20:25.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.492+0000 7f097ffff640 1 --2- 172.21.15.47:0/1889926469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09800c9400 0x7f09800b0c30 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f096c00f7d0 tx=0x7f096c00fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:25.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.492+0000 7f097d7fa640 1 -- 172.21.15.47:0/1889926469 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096c00fe80 con 0x7f09800c9400 2024-04-26T23:20:25.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.493+0000 7f097d7fa640 1 -- 172.21.15.47:0/1889926469 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f096c018070 con 0x7f09800c9400 2024-04-26T23:20:25.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.493+0000 7f09871e4640 1 -- 172.21.15.47:0/1889926469 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09800c8640 con 0x7f09800c9400 2024-04-26T23:20:25.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.493+0000 7f097d7fa640 1 -- 172.21.15.47:0/1889926469 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096c00d730 con 0x7f09800c9400 2024-04-26T23:20:25.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.493+0000 7f09871e4640 1 -- 172.21.15.47:0/1889926469 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09800c8c00 con 0x7f09800c9400 2024-04-26T23:20:25.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.494+0000 7f09871e4640 1 -- 172.21.15.47:0/1889926469 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09800694f0 con 0x7f09800c9400 2024-04-26T23:20:25.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.494+0000 7f097d7fa640 1 -- 172.21.15.47:0/1889926469 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f096c00d8d0 con 0x7f09800c9400 2024-04-26T23:20:25.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.494+0000 7f097d7fa640 1 --2- 172.21.15.47:0/1889926469 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f096406c8f0 0x7f096406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:25.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.495+0000 7f097d7fa640 1 -- 172.21.15.47:0/1889926469 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f096c08bb80 con 0x7f09800c9400 2024-04-26T23:20:25.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.495+0000 7f0984f80640 1 --2- 172.21.15.47:0/1889926469 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f096406c8f0 0x7f096406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:25.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.495+0000 7f0984f80640 1 --2- 172.21.15.47:0/1889926469 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f096406c8f0 0x7f096406edb0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0974009bd0 tx=0x7f0974008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:25.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.499+0000 7f097d7fa640 1 -- 172.21.15.47:0/1889926469 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f096c059eb0 con 0x7f09800c9400 2024-04-26T23:20:25.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.723+0000 7f09871e4640 1 -- 172.21.15.47:0/1889926469 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f09800cea10 con 0x7f09800c9400 2024-04-26T23:20:25.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.724+0000 7f097d7fa640 1 -- 172.21.15.47:0/1889926469 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5190 (secure 0 0 0) 0x7f096c0597b0 con 0x7f09800c9400 2024-04-26T23:20:25.725 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:20:25.725 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5865,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/805147373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":805147373},{"type":"v1","addr":"172.21.15.47:6833","nonce":805147373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5868,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3515322868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3515322868},{"type":"v1","addr":"172.21.15.47:6835","nonce":3515322868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":15008,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/4223205269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":4223205269},{"type":"v1","addr":"172.21.15.203:6837","nonce":4223205269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:20:21.712604+0000","modified":"2024-04-26T23:20:24.725165+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":15011},"failed":[],"damaged":[],"stopped":[],"info":{"gid_15011":{"gid":15011,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6836/1168022146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1168022146},{"type":"v1","addr":"172.21.15.203:6836","nonce":1168022146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":15011,"qdb_cluster":[15011]},"id":3}]} 2024-04-26T23:20:25.725 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 27 2024-04-26T23:20:25.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.726+0000 7f09871e4640 1 -- 172.21.15.47:0/1889926469 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f096406c8f0 msgr2=0x7f096406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:25.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.726+0000 7f09871e4640 1 --2- 172.21.15.47:0/1889926469 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f096406c8f0 0x7f096406edb0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0974009bd0 tx=0x7f0974008040 comp rx=0 tx=0).stop 2024-04-26T23:20:25.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.726+0000 7f09871e4640 1 -- 172.21.15.47:0/1889926469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09800c9400 msgr2=0x7f09800b0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:25.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.726+0000 7f09871e4640 1 --2- 172.21.15.47:0/1889926469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09800c9400 0x7f09800b0c30 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f096c00f7d0 tx=0x7f096c00fc90 comp rx=0 tx=0).stop 2024-04-26T23:20:25.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.726+0000 7f09871e4640 1 -- 172.21.15.47:0/1889926469 shutdown_connections 2024-04-26T23:20:25.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.726+0000 7f09871e4640 1 --2- 172.21.15.47:0/1889926469 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f096406c8f0 0x7f096406edb0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:25.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.726+0000 7f09871e4640 1 --2- 172.21.15.47:0/1889926469 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09800c9860 0x7f09800b1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:25.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.726+0000 7f09871e4640 1 --2- 172.21.15.47:0/1889926469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09800c9400 0x7f09800b0c30 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:25.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.726+0000 7f09871e4640 1 --2- 172.21.15.47:0/1889926469 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0970004d40 0x7f09800c68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:25.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.726+0000 7f09871e4640 1 -- 172.21.15.47:0/1889926469 >> 172.21.15.47:0/1889926469 conn(0x7f09800d2da0 msgr2=0x7f09800b3250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:25.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.726+0000 7f09871e4640 1 -- 172.21.15.47:0/1889926469 shutdown_connections 2024-04-26T23:20:25.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.726+0000 7f09871e4640 1 -- 172.21.15.47:0/1889926469 wait complete. 2024-04-26T23:20:25.738 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:20:25.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.838+0000 7f0fa4e72640 1 Processor -- start 2024-04-26T23:20:25.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.838+0000 7f0fa4e72640 1 -- start start 2024-04-26T23:20:25.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.839+0000 7f0fa4e72640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0fa00d3bc0 0x7f0fa00d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:25.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.839+0000 7f0fa4e72640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fa00b7e50 con 0x7f0fa00b5f20 2024-04-26T23:20:25.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.839+0000 7f0fa4e72640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fa00d1800 con 0x7f0fa0162880 2024-04-26T23:20:25.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.839+0000 7f0fa4e72640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f0fa00d0580 con 0x7f0fa00d3bc0 2024-04-26T23:20:25.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.839+0000 7f0f9ed9d640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0fa00b5f20 0x7f0fa00b8300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35776/0 (socket says 172.21.15.47:35776) 2024-04-26T23:20:25.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.839+0000 7f0f9ed9d640 1 -- 172.21.15.47:0/754397785 learned_addr learned my addr 172.21.15.47:0/754397785 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:25.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.840+0000 7f0f9dd9b640 1 --2- 172.21.15.47:0/754397785 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0fa00d3bc0 0x7f0fa00d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:25.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.840+0000 7f0f9d59a640 1 -- 172.21.15.47:0/754397785 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2886425024 0 0) 0x7f0fa00b7e50 con 0x7f0fa00b5f20 2024-04-26T23:20:25.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.840+0000 7f0f9d59a640 1 -- 172.21.15.47:0/754397785 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0f88003520 con 0x7f0fa00b5f20 2024-04-26T23:20:25.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.840+0000 7f0f9d59a640 1 -- 172.21.15.47:0/754397785 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3236403094 0 0) 0x7f0f98001110 con 0x7f0fa0162880 2024-04-26T23:20:25.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.840+0000 7f0f9d59a640 1 -- 172.21.15.47:0/754397785 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0fa00b7e50 con 0x7f0fa0162880 2024-04-26T23:20:25.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.840+0000 7f0f9d59a640 1 -- 172.21.15.47:0/754397785 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3640104937 0 0) 0x7f0f88003520 con 0x7f0fa00b5f20 2024-04-26T23:20:25.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.840+0000 7f0f9d59a640 1 -- 172.21.15.47:0/754397785 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0fa00d3bc0 msgr2=0x7f0fa00d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:25.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.840+0000 7f0f9d59a640 1 --2- 172.21.15.47:0/754397785 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0fa00d3bc0 0x7f0fa00d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:25.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.840+0000 7f0f9d59a640 1 -- 172.21.15.47:0/754397785 >> v1:172.21.15.203:6789/0 conn(0x7f0fa0162880 legacy=0x7f0fa00d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:25.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.840+0000 7f0f9d59a640 1 -- 172.21.15.47:0/754397785 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fa00d07d0 con 0x7f0fa00b5f20 2024-04-26T23:20:25.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.840+0000 7f0f9d59a640 1 -- 172.21.15.47:0/754397785 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7f0f900019f0 con 0x7f0fa00b5f20 2024-04-26T23:20:25.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.840+0000 7f0f9dd9b640 1 --2- 172.21.15.47:0/754397785 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0fa00d3bc0 0x7f0fa00d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:20:25.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.840+0000 7f0f9d59a640 1 -- 172.21.15.47:0/754397785 >> v1:172.21.15.47:6789/0 conn(0x7f0fa00b5f20 legacy=0x7f0fa00b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:25.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.840+0000 7f0f9d59a640 1 --2- 172.21.15.47:0/754397785 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f88003b70 0x7f0f88003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:25.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.840+0000 7f0f9d59a640 1 --2- 172.21.15.47:0/754397785 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0fa00d3bc0 0x7f0f88004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:25.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.840+0000 7f0f9d59a640 1 --2- 172.21.15.47:0/754397785 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f88004d40 0x7f0f880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:25.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.840+0000 7f0f9d59a640 1 -- 172.21.15.47:0/754397785 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa00d0580 con 0x7f0f88004d40 2024-04-26T23:20:25.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.840+0000 7f0f9d59a640 1 -- 172.21.15.47:0/754397785 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8800ba70 con 0x7f0f88003b70 2024-04-26T23:20:25.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.840+0000 7f0f9d59a640 1 -- 172.21.15.47:0/754397785 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f0f8800bbf0 con 0x7f0fa00d3bc0 2024-04-26T23:20:25.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.840+0000 7f0f9ed9d640 1 --2- 172.21.15.47:0/754397785 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f88004d40 0x7f0f880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:25.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.840+0000 7f0f9e59c640 1 --2- 172.21.15.47:0/754397785 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f88003b70 0x7f0f88003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:25.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.840+0000 7f0f9dd9b640 1 --2- 172.21.15.47:0/754397785 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0fa00d3bc0 0x7f0f88004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:25.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.840+0000 7f0f9ed9d640 1 -- 172.21.15.47:0/754397785 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0fa00d3bc0 msgr2=0x7f0f88004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:25.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.840+0000 7f0f9ed9d640 1 --2- 172.21.15.47:0/754397785 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0fa00d3bc0 0x7f0f88004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:25.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.840+0000 7f0f9ed9d640 1 -- 172.21.15.47:0/754397785 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f88003b70 msgr2=0x7f0f88003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:25.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.840+0000 7f0f9ed9d640 1 --2- 172.21.15.47:0/754397785 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f88003b70 0x7f0f88003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:25.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.841+0000 7f0f9ed9d640 1 -- 172.21.15.47:0/754397785 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f88003520 con 0x7f0f88004d40 2024-04-26T23:20:25.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.841+0000 7f0f9ed9d640 1 --2- 172.21.15.47:0/754397785 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f88004d40 0x7f0f880051c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f0f9000a840 tx=0x7f0f88005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e227a6d429d86f65 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:25.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.841+0000 7f0f9d59a640 1 -- 172.21.15.47:0/754397785 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9000a430 con 0x7f0f88004d40 2024-04-26T23:20:25.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.841+0000 7f0f9d59a640 1 -- 172.21.15.47:0/754397785 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f9002f070 con 0x7f0f88004d40 2024-04-26T23:20:25.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.841+0000 7f0f9d59a640 1 -- 172.21.15.47:0/754397785 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9002bc50 con 0x7f0f88004d40 2024-04-26T23:20:25.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.842+0000 7f0fa4e72640 1 -- 172.21.15.47:0/754397785 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f88004d40 msgr2=0x7f0f880051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:25.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.842+0000 7f0fa4e72640 1 --2- 172.21.15.47:0/754397785 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f88004d40 0x7f0f880051c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f0f9000a840 tx=0x7f0f88005830 comp rx=0 tx=0).stop 2024-04-26T23:20:25.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.842+0000 7f0f9ed9d640 1 -- 172.21.15.47:0/754397785 reap_dead start 2024-04-26T23:20:25.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.842+0000 7f0fa4e72640 1 -- 172.21.15.47:0/754397785 shutdown_connections 2024-04-26T23:20:25.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.842+0000 7f0fa4e72640 1 -- 172.21.15.47:0/754397785 >> 172.21.15.47:0/754397785 conn(0x7f0fa00d4f60 msgr2=0x7f0fa00d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:25.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.842+0000 7f0fa4e72640 1 -- 172.21.15.47:0/754397785 shutdown_connections 2024-04-26T23:20:25.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.842+0000 7f0fa4e72640 1 -- 172.21.15.47:0/754397785 wait complete. 2024-04-26T23:20:25.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.843+0000 7f0fa4e72640 1 Processor -- start 2024-04-26T23:20:25.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.843+0000 7f0fa4e72640 1 -- start start 2024-04-26T23:20:25.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.844+0000 7f0fa4e72640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f88004d40 0x7f0fa0156da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:25.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.844+0000 7f0fa4e72640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fa0159bf0 0x7f0fa01574a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:25.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.844+0000 7f0fa4e72640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0fa01f8d50 0x7f0fa00ba390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:25.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.844+0000 7f0fa4e72640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa0162c60 con 0x7f0fa0159bf0 2024-04-26T23:20:25.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.844+0000 7f0fa4e72640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa00d0d90 con 0x7f0fa01f8d50 2024-04-26T23:20:25.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.844+0000 7f0fa4e72640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f0fa0162de0 con 0x7f0f88004d40 2024-04-26T23:20:25.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.844+0000 7f0f9dd9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fa0159bf0 0x7f0fa01574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:25.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.844+0000 7f0f9dd9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fa0159bf0 0x7f0fa01574a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40780/0 (socket says 172.21.15.47:40780) 2024-04-26T23:20:25.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.844+0000 7f0f9dd9b640 1 -- 172.21.15.47:0/2069895035 learned_addr learned my addr 172.21.15.47:0/2069895035 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:25.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.844+0000 7f0f9e59c640 1 --2- 172.21.15.47:0/2069895035 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f88004d40 0x7f0fa0156da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:25.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.844+0000 7f0f9ed9d640 1 --2- 172.21.15.47:0/2069895035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0fa01f8d50 0x7f0fa00ba390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:25.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.845+0000 7f0f9dd9b640 1 -- 172.21.15.47:0/2069895035 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f88004d40 msgr2=0x7f0fa0156da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:25.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.845+0000 7f0f9dd9b640 1 --2- 172.21.15.47:0/2069895035 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f88004d40 0x7f0fa0156da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:25.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.845+0000 7f0f9dd9b640 1 -- 172.21.15.47:0/2069895035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0fa01f8d50 msgr2=0x7f0fa00ba390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:25.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.845+0000 7f0f9dd9b640 1 --2- 172.21.15.47:0/2069895035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0fa01f8d50 0x7f0fa00ba390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:25.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.845+0000 7f0f9dd9b640 1 -- 172.21.15.47:0/2069895035 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fa015a160 con 0x7f0fa0159bf0 2024-04-26T23:20:25.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.845+0000 7f0f9e59c640 1 --2- 172.21.15.47:0/2069895035 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f88004d40 0x7f0fa0156da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:20:25.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.845+0000 7f0f9dd9b640 1 --2- 172.21.15.47:0/2069895035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fa0159bf0 0x7f0fa01574a0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f0f940119d0 tx=0x7f0f94011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:25.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.845+0000 7f0f8effd640 1 -- 172.21.15.47:0/2069895035 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9400faf0 con 0x7f0fa0159bf0 2024-04-26T23:20:25.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.845+0000 7f0f8effd640 1 -- 172.21.15.47:0/2069895035 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f9400fc90 con 0x7f0fa0159bf0 2024-04-26T23:20:25.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.845+0000 7f0f8effd640 1 -- 172.21.15.47:0/2069895035 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f94004030 con 0x7f0fa0159bf0 2024-04-26T23:20:25.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.845+0000 7f0fa4e72640 1 -- 172.21.15.47:0/2069895035 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fa015a440 con 0x7f0fa0159bf0 2024-04-26T23:20:25.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.845+0000 7f0fa4e72640 1 -- 172.21.15.47:0/2069895035 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fa015aa00 con 0x7f0fa0159bf0 2024-04-26T23:20:25.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.846+0000 7f0fa4e72640 1 -- 172.21.15.47:0/2069895035 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fa00694f0 con 0x7f0fa0159bf0 2024-04-26T23:20:25.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.846+0000 7f0f8effd640 1 -- 172.21.15.47:0/2069895035 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0f94012040 con 0x7f0fa0159bf0 2024-04-26T23:20:25.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.847+0000 7f0f8effd640 1 --2- 172.21.15.47:0/2069895035 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f0f8006c8f0 0x7f0f8006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:25.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.847+0000 7f0f8effd640 1 -- 172.21.15.47:0/2069895035 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f0f9408c060 con 0x7f0fa0159bf0 2024-04-26T23:20:25.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.847+0000 7f0f9e59c640 1 --2- 172.21.15.47:0/2069895035 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f0f8006c8f0 0x7f0f8006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:25.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.847+0000 7f0f9e59c640 1 --2- 172.21.15.47:0/2069895035 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f0f8006c8f0 0x7f0f8006edb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0f98005430 tx=0x7f0f98009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:25.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:25.851+0000 7f0f8effd640 1 -- 172.21.15.47:0/2069895035 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0f94019070 con 0x7f0fa0159bf0 2024-04-26T23:20:26.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.037+0000 7f0fa4e72640 1 -- 172.21.15.47:0/2069895035 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0fa00d4380 con 0x7f0fa0159bf0 2024-04-26T23:20:26.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.038+0000 7f0f8effd640 1 -- 172.21.15.47:0/2069895035 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) ==== 74+0+16902 (secure 0 0 0) 0x7f0f94059cb0 con 0x7f0fa0159bf0 2024-04-26T23:20:26.039 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:20:26.040 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":54,"fsid":"399006d3-4db1-43e6-ba81-a79a276eabfd","created":"2024-04-26T23:18:06.731211+0000","modified":"2024-04-26T23:20:23.340537+0000","last_up_change":"2024-04-26T23:18:17.808736+0000","last_in_change":"2024-04-26T23:18:08.945859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:18:16.937492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:20:19.975691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-26T23:20:20.657778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ee091567-fea0-4a14-bf4e-0e16c20bb2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6819","nonce":326520779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6821","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6823","nonce":326520779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6829","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6831","nonce":326520779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6825","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6827","nonce":326520779}]},"public_addr":"172.21.15.47:6819/326520779","cluster_addr":"172.21.15.47:6823/326520779","heartbeat_back_addr":"172.21.15.47:6831/326520779","heartbeat_front_addr":"172.21.15.47:6827/326520779","state":["exists","up"]},{"osd":1,"uuid":"c14440a0-bba5-4931-a6db-b736c08e869b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6809","nonce":1912610931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6811","nonce":1912610931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6815","nonce":1912610931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6813","nonce":1912610931}]},"public_addr":"172.21.15.47:6809/1912610931","cluster_addr":"172.21.15.47:6811/1912610931","heartbeat_back_addr":"172.21.15.47:6815/1912610931","heartbeat_front_addr":"172.21.15.47:6813/1912610931","state":["exists","up"]},{"osd":2,"uuid":"3940f8de-c9b2-4cd3-a911-9c301729eb30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6801","nonce":2569869305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6803","nonce":2569869305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6807","nonce":2569869305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6805","nonce":2569869305}]},"public_addr":"172.21.15.47:6801/2569869305","cluster_addr":"172.21.15.47:6803/2569869305","heartbeat_back_addr":"172.21.15.47:6807/2569869305","heartbeat_front_addr":"172.21.15.47:6805/2569869305","state":["exists","up"]},{"osd":3,"uuid":"1286a67e-8958-426a-8c2e-bc85498c619c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6818","nonce":2791411683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6822","nonce":2791411683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6830","nonce":2791411683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6826","nonce":2791411683}]},"public_addr":"172.21.15.47:6818/2791411683","cluster_addr":"172.21.15.47:6822/2791411683","heartbeat_back_addr":"172.21.15.47:6830/2791411683","heartbeat_front_addr":"172.21.15.47:6826/2791411683","state":["exists","up"]},{"osd":4,"uuid":"d8ee2c20-e5b7-46c2-9999-b234070126cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6809","nonce":441313104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6811","nonce":441313104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6815","nonce":441313104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6813","nonce":441313104}]},"public_addr":"172.21.15.203:6809/441313104","cluster_addr":"172.21.15.203:6811/441313104","heartbeat_back_addr":"172.21.15.203:6815/441313104","heartbeat_front_addr":"172.21.15.203:6813/441313104","state":["exists","up"]},{"osd":5,"uuid":"b453a8f0-f324-4840-9ae0-d67bdadb9f8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6801","nonce":56838444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6803","nonce":56838444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6807","nonce":56838444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6805","nonce":56838444}]},"public_addr":"172.21.15.203:6801/56838444","cluster_addr":"172.21.15.203:6803/56838444","heartbeat_back_addr":"172.21.15.203:6807/56838444","heartbeat_front_addr":"172.21.15.203:6805/56838444","state":["exists","up"]},{"osd":6,"uuid":"a6d31951-2d4b-4e8d-a510-6283b01dc158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6825","nonce":2738517080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6827","nonce":2738517080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6831","nonce":2738517080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6829","nonce":2738517080}]},"public_addr":"172.21.15.203:6825/2738517080","cluster_addr":"172.21.15.203:6827/2738517080","heartbeat_back_addr":"172.21.15.203:6831/2738517080","heartbeat_front_addr":"172.21.15.203:6829/2738517080","state":["exists","up"]},{"osd":7,"uuid":"7119b8c0-fe96-402d-9e75-7b509d9d44d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6817","nonce":1315565058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6819","nonce":1315565058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6823","nonce":1315565058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6821","nonce":1315565058}]},"public_addr":"172.21.15.203:6817/1315565058","cluster_addr":"172.21.15.203:6819/1315565058","heartbeat_back_addr":"172.21.15.203:6823/1315565058","heartbeat_front_addr":"172.21.15.203:6821/1315565058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.121921+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.137494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.084814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.111593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.072430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.125066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.097867+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.057061+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:20:26.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.040+0000 7f0fa4e72640 1 -- 172.21.15.47:0/2069895035 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f0f8006c8f0 msgr2=0x7f0f8006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:26.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.040+0000 7f0fa4e72640 1 --2- 172.21.15.47:0/2069895035 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f0f8006c8f0 0x7f0f8006edb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0f98005430 tx=0x7f0f98009040 comp rx=0 tx=0).stop 2024-04-26T23:20:26.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.040+0000 7f0fa4e72640 1 -- 172.21.15.47:0/2069895035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fa0159bf0 msgr2=0x7f0fa01574a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:26.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.040+0000 7f0fa4e72640 1 --2- 172.21.15.47:0/2069895035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fa0159bf0 0x7f0fa01574a0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f0f940119d0 tx=0x7f0f94011e90 comp rx=0 tx=0).stop 2024-04-26T23:20:26.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.040+0000 7f0fa4e72640 1 -- 172.21.15.47:0/2069895035 shutdown_connections 2024-04-26T23:20:26.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.040+0000 7f0fa4e72640 1 --2- 172.21.15.47:0/2069895035 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f0f8006c8f0 0x7f0f8006edb0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:26.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.040+0000 7f0fa4e72640 1 --2- 172.21.15.47:0/2069895035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0fa01f8d50 0x7f0fa00ba390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:26.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.040+0000 7f0fa4e72640 1 --2- 172.21.15.47:0/2069895035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fa0159bf0 0x7f0fa01574a0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:26.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.040+0000 7f0fa4e72640 1 --2- 172.21.15.47:0/2069895035 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f88004d40 0x7f0fa0156da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:26.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.040+0000 7f0fa4e72640 1 -- 172.21.15.47:0/2069895035 >> 172.21.15.47:0/2069895035 conn(0x7f0fa00d4f60 msgr2=0x7f0fa015fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:26.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.040+0000 7f0fa4e72640 1 -- 172.21.15.47:0/2069895035 shutdown_connections 2024-04-26T23:20:26.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.040+0000 7f0fa4e72640 1 -- 172.21.15.47:0/2069895035 wait complete. 2024-04-26T23:20:26.052 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T23:20:26.052 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T23:20:26.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.152+0000 7f15c3b5f640 1 Processor -- start 2024-04-26T23:20:26.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.152+0000 7f15c3b5f640 1 -- start start 2024-04-26T23:20:26.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.153+0000 7f15c3b5f640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f15bc0d3bc0 0x7f15bc0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:26.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.153+0000 7f15c3b5f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15bc0b5f20 0x7f15bc0b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:26.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.153+0000 7f15c3b5f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15bc0b7e50 con 0x7f15bc162880 2024-04-26T23:20:26.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.153+0000 7f15c3b5f640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f15bc0d0580 con 0x7f15bc0d3bc0 2024-04-26T23:20:26.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.153+0000 7f15c3b5f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f15bc0d1800 con 0x7f15bc0b5f20 2024-04-26T23:20:26.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.153+0000 7f15c10fa640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f15bc0d3bc0 0x7f15bc0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:26.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.153+0000 7f15c10fa640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f15bc0d3bc0 0x7f15bc0d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40786/0 (socket says 172.21.15.47:40786) 2024-04-26T23:20:26.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.153+0000 7f15c10fa640 1 -- 172.21.15.47:0/1695567527 learned_addr learned my addr 172.21.15.47:0/1695567527 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:26.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.153+0000 7f15c20fc640 1 --2- 172.21.15.47:0/1695567527 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15bc0b5f20 0x7f15bc0b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:26.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.153+0000 7f15c10fa640 1 -- 172.21.15.47:0/1695567527 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15bc0b5f20 msgr2=0x7f15bc0b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:26.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.153+0000 7f15c10fa640 1 --2- 172.21.15.47:0/1695567527 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15bc0b5f20 0x7f15bc0b8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:26.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.153+0000 7f15c10fa640 1 -- 172.21.15.47:0/1695567527 >> v1:172.21.15.203:6789/0 conn(0x7f15bc162880 legacy=0x7f15bc0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:26.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.153+0000 7f15c10fa640 1 -- 172.21.15.47:0/1695567527 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15bc0d07d0 con 0x7f15bc0d3bc0 2024-04-26T23:20:26.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.154+0000 7f15c10fa640 1 --2- 172.21.15.47:0/1695567527 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15bc0d3bc0 0x7f15bc0d2cb0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f15b0009870 tx=0x7f15b002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a86269b10893922 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:26.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.154+0000 7f15c08f9640 1 -- 172.21.15.47:0/1695567527 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b0037070 con 0x7f15bc0d3bc0 2024-04-26T23:20:26.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.154+0000 7f15c08f9640 1 -- 172.21.15.47:0/1695567527 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15b0032440 con 0x7f15bc0d3bc0 2024-04-26T23:20:26.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.154+0000 7f15c08f9640 1 -- 172.21.15.47:0/1695567527 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b0032810 con 0x7f15bc0d3bc0 2024-04-26T23:20:26.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.155+0000 7f15c3b5f640 1 -- 172.21.15.47:0/1695567527 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15bc0d3bc0 msgr2=0x7f15bc0d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:26.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.155+0000 7f15c3b5f640 1 --2- 172.21.15.47:0/1695567527 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15bc0d3bc0 0x7f15bc0d2cb0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f15b0009870 tx=0x7f15b002a210 comp rx=0 tx=0).stop 2024-04-26T23:20:26.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.155+0000 7f15c3b5f640 1 -- 172.21.15.47:0/1695567527 shutdown_connections 2024-04-26T23:20:26.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.155+0000 7f15c3b5f640 1 --2- 172.21.15.47:0/1695567527 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15bc0b5f20 0x7f15bc0b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:26.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.155+0000 7f15c3b5f640 1 --2- 172.21.15.47:0/1695567527 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15bc0d3bc0 0x7f15bc0d2cb0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:26.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.155+0000 7f15c3b5f640 1 -- 172.21.15.47:0/1695567527 >> 172.21.15.47:0/1695567527 conn(0x7f15bc0d4f60 msgr2=0x7f15bc0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:26.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.155+0000 7f15c3b5f640 1 -- 172.21.15.47:0/1695567527 shutdown_connections 2024-04-26T23:20:26.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.155+0000 7f15c3b5f640 1 -- 172.21.15.47:0/1695567527 wait complete. 2024-04-26T23:20:26.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.156+0000 7f15c3b5f640 1 Processor -- start 2024-04-26T23:20:26.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.156+0000 7f15c3b5f640 1 -- start start 2024-04-26T23:20:26.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.157+0000 7f15c3b5f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15bc0b5f20 0x7f15bc1f46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:26.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.157+0000 7f15c3b5f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15bc0d3bc0 0x7f15bc0cda00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:26.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.157+0000 7f15c3b5f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15bc162880 0x7f15bc0b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:26.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.157+0000 7f15c3b5f640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f15bc162c60 con 0x7f15bc162880 2024-04-26T23:20:26.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.157+0000 7f15c3b5f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f15bc0d0d90 con 0x7f15bc0d3bc0 2024-04-26T23:20:26.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.157+0000 7f15c3b5f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f15bc162de0 con 0x7f15bc0b5f20 2024-04-26T23:20:26.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.157+0000 7f15c20fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15bc162880 0x7f15bc0b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:26.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.157+0000 7f15c20fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15bc162880 0x7f15bc0b69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40796/0 (socket says 172.21.15.47:40796) 2024-04-26T23:20:26.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.157+0000 7f15c20fc640 1 -- 172.21.15.47:0/4217220207 learned_addr learned my addr 172.21.15.47:0/4217220207 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:26.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.157+0000 7f15c18fb640 1 --2- 172.21.15.47:0/4217220207 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15bc0b5f20 0x7f15bc1f46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:26.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.157+0000 7f15c10fa640 1 --2- 172.21.15.47:0/4217220207 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15bc0d3bc0 0x7f15bc0cda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:26.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.157+0000 7f15c20fc640 1 -- 172.21.15.47:0/4217220207 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15bc0b5f20 msgr2=0x7f15bc1f46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:26.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.157+0000 7f15c20fc640 1 --2- 172.21.15.47:0/4217220207 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15bc0b5f20 0x7f15bc1f46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:26.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.157+0000 7f15c20fc640 1 -- 172.21.15.47:0/4217220207 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15bc0d3bc0 msgr2=0x7f15bc0cda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:26.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.157+0000 7f15c20fc640 1 --2- 172.21.15.47:0/4217220207 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15bc0d3bc0 0x7f15bc0cda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:26.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.157+0000 7f15c20fc640 1 -- 172.21.15.47:0/4217220207 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15bc0ce4c0 con 0x7f15bc162880 2024-04-26T23:20:26.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.158+0000 7f15c20fc640 1 --2- 172.21.15.47:0/4217220207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15bc162880 0x7f15bc0b69e0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f15ac00cc30 tx=0x7f15ac00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:26.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.158+0000 7f15ba7fc640 1 -- 172.21.15.47:0/4217220207 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac00ce40 con 0x7f15bc162880 2024-04-26T23:20:26.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.158+0000 7f15ba7fc640 1 -- 172.21.15.47:0/4217220207 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15ac003990 con 0x7f15bc162880 2024-04-26T23:20:26.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.158+0000 7f15c3b5f640 1 -- 172.21.15.47:0/4217220207 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15bc0ce7a0 con 0x7f15bc162880 2024-04-26T23:20:26.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.158+0000 7f15ba7fc640 1 -- 172.21.15.47:0/4217220207 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac003d40 con 0x7f15bc162880 2024-04-26T23:20:26.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.158+0000 7f15c3b5f640 1 -- 172.21.15.47:0/4217220207 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15bc159c90 con 0x7f15bc162880 2024-04-26T23:20:26.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.159+0000 7f15c3b5f640 1 -- 172.21.15.47:0/4217220207 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15bc0694f0 con 0x7f15bc162880 2024-04-26T23:20:26.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.159+0000 7f15ba7fc640 1 -- 172.21.15.47:0/4217220207 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f15ac003fe0 con 0x7f15bc162880 2024-04-26T23:20:26.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.160+0000 7f15ba7fc640 1 --2- 172.21.15.47:0/4217220207 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f15a806c8f0 0x7f15a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:26.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.160+0000 7f15ba7fc640 1 -- 172.21.15.47:0/4217220207 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f15ac08c1d0 con 0x7f15bc162880 2024-04-26T23:20:26.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.160+0000 7f15c18fb640 1 --2- 172.21.15.47:0/4217220207 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f15a806c8f0 0x7f15a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:26.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.160+0000 7f15c18fb640 1 --2- 172.21.15.47:0/4217220207 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f15a806c8f0 0x7f15a806edb0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f15b4004f60 tx=0x7f15b4007e30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:26.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.164+0000 7f15ba7fc640 1 -- 172.21.15.47:0/4217220207 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f15ac014030 con 0x7f15bc162880 2024-04-26T23:20:26.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.349+0000 7f15c3b5f640 1 -- 172.21.15.47:0/4217220207 --> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f15bc0cfd70 con 0x7f15a806c8f0 2024-04-26T23:20:26.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.356+0000 7f15ba7fc640 1 -- 172.21.15.47:0/4217220207 <== mgr.4099 v2:172.21.15.203:6832/992162732 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251905 (secure 0 0 0) 0x7f15bc0cfd70 con 0x7f15a806c8f0 2024-04-26T23:20:26.359 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:20:26.361 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-26T23:20:26.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.358+0000 7f15c3b5f640 1 -- 172.21.15.47:0/4217220207 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f15a806c8f0 msgr2=0x7f15a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:26.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.358+0000 7f15c3b5f640 1 --2- 172.21.15.47:0/4217220207 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f15a806c8f0 0x7f15a806edb0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f15b4004f60 tx=0x7f15b4007e30 comp rx=0 tx=0).stop 2024-04-26T23:20:26.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.358+0000 7f15c3b5f640 1 -- 172.21.15.47:0/4217220207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15bc162880 msgr2=0x7f15bc0b69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:26.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.358+0000 7f15c3b5f640 1 --2- 172.21.15.47:0/4217220207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15bc162880 0x7f15bc0b69e0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f15ac00cc30 tx=0x7f15ac00a5c0 comp rx=0 tx=0).stop 2024-04-26T23:20:26.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.358+0000 7f15c3b5f640 1 -- 172.21.15.47:0/4217220207 shutdown_connections 2024-04-26T23:20:26.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.358+0000 7f15c3b5f640 1 --2- 172.21.15.47:0/4217220207 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f15a806c8f0 0x7f15a806edb0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:26.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.358+0000 7f15c3b5f640 1 --2- 172.21.15.47:0/4217220207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15bc162880 0x7f15bc0b69e0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:26.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.358+0000 7f15c3b5f640 1 --2- 172.21.15.47:0/4217220207 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15bc0d3bc0 0x7f15bc0cda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:26.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.358+0000 7f15c3b5f640 1 --2- 172.21.15.47:0/4217220207 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15bc0b5f20 0x7f15bc1f46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:26.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.358+0000 7f15c3b5f640 1 -- 172.21.15.47:0/4217220207 >> 172.21.15.47:0/4217220207 conn(0x7f15bc0d4f60 msgr2=0x7f15bc0d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:26.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.358+0000 7f15c3b5f640 1 -- 172.21.15.47:0/4217220207 shutdown_connections 2024-04-26T23:20:26.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.358+0000 7f15c3b5f640 1 -- 172.21.15.47:0/4217220207 wait complete. 2024-04-26T23:20:26.371 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":111,"stamp":"2024-04-26T23:20:24.955404+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":111,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238912,"kb_used_data":6084,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735808,"statfs":{"total":773094113280,"available":772849467392,"internally_reserved":0,"allocated":6230016,"data_stored":3433533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.628307"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.359916+0000","last_change":"2024-04-26T23:20:22.351070+0000","last_active":"2024-04-26T23:20:22.359916+0000","last_peered":"2024-04-26T23:20:22.359916+0000","last_clean":"2024-04-26T23:20:22.359916+0000","last_became_active":"2024-04-26T23:20:22.350445+0000","last_became_peered":"2024-04-26T23:20:22.350445+0000","last_unstale":"2024-04-26T23:20:22.359916+0000","last_undegraded":"2024-04-26T23:20:22.359916+0000","last_fullsized":"2024-04-26T23:20:22.359916+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:24:59.174021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.366967+0000","last_change":"2024-04-26T23:20:22.365214+0000","last_active":"2024-04-26T23:20:22.366967+0000","last_peered":"2024-04-26T23:20:22.366967+0000","last_clean":"2024-04-26T23:20:22.366967+0000","last_became_active":"2024-04-26T23:20:22.364935+0000","last_became_peered":"2024-04-26T23:20:22.364935+0000","last_unstale":"2024-04-26T23:20:22.366967+0000","last_undegraded":"2024-04-26T23:20:22.366967+0000","last_fullsized":"2024-04-26T23:20:22.366967+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:22.991265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.362676+0000","last_change":"2024-04-26T23:20:22.359609+0000","last_active":"2024-04-26T23:20:22.362676+0000","last_peered":"2024-04-26T23:20:22.362676+0000","last_clean":"2024-04-26T23:20:22.362676+0000","last_became_active":"2024-04-26T23:20:22.359320+0000","last_became_peered":"2024-04-26T23:20:22.359320+0000","last_unstale":"2024-04-26T23:20:22.362676+0000","last_undegraded":"2024-04-26T23:20:22.362676+0000","last_fullsized":"2024-04-26T23:20:22.362676+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:40:57.749170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.353277+0000","last_change":"2024-04-26T23:20:22.358050+0000","last_active":"2024-04-26T23:20:23.353277+0000","last_peered":"2024-04-26T23:20:23.353277+0000","last_clean":"2024-04-26T23:20:23.353277+0000","last_became_active":"2024-04-26T23:20:22.356118+0000","last_became_peered":"2024-04-26T23:20:22.356118+0000","last_unstale":"2024-04-26T23:20:23.353277+0000","last_undegraded":"2024-04-26T23:20:23.353277+0000","last_fullsized":"2024-04-26T23:20:23.353277+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:53:38.181432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.362932+0000","last_change":"2024-04-26T23:20:22.360285+0000","last_active":"2024-04-26T23:20:22.362932+0000","last_peered":"2024-04-26T23:20:22.362932+0000","last_clean":"2024-04-26T23:20:22.362932+0000","last_became_active":"2024-04-26T23:20:22.359990+0000","last_became_peered":"2024-04-26T23:20:22.359990+0000","last_unstale":"2024-04-26T23:20:22.362932+0000","last_undegraded":"2024-04-26T23:20:22.362932+0000","last_fullsized":"2024-04-26T23:20:22.362932+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:31:47.385990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.349663+0000","last_change":"2024-04-26T23:20:22.347344+0000","last_active":"2024-04-26T23:20:22.349663+0000","last_peered":"2024-04-26T23:20:22.349663+0000","last_clean":"2024-04-26T23:20:22.349663+0000","last_became_active":"2024-04-26T23:20:22.347061+0000","last_became_peered":"2024-04-26T23:20:22.347061+0000","last_unstale":"2024-04-26T23:20:22.349663+0000","last_undegraded":"2024-04-26T23:20:22.349663+0000","last_fullsized":"2024-04-26T23:20:22.349663+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:04:00.567851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.513936+0000","last_change":"2024-04-26T23:20:22.358260+0000","last_active":"2024-04-26T23:20:23.513936+0000","last_peered":"2024-04-26T23:20:23.513936+0000","last_clean":"2024-04-26T23:20:23.513936+0000","last_became_active":"2024-04-26T23:20:22.357990+0000","last_became_peered":"2024-04-26T23:20:22.357990+0000","last_unstale":"2024-04-26T23:20:23.513936+0000","last_undegraded":"2024-04-26T23:20:23.513936+0000","last_fullsized":"2024-04-26T23:20:23.513936+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:49:27.891875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.361518+0000","last_change":"2024-04-26T23:20:22.359488+0000","last_active":"2024-04-26T23:20:22.361518+0000","last_peered":"2024-04-26T23:20:22.361518+0000","last_clean":"2024-04-26T23:20:22.361518+0000","last_became_active":"2024-04-26T23:20:22.359199+0000","last_became_peered":"2024-04-26T23:20:22.359199+0000","last_unstale":"2024-04-26T23:20:22.361518+0000","last_undegraded":"2024-04-26T23:20:22.361518+0000","last_fullsized":"2024-04-26T23:20:22.361518+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:30:38.323701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.514322+0000","last_change":"2024-04-26T23:20:22.366476+0000","last_active":"2024-04-26T23:20:23.514322+0000","last_peered":"2024-04-26T23:20:23.514322+0000","last_clean":"2024-04-26T23:20:23.514322+0000","last_became_active":"2024-04-26T23:20:22.366199+0000","last_became_peered":"2024-04-26T23:20:22.366199+0000","last_unstale":"2024-04-26T23:20:23.514322+0000","last_undegraded":"2024-04-26T23:20:23.514322+0000","last_fullsized":"2024-04-26T23:20:23.514322+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:47:38.765819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.358623+0000","last_change":"2024-04-26T23:20:22.354156+0000","last_active":"2024-04-26T23:20:22.358623+0000","last_peered":"2024-04-26T23:20:22.358623+0000","last_clean":"2024-04-26T23:20:22.358623+0000","last_became_active":"2024-04-26T23:20:22.351645+0000","last_became_peered":"2024-04-26T23:20:22.351645+0000","last_unstale":"2024-04-26T23:20:22.358623+0000","last_undegraded":"2024-04-26T23:20:22.358623+0000","last_fullsized":"2024-04-26T23:20:22.358623+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:23.179907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.356131+0000","last_change":"2024-04-26T23:20:22.351221+0000","last_active":"2024-04-26T23:20:22.356131+0000","last_peered":"2024-04-26T23:20:22.356131+0000","last_clean":"2024-04-26T23:20:22.356131+0000","last_became_active":"2024-04-26T23:20:22.350930+0000","last_became_peered":"2024-04-26T23:20:22.350930+0000","last_unstale":"2024-04-26T23:20:22.356131+0000","last_undegraded":"2024-04-26T23:20:22.356131+0000","last_fullsized":"2024-04-26T23:20:22.356131+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:10:14.051477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.353370+0000","last_change":"2024-04-26T23:20:22.348885+0000","last_active":"2024-04-26T23:20:23.353370+0000","last_peered":"2024-04-26T23:20:23.353370+0000","last_clean":"2024-04-26T23:20:23.353370+0000","last_became_active":"2024-04-26T23:20:22.348628+0000","last_became_peered":"2024-04-26T23:20:22.348628+0000","last_unstale":"2024-04-26T23:20:23.353370+0000","last_undegraded":"2024-04-26T23:20:23.353370+0000","last_fullsized":"2024-04-26T23:20:23.353370+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:37:16.426093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.353016+0000","last_change":"2024-04-26T23:20:22.353648+0000","last_active":"2024-04-26T23:20:23.353016+0000","last_peered":"2024-04-26T23:20:23.353016+0000","last_clean":"2024-04-26T23:20:23.353016+0000","last_became_active":"2024-04-26T23:20:22.353324+0000","last_became_peered":"2024-04-26T23:20:22.353324+0000","last_unstale":"2024-04-26T23:20:23.353016+0000","last_undegraded":"2024-04-26T23:20:23.353016+0000","last_fullsized":"2024-04-26T23:20:23.353016+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:23.179907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.514063+0000","last_change":"2024-04-26T23:20:22.361471+0000","last_active":"2024-04-26T23:20:23.514063+0000","last_peered":"2024-04-26T23:20:23.514063+0000","last_clean":"2024-04-26T23:20:23.514063+0000","last_became_active":"2024-04-26T23:20:22.361201+0000","last_became_peered":"2024-04-26T23:20:22.361201+0000","last_unstale":"2024-04-26T23:20:23.514063+0000","last_undegraded":"2024-04-26T23:20:23.514063+0000","last_fullsized":"2024-04-26T23:20:23.514063+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:29.660098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.513971+0000","last_change":"2024-04-26T23:20:22.361208+0000","last_active":"2024-04-26T23:20:23.513971+0000","last_peered":"2024-04-26T23:20:23.513971+0000","last_clean":"2024-04-26T23:20:23.513971+0000","last_became_active":"2024-04-26T23:20:22.360906+0000","last_became_peered":"2024-04-26T23:20:22.360906+0000","last_unstale":"2024-04-26T23:20:23.513971+0000","last_undegraded":"2024-04-26T23:20:23.513971+0000","last_fullsized":"2024-04-26T23:20:23.513971+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:53:01.027225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.356445+0000","last_change":"2024-04-26T23:20:22.354199+0000","last_active":"2024-04-26T23:20:22.356445+0000","last_peered":"2024-04-26T23:20:22.356445+0000","last_clean":"2024-04-26T23:20:22.356445+0000","last_became_active":"2024-04-26T23:20:22.353803+0000","last_became_peered":"2024-04-26T23:20:22.353803+0000","last_unstale":"2024-04-26T23:20:22.356445+0000","last_undegraded":"2024-04-26T23:20:22.356445+0000","last_fullsized":"2024-04-26T23:20:22.356445+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:56:10.119661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.365801+0000","last_change":"2024-04-26T23:20:22.357968+0000","last_active":"2024-04-26T23:20:22.365801+0000","last_peered":"2024-04-26T23:20:22.365801+0000","last_clean":"2024-04-26T23:20:22.365801+0000","last_became_active":"2024-04-26T23:20:22.357686+0000","last_became_peered":"2024-04-26T23:20:22.357686+0000","last_unstale":"2024-04-26T23:20:22.365801+0000","last_undegraded":"2024-04-26T23:20:22.365801+0000","last_fullsized":"2024-04-26T23:20:22.365801+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:37:16.426093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.354120+0000","last_change":"2024-04-26T23:20:22.351041+0000","last_active":"2024-04-26T23:20:22.354120+0000","last_peered":"2024-04-26T23:20:22.354120+0000","last_clean":"2024-04-26T23:20:22.354120+0000","last_became_active":"2024-04-26T23:20:22.350417+0000","last_became_peered":"2024-04-26T23:20:22.350417+0000","last_unstale":"2024-04-26T23:20:22.354120+0000","last_undegraded":"2024-04-26T23:20:22.354120+0000","last_fullsized":"2024-04-26T23:20:22.354120+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:15:39.108411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.359808+0000","last_change":"2024-04-26T23:20:22.351740+0000","last_active":"2024-04-26T23:20:22.359808+0000","last_peered":"2024-04-26T23:20:22.359808+0000","last_clean":"2024-04-26T23:20:22.359808+0000","last_became_active":"2024-04-26T23:20:22.351471+0000","last_became_peered":"2024-04-26T23:20:22.351471+0000","last_unstale":"2024-04-26T23:20:22.359808+0000","last_undegraded":"2024-04-26T23:20:22.359808+0000","last_fullsized":"2024-04-26T23:20:22.359808+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:39:23.094926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.361357+0000","last_change":"2024-04-26T23:20:22.357619+0000","last_active":"2024-04-26T23:20:22.361357+0000","last_peered":"2024-04-26T23:20:22.361357+0000","last_clean":"2024-04-26T23:20:22.361357+0000","last_became_active":"2024-04-26T23:20:22.357297+0000","last_became_peered":"2024-04-26T23:20:22.357297+0000","last_unstale":"2024-04-26T23:20:22.361357+0000","last_undegraded":"2024-04-26T23:20:22.361357+0000","last_fullsized":"2024-04-26T23:20:22.361357+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:47:59.715605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.362705+0000","last_change":"2024-04-26T23:20:22.360080+0000","last_active":"2024-04-26T23:20:22.362705+0000","last_peered":"2024-04-26T23:20:22.362705+0000","last_clean":"2024-04-26T23:20:22.362705+0000","last_became_active":"2024-04-26T23:20:22.359803+0000","last_became_peered":"2024-04-26T23:20:22.359803+0000","last_unstale":"2024-04-26T23:20:22.362705+0000","last_undegraded":"2024-04-26T23:20:22.362705+0000","last_fullsized":"2024-04-26T23:20:22.362705+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:54:52.957936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.353151+0000","last_change":"2024-04-26T23:20:22.350730+0000","last_active":"2024-04-26T23:20:22.353151+0000","last_peered":"2024-04-26T23:20:22.353151+0000","last_clean":"2024-04-26T23:20:22.353151+0000","last_became_active":"2024-04-26T23:20:22.350293+0000","last_became_peered":"2024-04-26T23:20:22.350293+0000","last_unstale":"2024-04-26T23:20:22.353151+0000","last_undegraded":"2024-04-26T23:20:22.353151+0000","last_fullsized":"2024-04-26T23:20:22.353151+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:54.051375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.352080+0000","last_change":"2024-04-26T23:20:22.349599+0000","last_active":"2024-04-26T23:20:22.352080+0000","last_peered":"2024-04-26T23:20:22.352080+0000","last_clean":"2024-04-26T23:20:22.352080+0000","last_became_active":"2024-04-26T23:20:22.348919+0000","last_became_peered":"2024-04-26T23:20:22.348919+0000","last_unstale":"2024-04-26T23:20:22.352080+0000","last_undegraded":"2024-04-26T23:20:22.352080+0000","last_fullsized":"2024-04-26T23:20:22.352080+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:19:55.719384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.352323+0000","last_change":"2024-04-26T23:20:22.351222+0000","last_active":"2024-04-26T23:20:23.352323+0000","last_peered":"2024-04-26T23:20:23.352323+0000","last_clean":"2024-04-26T23:20:23.352323+0000","last_became_active":"2024-04-26T23:20:22.350909+0000","last_became_peered":"2024-04-26T23:20:22.350909+0000","last_unstale":"2024-04-26T23:20:23.352323+0000","last_undegraded":"2024-04-26T23:20:23.352323+0000","last_fullsized":"2024-04-26T23:20:23.352323+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:03:59.571345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.359936+0000","last_change":"2024-04-26T23:20:22.354123+0000","last_active":"2024-04-26T23:20:22.359936+0000","last_peered":"2024-04-26T23:20:22.359936+0000","last_clean":"2024-04-26T23:20:22.359936+0000","last_became_active":"2024-04-26T23:20:22.353853+0000","last_became_peered":"2024-04-26T23:20:22.353853+0000","last_unstale":"2024-04-26T23:20:22.359936+0000","last_undegraded":"2024-04-26T23:20:22.359936+0000","last_fullsized":"2024-04-26T23:20:22.359936+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:25:54.195017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.353103+0000","last_change":"2024-04-26T23:20:22.357507+0000","last_active":"2024-04-26T23:20:23.353103+0000","last_peered":"2024-04-26T23:20:23.353103+0000","last_clean":"2024-04-26T23:20:23.353103+0000","last_became_active":"2024-04-26T23:20:22.356803+0000","last_became_peered":"2024-04-26T23:20:22.356803+0000","last_unstale":"2024-04-26T23:20:23.353103+0000","last_undegraded":"2024-04-26T23:20:23.353103+0000","last_fullsized":"2024-04-26T23:20:23.353103+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:33:15.921208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.356491+0000","last_change":"2024-04-26T23:20:22.355401+0000","last_active":"2024-04-26T23:20:22.356491+0000","last_peered":"2024-04-26T23:20:22.356491+0000","last_clean":"2024-04-26T23:20:22.356491+0000","last_became_active":"2024-04-26T23:20:22.355125+0000","last_became_peered":"2024-04-26T23:20:22.355125+0000","last_unstale":"2024-04-26T23:20:22.356491+0000","last_undegraded":"2024-04-26T23:20:22.356491+0000","last_fullsized":"2024-04-26T23:20:22.356491+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:54:52.957936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.362718+0000","last_change":"2024-04-26T23:20:22.360659+0000","last_active":"2024-04-26T23:20:22.362718+0000","last_peered":"2024-04-26T23:20:22.362718+0000","last_clean":"2024-04-26T23:20:22.362718+0000","last_became_active":"2024-04-26T23:20:22.360405+0000","last_became_peered":"2024-04-26T23:20:22.360405+0000","last_unstale":"2024-04-26T23:20:22.362718+0000","last_undegraded":"2024-04-26T23:20:22.362718+0000","last_fullsized":"2024-04-26T23:20:22.362718+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:21:38.353777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.366011+0000","last_change":"2024-04-26T23:20:22.364482+0000","last_active":"2024-04-26T23:20:22.366011+0000","last_peered":"2024-04-26T23:20:22.366011+0000","last_clean":"2024-04-26T23:20:22.366011+0000","last_became_active":"2024-04-26T23:20:22.363442+0000","last_became_peered":"2024-04-26T23:20:22.363442+0000","last_unstale":"2024-04-26T23:20:22.366011+0000","last_undegraded":"2024-04-26T23:20:22.366011+0000","last_fullsized":"2024-04-26T23:20:22.366011+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:58:40.176904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.361844+0000","last_change":"2024-04-26T23:20:22.358255+0000","last_active":"2024-04-26T23:20:22.361844+0000","last_peered":"2024-04-26T23:20:22.361844+0000","last_clean":"2024-04-26T23:20:22.361844+0000","last_became_active":"2024-04-26T23:20:22.357997+0000","last_became_peered":"2024-04-26T23:20:22.357997+0000","last_unstale":"2024-04-26T23:20:22.361844+0000","last_undegraded":"2024-04-26T23:20:22.361844+0000","last_fullsized":"2024-04-26T23:20:22.361844+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:23:40.504144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.349015+0000","last_change":"2024-04-26T23:20:22.347985+0000","last_active":"2024-04-26T23:20:22.349015+0000","last_peered":"2024-04-26T23:20:22.349015+0000","last_clean":"2024-04-26T23:20:22.349015+0000","last_became_active":"2024-04-26T23:20:22.347692+0000","last_became_peered":"2024-04-26T23:20:22.347692+0000","last_unstale":"2024-04-26T23:20:22.349015+0000","last_undegraded":"2024-04-26T23:20:22.349015+0000","last_fullsized":"2024-04-26T23:20:22.349015+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:52:13.172713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.356418+0000","last_change":"2024-04-26T23:20:22.352336+0000","last_active":"2024-04-26T23:20:22.356418+0000","last_peered":"2024-04-26T23:20:22.356418+0000","last_clean":"2024-04-26T23:20:22.356418+0000","last_became_active":"2024-04-26T23:20:22.352041+0000","last_became_peered":"2024-04-26T23:20:22.352041+0000","last_unstale":"2024-04-26T23:20:22.356418+0000","last_undegraded":"2024-04-26T23:20:22.356418+0000","last_fullsized":"2024-04-26T23:20:22.356418+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:23:40.504144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.348362+0000","last_change":"2024-04-26T23:20:22.347239+0000","last_active":"2024-04-26T23:20:22.348362+0000","last_peered":"2024-04-26T23:20:22.348362+0000","last_clean":"2024-04-26T23:20:22.348362+0000","last_became_active":"2024-04-26T23:20:22.346936+0000","last_became_peered":"2024-04-26T23:20:22.346936+0000","last_unstale":"2024-04-26T23:20:22.348362+0000","last_undegraded":"2024-04-26T23:20:22.348362+0000","last_fullsized":"2024-04-26T23:20:22.348362+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:17:35.154747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"53'4","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.367312+0000","last_change":"2024-04-26T23:20:21.350058+0000","last_active":"2024-04-26T23:20:22.367312+0000","last_peered":"2024-04-26T23:20:22.367312+0000","last_clean":"2024-04-26T23:20:22.367312+0000","last_became_active":"2024-04-26T23:20:21.349721+0000","last_became_peered":"2024-04-26T23:20:21.349721+0000","last_unstale":"2024-04-26T23:20:22.367312+0000","last_undegraded":"2024-04-26T23:20:22.367312+0000","last_fullsized":"2024-04-26T23:20:22.367312+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:47:37.763322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.352125+0000","last_change":"2024-04-26T23:20:22.346088+0000","last_active":"2024-04-26T23:20:22.352125+0000","last_peered":"2024-04-26T23:20:22.352125+0000","last_clean":"2024-04-26T23:20:22.352125+0000","last_became_active":"2024-04-26T23:20:22.345811+0000","last_became_peered":"2024-04-26T23:20:22.345811+0000","last_unstale":"2024-04-26T23:20:22.352125+0000","last_undegraded":"2024-04-26T23:20:22.352125+0000","last_fullsized":"2024-04-26T23:20:22.352125+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:03:59.571345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"53'1","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.364464+0000","last_change":"2024-04-26T23:20:21.350999+0000","last_active":"2024-04-26T23:20:22.364464+0000","last_peered":"2024-04-26T23:20:22.364464+0000","last_clean":"2024-04-26T23:20:22.364464+0000","last_became_active":"2024-04-26T23:20:21.350702+0000","last_became_peered":"2024-04-26T23:20:21.350702+0000","last_unstale":"2024-04-26T23:20:22.364464+0000","last_undegraded":"2024-04-26T23:20:22.364464+0000","last_fullsized":"2024-04-26T23:20:22.364464+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:23:39.501646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.348330+0000","last_change":"2024-04-26T23:20:22.345369+0000","last_active":"2024-04-26T23:20:22.348330+0000","last_peered":"2024-04-26T23:20:22.348330+0000","last_clean":"2024-04-26T23:20:22.348330+0000","last_became_active":"2024-04-26T23:20:22.345069+0000","last_became_peered":"2024-04-26T23:20:22.345069+0000","last_unstale":"2024-04-26T23:20:22.348330+0000","last_undegraded":"2024-04-26T23:20:22.348330+0000","last_fullsized":"2024-04-26T23:20:22.348330+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:57.150910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"53'2","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.363922+0000","last_change":"2024-04-26T23:20:21.340294+0000","last_active":"2024-04-26T23:20:22.363922+0000","last_peered":"2024-04-26T23:20:22.363922+0000","last_clean":"2024-04-26T23:20:22.363922+0000","last_became_active":"2024-04-26T23:20:21.339986+0000","last_became_peered":"2024-04-26T23:20:21.339986+0000","last_unstale":"2024-04-26T23:20:22.363922+0000","last_undegraded":"2024-04-26T23:20:22.363922+0000","last_fullsized":"2024-04-26T23:20:22.363922+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:15:38.105913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.364326+0000","last_change":"2024-04-26T23:20:22.357226+0000","last_active":"2024-04-26T23:20:22.364326+0000","last_peered":"2024-04-26T23:20:22.364326+0000","last_clean":"2024-04-26T23:20:22.364326+0000","last_became_active":"2024-04-26T23:20:22.356965+0000","last_became_peered":"2024-04-26T23:20:22.356965+0000","last_unstale":"2024-04-26T23:20:22.364326+0000","last_undegraded":"2024-04-26T23:20:22.364326+0000","last_fullsized":"2024-04-26T23:20:22.364326+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:36:13.984324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.353365+0000","last_change":"2024-04-26T23:20:21.347741+0000","last_active":"2024-04-26T23:20:23.353365+0000","last_peered":"2024-04-26T23:20:23.353365+0000","last_clean":"2024-04-26T23:20:23.353365+0000","last_became_active":"2024-04-26T23:20:21.347463+0000","last_became_peered":"2024-04-26T23:20:21.347463+0000","last_unstale":"2024-04-26T23:20:23.353365+0000","last_undegraded":"2024-04-26T23:20:23.353365+0000","last_fullsized":"2024-04-26T23:20:23.353365+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:17:34.152250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.351137+0000","last_change":"2024-04-26T23:20:22.349525+0000","last_active":"2024-04-26T23:20:22.351137+0000","last_peered":"2024-04-26T23:20:22.351137+0000","last_clean":"2024-04-26T23:20:22.351137+0000","last_became_active":"2024-04-26T23:20:22.349206+0000","last_became_peered":"2024-04-26T23:20:22.349206+0000","last_unstale":"2024-04-26T23:20:22.351137+0000","last_undegraded":"2024-04-26T23:20:22.351137+0000","last_fullsized":"2024-04-26T23:20:22.351137+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:53.946417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.514134+0000","last_change":"2024-04-26T23:20:21.350949+0000","last_active":"2024-04-26T23:20:23.514134+0000","last_peered":"2024-04-26T23:20:23.514134+0000","last_clean":"2024-04-26T23:20:23.514134+0000","last_became_active":"2024-04-26T23:20:21.350652+0000","last_became_peered":"2024-04-26T23:20:21.350652+0000","last_unstale":"2024-04-26T23:20:23.514134+0000","last_undegraded":"2024-04-26T23:20:23.514134+0000","last_fullsized":"2024-04-26T23:20:23.514134+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:19:16.521452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.361287+0000","last_change":"2024-04-26T23:20:22.358930+0000","last_active":"2024-04-26T23:20:22.361287+0000","last_peered":"2024-04-26T23:20:22.361287+0000","last_clean":"2024-04-26T23:20:22.361287+0000","last_became_active":"2024-04-26T23:20:22.358231+0000","last_became_peered":"2024-04-26T23:20:22.358231+0000","last_unstale":"2024-04-26T23:20:22.361287+0000","last_undegraded":"2024-04-26T23:20:22.361287+0000","last_fullsized":"2024-04-26T23:20:22.361287+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:53:38.181432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"53'1","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.356577+0000","last_change":"2024-04-26T23:20:21.347858+0000","last_active":"2024-04-26T23:20:22.356577+0000","last_peered":"2024-04-26T23:20:22.356577+0000","last_clean":"2024-04-26T23:20:22.356577+0000","last_became_active":"2024-04-26T23:20:21.347590+0000","last_became_peered":"2024-04-26T23:20:21.347590+0000","last_unstale":"2024-04-26T23:20:22.356577+0000","last_undegraded":"2024-04-26T23:20:22.356577+0000","last_fullsized":"2024-04-26T23:20:22.356577+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:54:17.051800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.352358+0000","last_change":"2024-04-26T23:20:22.351205+0000","last_active":"2024-04-26T23:20:23.352358+0000","last_peered":"2024-04-26T23:20:23.352358+0000","last_clean":"2024-04-26T23:20:23.352358+0000","last_became_active":"2024-04-26T23:20:22.350812+0000","last_became_peered":"2024-04-26T23:20:22.350812+0000","last_unstale":"2024-04-26T23:20:23.352358+0000","last_undegraded":"2024-04-26T23:20:23.352358+0000","last_fullsized":"2024-04-26T23:20:23.352358+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:10:14.051477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.353245+0000","last_change":"2024-04-26T23:20:21.346980+0000","last_active":"2024-04-26T23:20:23.353245+0000","last_peered":"2024-04-26T23:20:23.353245+0000","last_clean":"2024-04-26T23:20:23.353245+0000","last_became_active":"2024-04-26T23:20:21.346064+0000","last_became_peered":"2024-04-26T23:20:21.346064+0000","last_unstale":"2024-04-26T23:20:23.353245+0000","last_undegraded":"2024-04-26T23:20:23.353245+0000","last_fullsized":"2024-04-26T23:20:23.353245+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:56.148412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.356833+0000","last_change":"2024-04-26T23:20:22.355222+0000","last_active":"2024-04-26T23:20:22.356833+0000","last_peered":"2024-04-26T23:20:22.356833+0000","last_clean":"2024-04-26T23:20:22.356833+0000","last_became_active":"2024-04-26T23:20:22.354550+0000","last_became_peered":"2024-04-26T23:20:22.354550+0000","last_unstale":"2024-04-26T23:20:22.356833+0000","last_undegraded":"2024-04-26T23:20:22.356833+0000","last_fullsized":"2024-04-26T23:20:22.356833+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:10:01.361701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.513915+0000","last_change":"2024-04-26T23:20:21.354548+0000","last_active":"2024-04-26T23:20:23.513915+0000","last_peered":"2024-04-26T23:20:23.513915+0000","last_clean":"2024-04-26T23:20:23.513915+0000","last_became_active":"2024-04-26T23:20:21.354269+0000","last_became_peered":"2024-04-26T23:20:21.354269+0000","last_unstale":"2024-04-26T23:20:23.513915+0000","last_undegraded":"2024-04-26T23:20:23.513915+0000","last_fullsized":"2024-04-26T23:20:23.513915+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:07.701563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.514371+0000","last_change":"2024-04-26T23:20:22.362804+0000","last_active":"2024-04-26T23:20:23.514371+0000","last_peered":"2024-04-26T23:20:23.514371+0000","last_clean":"2024-04-26T23:20:23.514371+0000","last_became_active":"2024-04-26T23:20:22.362541+0000","last_became_peered":"2024-04-26T23:20:22.362541+0000","last_unstale":"2024-04-26T23:20:23.514371+0000","last_undegraded":"2024-04-26T23:20:23.514371+0000","last_fullsized":"2024-04-26T23:20:23.514371+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:54:18.054298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.341571+0000","last_change":"2024-04-26T23:20:21.348484+0000","last_active":"2024-04-26T23:20:22.341571+0000","last_peered":"2024-04-26T23:20:22.341571+0000","last_clean":"2024-04-26T23:20:22.341571+0000","last_became_active":"2024-04-26T23:20:21.348169+0000","last_became_peered":"2024-04-26T23:20:21.348169+0000","last_unstale":"2024-04-26T23:20:22.341571+0000","last_undegraded":"2024-04-26T23:20:22.341571+0000","last_fullsized":"2024-04-26T23:20:22.341571+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:47:37.763322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.354085+0000","last_change":"2024-04-26T23:20:22.351912+0000","last_active":"2024-04-26T23:20:22.354085+0000","last_peered":"2024-04-26T23:20:22.354085+0000","last_clean":"2024-04-26T23:20:22.354085+0000","last_became_active":"2024-04-26T23:20:22.351626+0000","last_became_peered":"2024-04-26T23:20:22.351626+0000","last_unstale":"2024-04-26T23:20:22.354085+0000","last_undegraded":"2024-04-26T23:20:22.354085+0000","last_fullsized":"2024-04-26T23:20:22.354085+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:17:35.154747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.348042+0000","last_change":"2024-04-26T23:20:21.348401+0000","last_active":"2024-04-26T23:20:22.348042+0000","last_peered":"2024-04-26T23:20:22.348042+0000","last_clean":"2024-04-26T23:20:22.348042+0000","last_became_active":"2024-04-26T23:20:21.348072+0000","last_became_peered":"2024-04-26T23:20:21.348072+0000","last_unstale":"2024-04-26T23:20:22.348042+0000","last_undegraded":"2024-04-26T23:20:22.348042+0000","last_fullsized":"2024-04-26T23:20:22.348042+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:56:09.117164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.362508+0000","last_change":"2024-04-26T23:20:22.357498+0000","last_active":"2024-04-26T23:20:22.362508+0000","last_peered":"2024-04-26T23:20:22.362508+0000","last_clean":"2024-04-26T23:20:22.362508+0000","last_became_active":"2024-04-26T23:20:22.357213+0000","last_became_peered":"2024-04-26T23:20:22.357213+0000","last_unstale":"2024-04-26T23:20:22.362508+0000","last_undegraded":"2024-04-26T23:20:22.362508+0000","last_fullsized":"2024-04-26T23:20:22.362508+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:42:05.897506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"53'3","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.362554+0000","last_change":"2024-04-26T23:20:21.354343+0000","last_active":"2024-04-26T23:20:22.362554+0000","last_peered":"2024-04-26T23:20:22.362554+0000","last_clean":"2024-04-26T23:20:22.362554+0000","last_became_active":"2024-04-26T23:20:21.354067+0000","last_became_peered":"2024-04-26T23:20:21.354067+0000","last_unstale":"2024-04-26T23:20:22.362554+0000","last_undegraded":"2024-04-26T23:20:22.362554+0000","last_fullsized":"2024-04-26T23:20:22.362554+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:54:17.051800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.354233+0000","last_change":"2024-04-26T23:20:22.351263+0000","last_active":"2024-04-26T23:20:22.354233+0000","last_peered":"2024-04-26T23:20:22.354233+0000","last_clean":"2024-04-26T23:20:22.354233+0000","last_became_active":"2024-04-26T23:20:22.350759+0000","last_became_peered":"2024-04-26T23:20:22.350759+0000","last_unstale":"2024-04-26T23:20:22.354233+0000","last_undegraded":"2024-04-26T23:20:22.354233+0000","last_fullsized":"2024-04-26T23:20:22.354233+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:57.150910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"53'6","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.514310+0000","last_change":"2024-04-26T23:20:21.348001+0000","last_active":"2024-04-26T23:20:23.514310+0000","last_peered":"2024-04-26T23:20:23.514310+0000","last_clean":"2024-04-26T23:20:23.514310+0000","last_became_active":"2024-04-26T23:20:21.347720+0000","last_became_peered":"2024-04-26T23:20:21.347720+0000","last_unstale":"2024-04-26T23:20:23.514310+0000","last_undegraded":"2024-04-26T23:20:23.514310+0000","last_fullsized":"2024-04-26T23:20:23.514310+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:35:33.072603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.352859+0000","last_change":"2024-04-26T23:20:22.341469+0000","last_active":"2024-04-26T23:20:23.352859+0000","last_peered":"2024-04-26T23:20:23.352859+0000","last_clean":"2024-04-26T23:20:23.352859+0000","last_became_active":"2024-04-26T23:20:22.341176+0000","last_became_peered":"2024-04-26T23:20:22.341176+0000","last_unstale":"2024-04-26T23:20:23.352859+0000","last_undegraded":"2024-04-26T23:20:23.352859+0000","last_fullsized":"2024-04-26T23:20:23.352859+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:10:01.361701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"53'2","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.367252+0000","last_change":"2024-04-26T23:20:21.349102+0000","last_active":"2024-04-26T23:20:22.367252+0000","last_peered":"2024-04-26T23:20:22.367252+0000","last_clean":"2024-04-26T23:20:22.367252+0000","last_became_active":"2024-04-26T23:20:21.348810+0000","last_became_peered":"2024-04-26T23:20:21.348810+0000","last_unstale":"2024-04-26T23:20:22.367252+0000","last_undegraded":"2024-04-26T23:20:22.367252+0000","last_fullsized":"2024-04-26T23:20:22.367252+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:54:51.955438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.355205+0000","last_change":"2024-04-26T23:20:22.353043+0000","last_active":"2024-04-26T23:20:22.355205+0000","last_peered":"2024-04-26T23:20:22.355205+0000","last_clean":"2024-04-26T23:20:22.355205+0000","last_became_active":"2024-04-26T23:20:22.352751+0000","last_became_peered":"2024-04-26T23:20:22.352751+0000","last_unstale":"2024-04-26T23:20:22.355205+0000","last_undegraded":"2024-04-26T23:20:22.355205+0000","last_fullsized":"2024-04-26T23:20:22.355205+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:35:14.922511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.352588+0000","last_change":"2024-04-26T23:20:21.346438+0000","last_active":"2024-04-26T23:20:23.352588+0000","last_peered":"2024-04-26T23:20:23.352588+0000","last_clean":"2024-04-26T23:20:23.352588+0000","last_became_active":"2024-04-26T23:20:21.346143+0000","last_became_peered":"2024-04-26T23:20:21.346143+0000","last_unstale":"2024-04-26T23:20:23.352588+0000","last_undegraded":"2024-04-26T23:20:23.352588+0000","last_fullsized":"2024-04-26T23:20:23.352588+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:07.701563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.513965+0000","last_change":"2024-04-26T23:20:22.364021+0000","last_active":"2024-04-26T23:20:23.513965+0000","last_peered":"2024-04-26T23:20:23.513965+0000","last_clean":"2024-04-26T23:20:23.513965+0000","last_became_active":"2024-04-26T23:20:22.363609+0000","last_became_peered":"2024-04-26T23:20:22.363609+0000","last_unstale":"2024-04-26T23:20:23.513965+0000","last_undegraded":"2024-04-26T23:20:23.513965+0000","last_fullsized":"2024-04-26T23:20:23.513965+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:42:05.897506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.352685+0000","last_change":"2024-04-26T23:20:21.345639+0000","last_active":"2024-04-26T23:20:23.352685+0000","last_peered":"2024-04-26T23:20:23.352685+0000","last_clean":"2024-04-26T23:20:23.352685+0000","last_became_active":"2024-04-26T23:20:21.345358+0000","last_became_peered":"2024-04-26T23:20:21.345358+0000","last_unstale":"2024-04-26T23:20:23.352685+0000","last_undegraded":"2024-04-26T23:20:23.352685+0000","last_fullsized":"2024-04-26T23:20:23.352685+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:09:27.893781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.353976+0000","last_change":"2024-04-26T23:20:22.349964+0000","last_active":"2024-04-26T23:20:22.353976+0000","last_peered":"2024-04-26T23:20:22.353976+0000","last_clean":"2024-04-26T23:20:22.353976+0000","last_became_active":"2024-04-26T23:20:22.349288+0000","last_became_peered":"2024-04-26T23:20:22.349288+0000","last_unstale":"2024-04-26T23:20:22.353976+0000","last_undegraded":"2024-04-26T23:20:22.353976+0000","last_fullsized":"2024-04-26T23:20:22.353976+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:58:08.886088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"53'8","reported_seq":20,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.361067+0000","last_change":"2024-04-26T23:20:21.350145+0000","last_active":"2024-04-26T23:20:22.361067+0000","last_peered":"2024-04-26T23:20:22.361067+0000","last_clean":"2024-04-26T23:20:22.361067+0000","last_became_active":"2024-04-26T23:20:21.349857+0000","last_became_peered":"2024-04-26T23:20:21.349857+0000","last_unstale":"2024-04-26T23:20:22.361067+0000","last_undegraded":"2024-04-26T23:20:22.361067+0000","last_fullsized":"2024-04-26T23:20:22.361067+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:28.657600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.361935+0000","last_change":"2024-04-26T23:20:22.358963+0000","last_active":"2024-04-26T23:20:22.361935+0000","last_peered":"2024-04-26T23:20:22.361935+0000","last_clean":"2024-04-26T23:20:22.361935+0000","last_became_active":"2024-04-26T23:20:22.358678+0000","last_became_peered":"2024-04-26T23:20:22.358678+0000","last_unstale":"2024-04-26T23:20:22.361935+0000","last_undegraded":"2024-04-26T23:20:22.361935+0000","last_fullsized":"2024-04-26T23:20:22.361935+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:56:10.119661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"53'1","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.365606+0000","last_change":"2024-04-26T23:20:21.348069+0000","last_active":"2024-04-26T23:20:22.365606+0000","last_peered":"2024-04-26T23:20:22.365606+0000","last_clean":"2024-04-26T23:20:22.365606+0000","last_became_active":"2024-04-26T23:20:21.347150+0000","last_became_peered":"2024-04-26T23:20:21.347150+0000","last_unstale":"2024-04-26T23:20:22.365606+0000","last_undegraded":"2024-04-26T23:20:22.365606+0000","last_fullsized":"2024-04-26T23:20:22.365606+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:40:56.746673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.352886+0000","last_change":"2024-04-26T23:20:22.349644+0000","last_active":"2024-04-26T23:20:23.352886+0000","last_peered":"2024-04-26T23:20:23.352886+0000","last_clean":"2024-04-26T23:20:23.352886+0000","last_became_active":"2024-04-26T23:20:22.346873+0000","last_became_peered":"2024-04-26T23:20:22.346873+0000","last_unstale":"2024-04-26T23:20:23.352886+0000","last_undegraded":"2024-04-26T23:20:23.352886+0000","last_fullsized":"2024-04-26T23:20:23.352886+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:55:06.694814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"53'2","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.361620+0000","last_change":"2024-04-26T23:20:21.348994+0000","last_active":"2024-04-26T23:20:22.361620+0000","last_peered":"2024-04-26T23:20:22.361620+0000","last_clean":"2024-04-26T23:20:22.361620+0000","last_became_active":"2024-04-26T23:20:21.348713+0000","last_became_peered":"2024-04-26T23:20:21.348713+0000","last_unstale":"2024-04-26T23:20:22.361620+0000","last_undegraded":"2024-04-26T23:20:22.361620+0000","last_fullsized":"2024-04-26T23:20:22.361620+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:19:54.716887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.365474+0000","last_change":"2024-04-26T23:20:22.362301+0000","last_active":"2024-04-26T23:20:22.365474+0000","last_peered":"2024-04-26T23:20:22.365474+0000","last_clean":"2024-04-26T23:20:22.365474+0000","last_became_active":"2024-04-26T23:20:22.361986+0000","last_became_peered":"2024-04-26T23:20:22.361986+0000","last_unstale":"2024-04-26T23:20:22.365474+0000","last_undegraded":"2024-04-26T23:20:22.365474+0000","last_fullsized":"2024-04-26T23:20:22.365474+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:31.965056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.352683+0000","last_change":"2024-04-26T23:20:21.351307+0000","last_active":"2024-04-26T23:20:23.352683+0000","last_peered":"2024-04-26T23:20:23.352683+0000","last_clean":"2024-04-26T23:20:23.352683+0000","last_became_active":"2024-04-26T23:20:21.350102+0000","last_became_peered":"2024-04-26T23:20:21.350102+0000","last_unstale":"2024-04-26T23:20:23.352683+0000","last_undegraded":"2024-04-26T23:20:23.352683+0000","last_fullsized":"2024-04-26T23:20:23.352683+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:15:38.105913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.354253+0000","last_change":"2024-04-26T23:20:22.352142+0000","last_active":"2024-04-26T23:20:22.354253+0000","last_peered":"2024-04-26T23:20:22.354253+0000","last_clean":"2024-04-26T23:20:22.354253+0000","last_became_active":"2024-04-26T23:20:22.351704+0000","last_became_peered":"2024-04-26T23:20:22.351704+0000","last_unstale":"2024-04-26T23:20:22.354253+0000","last_undegraded":"2024-04-26T23:20:22.354253+0000","last_fullsized":"2024-04-26T23:20:22.354253+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:04:00.567851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.339397+0000","last_change":"2024-04-26T23:20:21.352216+0000","last_active":"2024-04-26T23:20:22.339397+0000","last_peered":"2024-04-26T23:20:22.339397+0000","last_clean":"2024-04-26T23:20:22.339397+0000","last_became_active":"2024-04-26T23:20:21.351823+0000","last_became_peered":"2024-04-26T23:20:21.351823+0000","last_unstale":"2024-04-26T23:20:22.339397+0000","last_undegraded":"2024-04-26T23:20:22.339397+0000","last_fullsized":"2024-04-26T23:20:22.339397+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:58:07.883590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.353356+0000","last_change":"2024-04-26T23:20:22.358146+0000","last_active":"2024-04-26T23:20:23.353356+0000","last_peered":"2024-04-26T23:20:23.353356+0000","last_clean":"2024-04-26T23:20:23.353356+0000","last_became_active":"2024-04-26T23:20:22.357567+0000","last_became_peered":"2024-04-26T23:20:22.357567+0000","last_unstale":"2024-04-26T23:20:23.353356+0000","last_undegraded":"2024-04-26T23:20:23.353356+0000","last_fullsized":"2024-04-26T23:20:23.353356+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:30:38.323701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"53'2","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.367091+0000","last_change":"2024-04-26T23:20:21.353594+0000","last_active":"2024-04-26T23:20:22.367091+0000","last_peered":"2024-04-26T23:20:22.367091+0000","last_clean":"2024-04-26T23:20:22.367091+0000","last_became_active":"2024-04-26T23:20:21.353226+0000","last_became_peered":"2024-04-26T23:20:21.353226+0000","last_unstale":"2024-04-26T23:20:22.367091+0000","last_undegraded":"2024-04-26T23:20:22.367091+0000","last_fullsized":"2024-04-26T23:20:22.367091+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:36:12.981827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.356429+0000","last_change":"2024-04-26T23:20:22.354795+0000","last_active":"2024-04-26T23:20:22.356429+0000","last_peered":"2024-04-26T23:20:22.356429+0000","last_clean":"2024-04-26T23:20:22.356429+0000","last_became_active":"2024-04-26T23:20:22.354521+0000","last_became_peered":"2024-04-26T23:20:22.354521+0000","last_unstale":"2024-04-26T23:20:22.356429+0000","last_undegraded":"2024-04-26T23:20:22.356429+0000","last_fullsized":"2024-04-26T23:20:22.356429+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:40:57.749170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.337174+0000","last_change":"2024-04-26T23:20:21.349962+0000","last_active":"2024-04-26T23:20:22.337174+0000","last_peered":"2024-04-26T23:20:22.337174+0000","last_clean":"2024-04-26T23:20:22.337174+0000","last_became_active":"2024-04-26T23:20:21.349695+0000","last_became_peered":"2024-04-26T23:20:21.349695+0000","last_unstale":"2024-04-26T23:20:22.337174+0000","last_undegraded":"2024-04-26T23:20:22.337174+0000","last_fullsized":"2024-04-26T23:20:22.337174+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:43.643241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.349753+0000","last_change":"2024-04-26T23:20:22.347662+0000","last_active":"2024-04-26T23:20:22.349753+0000","last_peered":"2024-04-26T23:20:22.349753+0000","last_clean":"2024-04-26T23:20:22.349753+0000","last_became_active":"2024-04-26T23:20:22.346708+0000","last_became_peered":"2024-04-26T23:20:22.346708+0000","last_unstale":"2024-04-26T23:20:22.349753+0000","last_undegraded":"2024-04-26T23:20:22.349753+0000","last_fullsized":"2024-04-26T23:20:22.349753+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:35:14.922511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.352345+0000","last_change":"2024-04-26T23:20:21.355343+0000","last_active":"2024-04-26T23:20:23.352345+0000","last_peered":"2024-04-26T23:20:23.352345+0000","last_clean":"2024-04-26T23:20:23.352345+0000","last_became_active":"2024-04-26T23:20:21.355088+0000","last_became_peered":"2024-04-26T23:20:21.355088+0000","last_unstale":"2024-04-26T23:20:23.352345+0000","last_undegraded":"2024-04-26T23:20:23.352345+0000","last_fullsized":"2024-04-26T23:20:23.352345+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:58:07.883590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.353319+0000","last_change":"2024-04-26T23:20:22.357862+0000","last_active":"2024-04-26T23:20:23.353319+0000","last_peered":"2024-04-26T23:20:23.353319+0000","last_clean":"2024-04-26T23:20:23.353319+0000","last_became_active":"2024-04-26T23:20:22.356157+0000","last_became_peered":"2024-04-26T23:20:22.356157+0000","last_unstale":"2024-04-26T23:20:23.353319+0000","last_undegraded":"2024-04-26T23:20:23.353319+0000","last_fullsized":"2024-04-26T23:20:23.353319+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:42:39.240263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"53'2","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.365258+0000","last_change":"2024-04-26T23:20:21.352529+0000","last_active":"2024-04-26T23:20:22.365258+0000","last_peered":"2024-04-26T23:20:22.365258+0000","last_clean":"2024-04-26T23:20:22.365258+0000","last_became_active":"2024-04-26T23:20:21.352251+0000","last_became_peered":"2024-04-26T23:20:21.352251+0000","last_unstale":"2024-04-26T23:20:22.365258+0000","last_undegraded":"2024-04-26T23:20:22.365258+0000","last_fullsized":"2024-04-26T23:20:22.365258+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:02:58.932261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.353116+0000","last_change":"2024-04-26T23:20:22.344984+0000","last_active":"2024-04-26T23:20:23.353116+0000","last_peered":"2024-04-26T23:20:23.353116+0000","last_clean":"2024-04-26T23:20:23.353116+0000","last_became_active":"2024-04-26T23:20:22.344694+0000","last_became_peered":"2024-04-26T23:20:22.344694+0000","last_unstale":"2024-04-26T23:20:23.353116+0000","last_undegraded":"2024-04-26T23:20:23.353116+0000","last_fullsized":"2024-04-26T23:20:23.353116+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:47:59.715605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.339411+0000","last_change":"2024-04-26T23:20:21.345885+0000","last_active":"2024-04-26T23:20:22.339411+0000","last_peered":"2024-04-26T23:20:22.339411+0000","last_clean":"2024-04-26T23:20:22.339411+0000","last_became_active":"2024-04-26T23:20:21.345596+0000","last_became_peered":"2024-04-26T23:20:21.345596+0000","last_unstale":"2024-04-26T23:20:22.339411+0000","last_undegraded":"2024-04-26T23:20:22.339411+0000","last_fullsized":"2024-04-26T23:20:22.339411+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:09:27.893781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.362534+0000","last_change":"2024-04-26T23:20:22.358252+0000","last_active":"2024-04-26T23:20:22.362534+0000","last_peered":"2024-04-26T23:20:22.362534+0000","last_clean":"2024-04-26T23:20:22.362534+0000","last_became_active":"2024-04-26T23:20:22.357977+0000","last_became_peered":"2024-04-26T23:20:22.357977+0000","last_unstale":"2024-04-26T23:20:22.362534+0000","last_undegraded":"2024-04-26T23:20:22.362534+0000","last_fullsized":"2024-04-26T23:20:22.362534+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:53:01.027225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.352724+0000","last_change":"2024-04-26T23:20:21.349544+0000","last_active":"2024-04-26T23:20:23.352724+0000","last_peered":"2024-04-26T23:20:23.352724+0000","last_clean":"2024-04-26T23:20:23.352724+0000","last_became_active":"2024-04-26T23:20:21.349284+0000","last_became_peered":"2024-04-26T23:20:21.349284+0000","last_unstale":"2024-04-26T23:20:23.352724+0000","last_undegraded":"2024-04-26T23:20:23.352724+0000","last_fullsized":"2024-04-26T23:20:23.352724+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:04:42.194040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.365753+0000","last_change":"2024-04-26T23:20:22.364177+0000","last_active":"2024-04-26T23:20:22.365753+0000","last_peered":"2024-04-26T23:20:22.365753+0000","last_clean":"2024-04-26T23:20:22.365753+0000","last_became_active":"2024-04-26T23:20:22.363873+0000","last_became_peered":"2024-04-26T23:20:22.363873+0000","last_unstale":"2024-04-26T23:20:22.365753+0000","last_undegraded":"2024-04-26T23:20:22.365753+0000","last_fullsized":"2024-04-26T23:20:22.365753+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:53.946417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.353423+0000","last_change":"2024-04-26T23:20:21.346723+0000","last_active":"2024-04-26T23:20:23.353423+0000","last_peered":"2024-04-26T23:20:23.353423+0000","last_clean":"2024-04-26T23:20:23.353423+0000","last_became_active":"2024-04-26T23:20:21.346450+0000","last_became_peered":"2024-04-26T23:20:21.346450+0000","last_unstale":"2024-04-26T23:20:23.353423+0000","last_undegraded":"2024-04-26T23:20:23.353423+0000","last_fullsized":"2024-04-26T23:20:23.353423+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:03:59.565353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.359404+0000","last_change":"2024-04-26T23:20:22.356941+0000","last_active":"2024-04-26T23:20:22.359404+0000","last_peered":"2024-04-26T23:20:22.359404+0000","last_clean":"2024-04-26T23:20:22.359404+0000","last_became_active":"2024-04-26T23:20:22.353656+0000","last_became_peered":"2024-04-26T23:20:22.353656+0000","last_unstale":"2024-04-26T23:20:22.359404+0000","last_undegraded":"2024-04-26T23:20:22.359404+0000","last_fullsized":"2024-04-26T23:20:22.359404+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:33:15.921208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.336380+0000","last_change":"2024-04-26T23:20:21.343908+0000","last_active":"2024-04-26T23:20:22.336380+0000","last_peered":"2024-04-26T23:20:22.336380+0000","last_clean":"2024-04-26T23:20:22.336380+0000","last_became_active":"2024-04-26T23:20:21.343577+0000","last_became_peered":"2024-04-26T23:20:21.343577+0000","last_unstale":"2024-04-26T23:20:22.336380+0000","last_undegraded":"2024-04-26T23:20:22.336380+0000","last_fullsized":"2024-04-26T23:20:22.336380+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:58:18.678110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.354053+0000","last_change":"2024-04-26T23:20:22.351156+0000","last_active":"2024-04-26T23:20:22.354053+0000","last_peered":"2024-04-26T23:20:22.354053+0000","last_clean":"2024-04-26T23:20:22.354053+0000","last_became_active":"2024-04-26T23:20:22.350816+0000","last_became_peered":"2024-04-26T23:20:22.350816+0000","last_unstale":"2024-04-26T23:20:22.354053+0000","last_undegraded":"2024-04-26T23:20:22.354053+0000","last_fullsized":"2024-04-26T23:20:22.354053+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:09:28.896278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.514154+0000","last_change":"2024-04-26T23:20:21.348504+0000","last_active":"2024-04-26T23:20:23.514154+0000","last_peered":"2024-04-26T23:20:23.514154+0000","last_clean":"2024-04-26T23:20:23.514154+0000","last_became_active":"2024-04-26T23:20:21.348255+0000","last_became_peered":"2024-04-26T23:20:21.348255+0000","last_unstale":"2024-04-26T23:20:23.514154+0000","last_undegraded":"2024-04-26T23:20:23.514154+0000","last_fullsized":"2024-04-26T23:20:23.514154+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:14:21.417625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.358446+0000","last_change":"2024-04-26T23:20:22.355373+0000","last_active":"2024-04-26T23:20:22.358446+0000","last_peered":"2024-04-26T23:20:22.358446+0000","last_clean":"2024-04-26T23:20:22.358446+0000","last_became_active":"2024-04-26T23:20:22.355105+0000","last_became_peered":"2024-04-26T23:20:22.355105+0000","last_unstale":"2024-04-26T23:20:22.358446+0000","last_undegraded":"2024-04-26T23:20:22.358446+0000","last_fullsized":"2024-04-26T23:20:22.358446+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:49:27.891875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.336916+0000","last_change":"2024-04-26T23:20:21.338331+0000","last_active":"2024-04-26T23:20:22.336916+0000","last_peered":"2024-04-26T23:20:22.336916+0000","last_clean":"2024-04-26T23:20:22.336916+0000","last_became_active":"2024-04-26T23:20:21.338011+0000","last_became_peered":"2024-04-26T23:20:21.338011+0000","last_unstale":"2024-04-26T23:20:22.336916+0000","last_undegraded":"2024-04-26T23:20:22.336916+0000","last_fullsized":"2024-04-26T23:20:22.336916+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:18:51.810804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.356856+0000","last_change":"2024-04-26T23:20:22.355451+0000","last_active":"2024-04-26T23:20:22.356856+0000","last_peered":"2024-04-26T23:20:22.356856+0000","last_clean":"2024-04-26T23:20:22.356856+0000","last_became_active":"2024-04-26T23:20:22.354928+0000","last_became_peered":"2024-04-26T23:20:22.354928+0000","last_unstale":"2024-04-26T23:20:22.356856+0000","last_undegraded":"2024-04-26T23:20:22.356856+0000","last_fullsized":"2024-04-26T23:20:22.356856+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:55:06.694814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.344946+0000","last_change":"2024-04-26T23:20:21.353531+0000","last_active":"2024-04-26T23:20:22.344946+0000","last_peered":"2024-04-26T23:20:22.344946+0000","last_clean":"2024-04-26T23:20:22.344946+0000","last_became_active":"2024-04-26T23:20:21.353143+0000","last_became_peered":"2024-04-26T23:20:21.353143+0000","last_unstale":"2024-04-26T23:20:22.344946+0000","last_undegraded":"2024-04-26T23:20:22.344946+0000","last_fullsized":"2024-04-26T23:20:22.344946+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:35:13.920013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.361174+0000","last_change":"2024-04-26T23:20:22.358939+0000","last_active":"2024-04-26T23:20:22.361174+0000","last_peered":"2024-04-26T23:20:22.361174+0000","last_clean":"2024-04-26T23:20:22.361174+0000","last_became_active":"2024-04-26T23:20:22.357875+0000","last_became_peered":"2024-04-26T23:20:22.357875+0000","last_unstale":"2024-04-26T23:20:22.361174+0000","last_undegraded":"2024-04-26T23:20:22.361174+0000","last_fullsized":"2024-04-26T23:20:22.361174+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:42:39.240263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.352514+0000","last_change":"2024-04-26T23:20:21.347743+0000","last_active":"2024-04-26T23:20:23.352514+0000","last_peered":"2024-04-26T23:20:23.352514+0000","last_clean":"2024-04-26T23:20:23.352514+0000","last_became_active":"2024-04-26T23:20:21.347475+0000","last_became_peered":"2024-04-26T23:20:21.347475+0000","last_unstale":"2024-04-26T23:20:23.352514+0000","last_undegraded":"2024-04-26T23:20:23.352514+0000","last_fullsized":"2024-04-26T23:20:23.352514+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:35:33.072603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":230,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.339969+0000","last_change":"2024-04-26T23:18:18.862631+0000","last_active":"2024-04-26T23:20:22.339969+0000","last_peered":"2024-04-26T23:20:22.339969+0000","last_clean":"2024-04-26T23:20:22.339969+0000","last_became_active":"2024-04-26T23:18:18.861536+0000","last_became_peered":"2024-04-26T23:18:18.861536+0000","last_unstale":"2024-04-26T23:20:22.339969+0000","last_undegraded":"2024-04-26T23:20:22.339969+0000","last_fullsized":"2024-04-26T23:20:22.339969+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:17.808736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:17.808736+0000","last_clean_scrub_stamp":"2024-04-26T23:18:17.808736+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:59:26.359282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542172,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28228,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343612,"statfs":{"total":96636764160,"available":96607858688,"internally_reserved":0,"allocated":1232896,"data_stored":874501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542174,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":630784,"data_stored":283973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542173,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27652,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344188,"statfs":{"total":96636764160,"available":96608448512,"internally_reserved":0,"allocated":638976,"data_stored":284625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542173,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":643072,"data_stored":285585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31788,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340052,"statfs":{"total":96636764160,"available":96604213248,"internally_reserved":0,"allocated":618496,"data_stored":276806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31780,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340060,"statfs":{"total":96636764160,"available":96604221440,"internally_reserved":0,"allocated":618496,"data_stored":276806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31800,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340040,"statfs":{"total":96636764160,"available":96604200960,"internally_reserved":0,"allocated":634880,"data_stored":284063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32364,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339476,"statfs":{"total":96636764160,"available":96603623424,"internally_reserved":0,"allocated":1212416,"data_stored":867174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:20:26.375 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T23:20:26.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.511+0000 7fb64a859640 1 Processor -- start 2024-04-26T23:20:26.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.511+0000 7fb64a859640 1 -- start start 2024-04-26T23:20:26.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.512+0000 7fb64a859640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb644072950 0x7fb644167ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:26.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.512+0000 7fb64a859640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb64415d7c0 con 0x7fb644071e60 2024-04-26T23:20:26.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.512+0000 7fb64a859640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb64415daa0 con 0x7fb6440cc480 2024-04-26T23:20:26.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.512+0000 7fb64a859640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fb64415dd80 con 0x7fb644072950 2024-04-26T23:20:26.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.512+0000 7fb648df6640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb644072950 0x7fb644167ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:26.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.512+0000 7fb648df6640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb644072950 0x7fb644167ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40800/0 (socket says 172.21.15.47:40800) 2024-04-26T23:20:26.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.512+0000 7fb648df6640 1 -- 172.21.15.47:0/974007957 learned_addr learned my addr 172.21.15.47:0/974007957 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:26.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.512+0000 7fb648df6640 1 -- 172.21.15.47:0/974007957 >> v1:172.21.15.203:6789/0 conn(0x7fb6440cc480 legacy=0x7fb644072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:26.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.512+0000 7fb648df6640 1 -- 172.21.15.47:0/974007957 >> v1:172.21.15.47:6789/0 conn(0x7fb644071e60 legacy=0x7fb64406f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:26.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.512+0000 7fb648df6640 1 -- 172.21.15.47:0/974007957 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb64415e060 con 0x7fb644072950 2024-04-26T23:20:26.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.512+0000 7fb642ffd640 1 -- 172.21.15.47:0/974007957 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3698355736 0 0) 0x7fb64415d7c0 con 0x7fb644071e60 2024-04-26T23:20:26.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.513+0000 7fb648df6640 1 --2- 172.21.15.47:0/974007957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb644072950 0x7fb644167ae0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fb63c009e50 tx=0x7fb63c02a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fcc555a80bec066 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:26.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.513+0000 7fb642ffd640 1 -- 172.21.15.47:0/974007957 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c036070 con 0x7fb644072950 2024-04-26T23:20:26.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.513+0000 7fb642ffd640 1 -- 172.21.15.47:0/974007957 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb63c031600 con 0x7fb644072950 2024-04-26T23:20:26.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.513+0000 7fb642ffd640 1 -- 172.21.15.47:0/974007957 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c0319d0 con 0x7fb644072950 2024-04-26T23:20:26.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.514+0000 7fb64a859640 1 -- 172.21.15.47:0/974007957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb644072950 msgr2=0x7fb644167ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:26.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.514+0000 7fb64a859640 1 --2- 172.21.15.47:0/974007957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb644072950 0x7fb644167ae0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fb63c009e50 tx=0x7fb63c02a760 comp rx=0 tx=0).stop 2024-04-26T23:20:26.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.514+0000 7fb64a859640 1 -- 172.21.15.47:0/974007957 shutdown_connections 2024-04-26T23:20:26.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.514+0000 7fb64a859640 1 --2- 172.21.15.47:0/974007957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb644072950 0x7fb644167ae0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:26.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.514+0000 7fb64a859640 1 -- 172.21.15.47:0/974007957 >> 172.21.15.47:0/974007957 conn(0x7fb6440d0be0 msgr2=0x7fb6440d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:26.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.514+0000 7fb64a859640 1 -- 172.21.15.47:0/974007957 shutdown_connections 2024-04-26T23:20:26.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.514+0000 7fb64a859640 1 -- 172.21.15.47:0/974007957 wait complete. 2024-04-26T23:20:26.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.515+0000 7fb64a859640 1 Processor -- start 2024-04-26T23:20:26.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.515+0000 7fb64a859640 1 -- start start 2024-04-26T23:20:26.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.516+0000 7fb64a859640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb644071e60 0x7fb6440b4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:26.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.516+0000 7fb64a859640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb644072950 0x7fb6440b6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:26.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.516+0000 7fb64a859640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6440cc480 0x7fb644153080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:26.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.516+0000 7fb64a859640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb64415d570 con 0x7fb6440cc480 2024-04-26T23:20:26.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.516+0000 7fb64a859640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb64415d3f0 con 0x7fb644071e60 2024-04-26T23:20:26.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.516+0000 7fb64a859640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb64406fff0 con 0x7fb644072950 2024-04-26T23:20:26.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.516+0000 7fb648df6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6440cc480 0x7fb644153080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:26.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.516+0000 7fb648df6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6440cc480 0x7fb644153080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40806/0 (socket says 172.21.15.47:40806) 2024-04-26T23:20:26.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.516+0000 7fb648df6640 1 -- 172.21.15.47:0/1269241347 learned_addr learned my addr 172.21.15.47:0/1269241347 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:26.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.516+0000 7fb643fff640 1 --2- 172.21.15.47:0/1269241347 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb644071e60 0x7fb6440b4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:26.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.516+0000 7fb6437fe640 1 --2- 172.21.15.47:0/1269241347 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb644072950 0x7fb6440b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:26.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.516+0000 7fb648df6640 1 -- 172.21.15.47:0/1269241347 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb644072950 msgr2=0x7fb6440b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:26.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.516+0000 7fb648df6640 1 --2- 172.21.15.47:0/1269241347 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb644072950 0x7fb6440b6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:26.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.516+0000 7fb648df6640 1 -- 172.21.15.47:0/1269241347 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb644071e60 msgr2=0x7fb6440b4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:26.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.516+0000 7fb648df6640 1 --2- 172.21.15.47:0/1269241347 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb644071e60 0x7fb6440b4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:26.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.516+0000 7fb648df6640 1 -- 172.21.15.47:0/1269241347 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6440b52f0 con 0x7fb6440cc480 2024-04-26T23:20:26.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.516+0000 7fb6437fe640 1 --2- 172.21.15.47:0/1269241347 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb644072950 0x7fb6440b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:20:26.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.517+0000 7fb648df6640 1 --2- 172.21.15.47:0/1269241347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6440cc480 0x7fb644153080 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fb63c034040 tx=0x7fb63c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:26.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.517+0000 7fb640ff9640 1 -- 172.21.15.47:0/1269241347 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c041020 con 0x7fb6440cc480 2024-04-26T23:20:26.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.517+0000 7fb640ff9640 1 -- 172.21.15.47:0/1269241347 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb63c007780 con 0x7fb6440cc480 2024-04-26T23:20:26.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.517+0000 7fb640ff9640 1 -- 172.21.15.47:0/1269241347 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c036040 con 0x7fb6440cc480 2024-04-26T23:20:26.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.517+0000 7fb64a859640 1 -- 172.21.15.47:0/1269241347 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6440b55d0 con 0x7fb6440cc480 2024-04-26T23:20:26.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.517+0000 7fb64a859640 1 -- 172.21.15.47:0/1269241347 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6440b5b90 con 0x7fb6440cc480 2024-04-26T23:20:26.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.518+0000 7fb64a859640 1 -- 172.21.15.47:0/1269241347 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6440694f0 con 0x7fb6440cc480 2024-04-26T23:20:26.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.518+0000 7fb640ff9640 1 -- 172.21.15.47:0/1269241347 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb63c044020 con 0x7fb6440cc480 2024-04-26T23:20:26.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.519+0000 7fb640ff9640 1 --2- 172.21.15.47:0/1269241347 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb62406c8f0 0x7fb62406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:26.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.519+0000 7fb640ff9640 1 -- 172.21.15.47:0/1269241347 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fb63c0aca70 con 0x7fb6440cc480 2024-04-26T23:20:26.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.519+0000 7fb643fff640 1 --2- 172.21.15.47:0/1269241347 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb62406c8f0 0x7fb62406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:26.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.519+0000 7fb643fff640 1 --2- 172.21.15.47:0/1269241347 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb62406c8f0 0x7fb62406edb0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb634000f60 tx=0x7fb634009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:26.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.523+0000 7fb640ff9640 1 -- 172.21.15.47:0/1269241347 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb63c04b030 con 0x7fb6440cc480 2024-04-26T23:20:26.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.703+0000 7fb64a859640 1 -- 172.21.15.47:0/1269241347 --> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb644159c60 con 0x7fb62406c8f0 2024-04-26T23:20:26.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.709+0000 7fb640ff9640 1 -- 172.21.15.47:0/1269241347 <== mgr.4099 v2:172.21.15.203:6832/992162732 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251905 (secure 0 0 0) 0x7fb644159c60 con 0x7fb62406c8f0 2024-04-26T23:20:26.713 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:20:26.715 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-26T23:20:26.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.712+0000 7fb64a859640 1 -- 172.21.15.47:0/1269241347 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb62406c8f0 msgr2=0x7fb62406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:26.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.712+0000 7fb64a859640 1 --2- 172.21.15.47:0/1269241347 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb62406c8f0 0x7fb62406edb0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb634000f60 tx=0x7fb634009040 comp rx=0 tx=0).stop 2024-04-26T23:20:26.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.712+0000 7fb64a859640 1 -- 172.21.15.47:0/1269241347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6440cc480 msgr2=0x7fb644153080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:26.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.712+0000 7fb64a859640 1 --2- 172.21.15.47:0/1269241347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6440cc480 0x7fb644153080 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fb63c034040 tx=0x7fb63c004a40 comp rx=0 tx=0).stop 2024-04-26T23:20:26.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.713+0000 7fb64a859640 1 -- 172.21.15.47:0/1269241347 shutdown_connections 2024-04-26T23:20:26.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.713+0000 7fb64a859640 1 --2- 172.21.15.47:0/1269241347 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb62406c8f0 0x7fb62406edb0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:26.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.713+0000 7fb64a859640 1 --2- 172.21.15.47:0/1269241347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6440cc480 0x7fb644153080 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:26.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.713+0000 7fb64a859640 1 --2- 172.21.15.47:0/1269241347 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb644072950 0x7fb6440b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:26.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.713+0000 7fb64a859640 1 --2- 172.21.15.47:0/1269241347 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb644071e60 0x7fb6440b4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:26.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.713+0000 7fb64a859640 1 -- 172.21.15.47:0/1269241347 >> 172.21.15.47:0/1269241347 conn(0x7fb6440d0be0 msgr2=0x7fb64415a180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:26.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.713+0000 7fb64a859640 1 -- 172.21.15.47:0/1269241347 shutdown_connections 2024-04-26T23:20:26.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.713+0000 7fb64a859640 1 -- 172.21.15.47:0/1269241347 wait complete. 2024-04-26T23:20:26.724 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":111,"stamp":"2024-04-26T23:20:24.955404+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":111,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238912,"kb_used_data":6084,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735808,"statfs":{"total":773094113280,"available":772849467392,"internally_reserved":0,"allocated":6230016,"data_stored":3433533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.628307"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.359916+0000","last_change":"2024-04-26T23:20:22.351070+0000","last_active":"2024-04-26T23:20:22.359916+0000","last_peered":"2024-04-26T23:20:22.359916+0000","last_clean":"2024-04-26T23:20:22.359916+0000","last_became_active":"2024-04-26T23:20:22.350445+0000","last_became_peered":"2024-04-26T23:20:22.350445+0000","last_unstale":"2024-04-26T23:20:22.359916+0000","last_undegraded":"2024-04-26T23:20:22.359916+0000","last_fullsized":"2024-04-26T23:20:22.359916+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:24:59.174021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.366967+0000","last_change":"2024-04-26T23:20:22.365214+0000","last_active":"2024-04-26T23:20:22.366967+0000","last_peered":"2024-04-26T23:20:22.366967+0000","last_clean":"2024-04-26T23:20:22.366967+0000","last_became_active":"2024-04-26T23:20:22.364935+0000","last_became_peered":"2024-04-26T23:20:22.364935+0000","last_unstale":"2024-04-26T23:20:22.366967+0000","last_undegraded":"2024-04-26T23:20:22.366967+0000","last_fullsized":"2024-04-26T23:20:22.366967+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:22.991265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.362676+0000","last_change":"2024-04-26T23:20:22.359609+0000","last_active":"2024-04-26T23:20:22.362676+0000","last_peered":"2024-04-26T23:20:22.362676+0000","last_clean":"2024-04-26T23:20:22.362676+0000","last_became_active":"2024-04-26T23:20:22.359320+0000","last_became_peered":"2024-04-26T23:20:22.359320+0000","last_unstale":"2024-04-26T23:20:22.362676+0000","last_undegraded":"2024-04-26T23:20:22.362676+0000","last_fullsized":"2024-04-26T23:20:22.362676+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:40:57.749170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.353277+0000","last_change":"2024-04-26T23:20:22.358050+0000","last_active":"2024-04-26T23:20:23.353277+0000","last_peered":"2024-04-26T23:20:23.353277+0000","last_clean":"2024-04-26T23:20:23.353277+0000","last_became_active":"2024-04-26T23:20:22.356118+0000","last_became_peered":"2024-04-26T23:20:22.356118+0000","last_unstale":"2024-04-26T23:20:23.353277+0000","last_undegraded":"2024-04-26T23:20:23.353277+0000","last_fullsized":"2024-04-26T23:20:23.353277+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:53:38.181432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.362932+0000","last_change":"2024-04-26T23:20:22.360285+0000","last_active":"2024-04-26T23:20:22.362932+0000","last_peered":"2024-04-26T23:20:22.362932+0000","last_clean":"2024-04-26T23:20:22.362932+0000","last_became_active":"2024-04-26T23:20:22.359990+0000","last_became_peered":"2024-04-26T23:20:22.359990+0000","last_unstale":"2024-04-26T23:20:22.362932+0000","last_undegraded":"2024-04-26T23:20:22.362932+0000","last_fullsized":"2024-04-26T23:20:22.362932+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:31:47.385990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.349663+0000","last_change":"2024-04-26T23:20:22.347344+0000","last_active":"2024-04-26T23:20:22.349663+0000","last_peered":"2024-04-26T23:20:22.349663+0000","last_clean":"2024-04-26T23:20:22.349663+0000","last_became_active":"2024-04-26T23:20:22.347061+0000","last_became_peered":"2024-04-26T23:20:22.347061+0000","last_unstale":"2024-04-26T23:20:22.349663+0000","last_undegraded":"2024-04-26T23:20:22.349663+0000","last_fullsized":"2024-04-26T23:20:22.349663+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:04:00.567851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.513936+0000","last_change":"2024-04-26T23:20:22.358260+0000","last_active":"2024-04-26T23:20:23.513936+0000","last_peered":"2024-04-26T23:20:23.513936+0000","last_clean":"2024-04-26T23:20:23.513936+0000","last_became_active":"2024-04-26T23:20:22.357990+0000","last_became_peered":"2024-04-26T23:20:22.357990+0000","last_unstale":"2024-04-26T23:20:23.513936+0000","last_undegraded":"2024-04-26T23:20:23.513936+0000","last_fullsized":"2024-04-26T23:20:23.513936+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:49:27.891875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.361518+0000","last_change":"2024-04-26T23:20:22.359488+0000","last_active":"2024-04-26T23:20:22.361518+0000","last_peered":"2024-04-26T23:20:22.361518+0000","last_clean":"2024-04-26T23:20:22.361518+0000","last_became_active":"2024-04-26T23:20:22.359199+0000","last_became_peered":"2024-04-26T23:20:22.359199+0000","last_unstale":"2024-04-26T23:20:22.361518+0000","last_undegraded":"2024-04-26T23:20:22.361518+0000","last_fullsized":"2024-04-26T23:20:22.361518+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:30:38.323701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.514322+0000","last_change":"2024-04-26T23:20:22.366476+0000","last_active":"2024-04-26T23:20:23.514322+0000","last_peered":"2024-04-26T23:20:23.514322+0000","last_clean":"2024-04-26T23:20:23.514322+0000","last_became_active":"2024-04-26T23:20:22.366199+0000","last_became_peered":"2024-04-26T23:20:22.366199+0000","last_unstale":"2024-04-26T23:20:23.514322+0000","last_undegraded":"2024-04-26T23:20:23.514322+0000","last_fullsized":"2024-04-26T23:20:23.514322+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:47:38.765819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.358623+0000","last_change":"2024-04-26T23:20:22.354156+0000","last_active":"2024-04-26T23:20:22.358623+0000","last_peered":"2024-04-26T23:20:22.358623+0000","last_clean":"2024-04-26T23:20:22.358623+0000","last_became_active":"2024-04-26T23:20:22.351645+0000","last_became_peered":"2024-04-26T23:20:22.351645+0000","last_unstale":"2024-04-26T23:20:22.358623+0000","last_undegraded":"2024-04-26T23:20:22.358623+0000","last_fullsized":"2024-04-26T23:20:22.358623+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:23.179907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.356131+0000","last_change":"2024-04-26T23:20:22.351221+0000","last_active":"2024-04-26T23:20:22.356131+0000","last_peered":"2024-04-26T23:20:22.356131+0000","last_clean":"2024-04-26T23:20:22.356131+0000","last_became_active":"2024-04-26T23:20:22.350930+0000","last_became_peered":"2024-04-26T23:20:22.350930+0000","last_unstale":"2024-04-26T23:20:22.356131+0000","last_undegraded":"2024-04-26T23:20:22.356131+0000","last_fullsized":"2024-04-26T23:20:22.356131+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:10:14.051477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.353370+0000","last_change":"2024-04-26T23:20:22.348885+0000","last_active":"2024-04-26T23:20:23.353370+0000","last_peered":"2024-04-26T23:20:23.353370+0000","last_clean":"2024-04-26T23:20:23.353370+0000","last_became_active":"2024-04-26T23:20:22.348628+0000","last_became_peered":"2024-04-26T23:20:22.348628+0000","last_unstale":"2024-04-26T23:20:23.353370+0000","last_undegraded":"2024-04-26T23:20:23.353370+0000","last_fullsized":"2024-04-26T23:20:23.353370+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:37:16.426093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.353016+0000","last_change":"2024-04-26T23:20:22.353648+0000","last_active":"2024-04-26T23:20:23.353016+0000","last_peered":"2024-04-26T23:20:23.353016+0000","last_clean":"2024-04-26T23:20:23.353016+0000","last_became_active":"2024-04-26T23:20:22.353324+0000","last_became_peered":"2024-04-26T23:20:22.353324+0000","last_unstale":"2024-04-26T23:20:23.353016+0000","last_undegraded":"2024-04-26T23:20:23.353016+0000","last_fullsized":"2024-04-26T23:20:23.353016+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:23.179907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.514063+0000","last_change":"2024-04-26T23:20:22.361471+0000","last_active":"2024-04-26T23:20:23.514063+0000","last_peered":"2024-04-26T23:20:23.514063+0000","last_clean":"2024-04-26T23:20:23.514063+0000","last_became_active":"2024-04-26T23:20:22.361201+0000","last_became_peered":"2024-04-26T23:20:22.361201+0000","last_unstale":"2024-04-26T23:20:23.514063+0000","last_undegraded":"2024-04-26T23:20:23.514063+0000","last_fullsized":"2024-04-26T23:20:23.514063+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:29.660098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.513971+0000","last_change":"2024-04-26T23:20:22.361208+0000","last_active":"2024-04-26T23:20:23.513971+0000","last_peered":"2024-04-26T23:20:23.513971+0000","last_clean":"2024-04-26T23:20:23.513971+0000","last_became_active":"2024-04-26T23:20:22.360906+0000","last_became_peered":"2024-04-26T23:20:22.360906+0000","last_unstale":"2024-04-26T23:20:23.513971+0000","last_undegraded":"2024-04-26T23:20:23.513971+0000","last_fullsized":"2024-04-26T23:20:23.513971+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:53:01.027225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.356445+0000","last_change":"2024-04-26T23:20:22.354199+0000","last_active":"2024-04-26T23:20:22.356445+0000","last_peered":"2024-04-26T23:20:22.356445+0000","last_clean":"2024-04-26T23:20:22.356445+0000","last_became_active":"2024-04-26T23:20:22.353803+0000","last_became_peered":"2024-04-26T23:20:22.353803+0000","last_unstale":"2024-04-26T23:20:22.356445+0000","last_undegraded":"2024-04-26T23:20:22.356445+0000","last_fullsized":"2024-04-26T23:20:22.356445+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:56:10.119661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.365801+0000","last_change":"2024-04-26T23:20:22.357968+0000","last_active":"2024-04-26T23:20:22.365801+0000","last_peered":"2024-04-26T23:20:22.365801+0000","last_clean":"2024-04-26T23:20:22.365801+0000","last_became_active":"2024-04-26T23:20:22.357686+0000","last_became_peered":"2024-04-26T23:20:22.357686+0000","last_unstale":"2024-04-26T23:20:22.365801+0000","last_undegraded":"2024-04-26T23:20:22.365801+0000","last_fullsized":"2024-04-26T23:20:22.365801+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:37:16.426093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.354120+0000","last_change":"2024-04-26T23:20:22.351041+0000","last_active":"2024-04-26T23:20:22.354120+0000","last_peered":"2024-04-26T23:20:22.354120+0000","last_clean":"2024-04-26T23:20:22.354120+0000","last_became_active":"2024-04-26T23:20:22.350417+0000","last_became_peered":"2024-04-26T23:20:22.350417+0000","last_unstale":"2024-04-26T23:20:22.354120+0000","last_undegraded":"2024-04-26T23:20:22.354120+0000","last_fullsized":"2024-04-26T23:20:22.354120+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:15:39.108411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.359808+0000","last_change":"2024-04-26T23:20:22.351740+0000","last_active":"2024-04-26T23:20:22.359808+0000","last_peered":"2024-04-26T23:20:22.359808+0000","last_clean":"2024-04-26T23:20:22.359808+0000","last_became_active":"2024-04-26T23:20:22.351471+0000","last_became_peered":"2024-04-26T23:20:22.351471+0000","last_unstale":"2024-04-26T23:20:22.359808+0000","last_undegraded":"2024-04-26T23:20:22.359808+0000","last_fullsized":"2024-04-26T23:20:22.359808+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:39:23.094926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.361357+0000","last_change":"2024-04-26T23:20:22.357619+0000","last_active":"2024-04-26T23:20:22.361357+0000","last_peered":"2024-04-26T23:20:22.361357+0000","last_clean":"2024-04-26T23:20:22.361357+0000","last_became_active":"2024-04-26T23:20:22.357297+0000","last_became_peered":"2024-04-26T23:20:22.357297+0000","last_unstale":"2024-04-26T23:20:22.361357+0000","last_undegraded":"2024-04-26T23:20:22.361357+0000","last_fullsized":"2024-04-26T23:20:22.361357+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:47:59.715605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.362705+0000","last_change":"2024-04-26T23:20:22.360080+0000","last_active":"2024-04-26T23:20:22.362705+0000","last_peered":"2024-04-26T23:20:22.362705+0000","last_clean":"2024-04-26T23:20:22.362705+0000","last_became_active":"2024-04-26T23:20:22.359803+0000","last_became_peered":"2024-04-26T23:20:22.359803+0000","last_unstale":"2024-04-26T23:20:22.362705+0000","last_undegraded":"2024-04-26T23:20:22.362705+0000","last_fullsized":"2024-04-26T23:20:22.362705+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:54:52.957936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.353151+0000","last_change":"2024-04-26T23:20:22.350730+0000","last_active":"2024-04-26T23:20:22.353151+0000","last_peered":"2024-04-26T23:20:22.353151+0000","last_clean":"2024-04-26T23:20:22.353151+0000","last_became_active":"2024-04-26T23:20:22.350293+0000","last_became_peered":"2024-04-26T23:20:22.350293+0000","last_unstale":"2024-04-26T23:20:22.353151+0000","last_undegraded":"2024-04-26T23:20:22.353151+0000","last_fullsized":"2024-04-26T23:20:22.353151+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:54.051375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.352080+0000","last_change":"2024-04-26T23:20:22.349599+0000","last_active":"2024-04-26T23:20:22.352080+0000","last_peered":"2024-04-26T23:20:22.352080+0000","last_clean":"2024-04-26T23:20:22.352080+0000","last_became_active":"2024-04-26T23:20:22.348919+0000","last_became_peered":"2024-04-26T23:20:22.348919+0000","last_unstale":"2024-04-26T23:20:22.352080+0000","last_undegraded":"2024-04-26T23:20:22.352080+0000","last_fullsized":"2024-04-26T23:20:22.352080+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:19:55.719384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.352323+0000","last_change":"2024-04-26T23:20:22.351222+0000","last_active":"2024-04-26T23:20:23.352323+0000","last_peered":"2024-04-26T23:20:23.352323+0000","last_clean":"2024-04-26T23:20:23.352323+0000","last_became_active":"2024-04-26T23:20:22.350909+0000","last_became_peered":"2024-04-26T23:20:22.350909+0000","last_unstale":"2024-04-26T23:20:23.352323+0000","last_undegraded":"2024-04-26T23:20:23.352323+0000","last_fullsized":"2024-04-26T23:20:23.352323+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:03:59.571345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.359936+0000","last_change":"2024-04-26T23:20:22.354123+0000","last_active":"2024-04-26T23:20:22.359936+0000","last_peered":"2024-04-26T23:20:22.359936+0000","last_clean":"2024-04-26T23:20:22.359936+0000","last_became_active":"2024-04-26T23:20:22.353853+0000","last_became_peered":"2024-04-26T23:20:22.353853+0000","last_unstale":"2024-04-26T23:20:22.359936+0000","last_undegraded":"2024-04-26T23:20:22.359936+0000","last_fullsized":"2024-04-26T23:20:22.359936+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:25:54.195017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.353103+0000","last_change":"2024-04-26T23:20:22.357507+0000","last_active":"2024-04-26T23:20:23.353103+0000","last_peered":"2024-04-26T23:20:23.353103+0000","last_clean":"2024-04-26T23:20:23.353103+0000","last_became_active":"2024-04-26T23:20:22.356803+0000","last_became_peered":"2024-04-26T23:20:22.356803+0000","last_unstale":"2024-04-26T23:20:23.353103+0000","last_undegraded":"2024-04-26T23:20:23.353103+0000","last_fullsized":"2024-04-26T23:20:23.353103+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:33:15.921208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.356491+0000","last_change":"2024-04-26T23:20:22.355401+0000","last_active":"2024-04-26T23:20:22.356491+0000","last_peered":"2024-04-26T23:20:22.356491+0000","last_clean":"2024-04-26T23:20:22.356491+0000","last_became_active":"2024-04-26T23:20:22.355125+0000","last_became_peered":"2024-04-26T23:20:22.355125+0000","last_unstale":"2024-04-26T23:20:22.356491+0000","last_undegraded":"2024-04-26T23:20:22.356491+0000","last_fullsized":"2024-04-26T23:20:22.356491+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:54:52.957936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.362718+0000","last_change":"2024-04-26T23:20:22.360659+0000","last_active":"2024-04-26T23:20:22.362718+0000","last_peered":"2024-04-26T23:20:22.362718+0000","last_clean":"2024-04-26T23:20:22.362718+0000","last_became_active":"2024-04-26T23:20:22.360405+0000","last_became_peered":"2024-04-26T23:20:22.360405+0000","last_unstale":"2024-04-26T23:20:22.362718+0000","last_undegraded":"2024-04-26T23:20:22.362718+0000","last_fullsized":"2024-04-26T23:20:22.362718+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:21:38.353777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.366011+0000","last_change":"2024-04-26T23:20:22.364482+0000","last_active":"2024-04-26T23:20:22.366011+0000","last_peered":"2024-04-26T23:20:22.366011+0000","last_clean":"2024-04-26T23:20:22.366011+0000","last_became_active":"2024-04-26T23:20:22.363442+0000","last_became_peered":"2024-04-26T23:20:22.363442+0000","last_unstale":"2024-04-26T23:20:22.366011+0000","last_undegraded":"2024-04-26T23:20:22.366011+0000","last_fullsized":"2024-04-26T23:20:22.366011+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:58:40.176904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.361844+0000","last_change":"2024-04-26T23:20:22.358255+0000","last_active":"2024-04-26T23:20:22.361844+0000","last_peered":"2024-04-26T23:20:22.361844+0000","last_clean":"2024-04-26T23:20:22.361844+0000","last_became_active":"2024-04-26T23:20:22.357997+0000","last_became_peered":"2024-04-26T23:20:22.357997+0000","last_unstale":"2024-04-26T23:20:22.361844+0000","last_undegraded":"2024-04-26T23:20:22.361844+0000","last_fullsized":"2024-04-26T23:20:22.361844+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:23:40.504144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.349015+0000","last_change":"2024-04-26T23:20:22.347985+0000","last_active":"2024-04-26T23:20:22.349015+0000","last_peered":"2024-04-26T23:20:22.349015+0000","last_clean":"2024-04-26T23:20:22.349015+0000","last_became_active":"2024-04-26T23:20:22.347692+0000","last_became_peered":"2024-04-26T23:20:22.347692+0000","last_unstale":"2024-04-26T23:20:22.349015+0000","last_undegraded":"2024-04-26T23:20:22.349015+0000","last_fullsized":"2024-04-26T23:20:22.349015+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:52:13.172713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.356418+0000","last_change":"2024-04-26T23:20:22.352336+0000","last_active":"2024-04-26T23:20:22.356418+0000","last_peered":"2024-04-26T23:20:22.356418+0000","last_clean":"2024-04-26T23:20:22.356418+0000","last_became_active":"2024-04-26T23:20:22.352041+0000","last_became_peered":"2024-04-26T23:20:22.352041+0000","last_unstale":"2024-04-26T23:20:22.356418+0000","last_undegraded":"2024-04-26T23:20:22.356418+0000","last_fullsized":"2024-04-26T23:20:22.356418+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:23:40.504144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.348362+0000","last_change":"2024-04-26T23:20:22.347239+0000","last_active":"2024-04-26T23:20:22.348362+0000","last_peered":"2024-04-26T23:20:22.348362+0000","last_clean":"2024-04-26T23:20:22.348362+0000","last_became_active":"2024-04-26T23:20:22.346936+0000","last_became_peered":"2024-04-26T23:20:22.346936+0000","last_unstale":"2024-04-26T23:20:22.348362+0000","last_undegraded":"2024-04-26T23:20:22.348362+0000","last_fullsized":"2024-04-26T23:20:22.348362+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:17:35.154747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"53'4","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.367312+0000","last_change":"2024-04-26T23:20:21.350058+0000","last_active":"2024-04-26T23:20:22.367312+0000","last_peered":"2024-04-26T23:20:22.367312+0000","last_clean":"2024-04-26T23:20:22.367312+0000","last_became_active":"2024-04-26T23:20:21.349721+0000","last_became_peered":"2024-04-26T23:20:21.349721+0000","last_unstale":"2024-04-26T23:20:22.367312+0000","last_undegraded":"2024-04-26T23:20:22.367312+0000","last_fullsized":"2024-04-26T23:20:22.367312+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:47:37.763322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.352125+0000","last_change":"2024-04-26T23:20:22.346088+0000","last_active":"2024-04-26T23:20:22.352125+0000","last_peered":"2024-04-26T23:20:22.352125+0000","last_clean":"2024-04-26T23:20:22.352125+0000","last_became_active":"2024-04-26T23:20:22.345811+0000","last_became_peered":"2024-04-26T23:20:22.345811+0000","last_unstale":"2024-04-26T23:20:22.352125+0000","last_undegraded":"2024-04-26T23:20:22.352125+0000","last_fullsized":"2024-04-26T23:20:22.352125+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:03:59.571345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"53'1","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.364464+0000","last_change":"2024-04-26T23:20:21.350999+0000","last_active":"2024-04-26T23:20:22.364464+0000","last_peered":"2024-04-26T23:20:22.364464+0000","last_clean":"2024-04-26T23:20:22.364464+0000","last_became_active":"2024-04-26T23:20:21.350702+0000","last_became_peered":"2024-04-26T23:20:21.350702+0000","last_unstale":"2024-04-26T23:20:22.364464+0000","last_undegraded":"2024-04-26T23:20:22.364464+0000","last_fullsized":"2024-04-26T23:20:22.364464+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:23:39.501646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.348330+0000","last_change":"2024-04-26T23:20:22.345369+0000","last_active":"2024-04-26T23:20:22.348330+0000","last_peered":"2024-04-26T23:20:22.348330+0000","last_clean":"2024-04-26T23:20:22.348330+0000","last_became_active":"2024-04-26T23:20:22.345069+0000","last_became_peered":"2024-04-26T23:20:22.345069+0000","last_unstale":"2024-04-26T23:20:22.348330+0000","last_undegraded":"2024-04-26T23:20:22.348330+0000","last_fullsized":"2024-04-26T23:20:22.348330+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:57.150910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"53'2","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.363922+0000","last_change":"2024-04-26T23:20:21.340294+0000","last_active":"2024-04-26T23:20:22.363922+0000","last_peered":"2024-04-26T23:20:22.363922+0000","last_clean":"2024-04-26T23:20:22.363922+0000","last_became_active":"2024-04-26T23:20:21.339986+0000","last_became_peered":"2024-04-26T23:20:21.339986+0000","last_unstale":"2024-04-26T23:20:22.363922+0000","last_undegraded":"2024-04-26T23:20:22.363922+0000","last_fullsized":"2024-04-26T23:20:22.363922+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:15:38.105913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.364326+0000","last_change":"2024-04-26T23:20:22.357226+0000","last_active":"2024-04-26T23:20:22.364326+0000","last_peered":"2024-04-26T23:20:22.364326+0000","last_clean":"2024-04-26T23:20:22.364326+0000","last_became_active":"2024-04-26T23:20:22.356965+0000","last_became_peered":"2024-04-26T23:20:22.356965+0000","last_unstale":"2024-04-26T23:20:22.364326+0000","last_undegraded":"2024-04-26T23:20:22.364326+0000","last_fullsized":"2024-04-26T23:20:22.364326+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:36:13.984324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.353365+0000","last_change":"2024-04-26T23:20:21.347741+0000","last_active":"2024-04-26T23:20:23.353365+0000","last_peered":"2024-04-26T23:20:23.353365+0000","last_clean":"2024-04-26T23:20:23.353365+0000","last_became_active":"2024-04-26T23:20:21.347463+0000","last_became_peered":"2024-04-26T23:20:21.347463+0000","last_unstale":"2024-04-26T23:20:23.353365+0000","last_undegraded":"2024-04-26T23:20:23.353365+0000","last_fullsized":"2024-04-26T23:20:23.353365+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:17:34.152250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.351137+0000","last_change":"2024-04-26T23:20:22.349525+0000","last_active":"2024-04-26T23:20:22.351137+0000","last_peered":"2024-04-26T23:20:22.351137+0000","last_clean":"2024-04-26T23:20:22.351137+0000","last_became_active":"2024-04-26T23:20:22.349206+0000","last_became_peered":"2024-04-26T23:20:22.349206+0000","last_unstale":"2024-04-26T23:20:22.351137+0000","last_undegraded":"2024-04-26T23:20:22.351137+0000","last_fullsized":"2024-04-26T23:20:22.351137+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:53.946417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.514134+0000","last_change":"2024-04-26T23:20:21.350949+0000","last_active":"2024-04-26T23:20:23.514134+0000","last_peered":"2024-04-26T23:20:23.514134+0000","last_clean":"2024-04-26T23:20:23.514134+0000","last_became_active":"2024-04-26T23:20:21.350652+0000","last_became_peered":"2024-04-26T23:20:21.350652+0000","last_unstale":"2024-04-26T23:20:23.514134+0000","last_undegraded":"2024-04-26T23:20:23.514134+0000","last_fullsized":"2024-04-26T23:20:23.514134+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:19:16.521452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.361287+0000","last_change":"2024-04-26T23:20:22.358930+0000","last_active":"2024-04-26T23:20:22.361287+0000","last_peered":"2024-04-26T23:20:22.361287+0000","last_clean":"2024-04-26T23:20:22.361287+0000","last_became_active":"2024-04-26T23:20:22.358231+0000","last_became_peered":"2024-04-26T23:20:22.358231+0000","last_unstale":"2024-04-26T23:20:22.361287+0000","last_undegraded":"2024-04-26T23:20:22.361287+0000","last_fullsized":"2024-04-26T23:20:22.361287+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:53:38.181432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"53'1","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.356577+0000","last_change":"2024-04-26T23:20:21.347858+0000","last_active":"2024-04-26T23:20:22.356577+0000","last_peered":"2024-04-26T23:20:22.356577+0000","last_clean":"2024-04-26T23:20:22.356577+0000","last_became_active":"2024-04-26T23:20:21.347590+0000","last_became_peered":"2024-04-26T23:20:21.347590+0000","last_unstale":"2024-04-26T23:20:22.356577+0000","last_undegraded":"2024-04-26T23:20:22.356577+0000","last_fullsized":"2024-04-26T23:20:22.356577+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:54:17.051800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.352358+0000","last_change":"2024-04-26T23:20:22.351205+0000","last_active":"2024-04-26T23:20:23.352358+0000","last_peered":"2024-04-26T23:20:23.352358+0000","last_clean":"2024-04-26T23:20:23.352358+0000","last_became_active":"2024-04-26T23:20:22.350812+0000","last_became_peered":"2024-04-26T23:20:22.350812+0000","last_unstale":"2024-04-26T23:20:23.352358+0000","last_undegraded":"2024-04-26T23:20:23.352358+0000","last_fullsized":"2024-04-26T23:20:23.352358+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:10:14.051477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.353245+0000","last_change":"2024-04-26T23:20:21.346980+0000","last_active":"2024-04-26T23:20:23.353245+0000","last_peered":"2024-04-26T23:20:23.353245+0000","last_clean":"2024-04-26T23:20:23.353245+0000","last_became_active":"2024-04-26T23:20:21.346064+0000","last_became_peered":"2024-04-26T23:20:21.346064+0000","last_unstale":"2024-04-26T23:20:23.353245+0000","last_undegraded":"2024-04-26T23:20:23.353245+0000","last_fullsized":"2024-04-26T23:20:23.353245+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:56.148412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.356833+0000","last_change":"2024-04-26T23:20:22.355222+0000","last_active":"2024-04-26T23:20:22.356833+0000","last_peered":"2024-04-26T23:20:22.356833+0000","last_clean":"2024-04-26T23:20:22.356833+0000","last_became_active":"2024-04-26T23:20:22.354550+0000","last_became_peered":"2024-04-26T23:20:22.354550+0000","last_unstale":"2024-04-26T23:20:22.356833+0000","last_undegraded":"2024-04-26T23:20:22.356833+0000","last_fullsized":"2024-04-26T23:20:22.356833+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:10:01.361701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.513915+0000","last_change":"2024-04-26T23:20:21.354548+0000","last_active":"2024-04-26T23:20:23.513915+0000","last_peered":"2024-04-26T23:20:23.513915+0000","last_clean":"2024-04-26T23:20:23.513915+0000","last_became_active":"2024-04-26T23:20:21.354269+0000","last_became_peered":"2024-04-26T23:20:21.354269+0000","last_unstale":"2024-04-26T23:20:23.513915+0000","last_undegraded":"2024-04-26T23:20:23.513915+0000","last_fullsized":"2024-04-26T23:20:23.513915+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:07.701563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.514371+0000","last_change":"2024-04-26T23:20:22.362804+0000","last_active":"2024-04-26T23:20:23.514371+0000","last_peered":"2024-04-26T23:20:23.514371+0000","last_clean":"2024-04-26T23:20:23.514371+0000","last_became_active":"2024-04-26T23:20:22.362541+0000","last_became_peered":"2024-04-26T23:20:22.362541+0000","last_unstale":"2024-04-26T23:20:23.514371+0000","last_undegraded":"2024-04-26T23:20:23.514371+0000","last_fullsized":"2024-04-26T23:20:23.514371+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:54:18.054298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.341571+0000","last_change":"2024-04-26T23:20:21.348484+0000","last_active":"2024-04-26T23:20:22.341571+0000","last_peered":"2024-04-26T23:20:22.341571+0000","last_clean":"2024-04-26T23:20:22.341571+0000","last_became_active":"2024-04-26T23:20:21.348169+0000","last_became_peered":"2024-04-26T23:20:21.348169+0000","last_unstale":"2024-04-26T23:20:22.341571+0000","last_undegraded":"2024-04-26T23:20:22.341571+0000","last_fullsized":"2024-04-26T23:20:22.341571+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:47:37.763322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.354085+0000","last_change":"2024-04-26T23:20:22.351912+0000","last_active":"2024-04-26T23:20:22.354085+0000","last_peered":"2024-04-26T23:20:22.354085+0000","last_clean":"2024-04-26T23:20:22.354085+0000","last_became_active":"2024-04-26T23:20:22.351626+0000","last_became_peered":"2024-04-26T23:20:22.351626+0000","last_unstale":"2024-04-26T23:20:22.354085+0000","last_undegraded":"2024-04-26T23:20:22.354085+0000","last_fullsized":"2024-04-26T23:20:22.354085+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:17:35.154747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.348042+0000","last_change":"2024-04-26T23:20:21.348401+0000","last_active":"2024-04-26T23:20:22.348042+0000","last_peered":"2024-04-26T23:20:22.348042+0000","last_clean":"2024-04-26T23:20:22.348042+0000","last_became_active":"2024-04-26T23:20:21.348072+0000","last_became_peered":"2024-04-26T23:20:21.348072+0000","last_unstale":"2024-04-26T23:20:22.348042+0000","last_undegraded":"2024-04-26T23:20:22.348042+0000","last_fullsized":"2024-04-26T23:20:22.348042+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:56:09.117164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.362508+0000","last_change":"2024-04-26T23:20:22.357498+0000","last_active":"2024-04-26T23:20:22.362508+0000","last_peered":"2024-04-26T23:20:22.362508+0000","last_clean":"2024-04-26T23:20:22.362508+0000","last_became_active":"2024-04-26T23:20:22.357213+0000","last_became_peered":"2024-04-26T23:20:22.357213+0000","last_unstale":"2024-04-26T23:20:22.362508+0000","last_undegraded":"2024-04-26T23:20:22.362508+0000","last_fullsized":"2024-04-26T23:20:22.362508+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:42:05.897506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"53'3","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.362554+0000","last_change":"2024-04-26T23:20:21.354343+0000","last_active":"2024-04-26T23:20:22.362554+0000","last_peered":"2024-04-26T23:20:22.362554+0000","last_clean":"2024-04-26T23:20:22.362554+0000","last_became_active":"2024-04-26T23:20:21.354067+0000","last_became_peered":"2024-04-26T23:20:21.354067+0000","last_unstale":"2024-04-26T23:20:22.362554+0000","last_undegraded":"2024-04-26T23:20:22.362554+0000","last_fullsized":"2024-04-26T23:20:22.362554+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:54:17.051800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.354233+0000","last_change":"2024-04-26T23:20:22.351263+0000","last_active":"2024-04-26T23:20:22.354233+0000","last_peered":"2024-04-26T23:20:22.354233+0000","last_clean":"2024-04-26T23:20:22.354233+0000","last_became_active":"2024-04-26T23:20:22.350759+0000","last_became_peered":"2024-04-26T23:20:22.350759+0000","last_unstale":"2024-04-26T23:20:22.354233+0000","last_undegraded":"2024-04-26T23:20:22.354233+0000","last_fullsized":"2024-04-26T23:20:22.354233+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:57.150910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"53'6","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.514310+0000","last_change":"2024-04-26T23:20:21.348001+0000","last_active":"2024-04-26T23:20:23.514310+0000","last_peered":"2024-04-26T23:20:23.514310+0000","last_clean":"2024-04-26T23:20:23.514310+0000","last_became_active":"2024-04-26T23:20:21.347720+0000","last_became_peered":"2024-04-26T23:20:21.347720+0000","last_unstale":"2024-04-26T23:20:23.514310+0000","last_undegraded":"2024-04-26T23:20:23.514310+0000","last_fullsized":"2024-04-26T23:20:23.514310+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:35:33.072603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.352859+0000","last_change":"2024-04-26T23:20:22.341469+0000","last_active":"2024-04-26T23:20:23.352859+0000","last_peered":"2024-04-26T23:20:23.352859+0000","last_clean":"2024-04-26T23:20:23.352859+0000","last_became_active":"2024-04-26T23:20:22.341176+0000","last_became_peered":"2024-04-26T23:20:22.341176+0000","last_unstale":"2024-04-26T23:20:23.352859+0000","last_undegraded":"2024-04-26T23:20:23.352859+0000","last_fullsized":"2024-04-26T23:20:23.352859+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:10:01.361701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"53'2","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.367252+0000","last_change":"2024-04-26T23:20:21.349102+0000","last_active":"2024-04-26T23:20:22.367252+0000","last_peered":"2024-04-26T23:20:22.367252+0000","last_clean":"2024-04-26T23:20:22.367252+0000","last_became_active":"2024-04-26T23:20:21.348810+0000","last_became_peered":"2024-04-26T23:20:21.348810+0000","last_unstale":"2024-04-26T23:20:22.367252+0000","last_undegraded":"2024-04-26T23:20:22.367252+0000","last_fullsized":"2024-04-26T23:20:22.367252+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:54:51.955438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.355205+0000","last_change":"2024-04-26T23:20:22.353043+0000","last_active":"2024-04-26T23:20:22.355205+0000","last_peered":"2024-04-26T23:20:22.355205+0000","last_clean":"2024-04-26T23:20:22.355205+0000","last_became_active":"2024-04-26T23:20:22.352751+0000","last_became_peered":"2024-04-26T23:20:22.352751+0000","last_unstale":"2024-04-26T23:20:22.355205+0000","last_undegraded":"2024-04-26T23:20:22.355205+0000","last_fullsized":"2024-04-26T23:20:22.355205+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:35:14.922511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.352588+0000","last_change":"2024-04-26T23:20:21.346438+0000","last_active":"2024-04-26T23:20:23.352588+0000","last_peered":"2024-04-26T23:20:23.352588+0000","last_clean":"2024-04-26T23:20:23.352588+0000","last_became_active":"2024-04-26T23:20:21.346143+0000","last_became_peered":"2024-04-26T23:20:21.346143+0000","last_unstale":"2024-04-26T23:20:23.352588+0000","last_undegraded":"2024-04-26T23:20:23.352588+0000","last_fullsized":"2024-04-26T23:20:23.352588+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:07.701563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.513965+0000","last_change":"2024-04-26T23:20:22.364021+0000","last_active":"2024-04-26T23:20:23.513965+0000","last_peered":"2024-04-26T23:20:23.513965+0000","last_clean":"2024-04-26T23:20:23.513965+0000","last_became_active":"2024-04-26T23:20:22.363609+0000","last_became_peered":"2024-04-26T23:20:22.363609+0000","last_unstale":"2024-04-26T23:20:23.513965+0000","last_undegraded":"2024-04-26T23:20:23.513965+0000","last_fullsized":"2024-04-26T23:20:23.513965+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:42:05.897506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.352685+0000","last_change":"2024-04-26T23:20:21.345639+0000","last_active":"2024-04-26T23:20:23.352685+0000","last_peered":"2024-04-26T23:20:23.352685+0000","last_clean":"2024-04-26T23:20:23.352685+0000","last_became_active":"2024-04-26T23:20:21.345358+0000","last_became_peered":"2024-04-26T23:20:21.345358+0000","last_unstale":"2024-04-26T23:20:23.352685+0000","last_undegraded":"2024-04-26T23:20:23.352685+0000","last_fullsized":"2024-04-26T23:20:23.352685+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:09:27.893781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.353976+0000","last_change":"2024-04-26T23:20:22.349964+0000","last_active":"2024-04-26T23:20:22.353976+0000","last_peered":"2024-04-26T23:20:22.353976+0000","last_clean":"2024-04-26T23:20:22.353976+0000","last_became_active":"2024-04-26T23:20:22.349288+0000","last_became_peered":"2024-04-26T23:20:22.349288+0000","last_unstale":"2024-04-26T23:20:22.353976+0000","last_undegraded":"2024-04-26T23:20:22.353976+0000","last_fullsized":"2024-04-26T23:20:22.353976+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:58:08.886088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"53'8","reported_seq":20,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.361067+0000","last_change":"2024-04-26T23:20:21.350145+0000","last_active":"2024-04-26T23:20:22.361067+0000","last_peered":"2024-04-26T23:20:22.361067+0000","last_clean":"2024-04-26T23:20:22.361067+0000","last_became_active":"2024-04-26T23:20:21.349857+0000","last_became_peered":"2024-04-26T23:20:21.349857+0000","last_unstale":"2024-04-26T23:20:22.361067+0000","last_undegraded":"2024-04-26T23:20:22.361067+0000","last_fullsized":"2024-04-26T23:20:22.361067+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:28.657600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.361935+0000","last_change":"2024-04-26T23:20:22.358963+0000","last_active":"2024-04-26T23:20:22.361935+0000","last_peered":"2024-04-26T23:20:22.361935+0000","last_clean":"2024-04-26T23:20:22.361935+0000","last_became_active":"2024-04-26T23:20:22.358678+0000","last_became_peered":"2024-04-26T23:20:22.358678+0000","last_unstale":"2024-04-26T23:20:22.361935+0000","last_undegraded":"2024-04-26T23:20:22.361935+0000","last_fullsized":"2024-04-26T23:20:22.361935+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:56:10.119661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"53'1","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.365606+0000","last_change":"2024-04-26T23:20:21.348069+0000","last_active":"2024-04-26T23:20:22.365606+0000","last_peered":"2024-04-26T23:20:22.365606+0000","last_clean":"2024-04-26T23:20:22.365606+0000","last_became_active":"2024-04-26T23:20:21.347150+0000","last_became_peered":"2024-04-26T23:20:21.347150+0000","last_unstale":"2024-04-26T23:20:22.365606+0000","last_undegraded":"2024-04-26T23:20:22.365606+0000","last_fullsized":"2024-04-26T23:20:22.365606+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:40:56.746673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.352886+0000","last_change":"2024-04-26T23:20:22.349644+0000","last_active":"2024-04-26T23:20:23.352886+0000","last_peered":"2024-04-26T23:20:23.352886+0000","last_clean":"2024-04-26T23:20:23.352886+0000","last_became_active":"2024-04-26T23:20:22.346873+0000","last_became_peered":"2024-04-26T23:20:22.346873+0000","last_unstale":"2024-04-26T23:20:23.352886+0000","last_undegraded":"2024-04-26T23:20:23.352886+0000","last_fullsized":"2024-04-26T23:20:23.352886+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:55:06.694814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"53'2","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.361620+0000","last_change":"2024-04-26T23:20:21.348994+0000","last_active":"2024-04-26T23:20:22.361620+0000","last_peered":"2024-04-26T23:20:22.361620+0000","last_clean":"2024-04-26T23:20:22.361620+0000","last_became_active":"2024-04-26T23:20:21.348713+0000","last_became_peered":"2024-04-26T23:20:21.348713+0000","last_unstale":"2024-04-26T23:20:22.361620+0000","last_undegraded":"2024-04-26T23:20:22.361620+0000","last_fullsized":"2024-04-26T23:20:22.361620+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:19:54.716887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.365474+0000","last_change":"2024-04-26T23:20:22.362301+0000","last_active":"2024-04-26T23:20:22.365474+0000","last_peered":"2024-04-26T23:20:22.365474+0000","last_clean":"2024-04-26T23:20:22.365474+0000","last_became_active":"2024-04-26T23:20:22.361986+0000","last_became_peered":"2024-04-26T23:20:22.361986+0000","last_unstale":"2024-04-26T23:20:22.365474+0000","last_undegraded":"2024-04-26T23:20:22.365474+0000","last_fullsized":"2024-04-26T23:20:22.365474+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:31.965056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.352683+0000","last_change":"2024-04-26T23:20:21.351307+0000","last_active":"2024-04-26T23:20:23.352683+0000","last_peered":"2024-04-26T23:20:23.352683+0000","last_clean":"2024-04-26T23:20:23.352683+0000","last_became_active":"2024-04-26T23:20:21.350102+0000","last_became_peered":"2024-04-26T23:20:21.350102+0000","last_unstale":"2024-04-26T23:20:23.352683+0000","last_undegraded":"2024-04-26T23:20:23.352683+0000","last_fullsized":"2024-04-26T23:20:23.352683+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:15:38.105913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.354253+0000","last_change":"2024-04-26T23:20:22.352142+0000","last_active":"2024-04-26T23:20:22.354253+0000","last_peered":"2024-04-26T23:20:22.354253+0000","last_clean":"2024-04-26T23:20:22.354253+0000","last_became_active":"2024-04-26T23:20:22.351704+0000","last_became_peered":"2024-04-26T23:20:22.351704+0000","last_unstale":"2024-04-26T23:20:22.354253+0000","last_undegraded":"2024-04-26T23:20:22.354253+0000","last_fullsized":"2024-04-26T23:20:22.354253+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:04:00.567851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.339397+0000","last_change":"2024-04-26T23:20:21.352216+0000","last_active":"2024-04-26T23:20:22.339397+0000","last_peered":"2024-04-26T23:20:22.339397+0000","last_clean":"2024-04-26T23:20:22.339397+0000","last_became_active":"2024-04-26T23:20:21.351823+0000","last_became_peered":"2024-04-26T23:20:21.351823+0000","last_unstale":"2024-04-26T23:20:22.339397+0000","last_undegraded":"2024-04-26T23:20:22.339397+0000","last_fullsized":"2024-04-26T23:20:22.339397+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:58:07.883590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.353356+0000","last_change":"2024-04-26T23:20:22.358146+0000","last_active":"2024-04-26T23:20:23.353356+0000","last_peered":"2024-04-26T23:20:23.353356+0000","last_clean":"2024-04-26T23:20:23.353356+0000","last_became_active":"2024-04-26T23:20:22.357567+0000","last_became_peered":"2024-04-26T23:20:22.357567+0000","last_unstale":"2024-04-26T23:20:23.353356+0000","last_undegraded":"2024-04-26T23:20:23.353356+0000","last_fullsized":"2024-04-26T23:20:23.353356+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:30:38.323701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"53'2","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.367091+0000","last_change":"2024-04-26T23:20:21.353594+0000","last_active":"2024-04-26T23:20:22.367091+0000","last_peered":"2024-04-26T23:20:22.367091+0000","last_clean":"2024-04-26T23:20:22.367091+0000","last_became_active":"2024-04-26T23:20:21.353226+0000","last_became_peered":"2024-04-26T23:20:21.353226+0000","last_unstale":"2024-04-26T23:20:22.367091+0000","last_undegraded":"2024-04-26T23:20:22.367091+0000","last_fullsized":"2024-04-26T23:20:22.367091+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:36:12.981827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.356429+0000","last_change":"2024-04-26T23:20:22.354795+0000","last_active":"2024-04-26T23:20:22.356429+0000","last_peered":"2024-04-26T23:20:22.356429+0000","last_clean":"2024-04-26T23:20:22.356429+0000","last_became_active":"2024-04-26T23:20:22.354521+0000","last_became_peered":"2024-04-26T23:20:22.354521+0000","last_unstale":"2024-04-26T23:20:22.356429+0000","last_undegraded":"2024-04-26T23:20:22.356429+0000","last_fullsized":"2024-04-26T23:20:22.356429+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:40:57.749170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.337174+0000","last_change":"2024-04-26T23:20:21.349962+0000","last_active":"2024-04-26T23:20:22.337174+0000","last_peered":"2024-04-26T23:20:22.337174+0000","last_clean":"2024-04-26T23:20:22.337174+0000","last_became_active":"2024-04-26T23:20:21.349695+0000","last_became_peered":"2024-04-26T23:20:21.349695+0000","last_unstale":"2024-04-26T23:20:22.337174+0000","last_undegraded":"2024-04-26T23:20:22.337174+0000","last_fullsized":"2024-04-26T23:20:22.337174+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:29:43.643241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.349753+0000","last_change":"2024-04-26T23:20:22.347662+0000","last_active":"2024-04-26T23:20:22.349753+0000","last_peered":"2024-04-26T23:20:22.349753+0000","last_clean":"2024-04-26T23:20:22.349753+0000","last_became_active":"2024-04-26T23:20:22.346708+0000","last_became_peered":"2024-04-26T23:20:22.346708+0000","last_unstale":"2024-04-26T23:20:22.349753+0000","last_undegraded":"2024-04-26T23:20:22.349753+0000","last_fullsized":"2024-04-26T23:20:22.349753+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:35:14.922511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.352345+0000","last_change":"2024-04-26T23:20:21.355343+0000","last_active":"2024-04-26T23:20:23.352345+0000","last_peered":"2024-04-26T23:20:23.352345+0000","last_clean":"2024-04-26T23:20:23.352345+0000","last_became_active":"2024-04-26T23:20:21.355088+0000","last_became_peered":"2024-04-26T23:20:21.355088+0000","last_unstale":"2024-04-26T23:20:23.352345+0000","last_undegraded":"2024-04-26T23:20:23.352345+0000","last_fullsized":"2024-04-26T23:20:23.352345+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:58:07.883590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.353319+0000","last_change":"2024-04-26T23:20:22.357862+0000","last_active":"2024-04-26T23:20:23.353319+0000","last_peered":"2024-04-26T23:20:23.353319+0000","last_clean":"2024-04-26T23:20:23.353319+0000","last_became_active":"2024-04-26T23:20:22.356157+0000","last_became_peered":"2024-04-26T23:20:22.356157+0000","last_unstale":"2024-04-26T23:20:23.353319+0000","last_undegraded":"2024-04-26T23:20:23.353319+0000","last_fullsized":"2024-04-26T23:20:23.353319+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:42:39.240263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"53'2","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.365258+0000","last_change":"2024-04-26T23:20:21.352529+0000","last_active":"2024-04-26T23:20:22.365258+0000","last_peered":"2024-04-26T23:20:22.365258+0000","last_clean":"2024-04-26T23:20:22.365258+0000","last_became_active":"2024-04-26T23:20:21.352251+0000","last_became_peered":"2024-04-26T23:20:21.352251+0000","last_unstale":"2024-04-26T23:20:22.365258+0000","last_undegraded":"2024-04-26T23:20:22.365258+0000","last_fullsized":"2024-04-26T23:20:22.365258+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:02:58.932261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.353116+0000","last_change":"2024-04-26T23:20:22.344984+0000","last_active":"2024-04-26T23:20:23.353116+0000","last_peered":"2024-04-26T23:20:23.353116+0000","last_clean":"2024-04-26T23:20:23.353116+0000","last_became_active":"2024-04-26T23:20:22.344694+0000","last_became_peered":"2024-04-26T23:20:22.344694+0000","last_unstale":"2024-04-26T23:20:23.353116+0000","last_undegraded":"2024-04-26T23:20:23.353116+0000","last_fullsized":"2024-04-26T23:20:23.353116+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:47:59.715605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.339411+0000","last_change":"2024-04-26T23:20:21.345885+0000","last_active":"2024-04-26T23:20:22.339411+0000","last_peered":"2024-04-26T23:20:22.339411+0000","last_clean":"2024-04-26T23:20:22.339411+0000","last_became_active":"2024-04-26T23:20:21.345596+0000","last_became_peered":"2024-04-26T23:20:21.345596+0000","last_unstale":"2024-04-26T23:20:22.339411+0000","last_undegraded":"2024-04-26T23:20:22.339411+0000","last_fullsized":"2024-04-26T23:20:22.339411+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:09:27.893781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.362534+0000","last_change":"2024-04-26T23:20:22.358252+0000","last_active":"2024-04-26T23:20:22.362534+0000","last_peered":"2024-04-26T23:20:22.362534+0000","last_clean":"2024-04-26T23:20:22.362534+0000","last_became_active":"2024-04-26T23:20:22.357977+0000","last_became_peered":"2024-04-26T23:20:22.357977+0000","last_unstale":"2024-04-26T23:20:22.362534+0000","last_undegraded":"2024-04-26T23:20:22.362534+0000","last_fullsized":"2024-04-26T23:20:22.362534+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:53:01.027225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.352724+0000","last_change":"2024-04-26T23:20:21.349544+0000","last_active":"2024-04-26T23:20:23.352724+0000","last_peered":"2024-04-26T23:20:23.352724+0000","last_clean":"2024-04-26T23:20:23.352724+0000","last_became_active":"2024-04-26T23:20:21.349284+0000","last_became_peered":"2024-04-26T23:20:21.349284+0000","last_unstale":"2024-04-26T23:20:23.352724+0000","last_undegraded":"2024-04-26T23:20:23.352724+0000","last_fullsized":"2024-04-26T23:20:23.352724+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:04:42.194040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.365753+0000","last_change":"2024-04-26T23:20:22.364177+0000","last_active":"2024-04-26T23:20:22.365753+0000","last_peered":"2024-04-26T23:20:22.365753+0000","last_clean":"2024-04-26T23:20:22.365753+0000","last_became_active":"2024-04-26T23:20:22.363873+0000","last_became_peered":"2024-04-26T23:20:22.363873+0000","last_unstale":"2024-04-26T23:20:22.365753+0000","last_undegraded":"2024-04-26T23:20:22.365753+0000","last_fullsized":"2024-04-26T23:20:22.365753+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:53.946417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.353423+0000","last_change":"2024-04-26T23:20:21.346723+0000","last_active":"2024-04-26T23:20:23.353423+0000","last_peered":"2024-04-26T23:20:23.353423+0000","last_clean":"2024-04-26T23:20:23.353423+0000","last_became_active":"2024-04-26T23:20:21.346450+0000","last_became_peered":"2024-04-26T23:20:21.346450+0000","last_unstale":"2024-04-26T23:20:23.353423+0000","last_undegraded":"2024-04-26T23:20:23.353423+0000","last_fullsized":"2024-04-26T23:20:23.353423+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:03:59.565353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.359404+0000","last_change":"2024-04-26T23:20:22.356941+0000","last_active":"2024-04-26T23:20:22.359404+0000","last_peered":"2024-04-26T23:20:22.359404+0000","last_clean":"2024-04-26T23:20:22.359404+0000","last_became_active":"2024-04-26T23:20:22.353656+0000","last_became_peered":"2024-04-26T23:20:22.353656+0000","last_unstale":"2024-04-26T23:20:22.359404+0000","last_undegraded":"2024-04-26T23:20:22.359404+0000","last_fullsized":"2024-04-26T23:20:22.359404+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:33:15.921208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.336380+0000","last_change":"2024-04-26T23:20:21.343908+0000","last_active":"2024-04-26T23:20:22.336380+0000","last_peered":"2024-04-26T23:20:22.336380+0000","last_clean":"2024-04-26T23:20:22.336380+0000","last_became_active":"2024-04-26T23:20:21.343577+0000","last_became_peered":"2024-04-26T23:20:21.343577+0000","last_unstale":"2024-04-26T23:20:22.336380+0000","last_undegraded":"2024-04-26T23:20:22.336380+0000","last_fullsized":"2024-04-26T23:20:22.336380+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:58:18.678110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.354053+0000","last_change":"2024-04-26T23:20:22.351156+0000","last_active":"2024-04-26T23:20:22.354053+0000","last_peered":"2024-04-26T23:20:22.354053+0000","last_clean":"2024-04-26T23:20:22.354053+0000","last_became_active":"2024-04-26T23:20:22.350816+0000","last_became_peered":"2024-04-26T23:20:22.350816+0000","last_unstale":"2024-04-26T23:20:22.354053+0000","last_undegraded":"2024-04-26T23:20:22.354053+0000","last_fullsized":"2024-04-26T23:20:22.354053+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:09:28.896278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.514154+0000","last_change":"2024-04-26T23:20:21.348504+0000","last_active":"2024-04-26T23:20:23.514154+0000","last_peered":"2024-04-26T23:20:23.514154+0000","last_clean":"2024-04-26T23:20:23.514154+0000","last_became_active":"2024-04-26T23:20:21.348255+0000","last_became_peered":"2024-04-26T23:20:21.348255+0000","last_unstale":"2024-04-26T23:20:23.514154+0000","last_undegraded":"2024-04-26T23:20:23.514154+0000","last_fullsized":"2024-04-26T23:20:23.514154+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:14:21.417625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.358446+0000","last_change":"2024-04-26T23:20:22.355373+0000","last_active":"2024-04-26T23:20:22.358446+0000","last_peered":"2024-04-26T23:20:22.358446+0000","last_clean":"2024-04-26T23:20:22.358446+0000","last_became_active":"2024-04-26T23:20:22.355105+0000","last_became_peered":"2024-04-26T23:20:22.355105+0000","last_unstale":"2024-04-26T23:20:22.358446+0000","last_undegraded":"2024-04-26T23:20:22.358446+0000","last_fullsized":"2024-04-26T23:20:22.358446+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:49:27.891875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.336916+0000","last_change":"2024-04-26T23:20:21.338331+0000","last_active":"2024-04-26T23:20:22.336916+0000","last_peered":"2024-04-26T23:20:22.336916+0000","last_clean":"2024-04-26T23:20:22.336916+0000","last_became_active":"2024-04-26T23:20:21.338011+0000","last_became_peered":"2024-04-26T23:20:21.338011+0000","last_unstale":"2024-04-26T23:20:22.336916+0000","last_undegraded":"2024-04-26T23:20:22.336916+0000","last_fullsized":"2024-04-26T23:20:22.336916+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:18:51.810804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.356856+0000","last_change":"2024-04-26T23:20:22.355451+0000","last_active":"2024-04-26T23:20:22.356856+0000","last_peered":"2024-04-26T23:20:22.356856+0000","last_clean":"2024-04-26T23:20:22.356856+0000","last_became_active":"2024-04-26T23:20:22.354928+0000","last_became_peered":"2024-04-26T23:20:22.354928+0000","last_unstale":"2024-04-26T23:20:22.356856+0000","last_undegraded":"2024-04-26T23:20:22.356856+0000","last_fullsized":"2024-04-26T23:20:22.356856+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:55:06.694814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.344946+0000","last_change":"2024-04-26T23:20:21.353531+0000","last_active":"2024-04-26T23:20:22.344946+0000","last_peered":"2024-04-26T23:20:22.344946+0000","last_clean":"2024-04-26T23:20:22.344946+0000","last_became_active":"2024-04-26T23:20:21.353143+0000","last_became_peered":"2024-04-26T23:20:21.353143+0000","last_unstale":"2024-04-26T23:20:22.344946+0000","last_undegraded":"2024-04-26T23:20:22.344946+0000","last_fullsized":"2024-04-26T23:20:22.344946+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:35:13.920013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.361174+0000","last_change":"2024-04-26T23:20:22.358939+0000","last_active":"2024-04-26T23:20:22.361174+0000","last_peered":"2024-04-26T23:20:22.361174+0000","last_clean":"2024-04-26T23:20:22.361174+0000","last_became_active":"2024-04-26T23:20:22.357875+0000","last_became_peered":"2024-04-26T23:20:22.357875+0000","last_unstale":"2024-04-26T23:20:22.361174+0000","last_undegraded":"2024-04-26T23:20:22.361174+0000","last_fullsized":"2024-04-26T23:20:22.361174+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:21.326683+0000","last_clean_scrub_stamp":"2024-04-26T23:20:21.326683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:42:39.240263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T23:20:23.352514+0000","last_change":"2024-04-26T23:20:21.347743+0000","last_active":"2024-04-26T23:20:23.352514+0000","last_peered":"2024-04-26T23:20:23.352514+0000","last_clean":"2024-04-26T23:20:23.352514+0000","last_became_active":"2024-04-26T23:20:21.347475+0000","last_became_peered":"2024-04-26T23:20:21.347475+0000","last_unstale":"2024-04-26T23:20:23.352514+0000","last_undegraded":"2024-04-26T23:20:23.352514+0000","last_fullsized":"2024-04-26T23:20:23.352514+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:20:20.324186+0000","last_clean_scrub_stamp":"2024-04-26T23:20:20.324186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:35:33.072603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":230,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-26T23:20:22.339969+0000","last_change":"2024-04-26T23:18:18.862631+0000","last_active":"2024-04-26T23:20:22.339969+0000","last_peered":"2024-04-26T23:20:22.339969+0000","last_clean":"2024-04-26T23:20:22.339969+0000","last_became_active":"2024-04-26T23:18:18.861536+0000","last_became_peered":"2024-04-26T23:18:18.861536+0000","last_unstale":"2024-04-26T23:20:22.339969+0000","last_undegraded":"2024-04-26T23:20:22.339969+0000","last_fullsized":"2024-04-26T23:20:22.339969+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:17.808736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:17.808736+0000","last_clean_scrub_stamp":"2024-04-26T23:18:17.808736+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:59:26.359282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542172,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28228,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343612,"statfs":{"total":96636764160,"available":96607858688,"internally_reserved":0,"allocated":1232896,"data_stored":874501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542174,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":630784,"data_stored":283973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542173,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27652,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344188,"statfs":{"total":96636764160,"available":96608448512,"internally_reserved":0,"allocated":638976,"data_stored":284625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542173,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":643072,"data_stored":285585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31788,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340052,"statfs":{"total":96636764160,"available":96604213248,"internally_reserved":0,"allocated":618496,"data_stored":276806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31780,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340060,"statfs":{"total":96636764160,"available":96604221440,"internally_reserved":0,"allocated":618496,"data_stored":276806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31800,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340040,"statfs":{"total":96636764160,"available":96604200960,"internally_reserved":0,"allocated":634880,"data_stored":284063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32364,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339476,"statfs":{"total":96636764160,"available":96603623424,"internally_reserved":0,"allocated":1212416,"data_stored":867174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:20:26.728 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T23:20:26.728 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-26T23:20:26.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.872+0000 7f4924c09640 1 Processor -- start 2024-04-26T23:20:26.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.872+0000 7f4924c09640 1 -- start start 2024-04-26T23:20:26.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.872+0000 7f4924c09640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f49200d1a00 0x7f49200d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:26.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.872+0000 7f4924c09640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49200d2320 0x7f49201625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:26.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.872+0000 7f4924c09640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f49200b40d0 0x7f4920162b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:26.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.872+0000 7f4924c09640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f4920163070 con 0x7f49200b40d0 2024-04-26T23:20:26.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.872+0000 7f4924c09640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f49200d0a10 con 0x7f49200d1a00 2024-04-26T23:20:26.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.872+0000 7f4924c09640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f49200d0cf0 con 0x7f49200d2320 2024-04-26T23:20:26.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.873+0000 7f491ed9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f49200b40d0 0x7f4920162b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:26.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.873+0000 7f491ed9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f49200b40d0 0x7f4920162b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40818/0 (socket says 172.21.15.47:40818) 2024-04-26T23:20:26.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.873+0000 7f491ed9d640 1 -- 172.21.15.47:0/4123396186 learned_addr learned my addr 172.21.15.47:0/4123396186 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:26.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.873+0000 7f491e59c640 1 --2- 172.21.15.47:0/4123396186 >> v2:172.21.15.203:3300/0 conn(0x7f49200d1a00 0x7f49200d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:26.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.873+0000 7f4917fff640 1 --2- 172.21.15.47:0/4123396186 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49200d2320 0x7f49201625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:26.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.873+0000 7f491ed9d640 1 -- 172.21.15.47:0/4123396186 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49200d2320 msgr2=0x7f49201625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:26.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.873+0000 7f491ed9d640 1 --2- 172.21.15.47:0/4123396186 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49200d2320 0x7f49201625f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:26.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.873+0000 7f491ed9d640 1 -- 172.21.15.47:0/4123396186 >> v2:172.21.15.203:3300/0 conn(0x7f49200d1a00 msgr2=0x7f49200d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:26.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.873+0000 7f491ed9d640 1 --2- 172.21.15.47:0/4123396186 >> v2:172.21.15.203:3300/0 conn(0x7f49200d1a00 0x7f49200d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:26.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.873+0000 7f491ed9d640 1 -- 172.21.15.47:0/4123396186 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49200d0fd0 con 0x7f49200b40d0 2024-04-26T23:20:26.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.874+0000 7f491ed9d640 1 --2- 172.21.15.47:0/4123396186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49200b40d0 0x7f4920162b30 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f492006c650 tx=0x7f491802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a99066dbe8630ab7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:26.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.874+0000 7f491dd9b640 1 -- 172.21.15.47:0/4123396186 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4918036070 con 0x7f49200b40d0 2024-04-26T23:20:26.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.874+0000 7f491dd9b640 1 -- 172.21.15.47:0/4123396186 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4918031600 con 0x7f49200b40d0 2024-04-26T23:20:26.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.874+0000 7f491dd9b640 1 -- 172.21.15.47:0/4123396186 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49180319d0 con 0x7f49200b40d0 2024-04-26T23:20:26.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.874+0000 7f4924c09640 1 -- 172.21.15.47:0/4123396186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49200b40d0 msgr2=0x7f4920162b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:26.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.874+0000 7f4924c09640 1 --2- 172.21.15.47:0/4123396186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49200b40d0 0x7f4920162b30 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f492006c650 tx=0x7f491802a610 comp rx=0 tx=0).stop 2024-04-26T23:20:26.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.874+0000 7f4924c09640 1 -- 172.21.15.47:0/4123396186 shutdown_connections 2024-04-26T23:20:26.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.874+0000 7f4924c09640 1 --2- 172.21.15.47:0/4123396186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49200b40d0 0x7f4920162b30 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:26.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.874+0000 7f4924c09640 1 --2- 172.21.15.47:0/4123396186 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49200d2320 0x7f49201625f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:26.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.874+0000 7f4924c09640 1 --2- 172.21.15.47:0/4123396186 >> v2:172.21.15.203:3300/0 conn(0x7f49200d1a00 0x7f49200d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:26.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.874+0000 7f4924c09640 1 -- 172.21.15.47:0/4123396186 >> 172.21.15.47:0/4123396186 conn(0x7f49200d4fd0 msgr2=0x7f49200d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:26.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.874+0000 7f4924c09640 1 -- 172.21.15.47:0/4123396186 shutdown_connections 2024-04-26T23:20:26.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.874+0000 7f4924c09640 1 -- 172.21.15.47:0/4123396186 wait complete. 2024-04-26T23:20:26.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.876+0000 7f4924c09640 1 Processor -- start 2024-04-26T23:20:26.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.876+0000 7f4924c09640 1 -- start start 2024-04-26T23:20:26.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.876+0000 7f4924c09640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49200b40d0 0x7f49200cec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:26.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.876+0000 7f4924c09640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49200d1a00 0x7f49201f4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:26.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.876+0000 7f4924c09640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49200d2320 0x7f4920155230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:26.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.876+0000 7f4924c09640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f49200d3050 con 0x7f49200d2320 2024-04-26T23:20:26.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.876+0000 7f4924c09640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f49200d2ed0 con 0x7f49200d1a00 2024-04-26T23:20:26.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.876+0000 7f4924c09640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f49200d31d0 con 0x7f49200b40d0 2024-04-26T23:20:26.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.877+0000 7f491ed9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49200d2320 0x7f4920155230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:26.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.877+0000 7f491ed9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49200d2320 0x7f4920155230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40824/0 (socket says 172.21.15.47:40824) 2024-04-26T23:20:26.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.877+0000 7f491ed9d640 1 -- 172.21.15.47:0/2944337310 learned_addr learned my addr 172.21.15.47:0/2944337310 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:26.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.877+0000 7f491e59c640 1 --2- 172.21.15.47:0/2944337310 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49200b40d0 0x7f49200cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:26.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.877+0000 7f491ed9d640 1 -- 172.21.15.47:0/2944337310 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49200b40d0 msgr2=0x7f49200cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:26.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.877+0000 7f4917fff640 1 --2- 172.21.15.47:0/2944337310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49200d1a00 0x7f49201f4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:26.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.877+0000 7f491ed9d640 1 --2- 172.21.15.47:0/2944337310 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49200b40d0 0x7f49200cec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:26.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.877+0000 7f491ed9d640 1 -- 172.21.15.47:0/2944337310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49200d1a00 msgr2=0x7f49201f4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:26.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.877+0000 7f491ed9d640 1 --2- 172.21.15.47:0/2944337310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49200d1a00 0x7f49201f4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:26.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.877+0000 7f491ed9d640 1 -- 172.21.15.47:0/2944337310 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49200bad80 con 0x7f49200d2320 2024-04-26T23:20:26.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.877+0000 7f491ed9d640 1 --2- 172.21.15.47:0/2944337310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49200d2320 0x7f4920155230 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f4918009f40 tx=0x7f4918004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:26.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.877+0000 7f4916ffd640 1 -- 172.21.15.47:0/2944337310 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4918036050 con 0x7f49200d2320 2024-04-26T23:20:26.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.877+0000 7f4916ffd640 1 -- 172.21.15.47:0/2944337310 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4918042070 con 0x7f49200d2320 2024-04-26T23:20:26.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.877+0000 7f4924c09640 1 -- 172.21.15.47:0/2944337310 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49200baaa0 con 0x7f49200d2320 2024-04-26T23:20:26.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.877+0000 7f4916ffd640 1 -- 172.21.15.47:0/2944337310 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4918009510 con 0x7f49200d2320 2024-04-26T23:20:26.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.877+0000 7f4924c09640 1 -- 172.21.15.47:0/2944337310 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49200ce490 con 0x7f49200d2320 2024-04-26T23:20:26.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.878+0000 7f4924c09640 1 -- 172.21.15.47:0/2944337310 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49200694f0 con 0x7f49200d2320 2024-04-26T23:20:26.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.879+0000 7f4916ffd640 1 -- 172.21.15.47:0/2944337310 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4918007bf0 con 0x7f49200d2320 2024-04-26T23:20:26.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.879+0000 7f4916ffd640 1 --2- 172.21.15.47:0/2944337310 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f490006c8f0 0x7f490006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:26.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.879+0000 7f4916ffd640 1 -- 172.21.15.47:0/2944337310 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f49180aca90 con 0x7f49200d2320 2024-04-26T23:20:26.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.880+0000 7f491e59c640 1 --2- 172.21.15.47:0/2944337310 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f490006c8f0 0x7f490006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:26.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.880+0000 7f491e59c640 1 --2- 172.21.15.47:0/2944337310 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f490006c8f0 0x7f490006edb0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4910009990 tx=0x7f4910008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:26.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:26.884+0000 7f4916ffd640 1 -- 172.21.15.47:0/2944337310 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4918045360 con 0x7f49200d2320 2024-04-26T23:20:27.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.138+0000 7f4924c09640 1 -- 172.21.15.47:0/2944337310 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f49201626a0 con 0x7f49200d2320 2024-04-26T23:20:27.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.140+0000 7f4916ffd640 1 -- 172.21.15.47:0/2944337310 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) ==== 190+0+147 (secure 0 0 0) 0x7f491807ad30 con 0x7f49200d2320 2024-04-26T23:20:27.141 INFO:teuthology.orchestra.run.smithi047.stdout:[client.0] 2024-04-26T23:20:27.141 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQAKNixmA5QFHRAAT89WPyzDs0acPluyvU4svw== 2024-04-26T23:20:27.142 INFO:teuthology.orchestra.run.smithi047.stdout: caps mds = "allow" 2024-04-26T23:20:27.142 INFO:teuthology.orchestra.run.smithi047.stdout: caps mon = "allow r" 2024-04-26T23:20:27.142 INFO:teuthology.orchestra.run.smithi047.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-26T23:20:27.142 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.0 2024-04-26T23:20:27.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.142+0000 7f4924c09640 1 -- 172.21.15.47:0/2944337310 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f490006c8f0 msgr2=0x7f490006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:27.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.142+0000 7f4924c09640 1 --2- 172.21.15.47:0/2944337310 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f490006c8f0 0x7f490006edb0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4910009990 tx=0x7f4910008040 comp rx=0 tx=0).stop 2024-04-26T23:20:27.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.142+0000 7f4924c09640 1 -- 172.21.15.47:0/2944337310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49200d2320 msgr2=0x7f4920155230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:27.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.142+0000 7f4924c09640 1 --2- 172.21.15.47:0/2944337310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49200d2320 0x7f4920155230 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f4918009f40 tx=0x7f4918004a40 comp rx=0 tx=0).stop 2024-04-26T23:20:27.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.142+0000 7f4924c09640 1 -- 172.21.15.47:0/2944337310 shutdown_connections 2024-04-26T23:20:27.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.142+0000 7f4924c09640 1 --2- 172.21.15.47:0/2944337310 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f490006c8f0 0x7f490006edb0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:27.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.142+0000 7f4924c09640 1 --2- 172.21.15.47:0/2944337310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49200d2320 0x7f4920155230 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:27.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.142+0000 7f4924c09640 1 --2- 172.21.15.47:0/2944337310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49200d1a00 0x7f49201f4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:27.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.142+0000 7f4924c09640 1 --2- 172.21.15.47:0/2944337310 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49200b40d0 0x7f49200cec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:27.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.142+0000 7f4924c09640 1 -- 172.21.15.47:0/2944337310 >> 172.21.15.47:0/2944337310 conn(0x7f49200d4fd0 msgr2=0x7f492006d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:27.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.142+0000 7f4924c09640 1 -- 172.21.15.47:0/2944337310 shutdown_connections 2024-04-26T23:20:27.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.142+0000 7f4924c09640 1 -- 172.21.15.47:0/2944337310 wait complete. 2024-04-26T23:20:27.154 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:20:27.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.254+0000 7f085a0cc640 1 Processor -- start 2024-04-26T23:20:27.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.254+0000 7f085a0cc640 1 -- start start 2024-04-26T23:20:27.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.255+0000 7f085a0cc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08540b5f20 0x7f08540d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:27.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.255+0000 7f085a0cc640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08540b7e50 con 0x7f0854162880 2024-04-26T23:20:27.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.255+0000 7f085a0cc640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08540d1800 con 0x7f08540d3bc0 2024-04-26T23:20:27.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.255+0000 7f085a0cc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f08540d0580 con 0x7f08540b5f20 2024-04-26T23:20:27.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.255+0000 7f08537fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0854162880 0x7f08540d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35802/0 (socket says 172.21.15.47:35802) 2024-04-26T23:20:27.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.255+0000 7f08537fe640 1 -- 172.21.15.47:0/3187954631 learned_addr learned my addr 172.21.15.47:0/3187954631 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:27.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.255+0000 7f0853fff640 1 --2- 172.21.15.47:0/3187954631 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08540b5f20 0x7f08540d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:27.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.255+0000 7f08527fc640 1 -- 172.21.15.47:0/3187954631 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3691342590 0 0) 0x7f08540b7e50 con 0x7f0854162880 2024-04-26T23:20:27.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.255+0000 7f08527fc640 1 -- 172.21.15.47:0/3187954631 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f083c003520 con 0x7f0854162880 2024-04-26T23:20:27.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.255+0000 7f08527fc640 1 -- 172.21.15.47:0/3187954631 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3066493068 0 0) 0x7f0848001110 con 0x7f08540d3bc0 2024-04-26T23:20:27.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.255+0000 7f08527fc640 1 -- 172.21.15.47:0/3187954631 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f08540b7e50 con 0x7f08540d3bc0 2024-04-26T23:20:27.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.255+0000 7f0853fff640 1 -- 172.21.15.47:0/3187954631 >> v1:172.21.15.203:6789/0 conn(0x7f08540d3bc0 legacy=0x7f08540b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:27.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.255+0000 7f0853fff640 1 -- 172.21.15.47:0/3187954631 >> v1:172.21.15.47:6789/0 conn(0x7f0854162880 legacy=0x7f08540d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:27.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.255+0000 7f0853fff640 1 -- 172.21.15.47:0/3187954631 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08540d07d0 con 0x7f08540b5f20 2024-04-26T23:20:27.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.256+0000 7f0853fff640 1 --2- 172.21.15.47:0/3187954631 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08540b5f20 0x7f08540d2cb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f08540d0700 tx=0x7f084c02a6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=538e0ccbd79d7346 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:27.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.256+0000 7f08527fc640 1 -- 172.21.15.47:0/3187954631 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c007740 con 0x7f08540b5f20 2024-04-26T23:20:27.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.256+0000 7f08527fc640 1 -- 172.21.15.47:0/3187954631 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f084c0078e0 con 0x7f08540b5f20 2024-04-26T23:20:27.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.256+0000 7f08527fc640 1 -- 172.21.15.47:0/3187954631 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c007c90 con 0x7f08540b5f20 2024-04-26T23:20:27.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.257+0000 7f085a0cc640 1 -- 172.21.15.47:0/3187954631 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08540b5f20 msgr2=0x7f08540d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:27.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.257+0000 7f085a0cc640 1 --2- 172.21.15.47:0/3187954631 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08540b5f20 0x7f08540d2cb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f08540d0700 tx=0x7f084c02a6c0 comp rx=0 tx=0).stop 2024-04-26T23:20:27.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.257+0000 7f085a0cc640 1 -- 172.21.15.47:0/3187954631 shutdown_connections 2024-04-26T23:20:27.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.257+0000 7f085a0cc640 1 --2- 172.21.15.47:0/3187954631 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08540b5f20 0x7f08540d2cb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:27.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.257+0000 7f085a0cc640 1 -- 172.21.15.47:0/3187954631 >> 172.21.15.47:0/3187954631 conn(0x7f08540d4f60 msgr2=0x7f08540d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:27.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.257+0000 7f085a0cc640 1 -- 172.21.15.47:0/3187954631 shutdown_connections 2024-04-26T23:20:27.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.257+0000 7f085a0cc640 1 -- 172.21.15.47:0/3187954631 wait complete. 2024-04-26T23:20:27.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.258+0000 7f085a0cc640 1 Processor -- start 2024-04-26T23:20:27.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.258+0000 7f085a0cc640 1 -- start start 2024-04-26T23:20:27.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.259+0000 7f085a0cc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08540b5f20 0x7f08541f4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:27.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.259+0000 7f085a0cc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08540d3bc0 0x7f08541f7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:27.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.259+0000 7f085a0cc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0854162880 0x7f085415a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:27.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.259+0000 7f085a0cc640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0854162c60 con 0x7f08540b5f20 2024-04-26T23:20:27.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.259+0000 7f085a0cc640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f08540d0d90 con 0x7f0854162880 2024-04-26T23:20:27.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.259+0000 7f085a0cc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f0854162de0 con 0x7f08540d3bc0 2024-04-26T23:20:27.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.259+0000 7f08537fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08540b5f20 0x7f08541f4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:27.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.259+0000 7f08537fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08540b5f20 0x7f08541f4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40826/0 (socket says 172.21.15.47:40826) 2024-04-26T23:20:27.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.259+0000 7f08537fe640 1 -- 172.21.15.47:0/21943227 learned_addr learned my addr 172.21.15.47:0/21943227 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:27.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.259+0000 7f0853fff640 1 --2- 172.21.15.47:0/21943227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0854162880 0x7f085415a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:27.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.259+0000 7f0852ffd640 1 --2- 172.21.15.47:0/21943227 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08540d3bc0 0x7f08541f7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:27.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.260+0000 7f08537fe640 1 -- 172.21.15.47:0/21943227 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08540d3bc0 msgr2=0x7f08541f7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:27.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.260+0000 7f08537fe640 1 --2- 172.21.15.47:0/21943227 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08540d3bc0 0x7f08541f7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:27.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.260+0000 7f08537fe640 1 -- 172.21.15.47:0/21943227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0854162880 msgr2=0x7f085415a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:27.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.260+0000 7f08537fe640 1 --2- 172.21.15.47:0/21943227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0854162880 0x7f085415a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:27.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.260+0000 7f08537fe640 1 -- 172.21.15.47:0/21943227 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08541f8310 con 0x7f08540b5f20 2024-04-26T23:20:27.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.260+0000 7f0853fff640 1 --2- 172.21.15.47:0/21943227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0854162880 0x7f085415a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:20:27.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.260+0000 7f0852ffd640 1 --2- 172.21.15.47:0/21943227 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08540d3bc0 0x7f08541f7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:20:27.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.260+0000 7f08537fe640 1 --2- 172.21.15.47:0/21943227 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08540b5f20 0x7f08541f4f40 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f084400c460 tx=0x7f084400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:27.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.260+0000 7f083bfff640 1 -- 172.21.15.47:0/21943227 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0844017070 con 0x7f08540b5f20 2024-04-26T23:20:27.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.260+0000 7f083bfff640 1 -- 172.21.15.47:0/21943227 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0844005ce0 con 0x7f08540b5f20 2024-04-26T23:20:27.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.260+0000 7f085a0cc640 1 -- 172.21.15.47:0/21943227 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f085415b000 con 0x7f08540b5f20 2024-04-26T23:20:27.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.260+0000 7f083bfff640 1 -- 172.21.15.47:0/21943227 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0844007290 con 0x7f08540b5f20 2024-04-26T23:20:27.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.260+0000 7f085a0cc640 1 -- 172.21.15.47:0/21943227 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08540cd450 con 0x7f08540b5f20 2024-04-26T23:20:27.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.261+0000 7f085a0cc640 1 -- 172.21.15.47:0/21943227 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08540694f0 con 0x7f08540b5f20 2024-04-26T23:20:27.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.261+0000 7f083bfff640 1 -- 172.21.15.47:0/21943227 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f084401a070 con 0x7f08540b5f20 2024-04-26T23:20:27.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.262+0000 7f083bfff640 1 --2- 172.21.15.47:0/21943227 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f084006c8f0 0x7f084006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:27.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.262+0000 7f083bfff640 1 -- 172.21.15.47:0/21943227 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f084408b0d0 con 0x7f08540b5f20 2024-04-26T23:20:27.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.262+0000 7f0852ffd640 1 --2- 172.21.15.47:0/21943227 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f084006c8f0 0x7f084006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:27.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.263+0000 7f0852ffd640 1 --2- 172.21.15.47:0/21943227 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f084006c8f0 0x7f084006edb0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f085415d220 tx=0x7f0848009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:27.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.266+0000 7f083bfff640 1 -- 172.21.15.47:0/21943227 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f08440592f0 con 0x7f08540b5f20 2024-04-26T23:20:27.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.490+0000 7f085a0cc640 1 -- 172.21.15.47:0/21943227 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f085406ea20 con 0x7f08540b5f20 2024-04-26T23:20:27.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.490+0000 7f083bfff640 1 -- 172.21.15.47:0/21943227 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5190 (secure 0 0 0) 0x7f0844058bf0 con 0x7f08540b5f20 2024-04-26T23:20:27.491 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:20:27.492 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5865,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/805147373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":805147373},{"type":"v1","addr":"172.21.15.47:6833","nonce":805147373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5868,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3515322868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3515322868},{"type":"v1","addr":"172.21.15.47:6835","nonce":3515322868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":15008,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/4223205269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":4223205269},{"type":"v1","addr":"172.21.15.203:6837","nonce":4223205269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:20:21.712604+0000","modified":"2024-04-26T23:20:24.725165+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":15011},"failed":[],"damaged":[],"stopped":[],"info":{"gid_15011":{"gid":15011,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6836/1168022146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1168022146},{"type":"v1","addr":"172.21.15.203:6836","nonce":1168022146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":15011,"qdb_cluster":[15011]},"id":3}]} 2024-04-26T23:20:27.492 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 27 2024-04-26T23:20:27.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.492+0000 7f085a0cc640 1 -- 172.21.15.47:0/21943227 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f084006c8f0 msgr2=0x7f084006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:27.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.492+0000 7f085a0cc640 1 --2- 172.21.15.47:0/21943227 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f084006c8f0 0x7f084006edb0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f085415d220 tx=0x7f0848009040 comp rx=0 tx=0).stop 2024-04-26T23:20:27.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.492+0000 7f085a0cc640 1 -- 172.21.15.47:0/21943227 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08540b5f20 msgr2=0x7f08541f4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:27.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.492+0000 7f085a0cc640 1 --2- 172.21.15.47:0/21943227 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08540b5f20 0x7f08541f4f40 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f084400c460 tx=0x7f084400c920 comp rx=0 tx=0).stop 2024-04-26T23:20:27.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.492+0000 7f085a0cc640 1 -- 172.21.15.47:0/21943227 shutdown_connections 2024-04-26T23:20:27.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.492+0000 7f085a0cc640 1 --2- 172.21.15.47:0/21943227 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f084006c8f0 0x7f084006edb0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:27.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.492+0000 7f085a0cc640 1 --2- 172.21.15.47:0/21943227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0854162880 0x7f085415a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:27.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.492+0000 7f085a0cc640 1 --2- 172.21.15.47:0/21943227 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08540d3bc0 0x7f08541f7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:27.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.492+0000 7f085a0cc640 1 --2- 172.21.15.47:0/21943227 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08540b5f20 0x7f08541f4f40 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:27.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.492+0000 7f085a0cc640 1 -- 172.21.15.47:0/21943227 >> 172.21.15.47:0/21943227 conn(0x7f08540d4f60 msgr2=0x7f08540d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:27.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.492+0000 7f085a0cc640 1 -- 172.21.15.47:0/21943227 shutdown_connections 2024-04-26T23:20:27.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:27.492+0000 7f085a0cc640 1 -- 172.21.15.47:0/21943227 wait complete. 2024-04-26T23:20:27.907 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T23:20:21.712604+0000', 'modified': '2024-04-26T23:20:24.725165+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 15011}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_15011': {'gid': 15011, 'name': 'b', 'rank': 0, 'incarnation': 23, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.203:6836/1168022146', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6834', 'nonce': 1168022146}, {'type': 'v1', 'addr': '172.21.15.203:6836', 'nonce': 1168022146}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 15011, 'qdb_cluster': [15011]} 2024-04-26T23:20:27.907 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T23:20:27.907 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-26T23:20:28.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.011+0000 7fcc0fe52640 1 Processor -- start 2024-04-26T23:20:28.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.012+0000 7fcc0fe52640 1 -- start start 2024-04-26T23:20:28.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.013+0000 7fcc0fe52640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc0815d1e0 0x7fcc080d09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:28.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.013+0000 7fcc0fe52640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcc080d0f20 0x7fcc080d1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:28.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.013+0000 7fcc0fe52640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc080ce840 con 0x7fcc080d18c0 2024-04-26T23:20:28.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.013+0000 7fcc0fe52640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fcc080cede0 con 0x7fcc080d0f20 2024-04-26T23:20:28.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.013+0000 7fcc0fe52640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fcc08070650 con 0x7fcc0815d1e0 2024-04-26T23:20:28.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.013+0000 7fcc0d3ed640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcc080d0f20 0x7fcc080d1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:28.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.013+0000 7fcc0d3ed640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcc080d0f20 0x7fcc080d1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40834/0 (socket says 172.21.15.47:40834) 2024-04-26T23:20:28.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.013+0000 7fcc0d3ed640 1 -- 172.21.15.47:0/1163655417 learned_addr learned my addr 172.21.15.47:0/1163655417 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:28.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.013+0000 7fcc0dbee640 1 --2- 172.21.15.47:0/1163655417 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc0815d1e0 0x7fcc080d09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:28.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.013+0000 7fcc0d3ed640 1 -- 172.21.15.47:0/1163655417 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc0815d1e0 msgr2=0x7fcc080d09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:28.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.013+0000 7fcc0d3ed640 1 --2- 172.21.15.47:0/1163655417 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc0815d1e0 0x7fcc080d09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:28.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.013+0000 7fcc0d3ed640 1 -- 172.21.15.47:0/1163655417 >> v1:172.21.15.203:6789/0 conn(0x7fcc080d18c0 legacy=0x7fcc080cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:28.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.013+0000 7fcc0d3ed640 1 -- 172.21.15.47:0/1163655417 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc080ceb00 con 0x7fcc080d0f20 2024-04-26T23:20:28.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.013+0000 7fcc0dbee640 1 --2- 172.21.15.47:0/1163655417 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc0815d1e0 0x7fcc080d09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:28.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.014+0000 7fcc0d3ed640 1 --2- 172.21.15.47:0/1163655417 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc080d0f20 0x7fcc080d1380 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8009870 tx=0x7fcbf802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a64aadec489ee99 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:28.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.014+0000 7fcc0cbec640 1 -- 172.21.15.47:0/1163655417 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8037070 con 0x7fcc080d0f20 2024-04-26T23:20:28.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.014+0000 7fcc0cbec640 1 -- 172.21.15.47:0/1163655417 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbf8032440 con 0x7fcc080d0f20 2024-04-26T23:20:28.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.014+0000 7fcc0cbec640 1 -- 172.21.15.47:0/1163655417 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8032810 con 0x7fcc080d0f20 2024-04-26T23:20:28.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.014+0000 7fcc0fe52640 1 -- 172.21.15.47:0/1163655417 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc080d0f20 msgr2=0x7fcc080d1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:28.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.014+0000 7fcc0fe52640 1 --2- 172.21.15.47:0/1163655417 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc080d0f20 0x7fcc080d1380 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8009870 tx=0x7fcbf802a210 comp rx=0 tx=0).stop 2024-04-26T23:20:28.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.015+0000 7fcc0fe52640 1 -- 172.21.15.47:0/1163655417 shutdown_connections 2024-04-26T23:20:28.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.015+0000 7fcc0fe52640 1 --2- 172.21.15.47:0/1163655417 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc080d0f20 0x7fcc080d1380 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:28.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.015+0000 7fcc0fe52640 1 --2- 172.21.15.47:0/1163655417 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc0815d1e0 0x7fcc080d09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:28.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.015+0000 7fcc0fe52640 1 -- 172.21.15.47:0/1163655417 >> 172.21.15.47:0/1163655417 conn(0x7fcc080d2da0 msgr2=0x7fcc080d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:28.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.015+0000 7fcc0fe52640 1 -- 172.21.15.47:0/1163655417 shutdown_connections 2024-04-26T23:20:28.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.015+0000 7fcc0fe52640 1 -- 172.21.15.47:0/1163655417 wait complete. 2024-04-26T23:20:28.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.016+0000 7fcc0fe52640 1 Processor -- start 2024-04-26T23:20:28.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.016+0000 7fcc0fe52640 1 -- start start 2024-04-26T23:20:28.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.017+0000 7fcc0fe52640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc080d0f20 0x7fcc08153220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:28.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.017+0000 7fcc0fe52640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc080d18c0 0x7fcc080b5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:28.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.017+0000 7fcc0fe52640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc0815d1e0 0x7fcc080b1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:28.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.017+0000 7fcc0fe52640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fcc08071af0 con 0x7fcc080d0f20 2024-04-26T23:20:28.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.017+0000 7fcc0fe52640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcc080ce4c0 con 0x7fcc080d18c0 2024-04-26T23:20:28.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.017+0000 7fcc0fe52640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fcc08071c70 con 0x7fcc0815d1e0 2024-04-26T23:20:28.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.017+0000 7fcc0dbee640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc080d0f20 0x7fcc08153220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:28.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.017+0000 7fcc0dbee640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc080d0f20 0x7fcc08153220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40846/0 (socket says 172.21.15.47:40846) 2024-04-26T23:20:28.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.017+0000 7fcc0dbee640 1 -- 172.21.15.47:0/3763784678 learned_addr learned my addr 172.21.15.47:0/3763784678 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:28.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.017+0000 7fcc0dbee640 1 -- 172.21.15.47:0/3763784678 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc0815d1e0 msgr2=0x7fcc080b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:28.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.017+0000 7fcc0e3ef640 1 --2- 172.21.15.47:0/3763784678 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc0815d1e0 0x7fcc080b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:28.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.017+0000 7fcc0d3ed640 1 --2- 172.21.15.47:0/3763784678 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc080d18c0 0x7fcc080b5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:28.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.017+0000 7fcc0dbee640 1 --2- 172.21.15.47:0/3763784678 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc0815d1e0 0x7fcc080b1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:28.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.017+0000 7fcc0dbee640 1 -- 172.21.15.47:0/3763784678 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc080d18c0 msgr2=0x7fcc080b5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:28.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.017+0000 7fcc0dbee640 1 --2- 172.21.15.47:0/3763784678 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc080d18c0 0x7fcc080b5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:28.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.017+0000 7fcc0dbee640 1 -- 172.21.15.47:0/3763784678 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc08153760 con 0x7fcc080d0f20 2024-04-26T23:20:28.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.017+0000 7fcc0e3ef640 1 --2- 172.21.15.47:0/3763784678 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc0815d1e0 0x7fcc080b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:28.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.017+0000 7fcc0d3ed640 1 --2- 172.21.15.47:0/3763784678 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc080d18c0 0x7fcc080b5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:28.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.017+0000 7fcc0dbee640 1 --2- 172.21.15.47:0/3763784678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc080d0f20 0x7fcc08153220 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fcc0400cf90 tx=0x7fcc0400a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:28.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.017+0000 7fcbfe7fc640 1 -- 172.21.15.47:0/3763784678 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc0401a070 con 0x7fcc080d0f20 2024-04-26T23:20:28.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.017+0000 7fcbfe7fc640 1 -- 172.21.15.47:0/3763784678 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc0400d840 con 0x7fcc080d0f20 2024-04-26T23:20:28.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.017+0000 7fcc0fe52640 1 -- 172.21.15.47:0/3763784678 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc081f28a0 con 0x7fcc080d0f20 2024-04-26T23:20:28.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.017+0000 7fcbfe7fc640 1 -- 172.21.15.47:0/3763784678 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc0400dc10 con 0x7fcc080d0f20 2024-04-26T23:20:28.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.017+0000 7fcc0fe52640 1 -- 172.21.15.47:0/3763784678 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc081f2e40 con 0x7fcc080d0f20 2024-04-26T23:20:28.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.019+0000 7fcbfe7fc640 1 -- 172.21.15.47:0/3763784678 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcc0401d070 con 0x7fcc080d0f20 2024-04-26T23:20:28.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.020+0000 7fcbfe7fc640 1 --2- 172.21.15.47:0/3763784678 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fcbec06c8f0 0x7fcbec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:28.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.020+0000 7fcbfe7fc640 1 -- 172.21.15.47:0/3763784678 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fcc0408c4c0 con 0x7fcc080d0f20 2024-04-26T23:20:28.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.020+0000 7fcc0d3ed640 1 --2- 172.21.15.47:0/3763784678 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fcbec06c8f0 0x7fcbec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:28.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.021+0000 7fcc12b6b740 1 Processor -- start 2024-04-26T23:20:28.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.021+0000 7fcc0d3ed640 1 --2- 172.21.15.47:0/3763784678 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fcbec06c8f0 0x7fcbec06edb0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8034000 tx=0x7fcbf802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:28.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.021+0000 7fcc12b6b740 1 -- start start 2024-04-26T23:20:28.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.021+0000 7fcc12b6b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563bcbbaf440 0x563bcbbaf820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:28.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.021+0000 7fcc12b6b740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563bcbbccce0 0x563bcbbcd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:28.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.021+0000 7fcc12b6b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563bcbb85120 0x563bcbbcb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:28.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.021+0000 7fcc12b6b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x563bcbae9740 con 0x563bcbb85120 2024-04-26T23:20:28.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.021+0000 7fcc12b6b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x563bcba853a0 con 0x563bcbbaf440 2024-04-26T23:20:28.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.021+0000 7fcc12b6b740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x563bcbae5700 con 0x563bcbbccce0 2024-04-26T23:20:28.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.021+0000 7fcc0e3ef640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563bcbb85120 0x563bcbbcb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:28.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.021+0000 7fcc0e3ef640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563bcbb85120 0x563bcbbcb3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40854/0 (socket says 172.21.15.47:40854) 2024-04-26T23:20:28.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.021+0000 7fcc0e3ef640 1 -- 172.21.15.47:0/475149448 learned_addr learned my addr 172.21.15.47:0/475149448 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:28.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.021+0000 7fcc0dbee640 1 --2- 172.21.15.47:0/475149448 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563bcbbaf440 0x563bcbbaf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:28.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.022+0000 7fcc0d3ed640 1 --2- 172.21.15.47:0/475149448 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563bcbbccce0 0x563bcbbcd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:28.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.022+0000 7fcc0e3ef640 1 -- 172.21.15.47:0/475149448 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563bcbbccce0 msgr2=0x563bcbbcd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:28.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.022+0000 7fcc0e3ef640 1 --2- 172.21.15.47:0/475149448 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563bcbbccce0 0x563bcbbcd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:28.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.022+0000 7fcc0e3ef640 1 -- 172.21.15.47:0/475149448 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563bcbbaf440 msgr2=0x563bcbbaf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:28.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.022+0000 7fcc0e3ef640 1 --2- 172.21.15.47:0/475149448 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563bcbbaf440 0x563bcbbaf820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:28.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.022+0000 7fcc0e3ef640 1 -- 172.21.15.47:0/475149448 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bcbaa1750 con 0x563bcbb85120 2024-04-26T23:20:28.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.022+0000 7fcc0dbee640 1 --2- 172.21.15.47:0/475149448 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563bcbbaf440 0x563bcbbaf820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:20:28.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.022+0000 7fcc0e3ef640 1 --2- 172.21.15.47:0/475149448 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563bcbb85120 0x563bcbbcb3a0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fcc00003e80 tx=0x7fcc00004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68ba0b59f99809af server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:28.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.022+0000 7fcbeb7fe640 1 -- 172.21.15.47:0/475149448 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc00005270 con 0x563bcbb85120 2024-04-26T23:20:28.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.022+0000 7fcbeb7fe640 1 -- 172.21.15.47:0/475149448 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc00005430 con 0x563bcbb85120 2024-04-26T23:20:28.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.022+0000 7fcbeb7fe640 1 -- 172.21.15.47:0/475149448 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc0000db80 con 0x563bcbb85120 2024-04-26T23:20:28.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.023+0000 7fcc12b6b740 1 -- 172.21.15.47:0/475149448 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563bcbb85120 msgr2=0x563bcbbcb3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:28.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.023+0000 7fcc12b6b740 1 --2- 172.21.15.47:0/475149448 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563bcbb85120 0x563bcbbcb3a0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fcc00003e80 tx=0x7fcc00004340 comp rx=0 tx=0).stop 2024-04-26T23:20:28.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.023+0000 7fcc12b6b740 1 -- 172.21.15.47:0/475149448 shutdown_connections 2024-04-26T23:20:28.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.023+0000 7fcc12b6b740 1 --2- 172.21.15.47:0/475149448 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563bcbb85120 0x563bcbbcb3a0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:28.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.023+0000 7fcc12b6b740 1 --2- 172.21.15.47:0/475149448 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563bcbbccce0 0x563bcbbcd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:28.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.023+0000 7fcc12b6b740 1 --2- 172.21.15.47:0/475149448 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563bcbbaf440 0x563bcbbaf820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:28.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.023+0000 7fcc12b6b740 1 -- 172.21.15.47:0/475149448 >> 172.21.15.47:0/475149448 conn(0x563bcbb81a50 msgr2=0x563bcbbca250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:28.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.023+0000 7fcc12b6b740 1 -- 172.21.15.47:0/475149448 shutdown_connections 2024-04-26T23:20:28.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.023+0000 7fcc12b6b740 1 -- 172.21.15.47:0/475149448 wait complete. 2024-04-26T23:20:28.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.023+0000 7fcc12b6b740 1 -- start start 2024-04-26T23:20:28.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.023+0000 7fcc12b6b740 1 Processor -- start 2024-04-26T23:20:28.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.024+0000 7fcc12b6b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563bcbbaf440 0x563bcbbd1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:28.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.024+0000 7fcc12b6b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563bcbbccce0 0x563bcbbab0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:28.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.024+0000 7fcc12b6b740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563bcbbabd20 0x563bcbbdad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:28.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.024+0000 7fcc12b6b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x563bcbb85ec0 con 0x563bcbbccce0 2024-04-26T23:20:28.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.024+0000 7fcc12b6b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x563bcba70fe0 con 0x563bcbbaf440 2024-04-26T23:20:28.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.024+0000 7fcc12b6b740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x563bcbb7f980 con 0x563bcbbabd20 2024-04-26T23:20:28.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.024+0000 7fcc12b6b740 10 client.0 _ll_register_callbacks cb 0x563bcba87150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:20:28.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.024+0000 7fcc0d3ed640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563bcbbccce0 0x563bcbbab0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:28.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.024+0000 7fcc0d3ed640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563bcbbccce0 0x563bcbbab0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40860/0 (socket says 172.21.15.47:40860) 2024-04-26T23:20:28.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.024+0000 7fcc0d3ed640 1 -- 172.21.15.47:0/886378931 learned_addr learned my addr 172.21.15.47:0/886378931 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:28.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.025+0000 7fcc0dbee640 1 --2- 172.21.15.47:0/886378931 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563bcbbaf440 0x563bcbbd1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:28.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.025+0000 7fcc0e3ef640 1 --2- 172.21.15.47:0/886378931 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563bcbbabd20 0x563bcbbdad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:28.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.025+0000 7fcc0d3ed640 1 -- 172.21.15.47:0/886378931 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563bcbbabd20 msgr2=0x563bcbbdad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:28.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.025+0000 7fcc0d3ed640 1 --2- 172.21.15.47:0/886378931 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563bcbbabd20 0x563bcbbdad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:28.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.025+0000 7fcc0d3ed640 1 -- 172.21.15.47:0/886378931 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563bcbbaf440 msgr2=0x563bcbbd1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:28.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.025+0000 7fcc0d3ed640 1 --2- 172.21.15.47:0/886378931 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563bcbbaf440 0x563bcbbd1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:28.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.025+0000 7fcc0d3ed640 1 -- 172.21.15.47:0/886378931 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bcb7fb2d0 con 0x563bcbbccce0 2024-04-26T23:20:28.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.025+0000 7fcc0d3ed640 1 --2- 172.21.15.47:0/886378931 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563bcbbccce0 0x563bcbbab0d0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fcbf802a800 tx=0x7fcbf802c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7aa6e8acdd7dec4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:28.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.025+0000 7fcbe8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-26T23:20:28.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.025+0000 7fcbe8ff9640 1 -- 172.21.15.47:0/886378931 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8037040 con 0x563bcbbccce0 2024-04-26T23:20:28.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.025+0000 7fcbe8ff9640 1 -- 172.21.15.47:0/886378931 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fcbf803ecc0 con 0x563bcbbccce0 2024-04-26T23:20:28.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.025+0000 7fcc12b6b740 1 -- 172.21.15.47:0/886378931 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc00005430 con 0x563bcbbccce0 2024-04-26T23:20:28.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.025+0000 7fcbe8ff9640 1 -- 172.21.15.47:0/886378931 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcbf8057050 con 0x563bcbbccce0 2024-04-26T23:20:28.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.025+0000 7fcc12b6b740 10 client.6015 fetch_fsmap learned FSMap version 27 2024-04-26T23:20:28.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.025+0000 7fcc12b6b740 1 -- 172.21.15.47:0/886378931 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) -- 0x563bcb8c56b0 con 0x563bcbbccce0 2024-04-26T23:20:28.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.025+0000 7fcbe8ff9640 1 -- 172.21.15.47:0/886378931 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc00005430 con 0x563bcbbccce0 2024-04-26T23:20:28.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.025+0000 7fcbe8ff9640 1 -- 172.21.15.47:0/886378931 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 27) ==== 3494+0+0 (secure 0 0 0) 0x7fcbf805a020 con 0x563bcbbccce0 2024-04-26T23:20:28.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.026+0000 7fcc12b6b740 10 client.6015 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T23:20:28.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.026+0000 7fcc12b6b740 10 client.6015 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T23:20:28.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.026+0000 7fcc12b6b740 20 client.6015 populate_metadata read hostname 'smithi047' 2024-04-26T23:20:28.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.026+0000 7fcc12b6b740 1 --2- 172.21.15.47:0/886378931 >> [v2:172.21.15.203:6834/1168022146,v1:172.21.15.203:6836/1168022146] conn(0x563bcbbcff10 0x563bcbbd02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:28.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.026+0000 7fcc12b6b740 4 client.6015 mds_command: new command op to 15011 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:20:28.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.026+0000 7fcc12b6b740 1 -- 172.21.15.47:0/886378931 --> [v2:172.21.15.203:6834/1168022146,v1:172.21.15.203:6836/1168022146] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563bcbaf7350 con 0x563bcbbcff10 2024-04-26T23:20:28.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.026+0000 7fcc0dbee640 1 --2- 172.21.15.47:0/886378931 >> [v2:172.21.15.203:6834/1168022146,v1:172.21.15.203:6836/1168022146] conn(0x563bcbbcff10 0x563bcbbd02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:28.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.026+0000 7fcc0dbee640 1 --2- 172.21.15.47:0/886378931 >> [v2:172.21.15.203:6834/1168022146,v1:172.21.15.203:6836/1168022146] conn(0x563bcbbcff10 0x563bcbbd02f0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b0545f48254c0bb server_cookie=43bf7fd093c3efdc in_seq=0 out_seq=0 2024-04-26T23:20:28.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.026+0000 7fcbe8ff9640 10 client.6015 ms_handle_connect on v2:172.21.15.203:6834/1168022146 2024-04-26T23:20:28.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.027+0000 7fcbe8ff9640 1 -- 172.21.15.47:0/886378931 <== mds.0 v2:172.21.15.203:6834/1168022146 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563bcbaf7350 con 0x563bcbbcff10 2024-04-26T23:20:28.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.027+0000 7fcbe8ff9640 10 client.6015 handle_command_reply: tid=0 2024-04-26T23:20:28.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.027+0000 7fcc12b6b740 1 client.6015 shutdown 2024-04-26T23:20:28.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.027+0000 7fcc12b6b740 1 -- 172.21.15.47:0/886378931 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563bcbbccce0 msgr2=0x563bcbbab0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:28.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.027+0000 7fcc12b6b740 1 --2- 172.21.15.47:0/886378931 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563bcbbccce0 0x563bcbbab0d0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fcbf802a800 tx=0x7fcbf802c070 comp rx=0 tx=0).stop 2024-04-26T23:20:28.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.027+0000 7fcc12b6b740 1 -- 172.21.15.47:0/886378931 shutdown_connections 2024-04-26T23:20:28.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.027+0000 7fcc12b6b740 1 --2- 172.21.15.47:0/886378931 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563bcbbabd20 0x563bcbbdad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:28.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.027+0000 7fcc12b6b740 1 --2- 172.21.15.47:0/886378931 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563bcbbccce0 0x563bcbbab0d0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:28.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.027+0000 7fcc12b6b740 1 --2- 172.21.15.47:0/886378931 >> [v2:172.21.15.203:6834/1168022146,v1:172.21.15.203:6836/1168022146] conn(0x563bcbbcff10 0x563bcbbd02f0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:28.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.027+0000 7fcc12b6b740 1 --2- 172.21.15.47:0/886378931 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563bcbbaf440 0x563bcbbd1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:28.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.027+0000 7fcc12b6b740 1 -- 172.21.15.47:0/886378931 >> 172.21.15.47:0/886378931 conn(0x563bcbb85120 msgr2=0x563bcbbcd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:28.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.027+0000 7fcbe8ff9640 0 client.6015 ms_handle_reset on v2:172.21.15.203:6834/1168022146 2024-04-26T23:20:28.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.027+0000 7fcc12b6b740 1 -- 172.21.15.47:0/886378931 shutdown_connections 2024-04-26T23:20:28.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.027+0000 7fcc12b6b740 1 -- 172.21.15.47:0/886378931 wait complete. 2024-04-26T23:20:28.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.027+0000 7fcc12b6b740 20 client.6015 trim_cache size 0 max 16384 2024-04-26T23:20:28.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.046+0000 7fcc12b6b740 1 Processor -- start 2024-04-26T23:20:28.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.046+0000 7fcc12b6b740 1 -- start start 2024-04-26T23:20:28.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.046+0000 7fcc12b6b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563bcbb85120 0x563bcbbc1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:28.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.046+0000 7fcc12b6b740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563bcbbabd20 0x563bcbbca880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:28.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.046+0000 7fcc12b6b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563bcbbaf440 0x563bcbb844e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:28.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.046+0000 7fcc12b6b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fcc081fe9b0 con 0x563bcbbaf440 2024-04-26T23:20:28.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.046+0000 7fcc12b6b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcc080b8500 con 0x563bcbb85120 2024-04-26T23:20:28.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.046+0000 7fcc12b6b740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fcc081fe340 con 0x563bcbbabd20 2024-04-26T23:20:28.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.046+0000 7fcc0e3ef640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563bcbbaf440 0x563bcbb844e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:28.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.047+0000 7fcc0e3ef640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563bcbbaf440 0x563bcbb844e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40870/0 (socket says 172.21.15.47:40870) 2024-04-26T23:20:28.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.047+0000 7fcc0e3ef640 1 -- 172.21.15.47:0/652163480 learned_addr learned my addr 172.21.15.47:0/652163480 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:28.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.047+0000 7fcc0d3ed640 1 --2- 172.21.15.47:0/652163480 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563bcbbabd20 0x563bcbbca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:28.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.047+0000 7fcc0dbee640 1 --2- 172.21.15.47:0/652163480 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563bcbb85120 0x563bcbbc1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:28.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.047+0000 7fcc0e3ef640 1 -- 172.21.15.47:0/652163480 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563bcbbabd20 msgr2=0x563bcbbca880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:28.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.047+0000 7fcc0e3ef640 1 --2- 172.21.15.47:0/652163480 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563bcbbabd20 0x563bcbbca880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:28.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.047+0000 7fcc0e3ef640 1 -- 172.21.15.47:0/652163480 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563bcbb85120 msgr2=0x563bcbbc1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:28.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.047+0000 7fcc0e3ef640 1 --2- 172.21.15.47:0/652163480 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563bcbb85120 0x563bcbbc1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:28.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.047+0000 7fcc0e3ef640 1 -- 172.21.15.47:0/652163480 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bcbb7fd20 con 0x563bcbbaf440 2024-04-26T23:20:28.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.047+0000 7fcc0d3ed640 1 --2- 172.21.15.47:0/652163480 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563bcbbabd20 0x563bcbbca880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:28.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.047+0000 7fcc0dbee640 1 --2- 172.21.15.47:0/652163480 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563bcbb85120 0x563bcbbc1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:28.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.047+0000 7fcc0e3ef640 1 --2- 172.21.15.47:0/652163480 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563bcbbaf440 0x563bcbb844e0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fcc00007fd0 tx=0x7fcc00012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b81ef3740f898595 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:28.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.047+0000 7fcbeaffd640 1 -- 172.21.15.47:0/652163480 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc0000db40 con 0x563bcbbaf440 2024-04-26T23:20:28.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.047+0000 7fcbeaffd640 1 -- 172.21.15.47:0/652163480 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc0000dce0 con 0x563bcbbaf440 2024-04-26T23:20:28.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.047+0000 7fcbeaffd640 1 -- 172.21.15.47:0/652163480 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc00007270 con 0x563bcbbaf440 2024-04-26T23:20:28.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.048+0000 7fcc12b6b740 1 -- 172.21.15.47:0/652163480 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563bcbbaf440 msgr2=0x563bcbb844e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:28.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.048+0000 7fcc12b6b740 1 --2- 172.21.15.47:0/652163480 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563bcbbaf440 0x563bcbb844e0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fcc00007fd0 tx=0x7fcc00012000 comp rx=0 tx=0).stop 2024-04-26T23:20:28.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.048+0000 7fcc12b6b740 1 -- 172.21.15.47:0/652163480 shutdown_connections 2024-04-26T23:20:28.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.048+0000 7fcc12b6b740 1 --2- 172.21.15.47:0/652163480 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563bcbbaf440 0x563bcbb844e0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:28.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.048+0000 7fcc12b6b740 1 --2- 172.21.15.47:0/652163480 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563bcbbabd20 0x563bcbbca880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:28.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.048+0000 7fcc12b6b740 1 --2- 172.21.15.47:0/652163480 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563bcbb85120 0x563bcbbc1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:28.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.048+0000 7fcc12b6b740 1 -- 172.21.15.47:0/652163480 >> 172.21.15.47:0/652163480 conn(0x563bcbbcc1b0 msgr2=0x563bcbb85500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:28.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.048+0000 7fcc12b6b740 1 -- 172.21.15.47:0/652163480 shutdown_connections 2024-04-26T23:20:28.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.048+0000 7fcc12b6b740 1 -- 172.21.15.47:0/652163480 wait complete. 2024-04-26T23:20:28.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.048+0000 7fcc12b6b740 1 -- start start 2024-04-26T23:20:28.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.048+0000 7fcc12b6b740 1 Processor -- start 2024-04-26T23:20:28.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.049+0000 7fcc12b6b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563bcbb85120 0x563bcbc32980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:28.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.049+0000 7fcc12b6b740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563bcbbabd20 0x563bcbba92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:28.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.049+0000 7fcc12b6b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563bcbbaf440 0x563bcbb82ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:28.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.049+0000 7fcc12b6b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x563bcbbb1360 con 0x563bcbb85120 2024-04-26T23:20:28.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.049+0000 7fcc12b6b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x563bcbbdb250 con 0x563bcbbaf440 2024-04-26T23:20:28.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.049+0000 7fcc12b6b740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x563bcbbcb2f0 con 0x563bcbbabd20 2024-04-26T23:20:28.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.049+0000 7fcc12b6b740 10 client.0 _ll_register_callbacks cb 0x7fcbf80096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:20:28.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.049+0000 7fcc0dbee640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563bcbb85120 0x563bcbc32980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:28.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.049+0000 7fcc0dbee640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563bcbb85120 0x563bcbc32980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40882/0 (socket says 172.21.15.47:40882) 2024-04-26T23:20:28.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.049+0000 7fcc0dbee640 1 -- 172.21.15.47:0/2250432378 learned_addr learned my addr 172.21.15.47:0/2250432378 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:28.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.049+0000 7fcc0d3ed640 1 --2- 172.21.15.47:0/2250432378 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563bcbbabd20 0x563bcbba92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:28.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.049+0000 7fcc0e3ef640 1 --2- 172.21.15.47:0/2250432378 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563bcbbaf440 0x563bcbb82ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:28.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.049+0000 7fcc0dbee640 1 -- 172.21.15.47:0/2250432378 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563bcbbabd20 msgr2=0x563bcbba92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:28.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.049+0000 7fcc0dbee640 1 --2- 172.21.15.47:0/2250432378 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563bcbbabd20 0x563bcbba92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:28.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.049+0000 7fcc0dbee640 1 -- 172.21.15.47:0/2250432378 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563bcbbaf440 msgr2=0x563bcbb82ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:28.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.050+0000 7fcc0dbee640 1 --2- 172.21.15.47:0/2250432378 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563bcbbaf440 0x563bcbb82ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:28.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.050+0000 7fcc0dbee640 1 -- 172.21.15.47:0/2250432378 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bcbbedb60 con 0x563bcbb85120 2024-04-26T23:20:28.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.050+0000 7fcc0dbee640 1 --2- 172.21.15.47:0/2250432378 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563bcbb85120 0x563bcbc32980 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fcc04059e10 tx=0x7fcc04059f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cec6129620c785f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:28.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.050+0000 7fcbe8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-26T23:20:28.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.050+0000 7fcbe8ff9640 1 -- 172.21.15.47:0/2250432378 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc0405a370 con 0x563bcbb85120 2024-04-26T23:20:28.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.050+0000 7fcbe8ff9640 1 -- 172.21.15.47:0/2250432378 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fcc0405a510 con 0x563bcbb85120 2024-04-26T23:20:28.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.050+0000 7fcc12b6b740 1 -- 172.21.15.47:0/2250432378 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc0000dce0 con 0x563bcbb85120 2024-04-26T23:20:28.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.050+0000 7fcbe8ff9640 1 -- 172.21.15.47:0/2250432378 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcc040627b0 con 0x563bcbb85120 2024-04-26T23:20:28.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.050+0000 7fcc12b6b740 10 client.6021 fetch_fsmap learned FSMap version 27 2024-04-26T23:20:28.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.050+0000 7fcc12b6b740 1 -- 172.21.15.47:0/2250432378 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) -- 0x563bcbbee920 con 0x563bcbb85120 2024-04-26T23:20:28.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.050+0000 7fcbe8ff9640 1 -- 172.21.15.47:0/2250432378 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc0000dce0 con 0x563bcbb85120 2024-04-26T23:20:28.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.050+0000 7fcbe8ff9640 1 -- 172.21.15.47:0/2250432378 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 27) ==== 3494+0+0 (secure 0 0 0) 0x7fcc04059560 con 0x563bcbb85120 2024-04-26T23:20:28.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.050+0000 7fcc12b6b740 10 client.6021 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T23:20:28.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.050+0000 7fcc12b6b740 10 client.6021 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T23:20:28.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.050+0000 7fcc12b6b740 20 client.6021 populate_metadata read hostname 'smithi047' 2024-04-26T23:20:28.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.050+0000 7fcc12b6b740 1 --2- 172.21.15.47:0/2250432378 >> [v2:172.21.15.203:6834/1168022146,v1:172.21.15.203:6836/1168022146] conn(0x563bcbbccce0 0x563bcbbb14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:28.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.050+0000 7fcc12b6b740 4 client.6021 mds_command: new command op to 15011 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T23:20:28.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.050+0000 7fcc12b6b740 1 -- 172.21.15.47:0/2250432378 --> [v2:172.21.15.203:6834/1168022146,v1:172.21.15.203:6836/1168022146] -- command(tid 0: {"prefix": "status"}) -- 0x563bcbaf11c0 con 0x563bcbbccce0 2024-04-26T23:20:28.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.051+0000 7fcc0d3ed640 1 --2- 172.21.15.47:0/2250432378 >> [v2:172.21.15.203:6834/1168022146,v1:172.21.15.203:6836/1168022146] conn(0x563bcbbccce0 0x563bcbbb14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:28.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.051+0000 7fcbe8ff9640 10 client.6021 ms_handle_connect on v2:172.21.15.203:6834/1168022146 2024-04-26T23:20:28.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.051+0000 7fcc0d3ed640 1 --2- 172.21.15.47:0/2250432378 >> [v2:172.21.15.203:6834/1168022146,v1:172.21.15.203:6836/1168022146] conn(0x563bcbbccce0 0x563bcbbb14e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20cd3ee802858c02 server_cookie=1100eebbe46f8970 in_seq=0 out_seq=0 2024-04-26T23:20:28.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.051+0000 7fcbe8ff9640 1 -- 172.21.15.47:0/2250432378 <== mds.0 v2:172.21.15.203:6834/1168022146 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x563bcbaf11c0 con 0x563bcbbccce0 2024-04-26T23:20:28.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.051+0000 7fcbe8ff9640 10 client.6021 handle_command_reply: tid=0 2024-04-26T23:20:28.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.051+0000 7fcc12b6b740 1 client.6021 shutdown 2024-04-26T23:20:28.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.051+0000 7fcc12b6b740 1 -- 172.21.15.47:0/2250432378 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563bcbb85120 msgr2=0x563bcbc32980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:28.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.051+0000 7fcc12b6b740 1 --2- 172.21.15.47:0/2250432378 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563bcbb85120 0x563bcbc32980 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fcc04059e10 tx=0x7fcc04059f90 comp rx=0 tx=0).stop 2024-04-26T23:20:28.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.051+0000 7fcc12b6b740 1 -- 172.21.15.47:0/2250432378 shutdown_connections 2024-04-26T23:20:28.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.051+0000 7fcc12b6b740 1 --2- 172.21.15.47:0/2250432378 >> [v2:172.21.15.203:6834/1168022146,v1:172.21.15.203:6836/1168022146] conn(0x563bcbbccce0 0x563bcbbb14e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:28.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.051+0000 7fcc12b6b740 1 --2- 172.21.15.47:0/2250432378 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563bcbbaf440 0x563bcbb82ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:28.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.051+0000 7fcc12b6b740 1 --2- 172.21.15.47:0/2250432378 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563bcbbabd20 0x563bcbba92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:28.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.051+0000 7fcbe8ff9640 0 client.6021 ms_handle_reset on v2:172.21.15.203:6834/1168022146 2024-04-26T23:20:28.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.051+0000 7fcc12b6b740 1 --2- 172.21.15.47:0/2250432378 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563bcbb85120 0x563bcbc32980 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:28.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.051+0000 7fcc12b6b740 1 -- 172.21.15.47:0/2250432378 >> 172.21.15.47:0/2250432378 conn(0x563bcbbcc1b0 msgr2=0x563bcbb84770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:28.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.051+0000 7fcc12b6b740 1 -- 172.21.15.47:0/2250432378 shutdown_connections 2024-04-26T23:20:28.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.051+0000 7fcc12b6b740 1 -- 172.21.15.47:0/2250432378 wait complete. 2024-04-26T23:20:28.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.052+0000 7fcc12b6b740 20 client.6021 trim_cache size 0 max 16384 2024-04-26T23:20:28.061 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-26T23:20:28.061 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "399006d3-4db1-43e6-ba81-a79a276eabfd", 2024-04-26T23:20:28.061 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 0, 2024-04-26T23:20:28.061 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 15011, 2024-04-26T23:20:28.061 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-04-26T23:20:28.061 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-04-26T23:20:28.061 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-04-26T23:20:28.061 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 5.7180810490000002, 2024-04-26T23:20:28.061 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 27, 2024-04-26T23:20:28.061 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 54, 2024-04-26T23:20:28.061 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 54, 2024-04-26T23:20:28.061 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 11.044241759 2024-04-26T23:20:28.061 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-26T23:20:28.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.052+0000 7fcc0fe52640 1 -- 172.21.15.47:0/3763784678 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fcbec06c8f0 msgr2=0x7fcbec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:28.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.052+0000 7fcc0fe52640 1 --2- 172.21.15.47:0/3763784678 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fcbec06c8f0 0x7fcbec06edb0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8034000 tx=0x7fcbf802e000 comp rx=0 tx=0).stop 2024-04-26T23:20:28.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.052+0000 7fcc0fe52640 1 -- 172.21.15.47:0/3763784678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc080d0f20 msgr2=0x7fcc08153220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:28.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.052+0000 7fcc0fe52640 1 --2- 172.21.15.47:0/3763784678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc080d0f20 0x7fcc08153220 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fcc0400cf90 tx=0x7fcc0400a7d0 comp rx=0 tx=0).stop 2024-04-26T23:20:28.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.052+0000 7fcc0fe52640 1 -- 172.21.15.47:0/3763784678 shutdown_connections 2024-04-26T23:20:28.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.052+0000 7fcc0fe52640 1 --2- 172.21.15.47:0/3763784678 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fcbec06c8f0 0x7fcbec06edb0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:28.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.052+0000 7fcc0fe52640 1 --2- 172.21.15.47:0/3763784678 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc0815d1e0 0x7fcc080b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:28.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.052+0000 7fcc0fe52640 1 --2- 172.21.15.47:0/3763784678 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc080d18c0 0x7fcc080b5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:28.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.052+0000 7fcc0fe52640 1 --2- 172.21.15.47:0/3763784678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc080d0f20 0x7fcc08153220 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:28.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.052+0000 7fcc0fe52640 1 -- 172.21.15.47:0/3763784678 >> 172.21.15.47:0/3763784678 conn(0x7fcc080d2da0 msgr2=0x7fcc080b0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:28.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.052+0000 7fcc0fe52640 1 -- 172.21.15.47:0/3763784678 shutdown_connections 2024-04-26T23:20:28.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.052+0000 7fcc0fe52640 1 -- 172.21.15.47:0/3763784678 wait complete. 2024-04-26T23:20:28.064 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T23:20:28.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.164+0000 7f4ef7264640 1 Processor -- start 2024-04-26T23:20:28.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.164+0000 7f4ef7264640 1 -- start start 2024-04-26T23:20:28.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.165+0000 7f4ef7264640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4ef0162880 0x7f4ef00d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:28.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.165+0000 7f4ef7264640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ef00b7e50 con 0x7f4ef00b5f20 2024-04-26T23:20:28.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.165+0000 7f4ef7264640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ef00d1800 con 0x7f4ef00d3bc0 2024-04-26T23:20:28.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.165+0000 7f4ef7264640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f4ef00d0580 con 0x7f4ef0162880 2024-04-26T23:20:28.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.165+0000 7f4ef5000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4ef0162880 0x7f4ef00d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:28.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.165+0000 7f4ef5000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4ef0162880 0x7f4ef00d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40890/0 (socket says 172.21.15.47:40890) 2024-04-26T23:20:28.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.165+0000 7f4ef5000640 1 -- 172.21.15.47:0/2364659551 learned_addr learned my addr 172.21.15.47:0/2364659551 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:28.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.165+0000 7f4ef5000640 1 -- 172.21.15.47:0/2364659551 >> v1:172.21.15.203:6789/0 conn(0x7f4ef00d3bc0 legacy=0x7f4ef00b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:28.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.165+0000 7f4ef5000640 1 -- 172.21.15.47:0/2364659551 >> v1:172.21.15.47:6789/0 conn(0x7f4ef00b5f20 legacy=0x7f4ef00d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:28.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.165+0000 7f4eef7fe640 1 -- 172.21.15.47:0/2364659551 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2157137904 0 0) 0x7f4ef00b7e50 con 0x7f4ef00b5f20 2024-04-26T23:20:28.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.165+0000 7f4ef5000640 1 -- 172.21.15.47:0/2364659551 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ef00d07d0 con 0x7f4ef0162880 2024-04-26T23:20:28.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.166+0000 7f4ef5000640 1 --2- 172.21.15.47:0/2364659551 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ef0162880 0x7f4ef00d2cb0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4009870 tx=0x7f4ee402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=949ec2f633aae2bf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:28.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.166+0000 7f4eef7fe640 1 -- 172.21.15.47:0/2364659551 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4037070 con 0x7f4ef0162880 2024-04-26T23:20:28.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.166+0000 7f4eef7fe640 1 -- 172.21.15.47:0/2364659551 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ee402ae50 con 0x7f4ef0162880 2024-04-26T23:20:28.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.166+0000 7f4eef7fe640 1 -- 172.21.15.47:0/2364659551 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4032860 con 0x7f4ef0162880 2024-04-26T23:20:28.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.167+0000 7f4ef7264640 1 -- 172.21.15.47:0/2364659551 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ef0162880 msgr2=0x7f4ef00d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:28.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.167+0000 7f4ef7264640 1 --2- 172.21.15.47:0/2364659551 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ef0162880 0x7f4ef00d2cb0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4009870 tx=0x7f4ee402a300 comp rx=0 tx=0).stop 2024-04-26T23:20:28.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.167+0000 7f4ef7264640 1 -- 172.21.15.47:0/2364659551 shutdown_connections 2024-04-26T23:20:28.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.167+0000 7f4ef7264640 1 --2- 172.21.15.47:0/2364659551 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ef0162880 0x7f4ef00d2cb0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:28.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.167+0000 7f4ef7264640 1 -- 172.21.15.47:0/2364659551 >> 172.21.15.47:0/2364659551 conn(0x7f4ef00d4f60 msgr2=0x7f4ef00d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:28.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.167+0000 7f4ef7264640 1 -- 172.21.15.47:0/2364659551 shutdown_connections 2024-04-26T23:20:28.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.167+0000 7f4ef7264640 1 -- 172.21.15.47:0/2364659551 wait complete. 2024-04-26T23:20:28.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.168+0000 7f4ef7264640 1 Processor -- start 2024-04-26T23:20:28.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.168+0000 7f4ef7264640 1 -- start start 2024-04-26T23:20:28.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.169+0000 7f4ef7264640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ef00b5f20 0x7f4ef00ce650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:28.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.169+0000 7f4ef7264640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ef00d3bc0 0x7f4ef01f46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:28.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.169+0000 7f4ef7264640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ef0162880 0x7f4ef00cc960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:28.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.169+0000 7f4ef7264640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef0162c60 con 0x7f4ef00b5f20 2024-04-26T23:20:28.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.169+0000 7f4ef7264640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef00d0d90 con 0x7f4ef00d3bc0 2024-04-26T23:20:28.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.169+0000 7f4ef7264640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f4ef0162de0 con 0x7f4ef0162880 2024-04-26T23:20:28.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.169+0000 7f4ef5000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ef00b5f20 0x7f4ef00ce650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:28.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.169+0000 7f4ef5000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ef00b5f20 0x7f4ef00ce650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40898/0 (socket says 172.21.15.47:40898) 2024-04-26T23:20:28.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.169+0000 7f4ef5000640 1 -- 172.21.15.47:0/3491121979 learned_addr learned my addr 172.21.15.47:0/3491121979 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:28.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.169+0000 7f4ef5801640 1 --2- 172.21.15.47:0/3491121979 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ef0162880 0x7f4ef00cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:28.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.169+0000 7f4eeffff640 1 --2- 172.21.15.47:0/3491121979 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ef00d3bc0 0x7f4ef01f46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:28.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.169+0000 7f4ef5000640 1 -- 172.21.15.47:0/3491121979 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ef0162880 msgr2=0x7f4ef00cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:28.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.169+0000 7f4ef5000640 1 --2- 172.21.15.47:0/3491121979 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ef0162880 0x7f4ef00cc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:28.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.169+0000 7f4ef5000640 1 -- 172.21.15.47:0/3491121979 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ef00d3bc0 msgr2=0x7f4ef01f46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:28.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.169+0000 7f4ef5000640 1 --2- 172.21.15.47:0/3491121979 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ef00d3bc0 0x7f4ef01f46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:28.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.169+0000 7f4ef5000640 1 -- 172.21.15.47:0/3491121979 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ef00cd200 con 0x7f4ef00b5f20 2024-04-26T23:20:28.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.170+0000 7f4ef5000640 1 --2- 172.21.15.47:0/3491121979 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ef00b5f20 0x7f4ef00ce650 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4009870 tx=0x7f4ee40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:28.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.170+0000 7f4eed7fa640 1 -- 172.21.15.47:0/3491121979 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4041070 con 0x7f4ef00b5f20 2024-04-26T23:20:28.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.170+0000 7f4eed7fa640 1 -- 172.21.15.47:0/3491121979 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ee403c870 con 0x7f4ef00b5f20 2024-04-26T23:20:28.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.170+0000 7f4ef7264640 1 -- 172.21.15.47:0/3491121979 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ef00cd4e0 con 0x7f4ef00b5f20 2024-04-26T23:20:28.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.170+0000 7f4eed7fa640 1 -- 172.21.15.47:0/3491121979 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4037040 con 0x7f4ef00b5f20 2024-04-26T23:20:28.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.170+0000 7f4ef7264640 1 -- 172.21.15.47:0/3491121979 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ef00cdaa0 con 0x7f4ef00b5f20 2024-04-26T23:20:28.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.171+0000 7f4ef7264640 1 -- 172.21.15.47:0/3491121979 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ef00694f0 con 0x7f4ef00b5f20 2024-04-26T23:20:28.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.171+0000 7f4eed7fa640 1 -- 172.21.15.47:0/3491121979 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4ee4044070 con 0x7f4ef00b5f20 2024-04-26T23:20:28.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.172+0000 7f4eed7fa640 1 --2- 172.21.15.47:0/3491121979 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4ed406c8f0 0x7f4ed406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:28.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.172+0000 7f4eed7fa640 1 -- 172.21.15.47:0/3491121979 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f4ee40ac700 con 0x7f4ef00b5f20 2024-04-26T23:20:28.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.172+0000 7f4eeffff640 1 --2- 172.21.15.47:0/3491121979 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4ed406c8f0 0x7f4ed406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:28.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.172+0000 7f4eeffff640 1 --2- 172.21.15.47:0/3491121979 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4ed406c8f0 0x7f4ed406edb0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4ef00ba920 tx=0x7f4edc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:28.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.176+0000 7f4eed7fa640 1 -- 172.21.15.47:0/3491121979 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4ee407a470 con 0x7f4ef00b5f20 2024-04-26T23:20:28.230 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:20:28.232 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:20:28.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.401+0000 7f4ef7264640 1 -- 172.21.15.47:0/3491121979 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f4ef00d54e0 con 0x7f4ef00b5f20 2024-04-26T23:20:28.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.401+0000 7f4eed7fa640 1 -- 172.21.15.47:0/3491121979 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+73 (secure 0 0 0) 0x7f4ee407e770 con 0x7f4ef00b5f20 2024-04-26T23:20:28.403 INFO:teuthology.orchestra.run.smithi047.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T23:20:28.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.403+0000 7f4ef7264640 1 -- 172.21.15.47:0/3491121979 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4ed406c8f0 msgr2=0x7f4ed406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:28.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.403+0000 7f4ef7264640 1 --2- 172.21.15.47:0/3491121979 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4ed406c8f0 0x7f4ed406edb0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4ef00ba920 tx=0x7f4edc009040 comp rx=0 tx=0).stop 2024-04-26T23:20:28.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.403+0000 7f4ef7264640 1 -- 172.21.15.47:0/3491121979 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ef00b5f20 msgr2=0x7f4ef00ce650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:28.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.403+0000 7f4ef7264640 1 --2- 172.21.15.47:0/3491121979 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ef00b5f20 0x7f4ef00ce650 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4009870 tx=0x7f4ee40046c0 comp rx=0 tx=0).stop 2024-04-26T23:20:28.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.403+0000 7f4ef7264640 1 -- 172.21.15.47:0/3491121979 shutdown_connections 2024-04-26T23:20:28.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.403+0000 7f4ef7264640 1 --2- 172.21.15.47:0/3491121979 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4ed406c8f0 0x7f4ed406edb0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:28.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.403+0000 7f4ef7264640 1 --2- 172.21.15.47:0/3491121979 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ef0162880 0x7f4ef00cc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:28.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.403+0000 7f4ef7264640 1 --2- 172.21.15.47:0/3491121979 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ef00d3bc0 0x7f4ef01f46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:28.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.403+0000 7f4ef7264640 1 --2- 172.21.15.47:0/3491121979 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ef00b5f20 0x7f4ef00ce650 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:28.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.404+0000 7f4ef7264640 1 -- 172.21.15.47:0/3491121979 >> 172.21.15.47:0/3491121979 conn(0x7f4ef00d4f60 msgr2=0x7f4ef015f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:28.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.404+0000 7f4ef7264640 1 -- 172.21.15.47:0/3491121979 shutdown_connections 2024-04-26T23:20:28.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28.404+0000 7f4ef7264640 1 -- 172.21.15.47:0/3491121979 wait complete. 2024-04-26T23:20:28.416 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T23:20:28.416 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T23:20:28.416 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi047.front.sepia.ceph.com 2024-04-26T23:20:28.416 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T23:20:28.416 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T23:20:28.416 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T23:20:28.416 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T23:20:28.416 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T23:20:28.416 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:20:28.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:20:28.416 DEBUG:teuthology.orchestra.run.smithi047:> ip addr 2024-04-26T23:20:28.433 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T23:20:28.433 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T23:20:28.433 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T23:20:28.433 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:20:28.433 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2024-04-26T23:20:28.433 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:20:28.433 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:20:28.433 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:20:28.433 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f0 2024-04-26T23:20:28.433 INFO:teuthology.orchestra.run.smithi047.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:20:28.434 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:20:28.434 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f1 2024-04-26T23:20:28.434 INFO:teuthology.orchestra.run.smithi047.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T23:20:28.434 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2024-04-26T23:20:28.434 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f0 2024-04-26T23:20:28.434 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-26T23:20:28.434 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42571sec preferred_lft 42571sec 2024-04-26T23:20:28.434 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link noprefixroute 2024-04-26T23:20:28.434 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:20:28.434 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:20:28.434 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2024-04-26T23:20:28.434 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f1 2024-04-26T23:20:28.434 INFO:teuthology.orchestra.run.smithi047.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T23:20:28.434 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 32:8d:47:b1:d0:77 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:20:28.435 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T23:20:28.435 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:20:28.435 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::7c6d:6aff:fe00:4b30/64 scope link 2024-04-26T23:20:28.435 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:20:28.435 INFO:teuthology.orchestra.run.smithi047.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T23:20:28.435 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 32:8d:47:b1:d0:77 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T23:20:28.435 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::308d:47ff:feb1:d077/64 scope link 2024-04-26T23:20:28.435 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:20:28.435 INFO:teuthology.orchestra.run.smithi047.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T23:20:28.435 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether b2:9e:42:6a:5b:5d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T23:20:28.435 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::b09e:42ff:fe6a:5b5d/64 scope link 2024-04-26T23:20:28.435 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:20:28.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:20:28.436 DEBUG:teuthology.orchestra.run.smithi047:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T23:20:28.496 INFO:teuthology.orchestra.run.smithi047.stdout:1 2024-04-26T23:20:28.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:20:28.498 DEBUG:teuthology.orchestra.run.smithi047:> route 2024-04-26T23:20:28.554 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2024-04-26T23:20:28.554 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T23:20:28.554 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T23:20:28.554 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T23:20:28.554 INFO:teuthology.orchestra.run.smithi047.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T23:20:28.554 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:20:28.554 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-04-26T23:20:28.555 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T23:20:28.555 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T23:20:28.555 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T23:20:28.555 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-04-26T23:20:28.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:20:28.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:20:28.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:20:28.681 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-04-26T23:20:28.696 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T23:20:28.696 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T23:20:28.696 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:20:28.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:20:28.696 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set brx.0 up 2024-04-26T23:20:28.761 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:20:28.761 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T23:20:28.817 INFO:teuthology.orchestra.run.smithi047.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:20:28.817 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:20:28.817 DEBUG:teuthology.orchestra.run.smithi047:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T23:20:28.872 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T23:20:28.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:20:28.872 DEBUG:teuthology.orchestra.run.smithi047:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:20:28.954 INFO:teuthology.orchestra.run.smithi047.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:20:28.954 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:20:28.954 INFO:teuthology.orchestra.run.smithi047.stdout:invalid new device string format 2024-04-26T23:20:28.963 INFO:teuthology.orchestra.run.smithi047.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:20:28.963 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:20:28.963 INFO:teuthology.orchestra.run.smithi047.stdout:invalid new device string format 2024-04-26T23:20:28.963 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: resolved to: "172.21.15.47:3300,172.21.15.203:3300,172.21.15.203:3301" 2024-04-26T23:20:28.963 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:20:28.968 INFO:teuthology.orchestra.run.smithi047.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:20:28.968 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:20:28.968 INFO:teuthology.orchestra.run.smithi047.stdout:invalid new device string format 2024-04-26T23:20:28.968 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: resolved to: "172.21.15.47:3300,172.21.15.203:3300,172.21.15.203:3301" 2024-04-26T23:20:28.968 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:20:28.968 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: resolved to: "172.21.15.47:6789,172.21.15.203:6789,172.21.15.203:6790" 2024-04-26T23:20:28.968 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.47:6789,172.21.15.203:6789,172.21.15.203:6790:/ 2024-04-26T23:20:28.968 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=399006d3-4db1-43e6-ba81-a79a276eabfd" will pass to kernel 2024-04-26T23:20:28.969 INFO:teuthology.orchestra.run.smithi047.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T23:20:28.969 INFO:teuthology.orchestra.run.smithi047.stdout: You just mounted a file system that supports labels which does not 2024-04-26T23:20:28.969 INFO:teuthology.orchestra.run.smithi047.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T23:20:28.969 INFO:teuthology.orchestra.run.smithi047.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T23:20:28.969 INFO:teuthology.orchestra.run.smithi047.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T23:20:28.970 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T23:20:28.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:20:28.970 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T23:20:29.036 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T23:20:29.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:20:29.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:20:29.142 INFO:teuthology.orchestra.run.smithi047.stdout:client6033 2024-04-26T23:20:29.142 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:20:29.142 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:20:29.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:20:29.158 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch test_file) 2024-04-26T23:20:29.220 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow r pool=cephfs_data' 2024-04-26T23:20:29.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.367+0000 7f4372a9c640 1 Processor -- start 2024-04-26T23:20:29.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.367+0000 7f4372a9c640 1 -- start start 2024-04-26T23:20:29.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.368+0000 7f4372a9c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f436c0b10b0 0x7f436c0ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:29.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.368+0000 7f4372a9c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f436c06c950 0x7f436c15e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:29.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.368+0000 7f4372a9c640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f436c0cfd80 con 0x7f436c0b0380 2024-04-26T23:20:29.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.368+0000 7f4372a9c640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f436c0af8e0 con 0x7f436c06c950 2024-04-26T23:20:29.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.368+0000 7f4372a9c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f436c0d0020 con 0x7f436c0b10b0 2024-04-26T23:20:29.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.368+0000 7f4370838640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f436c0b0380 0x7f436c06c240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35820/0 (socket says 172.21.15.47:35820) 2024-04-26T23:20:29.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.368+0000 7f4370838640 1 -- 172.21.15.47:0/3978367617 learned_addr learned my addr 172.21.15.47:0/3978367617 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:29.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.368+0000 7f4371039640 1 --2- 172.21.15.47:0/3978367617 >> v2:172.21.15.203:3300/0 conn(0x7f436c06c950 0x7f436c15e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:29.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.368+0000 7f436b7fe640 1 -- 172.21.15.47:0/3978367617 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1411751183 0 0) 0x7f436c0cfd80 con 0x7f436c0b0380 2024-04-26T23:20:29.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.368+0000 7f436bfff640 1 --2- 172.21.15.47:0/3978367617 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f436c0b10b0 0x7f436c0ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:29.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.368+0000 7f436b7fe640 1 -- 172.21.15.47:0/3978367617 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f435c003520 con 0x7f436c0b0380 2024-04-26T23:20:29.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.368+0000 7f436b7fe640 1 -- 172.21.15.47:0/3978367617 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4041479122 0 0) 0x7f435c003520 con 0x7f436c0b0380 2024-04-26T23:20:29.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.368+0000 7f436b7fe640 1 -- 172.21.15.47:0/3978367617 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f436c0b10b0 msgr2=0x7f436c0ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:29.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.368+0000 7f436b7fe640 1 --2- 172.21.15.47:0/3978367617 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f436c0b10b0 0x7f436c0ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:29.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.368+0000 7f436b7fe640 1 -- 172.21.15.47:0/3978367617 >> v2:172.21.15.203:3300/0 conn(0x7f436c06c950 msgr2=0x7f436c15e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:29.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.368+0000 7f436b7fe640 1 --2- 172.21.15.47:0/3978367617 >> v2:172.21.15.203:3300/0 conn(0x7f436c06c950 0x7f436c15e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:29.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.368+0000 7f436b7fe640 1 -- 172.21.15.47:0/3978367617 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f436c0d0300 con 0x7f436c0b0380 2024-04-26T23:20:29.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.368+0000 7f436b7fe640 1 -- 172.21.15.47:0/3978367617 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7f4360001670 con 0x7f436c0b0380 2024-04-26T23:20:29.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.368+0000 7f436b7fe640 1 -- 172.21.15.47:0/3978367617 >> v1:172.21.15.47:6789/0 conn(0x7f436c0b0380 legacy=0x7f436c06c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:29.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.368+0000 7f436b7fe640 1 --2- 172.21.15.47:0/3978367617 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f435c0039b0 0x7f435c003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:29.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.368+0000 7f436b7fe640 1 --2- 172.21.15.47:0/3978367617 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f436c0b10b0 0x7f435c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:29.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.368+0000 7f436b7fe640 1 --2- 172.21.15.47:0/3978367617 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f435c004d40 0x7f435c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:29.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.368+0000 7f436b7fe640 1 -- 172.21.15.47:0/3978367617 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f436c0af8e0 con 0x7f435c004d40 2024-04-26T23:20:29.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.368+0000 7f436b7fe640 1 -- 172.21.15.47:0/3978367617 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f436c0d0020 con 0x7f435c0039b0 2024-04-26T23:20:29.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.368+0000 7f436b7fe640 1 -- 172.21.15.47:0/3978367617 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f435c00b9c0 con 0x7f436c0b10b0 2024-04-26T23:20:29.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.369+0000 7f4371039640 1 --2- 172.21.15.47:0/3978367617 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f435c004d40 0x7f435c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:29.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.369+0000 7f4370838640 1 --2- 172.21.15.47:0/3978367617 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f435c0039b0 0x7f435c003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:29.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.369+0000 7f436bfff640 1 --2- 172.21.15.47:0/3978367617 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f436c0b10b0 0x7f435c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:29.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.369+0000 7f4371039640 1 -- 172.21.15.47:0/3978367617 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f436c0b10b0 msgr2=0x7f435c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:29.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.369+0000 7f4371039640 1 --2- 172.21.15.47:0/3978367617 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f436c0b10b0 0x7f435c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:29.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.369+0000 7f4371039640 1 -- 172.21.15.47:0/3978367617 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f435c0039b0 msgr2=0x7f435c003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:29.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.369+0000 7f4371039640 1 --2- 172.21.15.47:0/3978367617 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f435c0039b0 0x7f435c003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:29.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.369+0000 7f4371039640 1 -- 172.21.15.47:0/3978367617 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f435c003520 con 0x7f435c004d40 2024-04-26T23:20:29.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.370+0000 7f4371039640 1 --2- 172.21.15.47:0/3978367617 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f435c004d40 0x7f435c0051c0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f4364009f90 tx=0x7f435c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e4c54c3b172e884 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:29.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.370+0000 7f436b7fe640 1 -- 172.21.15.47:0/3978367617 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4364009c10 con 0x7f435c004d40 2024-04-26T23:20:29.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.370+0000 7f436b7fe640 1 -- 172.21.15.47:0/3978367617 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4364036070 con 0x7f435c004d40 2024-04-26T23:20:29.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.370+0000 7f436b7fe640 1 -- 172.21.15.47:0/3978367617 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4364031880 con 0x7f435c004d40 2024-04-26T23:20:29.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.370+0000 7f4372a9c640 1 -- 172.21.15.47:0/3978367617 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f435c004d40 msgr2=0x7f435c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:29.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.370+0000 7f4372a9c640 1 --2- 172.21.15.47:0/3978367617 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f435c004d40 0x7f435c0051c0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f4364009f90 tx=0x7f435c005830 comp rx=0 tx=0).stop 2024-04-26T23:20:29.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.370+0000 7f4371039640 1 -- 172.21.15.47:0/3978367617 reap_dead start 2024-04-26T23:20:29.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.370+0000 7f4372a9c640 1 -- 172.21.15.47:0/3978367617 shutdown_connections 2024-04-26T23:20:29.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.370+0000 7f4372a9c640 1 -- 172.21.15.47:0/3978367617 >> 172.21.15.47:0/3978367617 conn(0x7f436c0d1cf0 msgr2=0x7f436c0d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:29.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.370+0000 7f4372a9c640 1 -- 172.21.15.47:0/3978367617 shutdown_connections 2024-04-26T23:20:29.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.370+0000 7f4372a9c640 1 -- 172.21.15.47:0/3978367617 wait complete. 2024-04-26T23:20:29.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.371+0000 7f4372a9c640 1 Processor -- start 2024-04-26T23:20:29.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.372+0000 7f4372a9c640 1 -- start start 2024-04-26T23:20:29.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.372+0000 7f4372a9c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f435c004d40 0x7f436c0b7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:29.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.372+0000 7f4372a9c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f436c0ca0e0 0x7f436c1541b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:29.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.372+0000 7f4372a9c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f436c154d20 0x7f436c06f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:29.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.372+0000 7f4372a9c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f436c0b0760 con 0x7f435c004d40 2024-04-26T23:20:29.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.372+0000 7f4372a9c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f436c0d1980 con 0x7f436c154d20 2024-04-26T23:20:29.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.372+0000 7f4372a9c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f436c0d15c0 con 0x7f436c0ca0e0 2024-04-26T23:20:29.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.372+0000 7f4370838640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f435c004d40 0x7f436c0b7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:29.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.372+0000 7f4370838640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f435c004d40 0x7f436c0b7f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40920/0 (socket says 172.21.15.47:40920) 2024-04-26T23:20:29.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.373+0000 7f4370838640 1 -- 172.21.15.47:0/259465853 learned_addr learned my addr 172.21.15.47:0/259465853 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:29.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.373+0000 7f436bfff640 1 --2- 172.21.15.47:0/259465853 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f436c0ca0e0 0x7f436c1541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:29.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.373+0000 7f4371039640 1 --2- 172.21.15.47:0/259465853 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f436c154d20 0x7f436c06f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:29.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.373+0000 7f4370838640 1 -- 172.21.15.47:0/259465853 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f436c0ca0e0 msgr2=0x7f436c1541b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:29.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.373+0000 7f4370838640 1 --2- 172.21.15.47:0/259465853 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f436c0ca0e0 0x7f436c1541b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:29.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.373+0000 7f4370838640 1 -- 172.21.15.47:0/259465853 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f436c154d20 msgr2=0x7f436c06f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:29.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.373+0000 7f4370838640 1 --2- 172.21.15.47:0/259465853 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f436c154d20 0x7f436c06f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:29.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.373+0000 7f4370838640 1 -- 172.21.15.47:0/259465853 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f436c0ca650 con 0x7f435c004d40 2024-04-26T23:20:29.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.373+0000 7f4370838640 1 --2- 172.21.15.47:0/259465853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f435c004d40 0x7f436c0b7f00 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f4360007370 tx=0x7f4360007770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:29.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.375+0000 7f43697fa640 1 -- 172.21.15.47:0/259465853 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436001a070 con 0x7f435c004d40 2024-04-26T23:20:29.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.375+0000 7f43697fa640 1 -- 172.21.15.47:0/259465853 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f436000f960 con 0x7f435c004d40 2024-04-26T23:20:29.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.375+0000 7f4372a9c640 1 -- 172.21.15.47:0/259465853 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f436c0ca930 con 0x7f435c004d40 2024-04-26T23:20:29.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.375+0000 7f43697fa640 1 -- 172.21.15.47:0/259465853 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436000fd30 con 0x7f435c004d40 2024-04-26T23:20:29.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.375+0000 7f4372a9c640 1 -- 172.21.15.47:0/259465853 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f436c0caef0 con 0x7f435c004d40 2024-04-26T23:20:29.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.376+0000 7f4372a9c640 1 -- 172.21.15.47:0/259465853 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f436c0694f0 con 0x7f435c004d40 2024-04-26T23:20:29.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.376+0000 7f43697fa640 1 -- 172.21.15.47:0/259465853 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f436001d070 con 0x7f435c004d40 2024-04-26T23:20:29.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.377+0000 7f43697fa640 1 --2- 172.21.15.47:0/259465853 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f435006c8f0 0x7f435006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:29.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.377+0000 7f43697fa640 1 -- 172.21.15.47:0/259465853 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f436008b670 con 0x7f435c004d40 2024-04-26T23:20:29.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.377+0000 7f436bfff640 1 --2- 172.21.15.47:0/259465853 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f435006c8f0 0x7f435006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:29.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.378+0000 7f436bfff640 1 --2- 172.21.15.47:0/259465853 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f435006c8f0 0x7f435006edb0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f436c160e20 tx=0x7f4358009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:29.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.382+0000 7f43697fa640 1 -- 172.21.15.47:0/259465853 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f43600592c0 con 0x7f435c004d40 2024-04-26T23:20:29.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.641+0000 7f4372a9c640 1 -- 172.21.15.47:0/259465853 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow r pool=cephfs_data"]} v 0) -- 0x7f436c0ce7b0 con 0x7f435c004d40 2024-04-26T23:20:29.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.643+0000 7f43697fa640 1 -- 172.21.15.47:0/259465853 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow r pool=cephfs_data"]}]=0 updated caps for client.0 v15) ==== 183+0+140 (secure 0 0 0) 0x7f436005d5c0 con 0x7f435c004d40 2024-04-26T23:20:29.644 INFO:teuthology.orchestra.run.smithi047.stdout:[client.0] 2024-04-26T23:20:29.644 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQAKNixmA5QFHRAAT89WPyzDs0acPluyvU4svw== 2024-04-26T23:20:29.644 INFO:teuthology.orchestra.run.smithi047.stdout: caps mds = "allow" 2024-04-26T23:20:29.644 INFO:teuthology.orchestra.run.smithi047.stdout: caps mon = "allow r" 2024-04-26T23:20:29.644 INFO:teuthology.orchestra.run.smithi047.stdout: caps osd = "allow r pool=cephfs_data" 2024-04-26T23:20:29.644 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.0 2024-04-26T23:20:29.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.645+0000 7f4372a9c640 1 -- 172.21.15.47:0/259465853 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f435006c8f0 msgr2=0x7f435006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:29.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.645+0000 7f4372a9c640 1 --2- 172.21.15.47:0/259465853 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f435006c8f0 0x7f435006edb0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f436c160e20 tx=0x7f4358009190 comp rx=0 tx=0).stop 2024-04-26T23:20:29.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.645+0000 7f4372a9c640 1 -- 172.21.15.47:0/259465853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f435c004d40 msgr2=0x7f436c0b7f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:29.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.645+0000 7f4372a9c640 1 --2- 172.21.15.47:0/259465853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f435c004d40 0x7f436c0b7f00 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f4360007370 tx=0x7f4360007770 comp rx=0 tx=0).stop 2024-04-26T23:20:29.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.645+0000 7f4372a9c640 1 -- 172.21.15.47:0/259465853 shutdown_connections 2024-04-26T23:20:29.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.645+0000 7f4372a9c640 1 --2- 172.21.15.47:0/259465853 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f435006c8f0 0x7f435006edb0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:29.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.645+0000 7f4372a9c640 1 --2- 172.21.15.47:0/259465853 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f436c154d20 0x7f436c06f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:29.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.645+0000 7f4372a9c640 1 --2- 172.21.15.47:0/259465853 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f436c0ca0e0 0x7f436c1541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:29.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.645+0000 7f4372a9c640 1 --2- 172.21.15.47:0/259465853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f435c004d40 0x7f436c0b7f00 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:29.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.645+0000 7f4372a9c640 1 -- 172.21.15.47:0/259465853 >> 172.21.15.47:0/259465853 conn(0x7f436c0d1cf0 msgr2=0x7f436c0b2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:29.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.645+0000 7f4372a9c640 1 -- 172.21.15.47:0/259465853 shutdown_connections 2024-04-26T23:20:29.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.645+0000 7f4372a9c640 1 -- 172.21.15.47:0/259465853 wait complete. 2024-04-26T23:20:29.657 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:20:29.657 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:20:29.675 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:20:29.675 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:20:29.732 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:20:29.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.871+0000 7fe0876cc640 1 Processor -- start 2024-04-26T23:20:29.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.871+0000 7fe0876cc640 1 -- start start 2024-04-26T23:20:29.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.872+0000 7fe0876cc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0800b2340 0x7fe0800cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:29.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.872+0000 7fe0876cc640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe0800cd930 0x7fe0800d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:29.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.872+0000 7fe0876cc640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0800d10d0 con 0x7fe080162810 2024-04-26T23:20:29.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.872+0000 7fe0876cc640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fe0800d0750 con 0x7fe0800cd930 2024-04-26T23:20:29.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.872+0000 7fe0876cc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fe0800d16a0 con 0x7fe0800b2340 2024-04-26T23:20:29.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.872+0000 7fe085c69640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe0800cd930 0x7fe0800d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:29.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.872+0000 7fe085c69640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe0800cd930 0x7fe0800d1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51324/0 (socket says 172.21.15.47:51324) 2024-04-26T23:20:29.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.872+0000 7fe085c69640 1 -- 172.21.15.47:0/866916556 learned_addr learned my addr 172.21.15.47:0/866916556 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:29.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.872+0000 7fe084c67640 1 --2- 172.21.15.47:0/866916556 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0800b2340 0x7fe0800cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:29.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.873+0000 7fe085c69640 1 -- 172.21.15.47:0/866916556 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0800b2340 msgr2=0x7fe0800cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:29.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.873+0000 7fe085c69640 1 --2- 172.21.15.47:0/866916556 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0800b2340 0x7fe0800cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:29.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.873+0000 7fe085c69640 1 -- 172.21.15.47:0/866916556 >> v1:172.21.15.203:6789/0 conn(0x7fe080162810 legacy=0x7fe080070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:29.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.873+0000 7fe085c69640 1 -- 172.21.15.47:0/866916556 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0800d1370 con 0x7fe0800cd930 2024-04-26T23:20:29.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.873+0000 7fe085c69640 1 --2- 172.21.15.47:0/866916556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0800cd930 0x7fe0800d1b30 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fe070009e50 tx=0x7fe07002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89a4dfdf283cfe6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:29.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.873+0000 7fe07ffff640 1 -- 172.21.15.47:0/866916556 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe070036070 con 0x7fe0800cd930 2024-04-26T23:20:29.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.873+0000 7fe07ffff640 1 -- 172.21.15.47:0/866916556 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe070031600 con 0x7fe0800cd930 2024-04-26T23:20:29.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.873+0000 7fe07ffff640 1 -- 172.21.15.47:0/866916556 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0700319d0 con 0x7fe0800cd930 2024-04-26T23:20:29.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.874+0000 7fe0876cc640 1 -- 172.21.15.47:0/866916556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0800cd930 msgr2=0x7fe0800d1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:29.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.874+0000 7fe0876cc640 1 --2- 172.21.15.47:0/866916556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0800cd930 0x7fe0800d1b30 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fe070009e50 tx=0x7fe07002a710 comp rx=0 tx=0).stop 2024-04-26T23:20:29.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.874+0000 7fe0876cc640 1 -- 172.21.15.47:0/866916556 shutdown_connections 2024-04-26T23:20:29.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.874+0000 7fe0876cc640 1 --2- 172.21.15.47:0/866916556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0800cd930 0x7fe0800d1b30 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:29.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.874+0000 7fe0876cc640 1 --2- 172.21.15.47:0/866916556 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0800b2340 0x7fe0800cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:29.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.874+0000 7fe0876cc640 1 -- 172.21.15.47:0/866916556 >> 172.21.15.47:0/866916556 conn(0x7fe0800d2da0 msgr2=0x7fe0800d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:29.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.874+0000 7fe0876cc640 1 -- 172.21.15.47:0/866916556 shutdown_connections 2024-04-26T23:20:29.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.874+0000 7fe0876cc640 1 -- 172.21.15.47:0/866916556 wait complete. 2024-04-26T23:20:29.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.875+0000 7fe0876cc640 1 Processor -- start 2024-04-26T23:20:29.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.875+0000 7fe0876cc640 1 -- start start 2024-04-26T23:20:29.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.876+0000 7fe0876cc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0800b2340 0x7fe08014ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:29.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.876+0000 7fe0876cc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0800cd930 0x7fe0801f5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:29.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.876+0000 7fe0876cc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe080162810 0x7fe0800cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:29.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.876+0000 7fe0876cc640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe0800d2070 con 0x7fe0800cd930 2024-04-26T23:20:29.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.876+0000 7fe0876cc640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe0800d0470 con 0x7fe080162810 2024-04-26T23:20:29.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.876+0000 7fe0876cc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fe0800d21f0 con 0x7fe0800b2340 2024-04-26T23:20:29.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.876+0000 7fe084c67640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0800cd930 0x7fe0801f5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:29.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.876+0000 7fe084c67640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0800cd930 0x7fe0801f5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51332/0 (socket says 172.21.15.47:51332) 2024-04-26T23:20:29.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.876+0000 7fe084c67640 1 -- 172.21.15.47:0/3074726270 learned_addr learned my addr 172.21.15.47:0/3074726270 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:29.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.876+0000 7fe085468640 1 --2- 172.21.15.47:0/3074726270 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0800b2340 0x7fe08014ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:29.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.876+0000 7fe085c69640 1 --2- 172.21.15.47:0/3074726270 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe080162810 0x7fe0800cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:29.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.876+0000 7fe084c67640 1 -- 172.21.15.47:0/3074726270 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0800b2340 msgr2=0x7fe08014ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:29.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.876+0000 7fe084c67640 1 --2- 172.21.15.47:0/3074726270 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0800b2340 0x7fe08014ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:29.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.876+0000 7fe084c67640 1 -- 172.21.15.47:0/3074726270 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe080162810 msgr2=0x7fe0800cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:29.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.876+0000 7fe084c67640 1 --2- 172.21.15.47:0/3074726270 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe080162810 0x7fe0800cbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:29.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.876+0000 7fe084c67640 1 -- 172.21.15.47:0/3074726270 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0800cc430 con 0x7fe0800cd930 2024-04-26T23:20:29.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.876+0000 7fe085468640 1 --2- 172.21.15.47:0/3074726270 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0800b2340 0x7fe08014ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:29.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.876+0000 7fe085c69640 1 --2- 172.21.15.47:0/3074726270 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe080162810 0x7fe0800cbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:29.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.877+0000 7fe084c67640 1 --2- 172.21.15.47:0/3074726270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0800cd930 0x7fe0801f5740 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fe07400d830 tx=0x7fe07400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:29.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.877+0000 7fe07dffb640 1 -- 172.21.15.47:0/3074726270 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe07400b6c0 con 0x7fe0800cd930 2024-04-26T23:20:29.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.877+0000 7fe07dffb640 1 -- 172.21.15.47:0/3074726270 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe074019070 con 0x7fe0800cd930 2024-04-26T23:20:29.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.877+0000 7fe07dffb640 1 -- 172.21.15.47:0/3074726270 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe07400ba80 con 0x7fe0800cd930 2024-04-26T23:20:29.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.877+0000 7fe0876cc640 1 -- 172.21.15.47:0/3074726270 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0800cc710 con 0x7fe0800cd930 2024-04-26T23:20:29.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.877+0000 7fe0876cc640 1 -- 172.21.15.47:0/3074726270 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0800cccd0 con 0x7fe0800cd930 2024-04-26T23:20:29.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.878+0000 7fe0876cc640 1 -- 172.21.15.47:0/3074726270 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0800694f0 con 0x7fe0800cd930 2024-04-26T23:20:29.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.878+0000 7fe07dffb640 1 -- 172.21.15.47:0/3074726270 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe07401e460 con 0x7fe0800cd930 2024-04-26T23:20:29.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.879+0000 7fe07dffb640 1 --2- 172.21.15.47:0/3074726270 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fe06c06c8f0 0x7fe06c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:29.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.879+0000 7fe07dffb640 1 -- 172.21.15.47:0/3074726270 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fe07408c330 con 0x7fe0800cd930 2024-04-26T23:20:29.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.879+0000 7fe085468640 1 --2- 172.21.15.47:0/3074726270 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fe06c06c8f0 0x7fe06c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:29.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.879+0000 7fe085468640 1 --2- 172.21.15.47:0/3074726270 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fe06c06c8f0 0x7fe06c06edb0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe0800c7100 tx=0x7fe078009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:29.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:29.883+0000 7fe07dffb640 1 -- 172.21.15.47:0/3074726270 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe07405a660 con 0x7fe0800cd930 2024-04-26T23:20:30.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.069+0000 7fe0876cc640 1 -- 172.21.15.47:0/3074726270 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe0800b4600 con 0x7fe0800cd930 2024-04-26T23:20:30.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.070+0000 7fe07dffb640 1 -- 172.21.15.47:0/3074726270 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) ==== 74+0+16902 (secure 0 0 0) 0x7fe074059f60 con 0x7fe0800cd930 2024-04-26T23:20:30.071 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:20:30.072 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":54,"fsid":"399006d3-4db1-43e6-ba81-a79a276eabfd","created":"2024-04-26T23:18:06.731211+0000","modified":"2024-04-26T23:20:23.340537+0000","last_up_change":"2024-04-26T23:18:17.808736+0000","last_in_change":"2024-04-26T23:18:08.945859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:18:16.937492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:20:19.975691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-26T23:20:20.657778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ee091567-fea0-4a14-bf4e-0e16c20bb2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6819","nonce":326520779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6821","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6823","nonce":326520779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6829","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6831","nonce":326520779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6825","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6827","nonce":326520779}]},"public_addr":"172.21.15.47:6819/326520779","cluster_addr":"172.21.15.47:6823/326520779","heartbeat_back_addr":"172.21.15.47:6831/326520779","heartbeat_front_addr":"172.21.15.47:6827/326520779","state":["exists","up"]},{"osd":1,"uuid":"c14440a0-bba5-4931-a6db-b736c08e869b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6809","nonce":1912610931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6811","nonce":1912610931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6815","nonce":1912610931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6813","nonce":1912610931}]},"public_addr":"172.21.15.47:6809/1912610931","cluster_addr":"172.21.15.47:6811/1912610931","heartbeat_back_addr":"172.21.15.47:6815/1912610931","heartbeat_front_addr":"172.21.15.47:6813/1912610931","state":["exists","up"]},{"osd":2,"uuid":"3940f8de-c9b2-4cd3-a911-9c301729eb30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6801","nonce":2569869305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6803","nonce":2569869305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6807","nonce":2569869305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6805","nonce":2569869305}]},"public_addr":"172.21.15.47:6801/2569869305","cluster_addr":"172.21.15.47:6803/2569869305","heartbeat_back_addr":"172.21.15.47:6807/2569869305","heartbeat_front_addr":"172.21.15.47:6805/2569869305","state":["exists","up"]},{"osd":3,"uuid":"1286a67e-8958-426a-8c2e-bc85498c619c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6818","nonce":2791411683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6822","nonce":2791411683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6830","nonce":2791411683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6826","nonce":2791411683}]},"public_addr":"172.21.15.47:6818/2791411683","cluster_addr":"172.21.15.47:6822/2791411683","heartbeat_back_addr":"172.21.15.47:6830/2791411683","heartbeat_front_addr":"172.21.15.47:6826/2791411683","state":["exists","up"]},{"osd":4,"uuid":"d8ee2c20-e5b7-46c2-9999-b234070126cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6809","nonce":441313104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6811","nonce":441313104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6815","nonce":441313104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6813","nonce":441313104}]},"public_addr":"172.21.15.203:6809/441313104","cluster_addr":"172.21.15.203:6811/441313104","heartbeat_back_addr":"172.21.15.203:6815/441313104","heartbeat_front_addr":"172.21.15.203:6813/441313104","state":["exists","up"]},{"osd":5,"uuid":"b453a8f0-f324-4840-9ae0-d67bdadb9f8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6801","nonce":56838444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6803","nonce":56838444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6807","nonce":56838444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6805","nonce":56838444}]},"public_addr":"172.21.15.203:6801/56838444","cluster_addr":"172.21.15.203:6803/56838444","heartbeat_back_addr":"172.21.15.203:6807/56838444","heartbeat_front_addr":"172.21.15.203:6805/56838444","state":["exists","up"]},{"osd":6,"uuid":"a6d31951-2d4b-4e8d-a510-6283b01dc158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6825","nonce":2738517080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6827","nonce":2738517080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6831","nonce":2738517080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6829","nonce":2738517080}]},"public_addr":"172.21.15.203:6825/2738517080","cluster_addr":"172.21.15.203:6827/2738517080","heartbeat_back_addr":"172.21.15.203:6831/2738517080","heartbeat_front_addr":"172.21.15.203:6829/2738517080","state":["exists","up"]},{"osd":7,"uuid":"7119b8c0-fe96-402d-9e75-7b509d9d44d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6817","nonce":1315565058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6819","nonce":1315565058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6823","nonce":1315565058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6821","nonce":1315565058}]},"public_addr":"172.21.15.203:6817/1315565058","cluster_addr":"172.21.15.203:6819/1315565058","heartbeat_back_addr":"172.21.15.203:6823/1315565058","heartbeat_front_addr":"172.21.15.203:6821/1315565058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.121921+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.137494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.084814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.111593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.072430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.125066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.097867+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.057061+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:20:30.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.072+0000 7fe0876cc640 1 -- 172.21.15.47:0/3074726270 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fe06c06c8f0 msgr2=0x7fe06c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:30.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.072+0000 7fe0876cc640 1 --2- 172.21.15.47:0/3074726270 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fe06c06c8f0 0x7fe06c06edb0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe0800c7100 tx=0x7fe078009040 comp rx=0 tx=0).stop 2024-04-26T23:20:30.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.072+0000 7fe0876cc640 1 -- 172.21.15.47:0/3074726270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0800cd930 msgr2=0x7fe0801f5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:30.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.072+0000 7fe0876cc640 1 --2- 172.21.15.47:0/3074726270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0800cd930 0x7fe0801f5740 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fe07400d830 tx=0x7fe07400dcf0 comp rx=0 tx=0).stop 2024-04-26T23:20:30.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.072+0000 7fe0876cc640 1 -- 172.21.15.47:0/3074726270 shutdown_connections 2024-04-26T23:20:30.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.072+0000 7fe0876cc640 1 --2- 172.21.15.47:0/3074726270 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fe06c06c8f0 0x7fe06c06edb0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:30.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.072+0000 7fe0876cc640 1 --2- 172.21.15.47:0/3074726270 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe080162810 0x7fe0800cbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:30.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.072+0000 7fe0876cc640 1 --2- 172.21.15.47:0/3074726270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0800cd930 0x7fe0801f5740 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:30.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.072+0000 7fe0876cc640 1 --2- 172.21.15.47:0/3074726270 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0800b2340 0x7fe08014ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:30.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.072+0000 7fe0876cc640 1 -- 172.21.15.47:0/3074726270 >> 172.21.15.47:0/3074726270 conn(0x7fe0800d2da0 msgr2=0x7fe0800b2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:30.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.072+0000 7fe0876cc640 1 -- 172.21.15.47:0/3074726270 shutdown_connections 2024-04-26T23:20:30.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.072+0000 7fe0876cc640 1 -- 172.21.15.47:0/3074726270 wait complete. 2024-04-26T23:20:30.084 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:18:16.937492+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:20:19.975691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:20:20.657778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:20:30.085 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:20:30.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.187+0000 7f38e1ac4640 1 Processor -- start 2024-04-26T23:20:30.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.187+0000 7f38e1ac4640 1 -- start start 2024-04-26T23:20:30.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.188+0000 7f38e1ac4640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f38dc15f170 0x7f38dc0d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:30.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.188+0000 7f38e1ac4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38dc0d3bc0 0x7f38dc0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:30.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.188+0000 7f38e1ac4640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38dc0d1040 con 0x7f38dc15f550 2024-04-26T23:20:30.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.188+0000 7f38e1ac4640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f38dc0c9430 con 0x7f38dc15f170 2024-04-26T23:20:30.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.188+0000 7f38e1ac4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f38dc0c9680 con 0x7f38dc0d3bc0 2024-04-26T23:20:30.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.188+0000 7f38db7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f38dc15f170 0x7f38dc0d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:30.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.188+0000 7f38daffd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f38dc15f550 0x7f38dc0d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37786/0 (socket says 172.21.15.47:37786) 2024-04-26T23:20:30.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.188+0000 7f38db7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f38dc15f170 0x7f38dc0d5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51338/0 (socket says 172.21.15.47:51338) 2024-04-26T23:20:30.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.188+0000 7f38daffd640 1 -- 172.21.15.47:0/1617402062 learned_addr learned my addr 172.21.15.47:0/1617402062 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:30.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.188+0000 7f38dbfff640 1 --2- 172.21.15.47:0/1617402062 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38dc0d3bc0 0x7f38dc0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:30.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.189+0000 7f38db7fe640 1 -- 172.21.15.47:0/1617402062 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38dc0d3bc0 msgr2=0x7f38dc0d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:30.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.189+0000 7f38db7fe640 1 --2- 172.21.15.47:0/1617402062 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38dc0d3bc0 0x7f38dc0d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:30.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.189+0000 7f38db7fe640 1 -- 172.21.15.47:0/1617402062 >> v1:172.21.15.47:6789/0 conn(0x7f38dc15f550 legacy=0x7f38dc0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:30.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.189+0000 7f38db7fe640 1 -- 172.21.15.47:0/1617402062 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38dc0c9960 con 0x7f38dc15f170 2024-04-26T23:20:30.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.189+0000 7f38da7fc640 1 -- 172.21.15.47:0/1617402062 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 765749169 0 0) 0x7f38cc001110 con 0x7f38dc15f550 2024-04-26T23:20:30.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.189+0000 7f38db7fe640 1 --2- 172.21.15.47:0/1617402062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38dc15f170 0x7f38dc0d5a80 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f38c40097d0 tx=0x7f38c402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7db0a601ba04cbc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:30.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.189+0000 7f38da7fc640 1 -- 172.21.15.47:0/1617402062 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c4037070 con 0x7f38dc15f170 2024-04-26T23:20:30.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.189+0000 7f38da7fc640 1 -- 172.21.15.47:0/1617402062 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38c4032440 con 0x7f38dc15f170 2024-04-26T23:20:30.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.189+0000 7f38da7fc640 1 -- 172.21.15.47:0/1617402062 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c4032810 con 0x7f38dc15f170 2024-04-26T23:20:30.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.190+0000 7f38e1ac4640 1 -- 172.21.15.47:0/1617402062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38dc15f170 msgr2=0x7f38dc0d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:30.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.190+0000 7f38e1ac4640 1 --2- 172.21.15.47:0/1617402062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38dc15f170 0x7f38dc0d5a80 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f38c40097d0 tx=0x7f38c402a1c0 comp rx=0 tx=0).stop 2024-04-26T23:20:30.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.190+0000 7f38e1ac4640 1 -- 172.21.15.47:0/1617402062 shutdown_connections 2024-04-26T23:20:30.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.190+0000 7f38e1ac4640 1 --2- 172.21.15.47:0/1617402062 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38dc0d3bc0 0x7f38dc0d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:30.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.190+0000 7f38e1ac4640 1 --2- 172.21.15.47:0/1617402062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38dc15f170 0x7f38dc0d5a80 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:30.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.190+0000 7f38e1ac4640 1 -- 172.21.15.47:0/1617402062 >> 172.21.15.47:0/1617402062 conn(0x7f38dc0d4f60 msgr2=0x7f38dc0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:30.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.190+0000 7f38e1ac4640 1 -- 172.21.15.47:0/1617402062 shutdown_connections 2024-04-26T23:20:30.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.190+0000 7f38e1ac4640 1 -- 172.21.15.47:0/1617402062 wait complete. 2024-04-26T23:20:30.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.191+0000 7f38e1ac4640 1 Processor -- start 2024-04-26T23:20:30.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.191+0000 7f38e1ac4640 1 -- start start 2024-04-26T23:20:30.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.192+0000 7f38e1ac4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38dc0d3bc0 0x7f38dc150010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:30.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.192+0000 7f38e1ac4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38dc15f170 0x7f38dc159460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:30.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.192+0000 7f38e1ac4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38dc1599a0 0x7f38dc15b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:30.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.192+0000 7f38e1ac4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f38dc0ca5c0 con 0x7f38dc15f170 2024-04-26T23:20:30.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.192+0000 7f38e1ac4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f38dc0ca440 con 0x7f38dc0d3bc0 2024-04-26T23:20:30.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.192+0000 7f38e1ac4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f38dc0ca740 con 0x7f38dc1599a0 2024-04-26T23:20:30.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.192+0000 7f38daffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38dc15f170 0x7f38dc159460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:30.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.192+0000 7f38daffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38dc15f170 0x7f38dc159460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51354/0 (socket says 172.21.15.47:51354) 2024-04-26T23:20:30.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.192+0000 7f38daffd640 1 -- 172.21.15.47:0/2906730619 learned_addr learned my addr 172.21.15.47:0/2906730619 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:30.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.192+0000 7f38dbfff640 1 --2- 172.21.15.47:0/2906730619 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38dc1599a0 0x7f38dc15b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:30.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.192+0000 7f38db7fe640 1 --2- 172.21.15.47:0/2906730619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38dc0d3bc0 0x7f38dc150010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:30.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.193+0000 7f38daffd640 1 -- 172.21.15.47:0/2906730619 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38dc1599a0 msgr2=0x7f38dc15b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:30.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.193+0000 7f38daffd640 1 --2- 172.21.15.47:0/2906730619 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38dc1599a0 0x7f38dc15b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:30.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.193+0000 7f38daffd640 1 -- 172.21.15.47:0/2906730619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38dc0d3bc0 msgr2=0x7f38dc150010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:30.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.193+0000 7f38daffd640 1 --2- 172.21.15.47:0/2906730619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38dc0d3bc0 0x7f38dc150010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:30.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.193+0000 7f38daffd640 1 -- 172.21.15.47:0/2906730619 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38dc15a1d0 con 0x7f38dc15f170 2024-04-26T23:20:30.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.193+0000 7f38db7fe640 1 --2- 172.21.15.47:0/2906730619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38dc0d3bc0 0x7f38dc150010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:20:30.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.193+0000 7f38daffd640 1 --2- 172.21.15.47:0/2906730619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38dc15f170 0x7f38dc159460 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f38cc00c3e0 tx=0x7f38cc00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:30.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.193+0000 7f38d3fff640 1 -- 172.21.15.47:0/2906730619 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38cc0074e0 con 0x7f38dc15f170 2024-04-26T23:20:30.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.193+0000 7f38d3fff640 1 -- 172.21.15.47:0/2906730619 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38cc0047e0 con 0x7f38dc15f170 2024-04-26T23:20:30.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.193+0000 7f38d3fff640 1 -- 172.21.15.47:0/2906730619 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38cc004bd0 con 0x7f38dc15f170 2024-04-26T23:20:30.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.193+0000 7f38e1ac4640 1 -- 172.21.15.47:0/2906730619 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38dc15a4b0 con 0x7f38dc15f170 2024-04-26T23:20:30.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.193+0000 7f38e1ac4640 1 -- 172.21.15.47:0/2906730619 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38dc15aa70 con 0x7f38dc15f170 2024-04-26T23:20:30.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.194+0000 7f38e1ac4640 1 -- 172.21.15.47:0/2906730619 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38dc0cf1e0 con 0x7f38dc15f170 2024-04-26T23:20:30.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.194+0000 7f38d3fff640 1 -- 172.21.15.47:0/2906730619 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f38cc011070 con 0x7f38dc15f170 2024-04-26T23:20:30.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.195+0000 7f38d3fff640 1 --2- 172.21.15.47:0/2906730619 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f38bc06c8f0 0x7f38bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:30.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.195+0000 7f38d3fff640 1 -- 172.21.15.47:0/2906730619 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f38cc08b810 con 0x7f38dc15f170 2024-04-26T23:20:30.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.195+0000 7f38db7fe640 1 --2- 172.21.15.47:0/2906730619 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f38bc06c8f0 0x7f38bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:30.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.196+0000 7f38db7fe640 1 --2- 172.21.15.47:0/2906730619 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f38bc06c8f0 0x7f38bc06edb0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f38c4034000 tx=0x7f38c402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:30.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.199+0000 7f38d3fff640 1 -- 172.21.15.47:0/2906730619 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f38dc0cf1e0 con 0x7f38dc15f170 2024-04-26T23:20:30.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.391+0000 7f38e1ac4640 1 -- 172.21.15.47:0/2906730619 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f38dc069c20 con 0x7f38dc15f170 2024-04-26T23:20:30.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.391+0000 7f38d3fff640 1 -- 172.21.15.47:0/2906730619 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) ==== 93+0+10 (secure 0 0 0) 0x7f38dc0d1040 con 0x7f38dc15f170 2024-04-26T23:20:30.393 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-26T23:20:30.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.393+0000 7f38e1ac4640 1 -- 172.21.15.47:0/2906730619 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f38bc06c8f0 msgr2=0x7f38bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:30.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.393+0000 7f38e1ac4640 1 --2- 172.21.15.47:0/2906730619 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f38bc06c8f0 0x7f38bc06edb0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f38c4034000 tx=0x7f38c402e000 comp rx=0 tx=0).stop 2024-04-26T23:20:30.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.393+0000 7f38e1ac4640 1 -- 172.21.15.47:0/2906730619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38dc15f170 msgr2=0x7f38dc159460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:30.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.393+0000 7f38e1ac4640 1 --2- 172.21.15.47:0/2906730619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38dc15f170 0x7f38dc159460 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f38cc00c3e0 tx=0x7f38cc00c8a0 comp rx=0 tx=0).stop 2024-04-26T23:20:30.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.393+0000 7f38e1ac4640 1 -- 172.21.15.47:0/2906730619 shutdown_connections 2024-04-26T23:20:30.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.393+0000 7f38e1ac4640 1 --2- 172.21.15.47:0/2906730619 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f38bc06c8f0 0x7f38bc06edb0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:30.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.393+0000 7f38e1ac4640 1 --2- 172.21.15.47:0/2906730619 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38dc1599a0 0x7f38dc15b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:30.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.393+0000 7f38e1ac4640 1 --2- 172.21.15.47:0/2906730619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38dc15f170 0x7f38dc159460 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:30.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.393+0000 7f38e1ac4640 1 --2- 172.21.15.47:0/2906730619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38dc0d3bc0 0x7f38dc150010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:30.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.393+0000 7f38e1ac4640 1 -- 172.21.15.47:0/2906730619 >> 172.21.15.47:0/2906730619 conn(0x7f38dc0d4f60 msgr2=0x7f38dc0d2010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:30.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.394+0000 7f38e1ac4640 1 -- 172.21.15.47:0/2906730619 shutdown_connections 2024-04-26T23:20:30.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.394+0000 7f38e1ac4640 1 -- 172.21.15.47:0/2906730619 wait complete. 2024-04-26T23:20:30.405 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:20:30.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.508+0000 7fc4f1d22640 1 Processor -- start 2024-04-26T23:20:30.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.508+0000 7fc4f1d22640 1 -- start start 2024-04-26T23:20:30.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.509+0000 7fc4f1d22640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc4ec15f550 0x7fc4ec0d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:30.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.509+0000 7fc4f1d22640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4ec0d1040 con 0x7fc4ec0d3bc0 2024-04-26T23:20:30.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.509+0000 7fc4f1d22640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4ec0c9680 con 0x7fc4ec15f170 2024-04-26T23:20:30.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.509+0000 7fc4f1d22640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fc4ec0c9430 con 0x7fc4ec15f550 2024-04-26T23:20:30.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.509+0000 7fc4eaffd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc4ec15f550 0x7fc4ec0d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:30.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.509+0000 7fc4ebfff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc4ec0d3bc0 0x7fc4ec0ca440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37794/0 (socket says 172.21.15.47:37794) 2024-04-26T23:20:30.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.509+0000 7fc4ebfff640 1 -- 172.21.15.47:0/1355577124 learned_addr learned my addr 172.21.15.47:0/1355577124 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:30.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.509+0000 7fc4eaffd640 1 -- 172.21.15.47:0/1355577124 >> v1:172.21.15.203:6789/0 conn(0x7fc4ec15f170 legacy=0x7fc4ec0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:30.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.510+0000 7fc4eaffd640 1 -- 172.21.15.47:0/1355577124 >> v1:172.21.15.47:6789/0 conn(0x7fc4ec0d3bc0 legacy=0x7fc4ec0ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:30.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.510+0000 7fc4ea7fc640 1 -- 172.21.15.47:0/1355577124 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3867798749 0 0) 0x7fc4e4001490 con 0x7fc4ec0d3bc0 2024-04-26T23:20:30.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.510+0000 7fc4eaffd640 1 -- 172.21.15.47:0/1355577124 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4ec0c9960 con 0x7fc4ec15f550 2024-04-26T23:20:30.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.510+0000 7fc4eaffd640 1 --2- 172.21.15.47:0/1355577124 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4ec15f550 0x7fc4ec0d5a80 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec0c95b0 tx=0x7fc4e002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2905e037ddac275 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:30.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.510+0000 7fc4ea7fc640 1 -- 172.21.15.47:0/1355577124 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e0037070 con 0x7fc4ec15f550 2024-04-26T23:20:30.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.510+0000 7fc4ea7fc640 1 -- 172.21.15.47:0/1355577124 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4e0032440 con 0x7fc4ec15f550 2024-04-26T23:20:30.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.510+0000 7fc4ea7fc640 1 -- 172.21.15.47:0/1355577124 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e0032810 con 0x7fc4ec15f550 2024-04-26T23:20:30.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.511+0000 7fc4f1d22640 1 -- 172.21.15.47:0/1355577124 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4ec15f550 msgr2=0x7fc4ec0d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:30.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.511+0000 7fc4f1d22640 1 --2- 172.21.15.47:0/1355577124 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4ec15f550 0x7fc4ec0d5a80 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec0c95b0 tx=0x7fc4e002a250 comp rx=0 tx=0).stop 2024-04-26T23:20:30.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.511+0000 7fc4f1d22640 1 -- 172.21.15.47:0/1355577124 shutdown_connections 2024-04-26T23:20:30.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.511+0000 7fc4f1d22640 1 --2- 172.21.15.47:0/1355577124 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4ec15f550 0x7fc4ec0d5a80 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:30.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.511+0000 7fc4f1d22640 1 -- 172.21.15.47:0/1355577124 >> 172.21.15.47:0/1355577124 conn(0x7fc4ec0d4f60 msgr2=0x7fc4ec0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:30.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.511+0000 7fc4f1d22640 1 -- 172.21.15.47:0/1355577124 shutdown_connections 2024-04-26T23:20:30.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.511+0000 7fc4f1d22640 1 -- 172.21.15.47:0/1355577124 wait complete. 2024-04-26T23:20:30.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.512+0000 7fc4f1d22640 1 Processor -- start 2024-04-26T23:20:30.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.512+0000 7fc4f1d22640 1 -- start start 2024-04-26T23:20:30.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.513+0000 7fc4f1d22640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4ec0d3bc0 0x7fc4ec15b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:30.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.513+0000 7fc4f1d22640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4ec15f170 0x7fc4ec0b9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:30.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.513+0000 7fc4f1d22640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4ec0b7b80 0x7fc4ec0ce170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:30.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.513+0000 7fc4f1d22640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ec0d2e30 con 0x7fc4ec0b7b80 2024-04-26T23:20:30.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.513+0000 7fc4f1d22640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ec0d2cb0 con 0x7fc4ec0d3bc0 2024-04-26T23:20:30.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.513+0000 7fc4f1d22640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc4ec0d2fb0 con 0x7fc4ec15f170 2024-04-26T23:20:30.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.513+0000 7fc4ebfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4ec0b7b80 0x7fc4ec0ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:30.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.513+0000 7fc4ebfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4ec0b7b80 0x7fc4ec0ce170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51382/0 (socket says 172.21.15.47:51382) 2024-04-26T23:20:30.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.513+0000 7fc4ebfff640 1 -- 172.21.15.47:0/442718101 learned_addr learned my addr 172.21.15.47:0/442718101 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:30.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.513+0000 7fc4eb7fe640 1 --2- 172.21.15.47:0/442718101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4ec0d3bc0 0x7fc4ec15b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:30.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.513+0000 7fc4eaffd640 1 --2- 172.21.15.47:0/442718101 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4ec15f170 0x7fc4ec0b9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:30.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.513+0000 7fc4ebfff640 1 -- 172.21.15.47:0/442718101 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4ec15f170 msgr2=0x7fc4ec0b9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:30.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.513+0000 7fc4ebfff640 1 --2- 172.21.15.47:0/442718101 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4ec15f170 0x7fc4ec0b9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:30.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.513+0000 7fc4ebfff640 1 -- 172.21.15.47:0/442718101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4ec0d3bc0 msgr2=0x7fc4ec15b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:30.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.513+0000 7fc4ebfff640 1 --2- 172.21.15.47:0/442718101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4ec0d3bc0 0x7fc4ec15b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:30.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.513+0000 7fc4ebfff640 1 -- 172.21.15.47:0/442718101 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4ec0b9600 con 0x7fc4ec0b7b80 2024-04-26T23:20:30.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.514+0000 7fc4ebfff640 1 --2- 172.21.15.47:0/442718101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4ec0b7b80 0x7fc4ec0ce170 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fc4e400c7f0 tx=0x7fc4e400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:30.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.514+0000 7fc4d3fff640 1 -- 172.21.15.47:0/442718101 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e4018070 con 0x7fc4ec0b7b80 2024-04-26T23:20:30.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.514+0000 7fc4d3fff640 1 -- 172.21.15.47:0/442718101 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4e400a5e0 con 0x7fc4ec0b7b80 2024-04-26T23:20:30.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.514+0000 7fc4d3fff640 1 -- 172.21.15.47:0/442718101 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e400aa10 con 0x7fc4ec0b7b80 2024-04-26T23:20:30.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.514+0000 7fc4f1d22640 1 -- 172.21.15.47:0/442718101 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4ec0b8080 con 0x7fc4ec0b7b80 2024-04-26T23:20:30.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.514+0000 7fc4f1d22640 1 -- 172.21.15.47:0/442718101 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4ec0b8640 con 0x7fc4ec0b7b80 2024-04-26T23:20:30.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.515+0000 7fc4f1d22640 1 -- 172.21.15.47:0/442718101 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4ec0694f0 con 0x7fc4ec0b7b80 2024-04-26T23:20:30.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.515+0000 7fc4d3fff640 1 -- 172.21.15.47:0/442718101 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc4e400abb0 con 0x7fc4ec0b7b80 2024-04-26T23:20:30.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.516+0000 7fc4d3fff640 1 --2- 172.21.15.47:0/442718101 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fc4d806c8f0 0x7fc4d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:30.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.516+0000 7fc4d3fff640 1 -- 172.21.15.47:0/442718101 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fc4e408b370 con 0x7fc4ec0b7b80 2024-04-26T23:20:30.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.516+0000 7fc4eb7fe640 1 --2- 172.21.15.47:0/442718101 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fc4d806c8f0 0x7fc4d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:30.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.517+0000 7fc4eb7fe640 1 --2- 172.21.15.47:0/442718101 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fc4d806c8f0 0x7fc4d806edb0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc000f60 tx=0x7fc4dc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:30.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.520+0000 7fc4d3fff640 1 -- 172.21.15.47:0/442718101 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc4ec0b9600 con 0x7fc4ec0b7b80 2024-04-26T23:20:30.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.704+0000 7fc4f1d22640 1 -- 172.21.15.47:0/442718101 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc4ec0d2760 con 0x7fc4ec0b7b80 2024-04-26T23:20:30.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.705+0000 7fc4d3fff640 1 -- 172.21.15.47:0/442718101 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v54) ==== 104+0+11 (secure 0 0 0) 0x7fc4ec0d1040 con 0x7fc4ec0b7b80 2024-04-26T23:20:30.706 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-26T23:20:30.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.707+0000 7fc4f1d22640 1 -- 172.21.15.47:0/442718101 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fc4d806c8f0 msgr2=0x7fc4d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:30.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.707+0000 7fc4f1d22640 1 --2- 172.21.15.47:0/442718101 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fc4d806c8f0 0x7fc4d806edb0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc000f60 tx=0x7fc4dc009040 comp rx=0 tx=0).stop 2024-04-26T23:20:30.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.707+0000 7fc4f1d22640 1 -- 172.21.15.47:0/442718101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4ec0b7b80 msgr2=0x7fc4ec0ce170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:30.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.707+0000 7fc4f1d22640 1 --2- 172.21.15.47:0/442718101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4ec0b7b80 0x7fc4ec0ce170 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fc4e400c7f0 tx=0x7fc4e400ccb0 comp rx=0 tx=0).stop 2024-04-26T23:20:30.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.707+0000 7fc4f1d22640 1 -- 172.21.15.47:0/442718101 shutdown_connections 2024-04-26T23:20:30.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.707+0000 7fc4f1d22640 1 --2- 172.21.15.47:0/442718101 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fc4d806c8f0 0x7fc4d806edb0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:30.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.707+0000 7fc4f1d22640 1 --2- 172.21.15.47:0/442718101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4ec0b7b80 0x7fc4ec0ce170 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:30.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.707+0000 7fc4f1d22640 1 --2- 172.21.15.47:0/442718101 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4ec15f170 0x7fc4ec0b9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:30.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.707+0000 7fc4f1d22640 1 --2- 172.21.15.47:0/442718101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4ec0d3bc0 0x7fc4ec15b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:30.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.707+0000 7fc4f1d22640 1 -- 172.21.15.47:0/442718101 >> 172.21.15.47:0/442718101 conn(0x7fc4ec0d4f60 msgr2=0x7fc4ec0d1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:30.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.707+0000 7fc4f1d22640 1 -- 172.21.15.47:0/442718101 shutdown_connections 2024-04-26T23:20:30.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.707+0000 7fc4f1d22640 1 -- 172.21.15.47:0/442718101 wait complete. 2024-04-26T23:20:30.719 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:20:30.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.822+0000 7f890fc9c640 1 Processor -- start 2024-04-26T23:20:30.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.822+0000 7f890fc9c640 1 -- start start 2024-04-26T23:20:30.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.823+0000 7f890fc9c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f89080cf5e0 0x7f89080ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:30.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.823+0000 7f890fc9c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89080cbbd0 0x7f89080cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:30.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.823+0000 7f890fc9c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f890815b9e0 con 0x7f890815b600 2024-04-26T23:20:30.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.823+0000 7f890fc9c640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f89080cb9b0 con 0x7f89080cf5e0 2024-04-26T23:20:30.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.823+0000 7f890fc9c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f89080d0740 con 0x7f89080cbbd0 2024-04-26T23:20:30.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.823+0000 7f890d237640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f89080cf5e0 0x7f89080ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:30.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.823+0000 7f890d237640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f89080cf5e0 0x7f89080ce9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51392/0 (socket says 172.21.15.47:51392) 2024-04-26T23:20:30.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.823+0000 7f890d237640 1 -- 172.21.15.47:0/179540438 learned_addr learned my addr 172.21.15.47:0/179540438 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:30.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.823+0000 7f890e239640 1 --2- 172.21.15.47:0/179540438 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89080cbbd0 0x7f89080cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:30.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.823+0000 7f890d237640 1 -- 172.21.15.47:0/179540438 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89080cbbd0 msgr2=0x7f89080cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:30.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.823+0000 7f890d237640 1 --2- 172.21.15.47:0/179540438 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89080cbbd0 0x7f89080cef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:30.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.823+0000 7f890d237640 1 -- 172.21.15.47:0/179540438 >> v1:172.21.15.203:6789/0 conn(0x7f890815b600 legacy=0x7f8908071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:30.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.823+0000 7f890d237640 1 -- 172.21.15.47:0/179540438 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89080cb360 con 0x7f89080cf5e0 2024-04-26T23:20:30.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.824+0000 7f890d237640 1 --2- 172.21.15.47:0/179540438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89080cf5e0 0x7f89080ce9d0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f88fc009870 tx=0x7f88fc02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a8bd4f2fa65ad30 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:30.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.824+0000 7f890ca36640 1 -- 172.21.15.47:0/179540438 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88fc037070 con 0x7f89080cf5e0 2024-04-26T23:20:30.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.824+0000 7f890ca36640 1 -- 172.21.15.47:0/179540438 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88fc032440 con 0x7f89080cf5e0 2024-04-26T23:20:30.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.824+0000 7f890ca36640 1 -- 172.21.15.47:0/179540438 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88fc032810 con 0x7f89080cf5e0 2024-04-26T23:20:30.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.824+0000 7f890fc9c640 1 -- 172.21.15.47:0/179540438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89080cf5e0 msgr2=0x7f89080ce9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:30.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.824+0000 7f890fc9c640 1 --2- 172.21.15.47:0/179540438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89080cf5e0 0x7f89080ce9d0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f88fc009870 tx=0x7f88fc02a210 comp rx=0 tx=0).stop 2024-04-26T23:20:30.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.824+0000 7f890fc9c640 1 -- 172.21.15.47:0/179540438 shutdown_connections 2024-04-26T23:20:30.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.824+0000 7f890fc9c640 1 --2- 172.21.15.47:0/179540438 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89080cbbd0 0x7f89080cef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:30.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.824+0000 7f890fc9c640 1 --2- 172.21.15.47:0/179540438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89080cf5e0 0x7f89080ce9d0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:30.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.824+0000 7f890fc9c640 1 -- 172.21.15.47:0/179540438 >> 172.21.15.47:0/179540438 conn(0x7f89080d0be0 msgr2=0x7f89080d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:30.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.825+0000 7f890fc9c640 1 -- 172.21.15.47:0/179540438 shutdown_connections 2024-04-26T23:20:30.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.825+0000 7f890fc9c640 1 -- 172.21.15.47:0/179540438 wait complete. 2024-04-26T23:20:30.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.826+0000 7f890fc9c640 1 Processor -- start 2024-04-26T23:20:30.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.826+0000 7f890fc9c640 1 -- start start 2024-04-26T23:20:30.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.826+0000 7f890fc9c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89080cbbd0 0x7f8908169df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:30.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.826+0000 7f890fc9c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89080cf5e0 0x7f89080cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:30.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.826+0000 7f890fc9c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f890815b600 0x7f89080b3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:30.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.826+0000 7f890fc9c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f89080d0470 con 0x7f89080cbbd0 2024-04-26T23:20:30.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.826+0000 7f890fc9c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f89080d02f0 con 0x7f89080cf5e0 2024-04-26T23:20:30.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.826+0000 7f890fc9c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f890806ddf0 con 0x7f890815b600 2024-04-26T23:20:30.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.827+0000 7f890da38640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89080cbbd0 0x7f8908169df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:30.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.827+0000 7f890da38640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89080cbbd0 0x7f8908169df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51406/0 (socket says 172.21.15.47:51406) 2024-04-26T23:20:30.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.827+0000 7f890da38640 1 -- 172.21.15.47:0/4227097051 learned_addr learned my addr 172.21.15.47:0/4227097051 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:30.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.827+0000 7f890e239640 1 --2- 172.21.15.47:0/4227097051 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f890815b600 0x7f89080b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:30.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.827+0000 7f890d237640 1 --2- 172.21.15.47:0/4227097051 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89080cf5e0 0x7f89080cddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:30.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.827+0000 7f890da38640 1 -- 172.21.15.47:0/4227097051 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f890815b600 msgr2=0x7f89080b3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:30.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.827+0000 7f890da38640 1 --2- 172.21.15.47:0/4227097051 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f890815b600 0x7f89080b3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:30.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.827+0000 7f890da38640 1 -- 172.21.15.47:0/4227097051 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89080cf5e0 msgr2=0x7f89080cddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:30.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.827+0000 7f890da38640 1 --2- 172.21.15.47:0/4227097051 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89080cf5e0 0x7f89080cddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:30.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.827+0000 7f890da38640 1 -- 172.21.15.47:0/4227097051 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89080b4d80 con 0x7f89080cbbd0 2024-04-26T23:20:30.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.827+0000 7f890da38640 1 --2- 172.21.15.47:0/4227097051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89080cbbd0 0x7f8908169df0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f890000ba50 tx=0x7f890000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:30.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.827+0000 7f89067fc640 1 -- 172.21.15.47:0/4227097051 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8900019070 con 0x7f89080cbbd0 2024-04-26T23:20:30.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.827+0000 7f89067fc640 1 -- 172.21.15.47:0/4227097051 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f890000cc20 con 0x7f89080cbbd0 2024-04-26T23:20:30.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.827+0000 7f890fc9c640 1 -- 172.21.15.47:0/4227097051 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89080b5060 con 0x7f89080cbbd0 2024-04-26T23:20:30.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.827+0000 7f89067fc640 1 -- 172.21.15.47:0/4227097051 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8900014680 con 0x7f89080cbbd0 2024-04-26T23:20:30.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.827+0000 7f890fc9c640 1 -- 172.21.15.47:0/4227097051 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89080b5620 con 0x7f89080cbbd0 2024-04-26T23:20:30.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.828+0000 7f890fc9c640 1 -- 172.21.15.47:0/4227097051 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89080694f0 con 0x7f89080cbbd0 2024-04-26T23:20:30.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.829+0000 7f89067fc640 1 -- 172.21.15.47:0/4227097051 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f890001c070 con 0x7f89080cbbd0 2024-04-26T23:20:30.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.829+0000 7f89067fc640 1 --2- 172.21.15.47:0/4227097051 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f88f406c920 0x7f88f406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:30.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.829+0000 7f89067fc640 1 -- 172.21.15.47:0/4227097051 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f890008b9c0 con 0x7f89080cbbd0 2024-04-26T23:20:30.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.830+0000 7f890d237640 1 --2- 172.21.15.47:0/4227097051 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f88f406c920 0x7f88f406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:30.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.830+0000 7f890d237640 1 --2- 172.21.15.47:0/4227097051 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f88f406c920 0x7f88f406ede0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f88fc034000 tx=0x7f88fc02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:30.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:30.834+0000 7f89067fc640 1 -- 172.21.15.47:0/4227097051 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8900059600 con 0x7f89080cbbd0 2024-04-26T23:20:31.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.020+0000 7f890fc9c640 1 -- 172.21.15.47:0/4227097051 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f89080ce7b0 con 0x7f89080cbbd0 2024-04-26T23:20:31.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.020+0000 7f89067fc640 1 -- 172.21.15.47:0/4227097051 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v54) ==== 100+0+11 (secure 0 0 0) 0x7f890005d900 con 0x7f89080cbbd0 2024-04-26T23:20:31.021 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-26T23:20:31.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.022+0000 7f890fc9c640 1 -- 172.21.15.47:0/4227097051 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f88f406c920 msgr2=0x7f88f406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:31.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.022+0000 7f890fc9c640 1 --2- 172.21.15.47:0/4227097051 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f88f406c920 0x7f88f406ede0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f88fc034000 tx=0x7f88fc02e000 comp rx=0 tx=0).stop 2024-04-26T23:20:31.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.022+0000 7f890fc9c640 1 -- 172.21.15.47:0/4227097051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89080cbbd0 msgr2=0x7f8908169df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:31.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.022+0000 7f890fc9c640 1 --2- 172.21.15.47:0/4227097051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89080cbbd0 0x7f8908169df0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f890000ba50 tx=0x7f890000bf10 comp rx=0 tx=0).stop 2024-04-26T23:20:31.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.022+0000 7f890fc9c640 1 -- 172.21.15.47:0/4227097051 shutdown_connections 2024-04-26T23:20:31.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.022+0000 7f890fc9c640 1 --2- 172.21.15.47:0/4227097051 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f88f406c920 0x7f88f406ede0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:31.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.022+0000 7f890fc9c640 1 --2- 172.21.15.47:0/4227097051 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f890815b600 0x7f89080b3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:31.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.022+0000 7f890fc9c640 1 --2- 172.21.15.47:0/4227097051 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89080cf5e0 0x7f89080cddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:31.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.022+0000 7f890fc9c640 1 --2- 172.21.15.47:0/4227097051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89080cbbd0 0x7f8908169df0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:31.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.022+0000 7f890fc9c640 1 -- 172.21.15.47:0/4227097051 >> 172.21.15.47:0/4227097051 conn(0x7f89080d0be0 msgr2=0x7f89080cf9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:31.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.022+0000 7f890fc9c640 1 -- 172.21.15.47:0/4227097051 shutdown_connections 2024-04-26T23:20:31.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.022+0000 7f890fc9c640 1 -- 172.21.15.47:0/4227097051 wait complete. 2024-04-26T23:20:31.034 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T23:20:31.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.134+0000 7f1ecd50a640 1 Processor -- start 2024-04-26T23:20:31.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.134+0000 7f1ecd50a640 1 -- start start 2024-04-26T23:20:31.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.135+0000 7f1ecd50a640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1ec80b4d40 0x7f1ec80b6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:31.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.135+0000 7f1ecd50a640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ec8072fb0 con 0x7f1ec80b6c50 2024-04-26T23:20:31.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.135+0000 7f1ecd50a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ec8073290 con 0x7f1ec80b0b70 2024-04-26T23:20:31.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.135+0000 7f1ecd50a640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f1ec8073bc0 con 0x7f1ec80b4d40 2024-04-26T23:20:31.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.135+0000 7f1ec67fc640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1ec80b4d40 0x7f1ec80b6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:31.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.135+0000 7f1ec77fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1ec80b6c50 0x7f1ec80aed90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37800/0 (socket says 172.21.15.47:37800) 2024-04-26T23:20:31.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.135+0000 7f1ec77fe640 1 -- 172.21.15.47:0/1959051041 learned_addr learned my addr 172.21.15.47:0/1959051041 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:31.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.136+0000 7f1ec67fc640 1 -- 172.21.15.47:0/1959051041 >> v1:172.21.15.203:6789/0 conn(0x7f1ec80b0b70 legacy=0x7f1ec80b0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:31.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.136+0000 7f1ec67fc640 1 -- 172.21.15.47:0/1959051041 >> v1:172.21.15.47:6789/0 conn(0x7f1ec80b6c50 legacy=0x7f1ec80aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:31.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.136+0000 7f1ec67fc640 1 -- 172.21.15.47:0/1959051041 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ec8073570 con 0x7f1ec80b4d40 2024-04-26T23:20:31.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.136+0000 7f1ec67fc640 1 --2- 172.21.15.47:0/1959051041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ec80b4d40 0x7f1ec80b6710 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc0097a0 tx=0x7f1ebc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce707feb3dfd9f2c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:31.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.137+0000 7f1ec5ffb640 1 -- 172.21.15.47:0/1959051041 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc037070 con 0x7f1ec80b4d40 2024-04-26T23:20:31.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.137+0000 7f1ec5ffb640 1 -- 172.21.15.47:0/1959051041 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ebc032440 con 0x7f1ec80b4d40 2024-04-26T23:20:31.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.137+0000 7f1ec5ffb640 1 -- 172.21.15.47:0/1959051041 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc032810 con 0x7f1ec80b4d40 2024-04-26T23:20:31.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.137+0000 7f1ecd50a640 1 -- 172.21.15.47:0/1959051041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ec80b4d40 msgr2=0x7f1ec80b6710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:31.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.137+0000 7f1ecd50a640 1 --2- 172.21.15.47:0/1959051041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ec80b4d40 0x7f1ec80b6710 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc0097a0 tx=0x7f1ebc02a1e0 comp rx=0 tx=0).stop 2024-04-26T23:20:31.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.137+0000 7f1ecd50a640 1 -- 172.21.15.47:0/1959051041 shutdown_connections 2024-04-26T23:20:31.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.137+0000 7f1ecd50a640 1 --2- 172.21.15.47:0/1959051041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ec80b4d40 0x7f1ec80b6710 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:31.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.137+0000 7f1ecd50a640 1 -- 172.21.15.47:0/1959051041 >> 172.21.15.47:0/1959051041 conn(0x7f1ec80b39b0 msgr2=0x7f1ec80b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:31.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.137+0000 7f1ecd50a640 1 -- 172.21.15.47:0/1959051041 shutdown_connections 2024-04-26T23:20:31.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.137+0000 7f1ecd50a640 1 -- 172.21.15.47:0/1959051041 wait complete. 2024-04-26T23:20:31.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.138+0000 7f1ecd50a640 1 Processor -- start 2024-04-26T23:20:31.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.139+0000 7f1ecd50a640 1 -- start start 2024-04-26T23:20:31.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.139+0000 7f1ecd50a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ec80b0b70 0x7f1ec81f9090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:31.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.139+0000 7f1ecd50a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ec80b4d40 0x7f1ec80d19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:31.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.139+0000 7f1ecd50a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ec80b6c50 0x7f1ec814ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:31.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.139+0000 7f1ecd50a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec81604e0 con 0x7f1ec80b0b70 2024-04-26T23:20:31.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.139+0000 7f1ecd50a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec8160360 con 0x7f1ec80b6c50 2024-04-26T23:20:31.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.139+0000 7f1ecd50a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f1ec80af4a0 con 0x7f1ec80b4d40 2024-04-26T23:20:31.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.139+0000 7f1ec6ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ec80b0b70 0x7f1ec81f9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:31.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.139+0000 7f1ec6ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ec80b0b70 0x7f1ec81f9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51438/0 (socket says 172.21.15.47:51438) 2024-04-26T23:20:31.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.139+0000 7f1ec6ffd640 1 -- 172.21.15.47:0/543943947 learned_addr learned my addr 172.21.15.47:0/543943947 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:31.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.140+0000 7f1ec77fe640 1 --2- 172.21.15.47:0/543943947 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ec80b6c50 0x7f1ec814ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:31.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.140+0000 7f1ec67fc640 1 --2- 172.21.15.47:0/543943947 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ec80b4d40 0x7f1ec80d19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:31.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.140+0000 7f1ec6ffd640 1 -- 172.21.15.47:0/543943947 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ec80b4d40 msgr2=0x7f1ec80d19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:31.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.140+0000 7f1ec6ffd640 1 --2- 172.21.15.47:0/543943947 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ec80b4d40 0x7f1ec80d19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:31.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.140+0000 7f1ec6ffd640 1 -- 172.21.15.47:0/543943947 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ec80b6c50 msgr2=0x7f1ec814ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:31.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.140+0000 7f1ec6ffd640 1 --2- 172.21.15.47:0/543943947 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ec80b6c50 0x7f1ec814ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:31.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.140+0000 7f1ec6ffd640 1 -- 172.21.15.47:0/543943947 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ec80d2200 con 0x7f1ec80b0b70 2024-04-26T23:20:31.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.140+0000 7f1ec67fc640 1 --2- 172.21.15.47:0/543943947 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ec80b4d40 0x7f1ec80d19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:31.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.140+0000 7f1ec77fe640 1 --2- 172.21.15.47:0/543943947 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ec80b6c50 0x7f1ec814ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:31.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.140+0000 7f1ec6ffd640 1 --2- 172.21.15.47:0/543943947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ec80b0b70 0x7f1ec81f9090 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f1ec000c7c0 tx=0x7f1ec000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:31.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.140+0000 7f1eaf7fe640 1 -- 172.21.15.47:0/543943947 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec0016020 con 0x7f1ec80b0b70 2024-04-26T23:20:31.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.140+0000 7f1eaf7fe640 1 -- 172.21.15.47:0/543943947 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ec000aa50 con 0x7f1ec80b0b70 2024-04-26T23:20:31.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.140+0000 7f1ecd50a640 1 -- 172.21.15.47:0/543943947 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ec814c320 con 0x7f1ec80b0b70 2024-04-26T23:20:31.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.140+0000 7f1eaf7fe640 1 -- 172.21.15.47:0/543943947 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec0012420 con 0x7f1ec80b0b70 2024-04-26T23:20:31.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.140+0000 7f1ecd50a640 1 -- 172.21.15.47:0/543943947 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ec814b840 con 0x7f1ec80b0b70 2024-04-26T23:20:31.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.141+0000 7f1ecd50a640 1 -- 172.21.15.47:0/543943947 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ec80694f0 con 0x7f1ec80b0b70 2024-04-26T23:20:31.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.142+0000 7f1eaf7fe640 1 -- 172.21.15.47:0/543943947 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1ec0019020 con 0x7f1ec80b0b70 2024-04-26T23:20:31.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.142+0000 7f1eaf7fe640 1 --2- 172.21.15.47:0/543943947 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1eb406c8f0 0x7f1eb406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:31.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.142+0000 7f1eaf7fe640 1 -- 172.21.15.47:0/543943947 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f1ec008ba50 con 0x7f1ec80b0b70 2024-04-26T23:20:31.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.142+0000 7f1ec67fc640 1 --2- 172.21.15.47:0/543943947 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1eb406c8f0 0x7f1eb406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:31.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.143+0000 7f1ec67fc640 1 --2- 172.21.15.47:0/543943947 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1eb406c8f0 0x7f1eb406edb0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc034000 tx=0x7f1ebc02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:31.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.147+0000 7f1eaf7fe640 1 -- 172.21.15.47:0/543943947 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1ec0020070 con 0x7f1ec80b0b70 2024-04-26T23:20:31.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.342+0000 7f1ecd50a640 1 -- 172.21.15.47:0/543943947 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1ec80d1ef0 con 0x7f1ec80b0b70 2024-04-26T23:20:31.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.343+0000 7f1eaf7fe640 1 -- 172.21.15.47:0/543943947 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v54) ==== 80+0+0 (secure 0 0 0) 0x7f1ec0059cf0 con 0x7f1ec80b0b70 2024-04-26T23:20:31.481 INFO:teuthology.orchestra.run.smithi047.stderr:listed 0 entries 2024-04-26T23:20:31.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.345+0000 7f1ecd50a640 1 -- 172.21.15.47:0/543943947 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1eb406c8f0 msgr2=0x7f1eb406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:31.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.345+0000 7f1ecd50a640 1 --2- 172.21.15.47:0/543943947 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1eb406c8f0 0x7f1eb406edb0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc034000 tx=0x7f1ebc02e000 comp rx=0 tx=0).stop 2024-04-26T23:20:31.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.345+0000 7f1ecd50a640 1 -- 172.21.15.47:0/543943947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ec80b0b70 msgr2=0x7f1ec81f9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:31.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.345+0000 7f1ecd50a640 1 --2- 172.21.15.47:0/543943947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ec80b0b70 0x7f1ec81f9090 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f1ec000c7c0 tx=0x7f1ec000cc80 comp rx=0 tx=0).stop 2024-04-26T23:20:31.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.345+0000 7f1ecd50a640 1 -- 172.21.15.47:0/543943947 shutdown_connections 2024-04-26T23:20:31.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.345+0000 7f1ecd50a640 1 --2- 172.21.15.47:0/543943947 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1eb406c8f0 0x7f1eb406edb0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:31.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.345+0000 7f1ecd50a640 1 --2- 172.21.15.47:0/543943947 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ec80b6c50 0x7f1ec814ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:31.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.345+0000 7f1ecd50a640 1 --2- 172.21.15.47:0/543943947 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ec80b4d40 0x7f1ec80d19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:31.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.345+0000 7f1ecd50a640 1 --2- 172.21.15.47:0/543943947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ec80b0b70 0x7f1ec81f9090 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:31.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.345+0000 7f1ecd50a640 1 -- 172.21.15.47:0/543943947 >> 172.21.15.47:0/543943947 conn(0x7f1ec80b39b0 msgr2=0x7f1ec80b28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:31.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.345+0000 7f1ecd50a640 1 -- 172.21.15.47:0/543943947 shutdown_connections 2024-04-26T23:20:31.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.345+0000 7f1ecd50a640 1 -- 172.21.15.47:0/543943947 wait complete. 2024-04-26T23:20:31.484 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T23:20:31.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:20:31.484 DEBUG:teuthology.orchestra.run.smithi047:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T23:20:31.515 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi047.front.sepia.ceph.com 2024-04-26T23:20:31.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:20:31.516 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T23:20:31.573 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T23:20:31.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.712+0000 7fb856c69640 1 Processor -- start 2024-04-26T23:20:31.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.712+0000 7fb856c69640 1 -- start start 2024-04-26T23:20:31.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.713+0000 7fb856c69640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8500b2340 0x7fb8500cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:31.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.713+0000 7fb856c69640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8500d10d0 con 0x7fb8500cd930 2024-04-26T23:20:31.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.713+0000 7fb856c69640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8500d1370 con 0x7fb850162810 2024-04-26T23:20:31.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.713+0000 7fb856c69640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb8500d0750 con 0x7fb8500b2340 2024-04-26T23:20:31.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.713+0000 7fb855206640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb8500cd930 0x7fb8500d1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37810/0 (socket says 172.21.15.47:37810) 2024-04-26T23:20:31.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.713+0000 7fb855206640 1 -- 172.21.15.47:0/2833890961 learned_addr learned my addr 172.21.15.47:0/2833890961 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:31.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.713+0000 7fb84ffff640 1 --2- 172.21.15.47:0/2833890961 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8500b2340 0x7fb8500cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:31.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.713+0000 7fb84f7fe640 1 -- 172.21.15.47:0/2833890961 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 431438331 0 0) 0x7fb8500d10d0 con 0x7fb8500cd930 2024-04-26T23:20:31.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.713+0000 7fb84f7fe640 1 -- 172.21.15.47:0/2833890961 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb838003520 con 0x7fb8500cd930 2024-04-26T23:20:31.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.714+0000 7fb84f7fe640 1 -- 172.21.15.47:0/2833890961 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3540612521 0 0) 0x7fb8500d1370 con 0x7fb850162810 2024-04-26T23:20:31.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.714+0000 7fb84f7fe640 1 -- 172.21.15.47:0/2833890961 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb8500d10d0 con 0x7fb850162810 2024-04-26T23:20:31.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.714+0000 7fb84f7fe640 1 -- 172.21.15.47:0/2833890961 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2539239661 0 0) 0x7fb838003520 con 0x7fb8500cd930 2024-04-26T23:20:31.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.714+0000 7fb84f7fe640 1 -- 172.21.15.47:0/2833890961 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8500b2340 msgr2=0x7fb8500cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:31.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.714+0000 7fb84f7fe640 1 --2- 172.21.15.47:0/2833890961 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8500b2340 0x7fb8500cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:31.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.714+0000 7fb84f7fe640 1 -- 172.21.15.47:0/2833890961 >> v1:172.21.15.203:6789/0 conn(0x7fb850162810 legacy=0x7fb850070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:31.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.714+0000 7fb84f7fe640 1 -- 172.21.15.47:0/2833890961 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8500d01b0 con 0x7fb8500cd930 2024-04-26T23:20:31.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.714+0000 7fb84f7fe640 1 -- 172.21.15.47:0/2833890961 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7fb8480019f0 con 0x7fb8500cd930 2024-04-26T23:20:31.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.714+0000 7fb84f7fe640 1 -- 172.21.15.47:0/2833890961 >> v1:172.21.15.47:6789/0 conn(0x7fb8500cd930 legacy=0x7fb8500d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:31.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.714+0000 7fb84f7fe640 1 --2- 172.21.15.47:0/2833890961 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb838003b70 0x7fb838003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:31.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.714+0000 7fb84f7fe640 1 --2- 172.21.15.47:0/2833890961 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb838004800 0x7fb838004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:31.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.714+0000 7fb84f7fe640 1 --2- 172.21.15.47:0/2833890961 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8500b2340 0x7fb8380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:31.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.714+0000 7fb84f7fe640 1 -- 172.21.15.47:0/2833890961 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb8500d0750 con 0x7fb838003b70 2024-04-26T23:20:31.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.714+0000 7fb84f7fe640 1 -- 172.21.15.47:0/2833890961 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb83800ba70 con 0x7fb838004800 2024-04-26T23:20:31.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.714+0000 7fb84f7fe640 1 -- 172.21.15.47:0/2833890961 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb83800bc70 con 0x7fb8500b2340 2024-04-26T23:20:31.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.714+0000 7fb854a05640 1 --2- 172.21.15.47:0/2833890961 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb838003b70 0x7fb838003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:31.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.714+0000 7fb855206640 1 --2- 172.21.15.47:0/2833890961 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8500b2340 0x7fb8380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:31.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.714+0000 7fb84ffff640 1 --2- 172.21.15.47:0/2833890961 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb838004800 0x7fb838004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:31.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.714+0000 7fb854a05640 1 -- 172.21.15.47:0/2833890961 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8500b2340 msgr2=0x7fb8380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:31.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.714+0000 7fb854a05640 1 --2- 172.21.15.47:0/2833890961 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8500b2340 0x7fb8380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:31.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.714+0000 7fb854a05640 1 -- 172.21.15.47:0/2833890961 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb838004800 msgr2=0x7fb838004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:31.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.714+0000 7fb854a05640 1 --2- 172.21.15.47:0/2833890961 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb838004800 0x7fb838004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:31.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.714+0000 7fb854a05640 1 -- 172.21.15.47:0/2833890961 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb838003520 con 0x7fb838003b70 2024-04-26T23:20:31.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.715+0000 7fb854a05640 1 --2- 172.21.15.47:0/2833890961 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb838003b70 0x7fb838003f90 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fb8400018a0 tx=0x7fb838005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce9e2a49a95f3a2c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:31.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.715+0000 7fb84f7fe640 1 -- 172.21.15.47:0/2833890961 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8400014f0 con 0x7fb838003b70 2024-04-26T23:20:31.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.715+0000 7fb84f7fe640 1 -- 172.21.15.47:0/2833890961 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8400052d0 con 0x7fb838003b70 2024-04-26T23:20:31.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.715+0000 7fb84f7fe640 1 -- 172.21.15.47:0/2833890961 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84002ac40 con 0x7fb838003b70 2024-04-26T23:20:31.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.716+0000 7fb856c69640 1 -- 172.21.15.47:0/2833890961 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb838003b70 msgr2=0x7fb838003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:31.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.716+0000 7fb856c69640 1 --2- 172.21.15.47:0/2833890961 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb838003b70 0x7fb838003f90 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fb8400018a0 tx=0x7fb838005830 comp rx=0 tx=0).stop 2024-04-26T23:20:31.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.716+0000 7fb855206640 1 -- 172.21.15.47:0/2833890961 reap_dead start 2024-04-26T23:20:31.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.716+0000 7fb856c69640 1 -- 172.21.15.47:0/2833890961 shutdown_connections 2024-04-26T23:20:31.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.716+0000 7fb856c69640 1 -- 172.21.15.47:0/2833890961 >> 172.21.15.47:0/2833890961 conn(0x7fb8500d2da0 msgr2=0x7fb8500d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:31.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.716+0000 7fb856c69640 1 -- 172.21.15.47:0/2833890961 shutdown_connections 2024-04-26T23:20:31.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.716+0000 7fb856c69640 1 -- 172.21.15.47:0/2833890961 wait complete. 2024-04-26T23:20:31.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.717+0000 7fb856c69640 1 Processor -- start 2024-04-26T23:20:31.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.717+0000 7fb856c69640 1 -- start start 2024-04-26T23:20:31.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.718+0000 7fb856c69640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb838003b70 0x7fb8501522c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:31.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.718+0000 7fb856c69640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb85006ea70 0x7fb85006eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:31.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.718+0000 7fb856c69640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8500c8720 0x7fb8500c8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:31.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.718+0000 7fb856c69640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb8500d2240 con 0x7fb8500c8720 2024-04-26T23:20:31.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.718+0000 7fb856c69640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb8500d16a0 con 0x7fb85006ea70 2024-04-26T23:20:31.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.718+0000 7fb856c69640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb8500d23c0 con 0x7fb838003b70 2024-04-26T23:20:31.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.718+0000 7fb855206640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8500c8720 0x7fb8500c8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:31.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.718+0000 7fb855206640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8500c8720 0x7fb8500c8ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51444/0 (socket says 172.21.15.47:51444) 2024-04-26T23:20:31.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.718+0000 7fb855206640 1 -- 172.21.15.47:0/3403189234 learned_addr learned my addr 172.21.15.47:0/3403189234 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:31.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.718+0000 7fb854a05640 1 --2- 172.21.15.47:0/3403189234 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb838003b70 0x7fb8501522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:31.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.718+0000 7fb84ffff640 1 --2- 172.21.15.47:0/3403189234 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb85006ea70 0x7fb85006eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:31.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.718+0000 7fb855206640 1 -- 172.21.15.47:0/3403189234 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb838003b70 msgr2=0x7fb8501522c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:31.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.718+0000 7fb855206640 1 --2- 172.21.15.47:0/3403189234 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb838003b70 0x7fb8501522c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:31.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.718+0000 7fb855206640 1 -- 172.21.15.47:0/3403189234 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb85006ea70 msgr2=0x7fb85006eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:31.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.718+0000 7fb855206640 1 --2- 172.21.15.47:0/3403189234 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb85006ea70 0x7fb85006eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:31.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.718+0000 7fb855206640 1 -- 172.21.15.47:0/3403189234 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8500cbc00 con 0x7fb8500c8720 2024-04-26T23:20:31.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.719+0000 7fb855206640 1 --2- 172.21.15.47:0/3403189234 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8500c8720 0x7fb8500c8ba0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fb8500cdd10 tx=0x7fb848007b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:31.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.719+0000 7fb84d7fa640 1 -- 172.21.15.47:0/3403189234 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84800ab90 con 0x7fb8500c8720 2024-04-26T23:20:31.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.719+0000 7fb84d7fa640 1 -- 172.21.15.47:0/3403189234 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb848018070 con 0x7fb8500c8720 2024-04-26T23:20:31.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.719+0000 7fb856c69640 1 -- 172.21.15.47:0/3403189234 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8500cbee0 con 0x7fb8500c8720 2024-04-26T23:20:31.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.719+0000 7fb84d7fa640 1 -- 172.21.15.47:0/3403189234 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8480133f0 con 0x7fb8500c8720 2024-04-26T23:20:31.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.719+0000 7fb856c69640 1 -- 172.21.15.47:0/3403189234 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8500cc4a0 con 0x7fb8500c8720 2024-04-26T23:20:31.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.720+0000 7fb856c69640 1 -- 172.21.15.47:0/3403189234 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8500694f0 con 0x7fb8500c8720 2024-04-26T23:20:31.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.720+0000 7fb84d7fa640 1 -- 172.21.15.47:0/3403189234 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb848013590 con 0x7fb8500c8720 2024-04-26T23:20:31.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.721+0000 7fb84d7fa640 1 --2- 172.21.15.47:0/3403189234 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb83c06c8f0 0x7fb83c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:31.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.721+0000 7fb84d7fa640 1 -- 172.21.15.47:0/3403189234 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fb84808c1a0 con 0x7fb8500c8720 2024-04-26T23:20:31.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.721+0000 7fb854a05640 1 --2- 172.21.15.47:0/3403189234 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb83c06c8f0 0x7fb83c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:31.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.722+0000 7fb854a05640 1 --2- 172.21.15.47:0/3403189234 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb83c06c8f0 0x7fb83c06edb0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb840032000 tx=0x7fb840009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:31.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.725+0000 7fb84d7fa640 1 -- 172.21.15.47:0/3403189234 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb84805a3a0 con 0x7fb8500c8720 2024-04-26T23:20:31.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.952+0000 7fb856c69640 1 -- 172.21.15.47:0/3403189234 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb8500c9be0 con 0x7fb8500c8720 2024-04-26T23:20:31.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.952+0000 7fb84d7fa640 1 -- 172.21.15.47:0/3403189234 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+73 (secure 0 0 0) 0x7fb848059ca0 con 0x7fb8500c8720 2024-04-26T23:20:31.953 INFO:teuthology.orchestra.run.smithi047.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T23:20:31.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.954+0000 7fb856c69640 1 -- 172.21.15.47:0/3403189234 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb83c06c8f0 msgr2=0x7fb83c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:31.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.954+0000 7fb856c69640 1 --2- 172.21.15.47:0/3403189234 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb83c06c8f0 0x7fb83c06edb0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb840032000 tx=0x7fb840009030 comp rx=0 tx=0).stop 2024-04-26T23:20:31.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.954+0000 7fb856c69640 1 -- 172.21.15.47:0/3403189234 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8500c8720 msgr2=0x7fb8500c8ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:31.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.954+0000 7fb856c69640 1 --2- 172.21.15.47:0/3403189234 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8500c8720 0x7fb8500c8ba0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fb8500cdd10 tx=0x7fb848007b10 comp rx=0 tx=0).stop 2024-04-26T23:20:31.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.954+0000 7fb856c69640 1 -- 172.21.15.47:0/3403189234 shutdown_connections 2024-04-26T23:20:31.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.954+0000 7fb856c69640 1 --2- 172.21.15.47:0/3403189234 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb83c06c8f0 0x7fb83c06edb0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:31.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.954+0000 7fb856c69640 1 --2- 172.21.15.47:0/3403189234 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8500c8720 0x7fb8500c8ba0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:31.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.954+0000 7fb856c69640 1 --2- 172.21.15.47:0/3403189234 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb85006ea70 0x7fb85006eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:31.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.954+0000 7fb856c69640 1 --2- 172.21.15.47:0/3403189234 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb838003b70 0x7fb8501522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:31.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.954+0000 7fb856c69640 1 -- 172.21.15.47:0/3403189234 >> 172.21.15.47:0/3403189234 conn(0x7fb8500d2da0 msgr2=0x7fb850070b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:31.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.954+0000 7fb856c69640 1 -- 172.21.15.47:0/3403189234 shutdown_connections 2024-04-26T23:20:31.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:31.954+0000 7fb856c69640 1 -- 172.21.15.47:0/3403189234 wait complete. 2024-04-26T23:20:31.966 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T23:20:31.966 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T23:20:31.966 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi047.front.sepia.ceph.com 2024-04-26T23:20:31.966 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T23:20:31.966 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T23:20:31.966 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T23:20:31.966 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T23:20:31.966 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T23:20:31.967 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:20:31.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:20:31.967 DEBUG:teuthology.orchestra.run.smithi047:> ip addr 2024-04-26T23:20:31.985 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T23:20:31.985 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T23:20:31.985 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T23:20:31.985 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:20:31.985 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2024-04-26T23:20:31.985 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:20:31.985 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:20:31.985 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:20:31.985 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f0 2024-04-26T23:20:31.985 INFO:teuthology.orchestra.run.smithi047.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:20:31.985 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:20:31.985 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f1 2024-04-26T23:20:31.986 INFO:teuthology.orchestra.run.smithi047.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T23:20:31.986 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2024-04-26T23:20:31.986 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f0 2024-04-26T23:20:31.986 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-26T23:20:31.986 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42567sec preferred_lft 42567sec 2024-04-26T23:20:31.986 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link noprefixroute 2024-04-26T23:20:31.986 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:20:31.986 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:20:31.986 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2024-04-26T23:20:31.986 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f1 2024-04-26T23:20:31.986 INFO:teuthology.orchestra.run.smithi047.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T23:20:31.986 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 32:8d:47:b1:d0:77 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:20:31.986 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T23:20:31.986 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:20:31.987 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::7c6d:6aff:fe00:4b30/64 scope link 2024-04-26T23:20:31.987 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:20:31.987 INFO:teuthology.orchestra.run.smithi047.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T23:20:31.987 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 32:8d:47:b1:d0:77 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T23:20:31.987 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::308d:47ff:feb1:d077/64 scope link 2024-04-26T23:20:31.987 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:20:31.987 INFO:teuthology.orchestra.run.smithi047.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T23:20:31.987 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether b2:9e:42:6a:5b:5d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T23:20:31.987 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::b09e:42ff:fe6a:5b5d/64 scope link 2024-04-26T23:20:31.987 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:20:31.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:20:31.987 DEBUG:teuthology.orchestra.run.smithi047:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T23:20:32.051 INFO:teuthology.orchestra.run.smithi047.stdout:1 2024-04-26T23:20:32.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:20:32.053 DEBUG:teuthology.orchestra.run.smithi047:> route 2024-04-26T23:20:32.110 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2024-04-26T23:20:32.110 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T23:20:32.110 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T23:20:32.110 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T23:20:32.111 INFO:teuthology.orchestra.run.smithi047.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T23:20:32.111 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:20:32.111 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-04-26T23:20:32.111 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T23:20:32.112 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T23:20:32.112 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T23:20:32.112 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-04-26T23:20:32.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:20:32.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:20:32.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:20:32.238 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-04-26T23:20:32.254 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T23:20:32.254 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T23:20:32.254 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:20:32.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:20:32.255 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set brx.0 up 2024-04-26T23:20:32.319 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:20:32.319 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T23:20:32.374 INFO:teuthology.orchestra.run.smithi047.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:20:32.375 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:20:32.375 DEBUG:teuthology.orchestra.run.smithi047:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T23:20:32.430 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T23:20:32.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:20:32.430 DEBUG:teuthology.orchestra.run.smithi047:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:20:32.513 INFO:teuthology.orchestra.run.smithi047.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:20:32.514 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:20:32.514 INFO:teuthology.orchestra.run.smithi047.stdout:invalid new device string format 2024-04-26T23:20:32.522 INFO:teuthology.orchestra.run.smithi047.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:20:32.522 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:20:32.522 INFO:teuthology.orchestra.run.smithi047.stdout:invalid new device string format 2024-04-26T23:20:32.522 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: resolved to: "172.21.15.47:3300,172.21.15.203:3300,172.21.15.203:3301" 2024-04-26T23:20:32.522 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:20:32.531 INFO:teuthology.orchestra.run.smithi047.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:20:32.531 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:20:32.531 INFO:teuthology.orchestra.run.smithi047.stdout:invalid new device string format 2024-04-26T23:20:32.531 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: resolved to: "172.21.15.47:3300,172.21.15.203:3300,172.21.15.203:3301" 2024-04-26T23:20:32.531 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:20:32.531 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: resolved to: "172.21.15.47:6789,172.21.15.203:6789,172.21.15.203:6790" 2024-04-26T23:20:32.531 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.47:6789,172.21.15.203:6789,172.21.15.203:6790:/ 2024-04-26T23:20:32.531 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=399006d3-4db1-43e6-ba81-a79a276eabfd" will pass to kernel 2024-04-26T23:20:32.532 INFO:teuthology.orchestra.run.smithi047.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T23:20:32.532 INFO:teuthology.orchestra.run.smithi047.stdout: You just mounted a file system that supports labels which does not 2024-04-26T23:20:32.532 INFO:teuthology.orchestra.run.smithi047.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T23:20:32.532 INFO:teuthology.orchestra.run.smithi047.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T23:20:32.532 INFO:teuthology.orchestra.run.smithi047.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T23:20:32.533 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T23:20:32.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:20:32.533 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T23:20:32.597 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T23:20:32.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:20:32.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:20:32.695 INFO:teuthology.orchestra.run.smithi047.stdout:client15128 2024-04-26T23:20:32.695 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:20:32.695 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:20:32.712 DEBUG:teuthology.orchestra.run.smithi047:> stdin-killer -- python3 -c ' 2024-04-26T23:20:32.712 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-04-26T23:20:32.712 DEBUG:teuthology.orchestra.run.smithi047:> import errno 2024-04-26T23:20:32.712 DEBUG:teuthology.orchestra.run.smithi047:> 2024-04-26T23:20:32.712 DEBUG:teuthology.orchestra.run.smithi047:> fd = os.open("/home/ubuntu/cephtest/mnt.0/test_file", os.O_RDWR) 2024-04-26T23:20:32.712 DEBUG:teuthology.orchestra.run.smithi047:> try: 2024-04-26T23:20:32.712 DEBUG:teuthology.orchestra.run.smithi047:> if False: 2024-04-26T23:20:32.712 DEBUG:teuthology.orchestra.run.smithi047:> ret = os.read(fd, 1024) 2024-04-26T23:20:32.712 DEBUG:teuthology.orchestra.run.smithi047:> else: 2024-04-26T23:20:32.713 DEBUG:teuthology.orchestra.run.smithi047:> os.write(fd, b'"'"'content'"'"') 2024-04-26T23:20:32.713 DEBUG:teuthology.orchestra.run.smithi047:> except OSError as e: 2024-04-26T23:20:32.713 DEBUG:teuthology.orchestra.run.smithi047:> if e.errno != errno.EPERM: 2024-04-26T23:20:32.713 DEBUG:teuthology.orchestra.run.smithi047:> raise 2024-04-26T23:20:32.713 DEBUG:teuthology.orchestra.run.smithi047:> else: 2024-04-26T23:20:32.713 DEBUG:teuthology.orchestra.run.smithi047:> raise RuntimeError("client does not check permission of data pool") 2024-04-26T23:20:32.713 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-04-26T23:20:32.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:20:32.830 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow w pool=cephfs_data' 2024-04-26T23:20:32.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.978+0000 7fdea5724640 1 Processor -- start 2024-04-26T23:20:32.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.978+0000 7fdea5724640 1 -- start start 2024-04-26T23:20:32.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.978+0000 7fdea5724640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdea00cfde0 0x7fdea00d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:32.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.978+0000 7fdea5724640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdea0070350 0x7fdea0070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:32.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.978+0000 7fdea5724640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdea00d2390 con 0x7fdea0071090 2024-04-26T23:20:32.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.978+0000 7fdea5724640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fdea00afab0 con 0x7fdea0070350 2024-04-26T23:20:32.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.978+0000 7fdea5724640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fdea006dd20 con 0x7fdea00cfde0 2024-04-26T23:20:32.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.979+0000 7fde9e7fc640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdea0070350 0x7fdea0070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:32.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.979+0000 7fde9e7fc640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdea0070350 0x7fdea0070b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51454/0 (socket says 172.21.15.47:51454) 2024-04-26T23:20:32.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.979+0000 7fde9e7fc640 1 -- 172.21.15.47:0/1911679715 learned_addr learned my addr 172.21.15.47:0/1911679715 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:32.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.979+0000 7fde9effd640 1 --2- 172.21.15.47:0/1911679715 >> v2:172.21.15.203:3300/0 conn(0x7fdea00cfde0 0x7fdea00d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:32.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.979+0000 7fde9e7fc640 1 -- 172.21.15.47:0/1911679715 >> v2:172.21.15.203:3300/0 conn(0x7fdea00cfde0 msgr2=0x7fdea00d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:32.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.979+0000 7fde9e7fc640 1 --2- 172.21.15.47:0/1911679715 >> v2:172.21.15.203:3300/0 conn(0x7fdea00cfde0 0x7fdea00d1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:32.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.979+0000 7fde9e7fc640 1 -- 172.21.15.47:0/1911679715 >> v1:172.21.15.203:6789/0 conn(0x7fdea0071090 legacy=0x7fdea015e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:32.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.979+0000 7fde9e7fc640 1 -- 172.21.15.47:0/1911679715 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdea006d3f0 con 0x7fdea0070350 2024-04-26T23:20:32.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.980+0000 7fde9e7fc640 1 --2- 172.21.15.47:0/1911679715 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdea0070350 0x7fdea0070b50 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fdea00762b0 tx=0x7fde9402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beaee85f9f7b1c89 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:32.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.980+0000 7fde9dffb640 1 -- 172.21.15.47:0/1911679715 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde94037070 con 0x7fdea0070350 2024-04-26T23:20:32.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.980+0000 7fde9dffb640 1 -- 172.21.15.47:0/1911679715 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde94032440 con 0x7fdea0070350 2024-04-26T23:20:32.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.980+0000 7fde9dffb640 1 -- 172.21.15.47:0/1911679715 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde94032810 con 0x7fdea0070350 2024-04-26T23:20:32.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.980+0000 7fdea5724640 1 -- 172.21.15.47:0/1911679715 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdea0070350 msgr2=0x7fdea0070b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:32.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.980+0000 7fdea5724640 1 --2- 172.21.15.47:0/1911679715 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdea0070350 0x7fdea0070b50 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fdea00762b0 tx=0x7fde9402a200 comp rx=0 tx=0).stop 2024-04-26T23:20:32.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.980+0000 7fdea5724640 1 -- 172.21.15.47:0/1911679715 shutdown_connections 2024-04-26T23:20:32.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.980+0000 7fdea5724640 1 --2- 172.21.15.47:0/1911679715 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdea0070350 0x7fdea0070b50 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:32.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.980+0000 7fdea5724640 1 --2- 172.21.15.47:0/1911679715 >> v2:172.21.15.203:3300/0 conn(0x7fdea00cfde0 0x7fdea00d1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:32.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.980+0000 7fdea5724640 1 -- 172.21.15.47:0/1911679715 >> 172.21.15.47:0/1911679715 conn(0x7fdea00d2d80 msgr2=0x7fdea00d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:32.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.980+0000 7fdea5724640 1 -- 172.21.15.47:0/1911679715 shutdown_connections 2024-04-26T23:20:32.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.980+0000 7fdea5724640 1 -- 172.21.15.47:0/1911679715 wait complete. 2024-04-26T23:20:32.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.982+0000 7fdea5724640 1 Processor -- start 2024-04-26T23:20:32.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.982+0000 7fdea5724640 1 -- start start 2024-04-26T23:20:32.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.982+0000 7fdea5724640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdea0070350 0x7fdea0161350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:32.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.982+0000 7fdea5724640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdea0071090 0x7fdea00a0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:32.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.982+0000 7fdea5724640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdea00cfde0 0x7fdea00a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:32.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.982+0000 7fdea5724640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fdea006d9b0 con 0x7fdea00cfde0 2024-04-26T23:20:32.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.982+0000 7fdea5724640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdea00714c0 con 0x7fdea0071090 2024-04-26T23:20:32.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.982+0000 7fdea5724640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fdea00d20a0 con 0x7fdea0070350 2024-04-26T23:20:32.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.983+0000 7fde9f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdea00cfde0 0x7fdea00a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:32.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.983+0000 7fde9f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdea00cfde0 0x7fdea00a05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51468/0 (socket says 172.21.15.47:51468) 2024-04-26T23:20:32.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.983+0000 7fde9f7fe640 1 -- 172.21.15.47:0/1020212900 learned_addr learned my addr 172.21.15.47:0/1020212900 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:32.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.983+0000 7fde9e7fc640 1 --2- 172.21.15.47:0/1020212900 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdea0071090 0x7fdea00a0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:32.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.983+0000 7fde9effd640 1 --2- 172.21.15.47:0/1020212900 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdea0070350 0x7fdea0161350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:32.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.983+0000 7fde9f7fe640 1 -- 172.21.15.47:0/1020212900 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdea0070350 msgr2=0x7fdea0161350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:32.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.983+0000 7fde9f7fe640 1 --2- 172.21.15.47:0/1020212900 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdea0070350 0x7fdea0161350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:32.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.983+0000 7fde9f7fe640 1 -- 172.21.15.47:0/1020212900 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdea0071090 msgr2=0x7fdea00a0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:32.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.983+0000 7fde9f7fe640 1 --2- 172.21.15.47:0/1020212900 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdea0071090 0x7fdea00a0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:32.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.983+0000 7fde9f7fe640 1 -- 172.21.15.47:0/1020212900 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdea00b6280 con 0x7fdea00cfde0 2024-04-26T23:20:32.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.983+0000 7fde9effd640 1 --2- 172.21.15.47:0/1020212900 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdea0070350 0x7fdea0161350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:20:32.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.983+0000 7fde9f7fe640 1 --2- 172.21.15.47:0/1020212900 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdea00cfde0 0x7fdea00a05b0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fde9800c6b0 tx=0x7fde9800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:32.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.983+0000 7fde877fe640 1 -- 172.21.15.47:0/1020212900 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde98003b70 con 0x7fdea00cfde0 2024-04-26T23:20:32.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.983+0000 7fde877fe640 1 -- 172.21.15.47:0/1020212900 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde98003d30 con 0x7fdea00cfde0 2024-04-26T23:20:32.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.983+0000 7fdea5724640 1 -- 172.21.15.47:0/1020212900 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdea00b6560 con 0x7fdea00cfde0 2024-04-26T23:20:32.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.983+0000 7fde877fe640 1 -- 172.21.15.47:0/1020212900 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde98005310 con 0x7fdea00cfde0 2024-04-26T23:20:32.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.983+0000 7fdea5724640 1 -- 172.21.15.47:0/1020212900 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdea01f5930 con 0x7fdea00cfde0 2024-04-26T23:20:32.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.984+0000 7fdea5724640 1 -- 172.21.15.47:0/1020212900 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdea00694f0 con 0x7fdea00cfde0 2024-04-26T23:20:32.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.985+0000 7fde877fe640 1 -- 172.21.15.47:0/1020212900 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fde980054b0 con 0x7fdea00cfde0 2024-04-26T23:20:32.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.985+0000 7fde877fe640 1 --2- 172.21.15.47:0/1020212900 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fde8c06c8f0 0x7fde8c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:32.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.985+0000 7fde877fe640 1 -- 172.21.15.47:0/1020212900 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fde9808bab0 con 0x7fdea00cfde0 2024-04-26T23:20:32.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.985+0000 7fde9effd640 1 --2- 172.21.15.47:0/1020212900 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fde8c06c8f0 0x7fde8c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:32.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.986+0000 7fde9effd640 1 --2- 172.21.15.47:0/1020212900 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fde8c06c8f0 0x7fde8c06edb0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fde90006fd0 tx=0x7fde90008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:32.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:32.989+0000 7fde877fe640 1 -- 172.21.15.47:0/1020212900 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fde98059d50 con 0x7fdea00cfde0 2024-04-26T23:20:33.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.246+0000 7fdea5724640 1 -- 172.21.15.47:0/1020212900 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow w pool=cephfs_data"]} v 0) -- 0x7fdea00b6c10 con 0x7fdea00cfde0 2024-04-26T23:20:33.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.248+0000 7fde877fe640 1 -- 172.21.15.47:0/1020212900 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow w pool=cephfs_data"]}]=0 updated caps for client.0 v16) ==== 183+0+140 (secure 0 0 0) 0x7fde98059650 con 0x7fdea00cfde0 2024-04-26T23:20:33.250 INFO:teuthology.orchestra.run.smithi047.stdout:[client.0] 2024-04-26T23:20:33.250 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQAKNixmA5QFHRAAT89WPyzDs0acPluyvU4svw== 2024-04-26T23:20:33.250 INFO:teuthology.orchestra.run.smithi047.stdout: caps mds = "allow" 2024-04-26T23:20:33.250 INFO:teuthology.orchestra.run.smithi047.stdout: caps mon = "allow r" 2024-04-26T23:20:33.250 INFO:teuthology.orchestra.run.smithi047.stdout: caps osd = "allow w pool=cephfs_data" 2024-04-26T23:20:33.250 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.0 2024-04-26T23:20:33.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.250+0000 7fdea5724640 1 -- 172.21.15.47:0/1020212900 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fde8c06c8f0 msgr2=0x7fde8c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:33.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.250+0000 7fdea5724640 1 --2- 172.21.15.47:0/1020212900 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fde8c06c8f0 0x7fde8c06edb0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fde90006fd0 tx=0x7fde90008040 comp rx=0 tx=0).stop 2024-04-26T23:20:33.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.250+0000 7fdea5724640 1 -- 172.21.15.47:0/1020212900 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdea00cfde0 msgr2=0x7fdea00a05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:33.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.251+0000 7fdea5724640 1 --2- 172.21.15.47:0/1020212900 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdea00cfde0 0x7fdea00a05b0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fde9800c6b0 tx=0x7fde9800cb70 comp rx=0 tx=0).stop 2024-04-26T23:20:33.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.251+0000 7fdea5724640 1 -- 172.21.15.47:0/1020212900 shutdown_connections 2024-04-26T23:20:33.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.251+0000 7fdea5724640 1 --2- 172.21.15.47:0/1020212900 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fde8c06c8f0 0x7fde8c06edb0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:33.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.251+0000 7fdea5724640 1 --2- 172.21.15.47:0/1020212900 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdea00cfde0 0x7fdea00a05b0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:33.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.251+0000 7fdea5724640 1 --2- 172.21.15.47:0/1020212900 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdea0071090 0x7fdea00a0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:33.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.251+0000 7fdea5724640 1 --2- 172.21.15.47:0/1020212900 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdea0070350 0x7fdea0161350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:33.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.251+0000 7fdea5724640 1 -- 172.21.15.47:0/1020212900 >> 172.21.15.47:0/1020212900 conn(0x7fdea00d2d80 msgr2=0x7fdea00cf0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:33.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.251+0000 7fdea5724640 1 -- 172.21.15.47:0/1020212900 shutdown_connections 2024-04-26T23:20:33.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.251+0000 7fdea5724640 1 -- 172.21.15.47:0/1020212900 wait complete. 2024-04-26T23:20:33.262 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:20:33.262 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:20:33.278 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:20:33.278 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:20:33.334 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:20:33.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.474+0000 7f451d9a4640 1 Processor -- start 2024-04-26T23:20:33.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.474+0000 7f451d9a4640 1 -- start start 2024-04-26T23:20:33.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.475+0000 7f451d9a4640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f45180d06f0 0x7f45180d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:33.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.475+0000 7f451d9a4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45180b2850 0x7f45180b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:33.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.475+0000 7f451d9a4640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45180d0ba0 con 0x7f451815f3a0 2024-04-26T23:20:33.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.475+0000 7f451d9a4640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f45180d0e20 con 0x7f45180d06f0 2024-04-26T23:20:33.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.475+0000 7f451d9a4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f45180b1d70 con 0x7f45180b2850 2024-04-26T23:20:33.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.475+0000 7f45167fc640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f45180d06f0 0x7f45180d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:33.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.475+0000 7f45167fc640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f45180d06f0 0x7f45180d1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51470/0 (socket says 172.21.15.47:51470) 2024-04-26T23:20:33.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.475+0000 7f45167fc640 1 -- 172.21.15.47:0/3464749017 learned_addr learned my addr 172.21.15.47:0/3464749017 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:33.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.476+0000 7f45177fe640 1 --2- 172.21.15.47:0/3464749017 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45180b2850 0x7f45180b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:33.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.476+0000 7f45167fc640 1 -- 172.21.15.47:0/3464749017 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45180b2850 msgr2=0x7f45180b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:33.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.476+0000 7f45167fc640 1 --2- 172.21.15.47:0/3464749017 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45180b2850 0x7f45180b2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:33.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.476+0000 7f45167fc640 1 -- 172.21.15.47:0/3464749017 >> v1:172.21.15.203:6789/0 conn(0x7f451815f3a0 legacy=0x7f45180d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:33.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.476+0000 7f45167fc640 1 -- 172.21.15.47:0/3464749017 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45180b2050 con 0x7f45180d06f0 2024-04-26T23:20:33.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.476+0000 7f45167fc640 1 --2- 172.21.15.47:0/3464749017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45180d06f0 0x7f45180d1e80 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f450c009870 tx=0x7f450c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cd3a1b120eaf108 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:33.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.477+0000 7f4515ffb640 1 -- 172.21.15.47:0/3464749017 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450c037070 con 0x7f45180d06f0 2024-04-26T23:20:33.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.477+0000 7f4515ffb640 1 -- 172.21.15.47:0/3464749017 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f450c032440 con 0x7f45180d06f0 2024-04-26T23:20:33.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.477+0000 7f4515ffb640 1 -- 172.21.15.47:0/3464749017 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450c032810 con 0x7f45180d06f0 2024-04-26T23:20:33.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.477+0000 7f451d9a4640 1 -- 172.21.15.47:0/3464749017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45180d06f0 msgr2=0x7f45180d1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:33.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.477+0000 7f451d9a4640 1 --2- 172.21.15.47:0/3464749017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45180d06f0 0x7f45180d1e80 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f450c009870 tx=0x7f450c02a270 comp rx=0 tx=0).stop 2024-04-26T23:20:33.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.477+0000 7f451d9a4640 1 -- 172.21.15.47:0/3464749017 shutdown_connections 2024-04-26T23:20:33.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.477+0000 7f451d9a4640 1 --2- 172.21.15.47:0/3464749017 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45180b2850 0x7f45180b2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:33.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.477+0000 7f451d9a4640 1 --2- 172.21.15.47:0/3464749017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45180d06f0 0x7f45180d1e80 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:33.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.477+0000 7f451d9a4640 1 -- 172.21.15.47:0/3464749017 >> 172.21.15.47:0/3464749017 conn(0x7f45180d4f90 msgr2=0x7f45180d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:33.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.477+0000 7f451d9a4640 1 -- 172.21.15.47:0/3464749017 shutdown_connections 2024-04-26T23:20:33.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.477+0000 7f451d9a4640 1 -- 172.21.15.47:0/3464749017 wait complete. 2024-04-26T23:20:33.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.478+0000 7f451d9a4640 1 Processor -- start 2024-04-26T23:20:33.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.479+0000 7f451d9a4640 1 -- start start 2024-04-26T23:20:33.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.479+0000 7f451d9a4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45180b2850 0x7f45180cf420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:33.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.479+0000 7f451d9a4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45180d06f0 0x7f4518158110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:33.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.479+0000 7f451d9a4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f451815f3a0 0x7f451809ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:33.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.479+0000 7f451d9a4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f451815f900 con 0x7f45180b2850 2024-04-26T23:20:33.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.479+0000 7f451d9a4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f451815f780 con 0x7f45180d06f0 2024-04-26T23:20:33.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.479+0000 7f451d9a4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f45180d23c0 con 0x7f451815f3a0 2024-04-26T23:20:33.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.479+0000 7f4516ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45180b2850 0x7f45180cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:33.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.479+0000 7f4516ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45180b2850 0x7f45180cf420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51486/0 (socket says 172.21.15.47:51486) 2024-04-26T23:20:33.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.479+0000 7f4516ffd640 1 -- 172.21.15.47:0/2155109616 learned_addr learned my addr 172.21.15.47:0/2155109616 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:33.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.480+0000 7f45167fc640 1 --2- 172.21.15.47:0/2155109616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45180d06f0 0x7f4518158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:33.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.480+0000 7f4516ffd640 1 -- 172.21.15.47:0/2155109616 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f451815f3a0 msgr2=0x7f451809ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:33.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.480+0000 7f45177fe640 1 --2- 172.21.15.47:0/2155109616 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f451815f3a0 0x7f451809ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:33.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.480+0000 7f4516ffd640 1 --2- 172.21.15.47:0/2155109616 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f451815f3a0 0x7f451809ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:33.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.480+0000 7f4516ffd640 1 -- 172.21.15.47:0/2155109616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45180d06f0 msgr2=0x7f4518158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:33.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.480+0000 7f4516ffd640 1 --2- 172.21.15.47:0/2155109616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45180d06f0 0x7f4518158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:33.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.480+0000 7f4516ffd640 1 -- 172.21.15.47:0/2155109616 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45180cd590 con 0x7f45180b2850 2024-04-26T23:20:33.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.480+0000 7f4516ffd640 1 --2- 172.21.15.47:0/2155109616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45180b2850 0x7f45180cf420 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f451000c790 tx=0x7f451000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:33.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.480+0000 7f44fffff640 1 -- 172.21.15.47:0/2155109616 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4510016020 con 0x7f45180b2850 2024-04-26T23:20:33.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.480+0000 7f44fffff640 1 -- 172.21.15.47:0/2155109616 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4510006d50 con 0x7f45180b2850 2024-04-26T23:20:33.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.480+0000 7f451d9a4640 1 -- 172.21.15.47:0/2155109616 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45180cd870 con 0x7f45180b2850 2024-04-26T23:20:33.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.480+0000 7f44fffff640 1 -- 172.21.15.47:0/2155109616 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4510003bb0 con 0x7f45180b2850 2024-04-26T23:20:33.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.480+0000 7f451d9a4640 1 -- 172.21.15.47:0/2155109616 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45180cde30 con 0x7f45180b2850 2024-04-26T23:20:33.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.481+0000 7f451d9a4640 1 -- 172.21.15.47:0/2155109616 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45180694f0 con 0x7f45180b2850 2024-04-26T23:20:33.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.481+0000 7f44fffff640 1 -- 172.21.15.47:0/2155109616 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f451000a850 con 0x7f45180b2850 2024-04-26T23:20:33.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.482+0000 7f44fffff640 1 --2- 172.21.15.47:0/2155109616 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f450406c8f0 0x7f450406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:33.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.482+0000 7f44fffff640 1 -- 172.21.15.47:0/2155109616 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f451008b9a0 con 0x7f45180b2850 2024-04-26T23:20:33.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.482+0000 7f45167fc640 1 --2- 172.21.15.47:0/2155109616 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f450406c8f0 0x7f450406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:33.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.483+0000 7f45167fc640 1 --2- 172.21.15.47:0/2155109616 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f450406c8f0 0x7f450406edb0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f450c034000 tx=0x7f450c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:33.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.486+0000 7f44fffff640 1 -- 172.21.15.47:0/2155109616 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4510020020 con 0x7f45180b2850 2024-04-26T23:20:33.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.672+0000 7f451d9a4640 1 -- 172.21.15.47:0/2155109616 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f45180b6860 con 0x7f45180b2850 2024-04-26T23:20:33.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.673+0000 7f44fffff640 1 -- 172.21.15.47:0/2155109616 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) ==== 74+0+16902 (secure 0 0 0) 0x7f45180d0ba0 con 0x7f45180b2850 2024-04-26T23:20:33.674 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:20:33.674 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":54,"fsid":"399006d3-4db1-43e6-ba81-a79a276eabfd","created":"2024-04-26T23:18:06.731211+0000","modified":"2024-04-26T23:20:23.340537+0000","last_up_change":"2024-04-26T23:18:17.808736+0000","last_in_change":"2024-04-26T23:18:08.945859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:18:16.937492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:20:19.975691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-26T23:20:20.657778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ee091567-fea0-4a14-bf4e-0e16c20bb2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6819","nonce":326520779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6821","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6823","nonce":326520779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6829","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6831","nonce":326520779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6825","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6827","nonce":326520779}]},"public_addr":"172.21.15.47:6819/326520779","cluster_addr":"172.21.15.47:6823/326520779","heartbeat_back_addr":"172.21.15.47:6831/326520779","heartbeat_front_addr":"172.21.15.47:6827/326520779","state":["exists","up"]},{"osd":1,"uuid":"c14440a0-bba5-4931-a6db-b736c08e869b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6809","nonce":1912610931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6811","nonce":1912610931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6815","nonce":1912610931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6813","nonce":1912610931}]},"public_addr":"172.21.15.47:6809/1912610931","cluster_addr":"172.21.15.47:6811/1912610931","heartbeat_back_addr":"172.21.15.47:6815/1912610931","heartbeat_front_addr":"172.21.15.47:6813/1912610931","state":["exists","up"]},{"osd":2,"uuid":"3940f8de-c9b2-4cd3-a911-9c301729eb30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6801","nonce":2569869305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6803","nonce":2569869305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6807","nonce":2569869305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6805","nonce":2569869305}]},"public_addr":"172.21.15.47:6801/2569869305","cluster_addr":"172.21.15.47:6803/2569869305","heartbeat_back_addr":"172.21.15.47:6807/2569869305","heartbeat_front_addr":"172.21.15.47:6805/2569869305","state":["exists","up"]},{"osd":3,"uuid":"1286a67e-8958-426a-8c2e-bc85498c619c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6818","nonce":2791411683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6822","nonce":2791411683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6830","nonce":2791411683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6826","nonce":2791411683}]},"public_addr":"172.21.15.47:6818/2791411683","cluster_addr":"172.21.15.47:6822/2791411683","heartbeat_back_addr":"172.21.15.47:6830/2791411683","heartbeat_front_addr":"172.21.15.47:6826/2791411683","state":["exists","up"]},{"osd":4,"uuid":"d8ee2c20-e5b7-46c2-9999-b234070126cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6809","nonce":441313104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6811","nonce":441313104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6815","nonce":441313104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6813","nonce":441313104}]},"public_addr":"172.21.15.203:6809/441313104","cluster_addr":"172.21.15.203:6811/441313104","heartbeat_back_addr":"172.21.15.203:6815/441313104","heartbeat_front_addr":"172.21.15.203:6813/441313104","state":["exists","up"]},{"osd":5,"uuid":"b453a8f0-f324-4840-9ae0-d67bdadb9f8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6801","nonce":56838444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6803","nonce":56838444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6807","nonce":56838444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6805","nonce":56838444}]},"public_addr":"172.21.15.203:6801/56838444","cluster_addr":"172.21.15.203:6803/56838444","heartbeat_back_addr":"172.21.15.203:6807/56838444","heartbeat_front_addr":"172.21.15.203:6805/56838444","state":["exists","up"]},{"osd":6,"uuid":"a6d31951-2d4b-4e8d-a510-6283b01dc158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6825","nonce":2738517080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6827","nonce":2738517080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6831","nonce":2738517080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6829","nonce":2738517080}]},"public_addr":"172.21.15.203:6825/2738517080","cluster_addr":"172.21.15.203:6827/2738517080","heartbeat_back_addr":"172.21.15.203:6831/2738517080","heartbeat_front_addr":"172.21.15.203:6829/2738517080","state":["exists","up"]},{"osd":7,"uuid":"7119b8c0-fe96-402d-9e75-7b509d9d44d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6817","nonce":1315565058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6819","nonce":1315565058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6823","nonce":1315565058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6821","nonce":1315565058}]},"public_addr":"172.21.15.203:6817/1315565058","cluster_addr":"172.21.15.203:6819/1315565058","heartbeat_back_addr":"172.21.15.203:6823/1315565058","heartbeat_front_addr":"172.21.15.203:6821/1315565058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.121921+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.137494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.084814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.111593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.072430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.125066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.097867+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.057061+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:20:33.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.675+0000 7f451d9a4640 1 -- 172.21.15.47:0/2155109616 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f450406c8f0 msgr2=0x7f450406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:33.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.675+0000 7f451d9a4640 1 --2- 172.21.15.47:0/2155109616 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f450406c8f0 0x7f450406edb0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f450c034000 tx=0x7f450c02e000 comp rx=0 tx=0).stop 2024-04-26T23:20:33.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.675+0000 7f451d9a4640 1 -- 172.21.15.47:0/2155109616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45180b2850 msgr2=0x7f45180cf420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:33.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.675+0000 7f451d9a4640 1 --2- 172.21.15.47:0/2155109616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45180b2850 0x7f45180cf420 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f451000c790 tx=0x7f451000cc50 comp rx=0 tx=0).stop 2024-04-26T23:20:33.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.675+0000 7f451d9a4640 1 -- 172.21.15.47:0/2155109616 shutdown_connections 2024-04-26T23:20:33.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.675+0000 7f451d9a4640 1 --2- 172.21.15.47:0/2155109616 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f450406c8f0 0x7f450406edb0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:33.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.675+0000 7f451d9a4640 1 --2- 172.21.15.47:0/2155109616 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f451815f3a0 0x7f451809ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:33.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.675+0000 7f451d9a4640 1 --2- 172.21.15.47:0/2155109616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45180d06f0 0x7f4518158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:33.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.675+0000 7f451d9a4640 1 --2- 172.21.15.47:0/2155109616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45180b2850 0x7f45180cf420 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:33.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.675+0000 7f451d9a4640 1 -- 172.21.15.47:0/2155109616 >> 172.21.15.47:0/2155109616 conn(0x7f45180d4f90 msgr2=0x7f451806cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:33.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.675+0000 7f451d9a4640 1 -- 172.21.15.47:0/2155109616 shutdown_connections 2024-04-26T23:20:33.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.675+0000 7f451d9a4640 1 -- 172.21.15.47:0/2155109616 wait complete. 2024-04-26T23:20:33.687 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:18:16.937492+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:20:19.975691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:20:20.657778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:20:33.688 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:20:33.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.791+0000 7f70560c8640 1 Processor -- start 2024-04-26T23:20:33.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.791+0000 7f70560c8640 1 -- start start 2024-04-26T23:20:33.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.791+0000 7f70560c8640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f70500b56c0 0x7f70500afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:33.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.791+0000 7f70560c8640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70500b5e30 con 0x7f7050072fb0 2024-04-26T23:20:33.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.791+0000 7f70560c8640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70500b6110 con 0x7f7050070750 2024-04-26T23:20:33.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.791+0000 7f70560c8640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f705006f2b0 con 0x7f70500b56c0 2024-04-26T23:20:33.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.792+0000 7f704f7fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7050072fb0 0x7f7050073390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37826/0 (socket says 172.21.15.47:37826) 2024-04-26T23:20:33.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.792+0000 7f704f7fe640 1 -- 172.21.15.47:0/2950776480 learned_addr learned my addr 172.21.15.47:0/2950776480 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:33.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.792+0000 7f704ffff640 1 --2- 172.21.15.47:0/2950776480 >> v2:172.21.15.203:3300/0 conn(0x7f70500b56c0 0x7f70500afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:33.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.792+0000 7f704e7fc640 1 -- 172.21.15.47:0/2950776480 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3678910644 0 0) 0x7f70500b5e30 con 0x7f7050072fb0 2024-04-26T23:20:33.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.792+0000 7f704e7fc640 1 -- 172.21.15.47:0/2950776480 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7038003520 con 0x7f7050072fb0 2024-04-26T23:20:33.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.792+0000 7f704e7fc640 1 -- 172.21.15.47:0/2950776480 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1542585906 0 0) 0x7f70500b6110 con 0x7f7050070750 2024-04-26T23:20:33.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.792+0000 7f704e7fc640 1 -- 172.21.15.47:0/2950776480 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f70500b5e30 con 0x7f7050070750 2024-04-26T23:20:33.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.792+0000 7f704ffff640 1 -- 172.21.15.47:0/2950776480 >> v1:172.21.15.203:6789/0 conn(0x7f7050070750 legacy=0x7f70500af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:33.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.792+0000 7f704e7fc640 1 -- 172.21.15.47:0/2950776480 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3615158318 0 0) 0x7f7038003520 con 0x7f7050072fb0 2024-04-26T23:20:33.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.792+0000 7f704ffff640 1 -- 172.21.15.47:0/2950776480 >> v1:172.21.15.47:6789/0 conn(0x7f7050072fb0 legacy=0x7f7050073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:33.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.792+0000 7f704ffff640 1 -- 172.21.15.47:0/2950776480 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70500b63f0 con 0x7f70500b56c0 2024-04-26T23:20:33.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.793+0000 7f704ffff640 1 --2- 172.21.15.47:0/2950776480 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70500b56c0 0x7f70500afdf0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f70480097a0 tx=0x7f704802a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2c1d037d9959cbd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:33.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.793+0000 7f704e7fc640 1 -- 172.21.15.47:0/2950776480 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7048037070 con 0x7f70500b56c0 2024-04-26T23:20:33.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.793+0000 7f704e7fc640 1 -- 172.21.15.47:0/2950776480 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f704802ac50 con 0x7f70500b56c0 2024-04-26T23:20:33.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.793+0000 7f704e7fc640 1 -- 172.21.15.47:0/2950776480 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70400019f0 con 0x7f70500b56c0 2024-04-26T23:20:33.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.794+0000 7f70560c8640 1 -- 172.21.15.47:0/2950776480 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70500b56c0 msgr2=0x7f70500afdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:33.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.794+0000 7f70560c8640 1 --2- 172.21.15.47:0/2950776480 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70500b56c0 0x7f70500afdf0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f70480097a0 tx=0x7f704802a1c0 comp rx=0 tx=0).stop 2024-04-26T23:20:33.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.794+0000 7f70560c8640 1 -- 172.21.15.47:0/2950776480 shutdown_connections 2024-04-26T23:20:33.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.794+0000 7f70560c8640 1 --2- 172.21.15.47:0/2950776480 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70500b56c0 0x7f70500afdf0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:33.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.794+0000 7f70560c8640 1 -- 172.21.15.47:0/2950776480 >> 172.21.15.47:0/2950776480 conn(0x7f70500b39b0 msgr2=0x7f70500b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:33.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.794+0000 7f70560c8640 1 -- 172.21.15.47:0/2950776480 shutdown_connections 2024-04-26T23:20:33.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.794+0000 7f70560c8640 1 -- 172.21.15.47:0/2950776480 wait complete. 2024-04-26T23:20:33.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.795+0000 7f70560c8640 1 Processor -- start 2024-04-26T23:20:33.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.795+0000 7f70560c8640 1 -- start start 2024-04-26T23:20:33.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.796+0000 7f70560c8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7050070750 0x7f70501f3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:33.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.796+0000 7f70560c8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7050072fb0 0x7f705014f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:33.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.796+0000 7f70560c8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70500b56c0 0x7f705015dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:33.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.796+0000 7f70560c8640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f705015c2e0 con 0x7f70500b56c0 2024-04-26T23:20:33.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.796+0000 7f70560c8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f705015c160 con 0x7f7050072fb0 2024-04-26T23:20:33.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.796+0000 7f70560c8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f70500b0330 con 0x7f7050070750 2024-04-26T23:20:33.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.796+0000 7f704ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70500b56c0 0x7f705015dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:33.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.796+0000 7f704ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70500b56c0 0x7f705015dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51488/0 (socket says 172.21.15.47:51488) 2024-04-26T23:20:33.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.796+0000 7f704ffff640 1 -- 172.21.15.47:0/729871284 learned_addr learned my addr 172.21.15.47:0/729871284 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:33.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.796+0000 7f704f7fe640 1 --2- 172.21.15.47:0/729871284 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7050070750 0x7f70501f3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:33.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.796+0000 7f704effd640 1 --2- 172.21.15.47:0/729871284 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7050072fb0 0x7f705014f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:33.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.796+0000 7f704ffff640 1 -- 172.21.15.47:0/729871284 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7050070750 msgr2=0x7f70501f3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:33.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.796+0000 7f704ffff640 1 --2- 172.21.15.47:0/729871284 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7050070750 0x7f70501f3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:33.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.796+0000 7f704ffff640 1 -- 172.21.15.47:0/729871284 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7050072fb0 msgr2=0x7f705014f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:33.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.796+0000 7f704ffff640 1 --2- 172.21.15.47:0/729871284 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7050072fb0 0x7f705014f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:33.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.796+0000 7f704ffff640 1 -- 172.21.15.47:0/729871284 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f705009bd10 con 0x7f70500b56c0 2024-04-26T23:20:33.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.797+0000 7f704ffff640 1 --2- 172.21.15.47:0/729871284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70500b56c0 0x7f705015dec0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f7048035040 tx=0x7f70480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:33.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.797+0000 7f7037fff640 1 -- 172.21.15.47:0/729871284 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7048041070 con 0x7f70500b56c0 2024-04-26T23:20:33.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.797+0000 7f7037fff640 1 -- 172.21.15.47:0/729871284 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f704803c460 con 0x7f70500b56c0 2024-04-26T23:20:33.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.797+0000 7f7037fff640 1 -- 172.21.15.47:0/729871284 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7048037040 con 0x7f70500b56c0 2024-04-26T23:20:33.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.797+0000 7f70560c8640 1 -- 172.21.15.47:0/729871284 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f705009bff0 con 0x7f70500b56c0 2024-04-26T23:20:33.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.797+0000 7f70560c8640 1 -- 172.21.15.47:0/729871284 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f705009c5b0 con 0x7f70500b56c0 2024-04-26T23:20:33.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.798+0000 7f70560c8640 1 -- 172.21.15.47:0/729871284 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70500694f0 con 0x7f70500b56c0 2024-04-26T23:20:33.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.798+0000 7f7037fff640 1 -- 172.21.15.47:0/729871284 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7048044070 con 0x7f70500b56c0 2024-04-26T23:20:33.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.799+0000 7f7037fff640 1 --2- 172.21.15.47:0/729871284 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f703c06c8f0 0x7f703c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:33.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.799+0000 7f7037fff640 1 -- 172.21.15.47:0/729871284 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f70480ac140 con 0x7f70500b56c0 2024-04-26T23:20:33.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.799+0000 7f704f7fe640 1 --2- 172.21.15.47:0/729871284 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f703c06c8f0 0x7f703c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:33.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.800+0000 7f704f7fe640 1 --2- 172.21.15.47:0/729871284 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f703c06c8f0 0x7f703c06edb0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7040001b60 tx=0x7f7040009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:33.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.803+0000 7f7037fff640 1 -- 172.21.15.47:0/729871284 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7048079e20 con 0x7f70500b56c0 2024-04-26T23:20:33.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.987+0000 7f70560c8640 1 -- 172.21.15.47:0/729871284 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f705015ede0 con 0x7f70500b56c0 2024-04-26T23:20:33.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.987+0000 7f7037fff640 1 -- 172.21.15.47:0/729871284 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) ==== 93+0+10 (secure 0 0 0) 0x7f704807e120 con 0x7f70500b56c0 2024-04-26T23:20:33.988 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-26T23:20:33.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.989+0000 7f70560c8640 1 -- 172.21.15.47:0/729871284 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f703c06c8f0 msgr2=0x7f703c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:33.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.989+0000 7f70560c8640 1 --2- 172.21.15.47:0/729871284 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f703c06c8f0 0x7f703c06edb0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7040001b60 tx=0x7f7040009040 comp rx=0 tx=0).stop 2024-04-26T23:20:33.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.989+0000 7f70560c8640 1 -- 172.21.15.47:0/729871284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70500b56c0 msgr2=0x7f705015dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:33.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.989+0000 7f70560c8640 1 --2- 172.21.15.47:0/729871284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70500b56c0 0x7f705015dec0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f7048035040 tx=0x7f70480046c0 comp rx=0 tx=0).stop 2024-04-26T23:20:33.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.989+0000 7f70560c8640 1 -- 172.21.15.47:0/729871284 shutdown_connections 2024-04-26T23:20:33.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.989+0000 7f70560c8640 1 --2- 172.21.15.47:0/729871284 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f703c06c8f0 0x7f703c06edb0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:33.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.989+0000 7f70560c8640 1 --2- 172.21.15.47:0/729871284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70500b56c0 0x7f705015dec0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:33.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.989+0000 7f70560c8640 1 --2- 172.21.15.47:0/729871284 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7050072fb0 0x7f705014f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:33.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.989+0000 7f70560c8640 1 --2- 172.21.15.47:0/729871284 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7050070750 0x7f70501f3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:33.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.989+0000 7f70560c8640 1 -- 172.21.15.47:0/729871284 >> 172.21.15.47:0/729871284 conn(0x7f70500b39b0 msgr2=0x7f7050073390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:33.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.989+0000 7f70560c8640 1 -- 172.21.15.47:0/729871284 shutdown_connections 2024-04-26T23:20:33.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:33.989+0000 7f70560c8640 1 -- 172.21.15.47:0/729871284 wait complete. 2024-04-26T23:20:34.001 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:20:34.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.103+0000 7f7b63f9d640 1 Processor -- start 2024-04-26T23:20:34.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.103+0000 7f7b63f9d640 1 -- start start 2024-04-26T23:20:34.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.104+0000 7f7b63f9d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b5c0b4360 0x7f7b5c0b2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:34.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.104+0000 7f7b63f9d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7b5c0d0d60 0x7f7b5c0b2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:34.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.104+0000 7f7b63f9d640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b5c0ce5e0 con 0x7f7b5c0d1240 2024-04-26T23:20:34.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.104+0000 7f7b63f9d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f7b5c0cf210 con 0x7f7b5c0d0d60 2024-04-26T23:20:34.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.104+0000 7f7b63f9d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f7b5c0ce8c0 con 0x7f7b5c0b4360 2024-04-26T23:20:34.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.105+0000 7f7b61538640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b5c0b4360 0x7f7b5c0b2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:34.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.105+0000 7f7b6253a640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7b5c0d0d60 0x7f7b5c0b2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:34.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.105+0000 7f7b61d39640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f7b5c0d1240 0x7f7b5c0d1620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.47:59366/0 (socket says 172.21.15.47:59366) 2024-04-26T23:20:34.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.105+0000 7f7b61d39640 1 -- 172.21.15.47:0/3774799702 learned_addr learned my addr 172.21.15.47:0/3774799702 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:34.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.105+0000 7f7b60d37640 1 -- 172.21.15.47:0/3774799702 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2741928997 0 0) 0x7f7b5c0ce5e0 con 0x7f7b5c0d1240 2024-04-26T23:20:34.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.105+0000 7f7b61538640 1 -- 172.21.15.47:0/3774799702 >> v2:172.21.15.203:3300/0 conn(0x7f7b5c0d0d60 msgr2=0x7f7b5c0b2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:34.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.105+0000 7f7b61538640 1 --2- 172.21.15.47:0/3774799702 >> v2:172.21.15.203:3300/0 conn(0x7f7b5c0d0d60 0x7f7b5c0b2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:34.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.105+0000 7f7b61538640 1 -- 172.21.15.47:0/3774799702 >> v1:172.21.15.203:6789/0 conn(0x7f7b5c0d1240 legacy=0x7f7b5c0d1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:34.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.105+0000 7f7b61538640 1 -- 172.21.15.47:0/3774799702 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b5c160ab0 con 0x7f7b5c0b4360 2024-04-26T23:20:34.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.105+0000 7f7b6253a640 1 --2- 172.21.15.47:0/3774799702 >> v2:172.21.15.203:3300/0 conn(0x7f7b5c0d0d60 0x7f7b5c0b2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:20:34.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.106+0000 7f7b61538640 1 --2- 172.21.15.47:0/3774799702 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b5c0b4360 0x7f7b5c0b2600 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7b500097a0 tx=0x7f7b5002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c591123858454754 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:34.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.106+0000 7f7b60d37640 1 -- 172.21.15.47:0/3774799702 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b50037070 con 0x7f7b5c0b4360 2024-04-26T23:20:34.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.106+0000 7f7b60d37640 1 -- 172.21.15.47:0/3774799702 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b50032440 con 0x7f7b5c0b4360 2024-04-26T23:20:34.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.106+0000 7f7b60d37640 1 -- 172.21.15.47:0/3774799702 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b50032810 con 0x7f7b5c0b4360 2024-04-26T23:20:34.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.106+0000 7f7b63f9d640 1 -- 172.21.15.47:0/3774799702 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b5c0b4360 msgr2=0x7f7b5c0b2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:34.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.106+0000 7f7b63f9d640 1 --2- 172.21.15.47:0/3774799702 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b5c0b4360 0x7f7b5c0b2600 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7b500097a0 tx=0x7f7b5002a180 comp rx=0 tx=0).stop 2024-04-26T23:20:34.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.107+0000 7f7b63f9d640 1 -- 172.21.15.47:0/3774799702 shutdown_connections 2024-04-26T23:20:34.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.107+0000 7f7b63f9d640 1 --2- 172.21.15.47:0/3774799702 >> v2:172.21.15.203:3300/0 conn(0x7f7b5c0d0d60 0x7f7b5c0b2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:34.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.107+0000 7f7b63f9d640 1 --2- 172.21.15.47:0/3774799702 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b5c0b4360 0x7f7b5c0b2600 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:34.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.107+0000 7f7b63f9d640 1 -- 172.21.15.47:0/3774799702 >> 172.21.15.47:0/3774799702 conn(0x7f7b5c0d2e60 msgr2=0x7f7b5c0d3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:34.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.107+0000 7f7b63f9d640 1 -- 172.21.15.47:0/3774799702 shutdown_connections 2024-04-26T23:20:34.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.107+0000 7f7b63f9d640 1 -- 172.21.15.47:0/3774799702 wait complete. 2024-04-26T23:20:34.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.108+0000 7f7b63f9d640 1 Processor -- start 2024-04-26T23:20:34.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.108+0000 7f7b63f9d640 1 -- start start 2024-04-26T23:20:34.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.108+0000 7f7b63f9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b5c0d0d60 0x7f7b5c158660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:34.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.108+0000 7f7b63f9d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b5c0d1240 0x7f7b5c1f3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:34.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.108+0000 7f7b63f9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b5c0cc460 0x7f7b5c0c8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:34.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.108+0000 7f7b63f9d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5c0c6790 con 0x7f7b5c0cc460 2024-04-26T23:20:34.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.108+0000 7f7b63f9d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5c0c6610 con 0x7f7b5c0d0d60 2024-04-26T23:20:34.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.108+0000 7f7b63f9d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f7b5c0b3080 con 0x7f7b5c0d1240 2024-04-26T23:20:34.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.109+0000 7f7b6253a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b5c0cc460 0x7f7b5c0c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:34.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.109+0000 7f7b6253a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b5c0cc460 0x7f7b5c0c8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51498/0 (socket says 172.21.15.47:51498) 2024-04-26T23:20:34.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.109+0000 7f7b6253a640 1 -- 172.21.15.47:0/4260954487 learned_addr learned my addr 172.21.15.47:0/4260954487 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:34.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.109+0000 7f7b61d39640 1 --2- 172.21.15.47:0/4260954487 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b5c0d0d60 0x7f7b5c158660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:34.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.109+0000 7f7b61538640 1 --2- 172.21.15.47:0/4260954487 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b5c0d1240 0x7f7b5c1f3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:34.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.109+0000 7f7b6253a640 1 -- 172.21.15.47:0/4260954487 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b5c0d1240 msgr2=0x7f7b5c1f3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:34.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.109+0000 7f7b6253a640 1 --2- 172.21.15.47:0/4260954487 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b5c0d1240 0x7f7b5c1f3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:34.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.109+0000 7f7b6253a640 1 -- 172.21.15.47:0/4260954487 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b5c0d0d60 msgr2=0x7f7b5c158660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:34.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.109+0000 7f7b6253a640 1 --2- 172.21.15.47:0/4260954487 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b5c0d0d60 0x7f7b5c158660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:34.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.109+0000 7f7b6253a640 1 -- 172.21.15.47:0/4260954487 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b5c1f3e40 con 0x7f7b5c0cc460 2024-04-26T23:20:34.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.109+0000 7f7b6253a640 1 --2- 172.21.15.47:0/4260954487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b5c0cc460 0x7f7b5c0c8510 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c009f00 tx=0x7f7b4c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:34.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.109+0000 7f7b5a7fc640 1 -- 172.21.15.47:0/4260954487 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c018030 con 0x7f7b5c0cc460 2024-04-26T23:20:34.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.109+0000 7f7b5a7fc640 1 -- 172.21.15.47:0/4260954487 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b4c00ce50 con 0x7f7b5c0cc460 2024-04-26T23:20:34.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.109+0000 7f7b63f9d640 1 -- 172.21.15.47:0/4260954487 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b5c0c6bb0 con 0x7f7b5c0cc460 2024-04-26T23:20:34.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.109+0000 7f7b5a7fc640 1 -- 172.21.15.47:0/4260954487 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c003bb0 con 0x7f7b5c0cc460 2024-04-26T23:20:34.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.109+0000 7f7b63f9d640 1 -- 172.21.15.47:0/4260954487 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b5c0c7170 con 0x7f7b5c0cc460 2024-04-26T23:20:34.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.110+0000 7f7b63f9d640 1 -- 172.21.15.47:0/4260954487 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b5c0694f0 con 0x7f7b5c0cc460 2024-04-26T23:20:34.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.111+0000 7f7b5a7fc640 1 -- 172.21.15.47:0/4260954487 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7b4c01b070 con 0x7f7b5c0cc460 2024-04-26T23:20:34.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.111+0000 7f7b5a7fc640 1 --2- 172.21.15.47:0/4260954487 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f7b4806c8f0 0x7f7b4806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:34.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.111+0000 7f7b5a7fc640 1 -- 172.21.15.47:0/4260954487 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f7b4c08c580 con 0x7f7b5c0cc460 2024-04-26T23:20:34.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.112+0000 7f7b61d39640 1 --2- 172.21.15.47:0/4260954487 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f7b4806c8f0 0x7f7b4806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:34.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.112+0000 7f7b61d39640 1 --2- 172.21.15.47:0/4260954487 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f7b4806c8f0 0x7f7b4806edb0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7b54007fd0 tx=0x7f7b54007de0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:34.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.116+0000 7f7b5a7fc640 1 -- 172.21.15.47:0/4260954487 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7b4c010040 con 0x7f7b5c0cc460 2024-04-26T23:20:34.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.299+0000 7f7b63f9d640 1 -- 172.21.15.47:0/4260954487 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7b5c0d3450 con 0x7f7b5c0cc460 2024-04-26T23:20:34.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.299+0000 7f7b5a7fc640 1 -- 172.21.15.47:0/4260954487 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v54) ==== 104+0+11 (secure 0 0 0) 0x7f7b4c05a260 con 0x7f7b5c0cc460 2024-04-26T23:20:34.300 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-26T23:20:34.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.301+0000 7f7b63f9d640 1 -- 172.21.15.47:0/4260954487 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f7b4806c8f0 msgr2=0x7f7b4806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:34.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.301+0000 7f7b63f9d640 1 --2- 172.21.15.47:0/4260954487 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f7b4806c8f0 0x7f7b4806edb0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7b54007fd0 tx=0x7f7b54007de0 comp rx=0 tx=0).stop 2024-04-26T23:20:34.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.301+0000 7f7b63f9d640 1 -- 172.21.15.47:0/4260954487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b5c0cc460 msgr2=0x7f7b5c0c8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:34.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.301+0000 7f7b63f9d640 1 --2- 172.21.15.47:0/4260954487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b5c0cc460 0x7f7b5c0c8510 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c009f00 tx=0x7f7b4c00c490 comp rx=0 tx=0).stop 2024-04-26T23:20:34.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.301+0000 7f7b63f9d640 1 -- 172.21.15.47:0/4260954487 shutdown_connections 2024-04-26T23:20:34.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.301+0000 7f7b63f9d640 1 --2- 172.21.15.47:0/4260954487 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f7b4806c8f0 0x7f7b4806edb0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:34.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.301+0000 7f7b63f9d640 1 --2- 172.21.15.47:0/4260954487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b5c0cc460 0x7f7b5c0c8510 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:34.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.301+0000 7f7b63f9d640 1 --2- 172.21.15.47:0/4260954487 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b5c0d1240 0x7f7b5c1f3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:34.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.301+0000 7f7b63f9d640 1 --2- 172.21.15.47:0/4260954487 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b5c0d0d60 0x7f7b5c158660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:34.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.301+0000 7f7b63f9d640 1 -- 172.21.15.47:0/4260954487 >> 172.21.15.47:0/4260954487 conn(0x7f7b5c0d2e60 msgr2=0x7f7b5c071100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:34.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.301+0000 7f7b63f9d640 1 -- 172.21.15.47:0/4260954487 shutdown_connections 2024-04-26T23:20:34.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.301+0000 7f7b63f9d640 1 -- 172.21.15.47:0/4260954487 wait complete. 2024-04-26T23:20:34.312 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:20:34.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.416+0000 7fcb8c064640 1 Processor -- start 2024-04-26T23:20:34.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.416+0000 7fcb8c064640 1 -- start start 2024-04-26T23:20:34.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.417+0000 7fcb8c064640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcb8415f170 0x7fcb840d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:34.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.417+0000 7fcb8c064640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcb8415f550 0x7fcb840d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:34.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.417+0000 7fcb8c064640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb840d34b0 0x7fcb840d3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:34.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.417+0000 7fcb8c064640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fcb840d3e60 con 0x7fcb8415f550 2024-04-26T23:20:34.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.417+0000 7fcb8c064640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fcb840c9430 con 0x7fcb8415f170 2024-04-26T23:20:34.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.417+0000 7fcb8c064640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fcb840d1040 con 0x7fcb840d34b0 2024-04-26T23:20:34.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.417+0000 7fcb895ff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcb8415f550 0x7fcb840d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:34.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.417+0000 7fcb895ff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcb8415f550 0x7fcb840d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51502/0 (socket says 172.21.15.47:51502) 2024-04-26T23:20:34.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.417+0000 7fcb895ff640 1 -- 172.21.15.47:0/30111124 learned_addr learned my addr 172.21.15.47:0/30111124 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:34.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.417+0000 7fcb89e00640 1 --2- 172.21.15.47:0/30111124 >> v2:172.21.15.203:3300/0 conn(0x7fcb8415f170 0x7fcb840d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:34.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.418+0000 7fcb8a601640 1 --2- 172.21.15.47:0/30111124 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb840d34b0 0x7fcb840d3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:34.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.418+0000 7fcb895ff640 1 -- 172.21.15.47:0/30111124 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb840d34b0 msgr2=0x7fcb840d3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:34.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.418+0000 7fcb895ff640 1 --2- 172.21.15.47:0/30111124 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb840d34b0 0x7fcb840d3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:34.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.418+0000 7fcb895ff640 1 -- 172.21.15.47:0/30111124 >> v2:172.21.15.203:3300/0 conn(0x7fcb8415f170 msgr2=0x7fcb840d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:34.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.418+0000 7fcb895ff640 1 --2- 172.21.15.47:0/30111124 >> v2:172.21.15.203:3300/0 conn(0x7fcb8415f170 0x7fcb840d5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:34.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.418+0000 7fcb895ff640 1 -- 172.21.15.47:0/30111124 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb840c9680 con 0x7fcb8415f550 2024-04-26T23:20:34.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.418+0000 7fcb89e00640 1 --2- 172.21.15.47:0/30111124 >> v2:172.21.15.203:3300/0 conn(0x7fcb8415f170 0x7fcb840d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:20:34.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.418+0000 7fcb895ff640 1 --2- 172.21.15.47:0/30111124 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb8415f550 0x7fcb840d2cb0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fcb74009870 tx=0x7fcb7402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceed1e70a721c2c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:34.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.419+0000 7fcb88dfe640 1 -- 172.21.15.47:0/30111124 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb74037070 con 0x7fcb8415f550 2024-04-26T23:20:34.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.419+0000 7fcb88dfe640 1 -- 172.21.15.47:0/30111124 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb74032440 con 0x7fcb8415f550 2024-04-26T23:20:34.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.419+0000 7fcb88dfe640 1 -- 172.21.15.47:0/30111124 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb74032810 con 0x7fcb8415f550 2024-04-26T23:20:34.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.419+0000 7fcb8c064640 1 -- 172.21.15.47:0/30111124 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb8415f550 msgr2=0x7fcb840d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:34.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.419+0000 7fcb8c064640 1 --2- 172.21.15.47:0/30111124 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb8415f550 0x7fcb840d2cb0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fcb74009870 tx=0x7fcb7402a210 comp rx=0 tx=0).stop 2024-04-26T23:20:34.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.419+0000 7fcb8c064640 1 -- 172.21.15.47:0/30111124 shutdown_connections 2024-04-26T23:20:34.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.419+0000 7fcb8c064640 1 --2- 172.21.15.47:0/30111124 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb8415f550 0x7fcb840d2cb0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:34.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.419+0000 7fcb8c064640 1 --2- 172.21.15.47:0/30111124 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb840d34b0 0x7fcb840d3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:34.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.419+0000 7fcb8c064640 1 --2- 172.21.15.47:0/30111124 >> v2:172.21.15.203:3300/0 conn(0x7fcb8415f170 0x7fcb840d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:34.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.419+0000 7fcb8c064640 1 -- 172.21.15.47:0/30111124 >> 172.21.15.47:0/30111124 conn(0x7fcb840d4f60 msgr2=0x7fcb840d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:34.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.419+0000 7fcb8c064640 1 -- 172.21.15.47:0/30111124 shutdown_connections 2024-04-26T23:20:34.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.419+0000 7fcb8c064640 1 -- 172.21.15.47:0/30111124 wait complete. 2024-04-26T23:20:34.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.421+0000 7fcb8c064640 1 Processor -- start 2024-04-26T23:20:34.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.421+0000 7fcb8c064640 1 -- start start 2024-04-26T23:20:34.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.421+0000 7fcb8c064640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb840d34b0 0x7fcb840ce320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:34.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.421+0000 7fcb8c064640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb8415f170 0x7fcb840cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:34.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.421+0000 7fcb8c064640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb8415bf10 0x7fcb840ba0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:34.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.421+0000 7fcb8c064640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fcb840cfc50 con 0x7fcb840d34b0 2024-04-26T23:20:34.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.421+0000 7fcb8c064640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcb840c9c40 con 0x7fcb8415bf10 2024-04-26T23:20:34.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.421+0000 7fcb8c064640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fcb840cfdd0 con 0x7fcb8415f170 2024-04-26T23:20:34.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.421+0000 7fcb89e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb840d34b0 0x7fcb840ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:34.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.421+0000 7fcb89e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb840d34b0 0x7fcb840ce320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51512/0 (socket says 172.21.15.47:51512) 2024-04-26T23:20:34.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.421+0000 7fcb89e00640 1 -- 172.21.15.47:0/1613247064 learned_addr learned my addr 172.21.15.47:0/1613247064 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:34.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.422+0000 7fcb8a601640 1 --2- 172.21.15.47:0/1613247064 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb8415bf10 0x7fcb840ba0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:34.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.422+0000 7fcb89e00640 1 -- 172.21.15.47:0/1613247064 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb8415f170 msgr2=0x7fcb840cbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:34.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.422+0000 7fcb89e00640 1 --2- 172.21.15.47:0/1613247064 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb8415f170 0x7fcb840cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:34.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.422+0000 7fcb89e00640 1 -- 172.21.15.47:0/1613247064 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb8415bf10 msgr2=0x7fcb840ba0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:34.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.422+0000 7fcb89e00640 1 --2- 172.21.15.47:0/1613247064 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb8415bf10 0x7fcb840ba0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:34.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.422+0000 7fcb89e00640 1 -- 172.21.15.47:0/1613247064 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb840ce860 con 0x7fcb840d34b0 2024-04-26T23:20:34.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.422+0000 7fcb8a601640 1 --2- 172.21.15.47:0/1613247064 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb8415bf10 0x7fcb840ba0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:34.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.422+0000 7fcb89e00640 1 --2- 172.21.15.47:0/1613247064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb840d34b0 0x7fcb840ce320 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fcb80009fc0 tx=0x7fcb8000d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:34.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.422+0000 7fcb7a7fc640 1 -- 172.21.15.47:0/1613247064 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8001a070 con 0x7fcb840d34b0 2024-04-26T23:20:34.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.422+0000 7fcb7a7fc640 1 -- 172.21.15.47:0/1613247064 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb80003bb0 con 0x7fcb840d34b0 2024-04-26T23:20:34.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.422+0000 7fcb8c064640 1 -- 172.21.15.47:0/1613247064 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb840cc7d0 con 0x7fcb840d34b0 2024-04-26T23:20:34.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.422+0000 7fcb7a7fc640 1 -- 172.21.15.47:0/1613247064 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb80004030 con 0x7fcb840d34b0 2024-04-26T23:20:34.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.422+0000 7fcb8c064640 1 -- 172.21.15.47:0/1613247064 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb840ccd90 con 0x7fcb840d34b0 2024-04-26T23:20:34.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.423+0000 7fcb8c064640 1 -- 172.21.15.47:0/1613247064 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb840694f0 con 0x7fcb840d34b0 2024-04-26T23:20:34.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.424+0000 7fcb7a7fc640 1 -- 172.21.15.47:0/1613247064 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcb8001d070 con 0x7fcb840d34b0 2024-04-26T23:20:34.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.424+0000 7fcb7a7fc640 1 --2- 172.21.15.47:0/1613247064 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fcb6806c8f0 0x7fcb6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:34.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.424+0000 7fcb7a7fc640 1 -- 172.21.15.47:0/1613247064 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fcb8008bf00 con 0x7fcb840d34b0 2024-04-26T23:20:34.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.424+0000 7fcb895ff640 1 --2- 172.21.15.47:0/1613247064 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fcb6806c8f0 0x7fcb6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:34.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.425+0000 7fcb895ff640 1 --2- 172.21.15.47:0/1613247064 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fcb6806c8f0 0x7fcb6806edb0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fcb74034000 tx=0x7fcb7402c000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:34.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.428+0000 7fcb7a7fc640 1 -- 172.21.15.47:0/1613247064 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcb80010070 con 0x7fcb840d34b0 2024-04-26T23:20:34.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.612+0000 7fcb8c064640 1 -- 172.21.15.47:0/1613247064 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcb840cb950 con 0x7fcb840d34b0 2024-04-26T23:20:34.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.613+0000 7fcb7a7fc640 1 -- 172.21.15.47:0/1613247064 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v54) ==== 100+0+11 (secure 0 0 0) 0x7fcb80059be0 con 0x7fcb840d34b0 2024-04-26T23:20:34.614 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-26T23:20:34.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.615+0000 7fcb8c064640 1 -- 172.21.15.47:0/1613247064 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fcb6806c8f0 msgr2=0x7fcb6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:34.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.615+0000 7fcb8c064640 1 --2- 172.21.15.47:0/1613247064 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fcb6806c8f0 0x7fcb6806edb0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fcb74034000 tx=0x7fcb7402c000 comp rx=0 tx=0).stop 2024-04-26T23:20:34.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.615+0000 7fcb8c064640 1 -- 172.21.15.47:0/1613247064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb840d34b0 msgr2=0x7fcb840ce320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:34.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.615+0000 7fcb8c064640 1 --2- 172.21.15.47:0/1613247064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb840d34b0 0x7fcb840ce320 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fcb80009fc0 tx=0x7fcb8000d560 comp rx=0 tx=0).stop 2024-04-26T23:20:34.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.615+0000 7fcb8c064640 1 -- 172.21.15.47:0/1613247064 shutdown_connections 2024-04-26T23:20:34.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.615+0000 7fcb8c064640 1 --2- 172.21.15.47:0/1613247064 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fcb6806c8f0 0x7fcb6806edb0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:34.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.615+0000 7fcb8c064640 1 --2- 172.21.15.47:0/1613247064 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb8415bf10 0x7fcb840ba0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:34.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.615+0000 7fcb8c064640 1 --2- 172.21.15.47:0/1613247064 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb8415f170 0x7fcb840cbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:34.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.615+0000 7fcb8c064640 1 --2- 172.21.15.47:0/1613247064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb840d34b0 0x7fcb840ce320 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:34.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.615+0000 7fcb8c064640 1 -- 172.21.15.47:0/1613247064 >> 172.21.15.47:0/1613247064 conn(0x7fcb840d4f60 msgr2=0x7fcb840c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:34.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.615+0000 7fcb8c064640 1 -- 172.21.15.47:0/1613247064 shutdown_connections 2024-04-26T23:20:34.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.615+0000 7fcb8c064640 1 -- 172.21.15.47:0/1613247064 wait complete. 2024-04-26T23:20:34.627 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T23:20:34.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.727+0000 7f73bf1f7640 1 Processor -- start 2024-04-26T23:20:34.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.728+0000 7f73bf1f7640 1 -- start start 2024-04-26T23:20:34.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.728+0000 7f73bf1f7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73b806c950 0x7f73b8160ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:34.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.728+0000 7f73bf1f7640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f73b8070930 0x7f73b80aed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:34.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.728+0000 7f73bf1f7640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73b80b0990 con 0x7f73b815d100 2024-04-26T23:20:34.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.728+0000 7f73bf1f7640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f73b80af2d0 con 0x7f73b8070930 2024-04-26T23:20:34.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.728+0000 7f73bf1f7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f73b80b0c70 con 0x7f73b806c950 2024-04-26T23:20:34.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.729+0000 7f73bd794640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f73b8070930 0x7f73b80aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:34.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.729+0000 7f73b7fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73b806c950 0x7f73b8160ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:34.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.729+0000 7f73b7fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73b806c950 0x7f73b8160ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.47:60892/0 (socket says 172.21.15.47:60892) 2024-04-26T23:20:34.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.729+0000 7f73bd794640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f73b8070930 0x7f73b80aed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.47:52566/0 (socket says 172.21.15.47:52566) 2024-04-26T23:20:34.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.729+0000 7f73bcf93640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f73b815d100 0x7f73b806c240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.47:59372/0 (socket says 172.21.15.47:59372) 2024-04-26T23:20:34.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.729+0000 7f73b7fff640 1 -- 172.21.15.47:0/1451579790 learned_addr learned my addr 172.21.15.47:0/1451579790 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:34.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.729+0000 7f73b77fe640 1 -- 172.21.15.47:0/1451579790 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 780579250 0 0) 0x7f73b80b0990 con 0x7f73b815d100 2024-04-26T23:20:34.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.729+0000 7f73b77fe640 1 -- 172.21.15.47:0/1451579790 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f73a0003520 con 0x7f73b815d100 2024-04-26T23:20:34.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.729+0000 7f73b7fff640 1 -- 172.21.15.47:0/1451579790 >> v2:172.21.15.203:3300/0 conn(0x7f73b8070930 msgr2=0x7f73b80aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:34.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.729+0000 7f73b7fff640 1 --2- 172.21.15.47:0/1451579790 >> v2:172.21.15.203:3300/0 conn(0x7f73b8070930 0x7f73b80aed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:34.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.729+0000 7f73b7fff640 1 -- 172.21.15.47:0/1451579790 >> v1:172.21.15.203:6789/0 conn(0x7f73b815d100 legacy=0x7f73b806c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:34.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.729+0000 7f73b7fff640 1 -- 172.21.15.47:0/1451579790 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73b80b0f50 con 0x7f73b806c950 2024-04-26T23:20:34.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.729+0000 7f73bd794640 1 --2- 172.21.15.47:0/1451579790 >> v2:172.21.15.203:3300/0 conn(0x7f73b8070930 0x7f73b80aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:20:34.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.730+0000 7f73b7fff640 1 --2- 172.21.15.47:0/1451579790 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73b806c950 0x7f73b8160ba0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f73ac009870 tx=0x7f73ac02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57b8d559f09c1a98 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:34.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.730+0000 7f73b77fe640 1 -- 172.21.15.47:0/1451579790 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73ac037070 con 0x7f73b806c950 2024-04-26T23:20:34.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.730+0000 7f73b77fe640 1 -- 172.21.15.47:0/1451579790 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73ac032440 con 0x7f73b806c950 2024-04-26T23:20:34.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.730+0000 7f73b77fe640 1 -- 172.21.15.47:0/1451579790 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73ac0327f0 con 0x7f73b806c950 2024-04-26T23:20:34.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.731+0000 7f73bf1f7640 1 -- 172.21.15.47:0/1451579790 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73b806c950 msgr2=0x7f73b8160ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:34.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.731+0000 7f73bf1f7640 1 --2- 172.21.15.47:0/1451579790 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73b806c950 0x7f73b8160ba0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f73ac009870 tx=0x7f73ac02a270 comp rx=0 tx=0).stop 2024-04-26T23:20:34.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.731+0000 7f73bf1f7640 1 -- 172.21.15.47:0/1451579790 shutdown_connections 2024-04-26T23:20:34.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.731+0000 7f73bf1f7640 1 --2- 172.21.15.47:0/1451579790 >> v2:172.21.15.203:3300/0 conn(0x7f73b8070930 0x7f73b80aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:34.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.731+0000 7f73bf1f7640 1 --2- 172.21.15.47:0/1451579790 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73b806c950 0x7f73b8160ba0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:34.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.731+0000 7f73bf1f7640 1 -- 172.21.15.47:0/1451579790 >> 172.21.15.47:0/1451579790 conn(0x7f73b80b39b0 msgr2=0x7f73b80b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:34.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.731+0000 7f73bf1f7640 1 -- 172.21.15.47:0/1451579790 shutdown_connections 2024-04-26T23:20:34.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.731+0000 7f73bf1f7640 1 -- 172.21.15.47:0/1451579790 wait complete. 2024-04-26T23:20:34.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.732+0000 7f73bf1f7640 1 Processor -- start 2024-04-26T23:20:34.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.732+0000 7f73bf1f7640 1 -- start start 2024-04-26T23:20:34.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.733+0000 7f73bf1f7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73b806c950 0x7f73b815fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:34.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.733+0000 7f73bf1f7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73b8070930 0x7f73b8160040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:34.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.733+0000 7f73bf1f7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73b815d100 0x7f73b8152450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:34.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.733+0000 7f73bf1f7640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f73b8073160 con 0x7f73b806c950 2024-04-26T23:20:34.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.733+0000 7f73bf1f7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f73b80b1510 con 0x7f73b815d100 2024-04-26T23:20:34.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.733+0000 7f73bf1f7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f73b80732e0 con 0x7f73b8070930 2024-04-26T23:20:34.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.733+0000 7f73bcf93640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73b806c950 0x7f73b815fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:34.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.733+0000 7f73bcf93640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73b806c950 0x7f73b815fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51516/0 (socket says 172.21.15.47:51516) 2024-04-26T23:20:34.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.733+0000 7f73bcf93640 1 -- 172.21.15.47:0/1661017323 learned_addr learned my addr 172.21.15.47:0/1661017323 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:34.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.733+0000 7f73b7fff640 1 --2- 172.21.15.47:0/1661017323 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73b8070930 0x7f73b8160040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:34.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.733+0000 7f73bd794640 1 --2- 172.21.15.47:0/1661017323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73b815d100 0x7f73b8152450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:34.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.733+0000 7f73bcf93640 1 -- 172.21.15.47:0/1661017323 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73b8070930 msgr2=0x7f73b8160040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:34.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.733+0000 7f73bcf93640 1 --2- 172.21.15.47:0/1661017323 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73b8070930 0x7f73b8160040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:34.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.733+0000 7f73bcf93640 1 -- 172.21.15.47:0/1661017323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73b815d100 msgr2=0x7f73b8152450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:34.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.733+0000 7f73bcf93640 1 --2- 172.21.15.47:0/1661017323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73b815d100 0x7f73b8152450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:34.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.733+0000 7f73bcf93640 1 -- 172.21.15.47:0/1661017323 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73b8153180 con 0x7f73b806c950 2024-04-26T23:20:34.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.733+0000 7f73bcf93640 1 --2- 172.21.15.47:0/1661017323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73b806c950 0x7f73b815fb00 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f73b000c700 tx=0x7f73b000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:34.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.734+0000 7f73b57fa640 1 -- 172.21.15.47:0/1661017323 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b0017070 con 0x7f73b806c950 2024-04-26T23:20:34.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.734+0000 7f73b57fa640 1 -- 172.21.15.47:0/1661017323 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73b000ce10 con 0x7f73b806c950 2024-04-26T23:20:34.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.734+0000 7f73bf1f7640 1 -- 172.21.15.47:0/1661017323 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73b8153460 con 0x7f73b806c950 2024-04-26T23:20:34.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.734+0000 7f73b57fa640 1 -- 172.21.15.47:0/1661017323 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b000a860 con 0x7f73b806c950 2024-04-26T23:20:34.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.734+0000 7f73bf1f7640 1 -- 172.21.15.47:0/1661017323 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73b80b51f0 con 0x7f73b806c950 2024-04-26T23:20:34.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.735+0000 7f73bf1f7640 1 -- 172.21.15.47:0/1661017323 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73b8153980 con 0x7f73b806c950 2024-04-26T23:20:34.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.735+0000 7f73b57fa640 1 -- 172.21.15.47:0/1661017323 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f73b001a070 con 0x7f73b806c950 2024-04-26T23:20:34.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.736+0000 7f73b57fa640 1 --2- 172.21.15.47:0/1661017323 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f73a406c8f0 0x7f73a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:34.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.736+0000 7f73b57fa640 1 -- 172.21.15.47:0/1661017323 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f73b008b320 con 0x7f73b806c950 2024-04-26T23:20:34.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.736+0000 7f73b7fff640 1 --2- 172.21.15.47:0/1661017323 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f73a406c8f0 0x7f73a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:34.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.736+0000 7f73b7fff640 1 --2- 172.21.15.47:0/1661017323 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f73a406c8f0 0x7f73a406edb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f73ac034000 tx=0x7f73ac02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:34.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.740+0000 7f73b57fa640 1 -- 172.21.15.47:0/1661017323 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f73b0058ed0 con 0x7f73b806c950 2024-04-26T23:20:34.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.937+0000 7f73bf1f7640 1 -- 172.21.15.47:0/1661017323 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f73b80694f0 con 0x7f73b806c950 2024-04-26T23:20:34.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.937+0000 7f73b57fa640 1 -- 172.21.15.47:0/1661017323 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v54) ==== 80+0+0 (secure 0 0 0) 0x7f73b005d1d0 con 0x7f73b806c950 2024-04-26T23:20:34.938 INFO:teuthology.orchestra.run.smithi047.stderr:listed 0 entries 2024-04-26T23:20:34.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.939+0000 7f73bf1f7640 1 -- 172.21.15.47:0/1661017323 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f73a406c8f0 msgr2=0x7f73a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:34.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.939+0000 7f73bf1f7640 1 --2- 172.21.15.47:0/1661017323 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f73a406c8f0 0x7f73a406edb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f73ac034000 tx=0x7f73ac02e000 comp rx=0 tx=0).stop 2024-04-26T23:20:34.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.939+0000 7f73bf1f7640 1 -- 172.21.15.47:0/1661017323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73b806c950 msgr2=0x7f73b815fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:34.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.939+0000 7f73bf1f7640 1 --2- 172.21.15.47:0/1661017323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73b806c950 0x7f73b815fb00 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f73b000c700 tx=0x7f73b000cbc0 comp rx=0 tx=0).stop 2024-04-26T23:20:34.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.939+0000 7f73bf1f7640 1 -- 172.21.15.47:0/1661017323 shutdown_connections 2024-04-26T23:20:34.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.939+0000 7f73bf1f7640 1 --2- 172.21.15.47:0/1661017323 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f73a406c8f0 0x7f73a406edb0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:34.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.939+0000 7f73bf1f7640 1 --2- 172.21.15.47:0/1661017323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73b815d100 0x7f73b8152450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:34.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.939+0000 7f73bf1f7640 1 --2- 172.21.15.47:0/1661017323 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73b8070930 0x7f73b8160040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:34.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.939+0000 7f73bf1f7640 1 --2- 172.21.15.47:0/1661017323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73b806c950 0x7f73b815fb00 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:34.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.939+0000 7f73bf1f7640 1 -- 172.21.15.47:0/1661017323 >> 172.21.15.47:0/1661017323 conn(0x7f73b80b39b0 msgr2=0x7f73b806c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:34.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.939+0000 7f73bf1f7640 1 -- 172.21.15.47:0/1661017323 shutdown_connections 2024-04-26T23:20:34.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:34.939+0000 7f73bf1f7640 1 -- 172.21.15.47:0/1661017323 wait complete. 2024-04-26T23:20:34.951 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T23:20:34.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:20:34.951 DEBUG:teuthology.orchestra.run.smithi047:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T23:20:34.979 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi047.front.sepia.ceph.com 2024-04-26T23:20:34.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:20:34.979 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T23:20:35.036 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T23:20:35.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.175+0000 7f830d512640 1 Processor -- start 2024-04-26T23:20:35.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.175+0000 7f830d512640 1 -- start start 2024-04-26T23:20:35.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.176+0000 7f830d512640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f83080d0070 0x7f83080d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:35.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.176+0000 7f830d512640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f830806e270 0x7f83080ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:35.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.176+0000 7f830d512640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83080d2420 con 0x7f830806d560 2024-04-26T23:20:35.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.176+0000 7f830d512640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f8308162d00 con 0x7f83080d0070 2024-04-26T23:20:35.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.176+0000 7f830d512640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f83080d2d10 con 0x7f830806e270 2024-04-26T23:20:35.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.177+0000 7f83077fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f830806e270 0x7f83080ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:35.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.177+0000 7f8306ffd640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f83080d0070 0x7f83080d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:35.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.177+0000 7f83077fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f830806e270 0x7f83080ce190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.47:60910/0 (socket says 172.21.15.47:60910) 2024-04-26T23:20:35.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.177+0000 7f83077fe640 1 -- 172.21.15.47:0/1745542032 learned_addr learned my addr 172.21.15.47:0/1745542032 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:35.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.177+0000 7f83077fe640 1 -- 172.21.15.47:0/1745542032 >> v2:172.21.15.203:3300/0 conn(0x7f83080d0070 msgr2=0x7f83080d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:35.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.177+0000 7f83077fe640 1 --2- 172.21.15.47:0/1745542032 >> v2:172.21.15.203:3300/0 conn(0x7f83080d0070 0x7f83080d1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:35.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.177+0000 7f83077fe640 1 -- 172.21.15.47:0/1745542032 >> v1:172.21.15.203:6789/0 conn(0x7f830806d560 legacy=0x7f830806db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:35.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.177+0000 7f83077fe640 1 -- 172.21.15.47:0/1745542032 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83080cd6b0 con 0x7f830806e270 2024-04-26T23:20:35.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.177+0000 7f8306ffd640 1 --2- 172.21.15.47:0/1745542032 >> v2:172.21.15.203:3300/0 conn(0x7f83080d0070 0x7f83080d1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:20:35.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.178+0000 7f83077fe640 1 --2- 172.21.15.47:0/1745542032 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f830806e270 0x7f83080ce190 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f82f8009e50 tx=0x7f82f802a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8912a175a804249c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:35.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.178+0000 7f8305ffb640 1 -- 172.21.15.47:0/1745542032 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f8036070 con 0x7f830806e270 2024-04-26T23:20:35.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.178+0000 7f8305ffb640 1 -- 172.21.15.47:0/1745542032 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82f8031600 con 0x7f830806e270 2024-04-26T23:20:35.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.178+0000 7f8305ffb640 1 -- 172.21.15.47:0/1745542032 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f80319d0 con 0x7f830806e270 2024-04-26T23:20:35.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.178+0000 7f830d512640 1 -- 172.21.15.47:0/1745542032 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f830806e270 msgr2=0x7f83080ce190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:35.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.178+0000 7f830d512640 1 --2- 172.21.15.47:0/1745542032 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f830806e270 0x7f83080ce190 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f82f8009e50 tx=0x7f82f802a710 comp rx=0 tx=0).stop 2024-04-26T23:20:35.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.179+0000 7f830d512640 1 -- 172.21.15.47:0/1745542032 shutdown_connections 2024-04-26T23:20:35.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.179+0000 7f830d512640 1 --2- 172.21.15.47:0/1745542032 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f830806e270 0x7f83080ce190 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:35.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.179+0000 7f830d512640 1 --2- 172.21.15.47:0/1745542032 >> v2:172.21.15.203:3300/0 conn(0x7f83080d0070 0x7f83080d1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:35.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.179+0000 7f830d512640 1 -- 172.21.15.47:0/1745542032 >> 172.21.15.47:0/1745542032 conn(0x7f83080d2ec0 msgr2=0x7f83080d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:35.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.179+0000 7f830d512640 1 -- 172.21.15.47:0/1745542032 shutdown_connections 2024-04-26T23:20:35.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.179+0000 7f830d512640 1 -- 172.21.15.47:0/1745542032 wait complete. 2024-04-26T23:20:35.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.180+0000 7f830d512640 1 Processor -- start 2024-04-26T23:20:35.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.180+0000 7f830d512640 1 -- start start 2024-04-26T23:20:35.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.181+0000 7f830d512640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f830806d560 0x7f830815b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:35.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.181+0000 7f830d512640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f830806e270 0x7f83080cbd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:35.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.181+0000 7f830d512640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83080d0070 0x7f830815a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:35.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.181+0000 7f830d512640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f83080ce6d0 con 0x7f830806d560 2024-04-26T23:20:35.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.181+0000 7f830d512640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f83080cdc70 con 0x7f83080d0070 2024-04-26T23:20:35.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.181+0000 7f830d512640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f83080ce850 con 0x7f830806e270 2024-04-26T23:20:35.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.181+0000 7f8306ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f830806d560 0x7f830815b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:35.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.181+0000 7f8306ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f830806d560 0x7f830815b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51520/0 (socket says 172.21.15.47:51520) 2024-04-26T23:20:35.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.181+0000 7f8306ffd640 1 -- 172.21.15.47:0/2491624716 learned_addr learned my addr 172.21.15.47:0/2491624716 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:35.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.181+0000 7f83077fe640 1 --2- 172.21.15.47:0/2491624716 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83080d0070 0x7f830815a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:35.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.181+0000 7f83067fc640 1 --2- 172.21.15.47:0/2491624716 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f830806e270 0x7f83080cbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:35.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.181+0000 7f8306ffd640 1 -- 172.21.15.47:0/2491624716 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f830806e270 msgr2=0x7f83080cbd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:35.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.181+0000 7f8306ffd640 1 --2- 172.21.15.47:0/2491624716 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f830806e270 0x7f83080cbd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:35.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.181+0000 7f8306ffd640 1 -- 172.21.15.47:0/2491624716 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83080d0070 msgr2=0x7f830815a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:35.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.181+0000 7f8306ffd640 1 --2- 172.21.15.47:0/2491624716 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83080d0070 0x7f830815a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:35.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.181+0000 7f8306ffd640 1 -- 172.21.15.47:0/2491624716 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f830815b3e0 con 0x7f830806d560 2024-04-26T23:20:35.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.181+0000 7f83067fc640 1 --2- 172.21.15.47:0/2491624716 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f830806e270 0x7f83080cbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:35.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.181+0000 7f8306ffd640 1 --2- 172.21.15.47:0/2491624716 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f830806d560 0x7f830815b720 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f830000e960 tx=0x7f830000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:35.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.181+0000 7f82ff7fe640 1 -- 172.21.15.47:0/2491624716 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8300017070 con 0x7f830806d560 2024-04-26T23:20:35.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.181+0000 7f82ff7fe640 1 -- 172.21.15.47:0/2491624716 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f830000ca00 con 0x7f830806d560 2024-04-26T23:20:35.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.182+0000 7f830d512640 1 -- 172.21.15.47:0/2491624716 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f830815ae80 con 0x7f830806d560 2024-04-26T23:20:35.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.182+0000 7f82ff7fe640 1 -- 172.21.15.47:0/2491624716 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830000cdd0 con 0x7f830806d560 2024-04-26T23:20:35.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.182+0000 7f830d512640 1 -- 172.21.15.47:0/2491624716 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83080c6d10 con 0x7f830806d560 2024-04-26T23:20:35.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.183+0000 7f830d512640 1 -- 172.21.15.47:0/2491624716 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83080694f0 con 0x7f830806d560 2024-04-26T23:20:35.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.183+0000 7f82ff7fe640 1 -- 172.21.15.47:0/2491624716 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f830001a070 con 0x7f830806d560 2024-04-26T23:20:35.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.183+0000 7f82ff7fe640 1 --2- 172.21.15.47:0/2491624716 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f82e806c8f0 0x7f82e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:35.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.183+0000 7f82ff7fe640 1 -- 172.21.15.47:0/2491624716 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f830008bbc0 con 0x7f830806d560 2024-04-26T23:20:35.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.184+0000 7f83067fc640 1 --2- 172.21.15.47:0/2491624716 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f82e806c8f0 0x7f82e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:35.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.184+0000 7f83067fc640 1 --2- 172.21.15.47:0/2491624716 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f82e806c8f0 0x7f82e806edb0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f83080cf3a0 tx=0x7f82f0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:35.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.188+0000 7f82ff7fe640 1 -- 172.21.15.47:0/2491624716 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f830815ae80 con 0x7f830806d560 2024-04-26T23:20:35.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.413+0000 7f830d512640 1 -- 172.21.15.47:0/2491624716 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f83080d0450 con 0x7f830806d560 2024-04-26T23:20:35.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.413+0000 7f82ff7fe640 1 -- 172.21.15.47:0/2491624716 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+73 (secure 0 0 0) 0x7f8300059e60 con 0x7f830806d560 2024-04-26T23:20:35.415 INFO:teuthology.orchestra.run.smithi047.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T23:20:35.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.415+0000 7f830d512640 1 -- 172.21.15.47:0/2491624716 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f82e806c8f0 msgr2=0x7f82e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:35.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.415+0000 7f830d512640 1 --2- 172.21.15.47:0/2491624716 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f82e806c8f0 0x7f82e806edb0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f83080cf3a0 tx=0x7f82f0009040 comp rx=0 tx=0).stop 2024-04-26T23:20:35.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.415+0000 7f830d512640 1 -- 172.21.15.47:0/2491624716 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f830806d560 msgr2=0x7f830815b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:35.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.415+0000 7f830d512640 1 --2- 172.21.15.47:0/2491624716 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f830806d560 0x7f830815b720 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f830000e960 tx=0x7f830000ee20 comp rx=0 tx=0).stop 2024-04-26T23:20:35.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.415+0000 7f830d512640 1 -- 172.21.15.47:0/2491624716 shutdown_connections 2024-04-26T23:20:35.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.415+0000 7f830d512640 1 --2- 172.21.15.47:0/2491624716 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f82e806c8f0 0x7f82e806edb0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:35.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.415+0000 7f830d512640 1 --2- 172.21.15.47:0/2491624716 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83080d0070 0x7f830815a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:35.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.415+0000 7f830d512640 1 --2- 172.21.15.47:0/2491624716 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f830806e270 0x7f83080cbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:35.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.415+0000 7f830d512640 1 --2- 172.21.15.47:0/2491624716 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f830806d560 0x7f830815b720 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:35.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.415+0000 7f830d512640 1 -- 172.21.15.47:0/2491624716 >> 172.21.15.47:0/2491624716 conn(0x7f83080d2ec0 msgr2=0x7f830806db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:35.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.415+0000 7f830d512640 1 -- 172.21.15.47:0/2491624716 shutdown_connections 2024-04-26T23:20:35.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35.415+0000 7f830d512640 1 -- 172.21.15.47:0/2491624716 wait complete. 2024-04-26T23:20:35.427 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T23:20:35.427 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T23:20:35.427 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi047.front.sepia.ceph.com 2024-04-26T23:20:35.427 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T23:20:35.427 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T23:20:35.428 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T23:20:35.428 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T23:20:35.428 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T23:20:35.428 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:20:35.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:20:35.428 DEBUG:teuthology.orchestra.run.smithi047:> ip addr 2024-04-26T23:20:35.444 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T23:20:35.444 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T23:20:35.444 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T23:20:35.444 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:20:35.444 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2024-04-26T23:20:35.444 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:20:35.444 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:20:35.444 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:20:35.444 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f0 2024-04-26T23:20:35.444 INFO:teuthology.orchestra.run.smithi047.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:20:35.444 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:20:35.445 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f1 2024-04-26T23:20:35.445 INFO:teuthology.orchestra.run.smithi047.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T23:20:35.445 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2024-04-26T23:20:35.445 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f0 2024-04-26T23:20:35.445 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-26T23:20:35.445 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42564sec preferred_lft 42564sec 2024-04-26T23:20:35.445 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link noprefixroute 2024-04-26T23:20:35.445 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:20:35.445 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:20:35.445 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2024-04-26T23:20:35.445 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f1 2024-04-26T23:20:35.445 INFO:teuthology.orchestra.run.smithi047.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T23:20:35.445 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 32:8d:47:b1:d0:77 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:20:35.445 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T23:20:35.445 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:20:35.446 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::7c6d:6aff:fe00:4b30/64 scope link 2024-04-26T23:20:35.446 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:20:35.446 INFO:teuthology.orchestra.run.smithi047.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T23:20:35.446 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 32:8d:47:b1:d0:77 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T23:20:35.446 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::308d:47ff:feb1:d077/64 scope link 2024-04-26T23:20:35.446 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:20:35.446 INFO:teuthology.orchestra.run.smithi047.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T23:20:35.446 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether b2:9e:42:6a:5b:5d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T23:20:35.446 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::b09e:42ff:fe6a:5b5d/64 scope link 2024-04-26T23:20:35.447 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:20:35.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:20:35.447 DEBUG:teuthology.orchestra.run.smithi047:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T23:20:35.507 INFO:teuthology.orchestra.run.smithi047.stdout:1 2024-04-26T23:20:35.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:20:35.508 DEBUG:teuthology.orchestra.run.smithi047:> route 2024-04-26T23:20:35.565 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2024-04-26T23:20:35.565 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T23:20:35.565 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T23:20:35.565 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T23:20:35.565 INFO:teuthology.orchestra.run.smithi047.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T23:20:35.565 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:20:35.566 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-04-26T23:20:35.566 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T23:20:35.566 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T23:20:35.566 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T23:20:35.566 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-04-26T23:20:35.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:20:35.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:20:35.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:20:35.692 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-04-26T23:20:35.708 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T23:20:35.708 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T23:20:35.709 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:20:35.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:20:35.709 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set brx.0 up 2024-04-26T23:20:35.773 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:20:35.773 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T23:20:35.830 INFO:teuthology.orchestra.run.smithi047.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:20:35.830 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:20:35.830 DEBUG:teuthology.orchestra.run.smithi047:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T23:20:35.885 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T23:20:35.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:20:35.885 DEBUG:teuthology.orchestra.run.smithi047:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:20:35.968 INFO:teuthology.orchestra.run.smithi047.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:20:35.968 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:20:35.968 INFO:teuthology.orchestra.run.smithi047.stdout:invalid new device string format 2024-04-26T23:20:35.976 INFO:teuthology.orchestra.run.smithi047.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:20:35.977 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:20:35.977 INFO:teuthology.orchestra.run.smithi047.stdout:invalid new device string format 2024-04-26T23:20:35.977 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: resolved to: "172.21.15.47:3300,172.21.15.203:3300,172.21.15.203:3301" 2024-04-26T23:20:35.977 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:20:35.983 INFO:teuthology.orchestra.run.smithi047.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T23:20:35.983 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T23:20:35.983 INFO:teuthology.orchestra.run.smithi047.stdout:invalid new device string format 2024-04-26T23:20:35.983 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: resolved to: "172.21.15.47:3300,172.21.15.203:3300,172.21.15.203:3301" 2024-04-26T23:20:35.983 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T23:20:35.983 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: resolved to: "172.21.15.47:6789,172.21.15.203:6789,172.21.15.203:6790" 2024-04-26T23:20:35.983 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.47:6789,172.21.15.203:6789,172.21.15.203:6790:/ 2024-04-26T23:20:35.984 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=399006d3-4db1-43e6-ba81-a79a276eabfd" will pass to kernel 2024-04-26T23:20:35.984 INFO:teuthology.orchestra.run.smithi047.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T23:20:35.984 INFO:teuthology.orchestra.run.smithi047.stdout: You just mounted a file system that supports labels which does not 2024-04-26T23:20:35.984 INFO:teuthology.orchestra.run.smithi047.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T23:20:35.985 INFO:teuthology.orchestra.run.smithi047.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T23:20:35.985 INFO:teuthology.orchestra.run.smithi047.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T23:20:35.986 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T23:20:35.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:20:35.986 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T23:20:36.051 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T23:20:36.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:20:36.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:20:36.149 INFO:teuthology.orchestra.run.smithi047.stdout:client5602 2024-04-26T23:20:36.149 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:20:36.149 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:20:36.165 DEBUG:teuthology.orchestra.run.smithi047:> stdin-killer -- python3 -c ' 2024-04-26T23:20:36.165 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-04-26T23:20:36.165 DEBUG:teuthology.orchestra.run.smithi047:> import errno 2024-04-26T23:20:36.165 DEBUG:teuthology.orchestra.run.smithi047:> 2024-04-26T23:20:36.165 DEBUG:teuthology.orchestra.run.smithi047:> fd = os.open("/home/ubuntu/cephtest/mnt.0/test_file", os.O_RDWR) 2024-04-26T23:20:36.165 DEBUG:teuthology.orchestra.run.smithi047:> try: 2024-04-26T23:20:36.165 DEBUG:teuthology.orchestra.run.smithi047:> if True: 2024-04-26T23:20:36.165 DEBUG:teuthology.orchestra.run.smithi047:> ret = os.read(fd, 1024) 2024-04-26T23:20:36.165 DEBUG:teuthology.orchestra.run.smithi047:> else: 2024-04-26T23:20:36.165 DEBUG:teuthology.orchestra.run.smithi047:> os.write(fd, b'"'"'content'"'"') 2024-04-26T23:20:36.165 DEBUG:teuthology.orchestra.run.smithi047:> except OSError as e: 2024-04-26T23:20:36.166 DEBUG:teuthology.orchestra.run.smithi047:> if e.errno != errno.EPERM: 2024-04-26T23:20:36.166 DEBUG:teuthology.orchestra.run.smithi047:> raise 2024-04-26T23:20:36.166 DEBUG:teuthology.orchestra.run.smithi047:> else: 2024-04-26T23:20:36.166 DEBUG:teuthology.orchestra.run.smithi047:> raise RuntimeError("client does not check permission of data pool") 2024-04-26T23:20:36.166 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-04-26T23:20:36.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:20:36.284 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2024-04-26T23:20:36.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.430+0000 7f1b192dc640 1 Processor -- start 2024-04-26T23:20:36.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.431+0000 7f1b192dc640 1 -- start start 2024-04-26T23:20:36.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.431+0000 7f1b192dc640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1b1415f170 0x7f1b140d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:36.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.431+0000 7f1b192dc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b1415f550 0x7f1b140d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:36.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.431+0000 7f1b192dc640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1b140d34b0 0x7f1b140d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:36.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.431+0000 7f1b192dc640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f1b140c9430 con 0x7f1b140d34b0 2024-04-26T23:20:36.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.431+0000 7f1b192dc640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f1b140d3e20 con 0x7f1b1415f170 2024-04-26T23:20:36.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.431+0000 7f1b192dc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f1b140d1040 con 0x7f1b1415f550 2024-04-26T23:20:36.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.431+0000 7f1b137fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1b140d34b0 0x7f1b140d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:36.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.432+0000 7f1b137fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1b140d34b0 0x7f1b140d38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51528/0 (socket says 172.21.15.47:51528) 2024-04-26T23:20:36.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.432+0000 7f1b137fe640 1 -- 172.21.15.47:0/2083178725 learned_addr learned my addr 172.21.15.47:0/2083178725 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:36.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.432+0000 7f1b127fc640 1 --2- 172.21.15.47:0/2083178725 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b1415f550 0x7f1b140d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:36.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.432+0000 7f1b12ffd640 1 --2- 172.21.15.47:0/2083178725 >> v2:172.21.15.203:3300/0 conn(0x7f1b1415f170 0x7f1b140d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:36.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.432+0000 7f1b137fe640 1 -- 172.21.15.47:0/2083178725 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b1415f550 msgr2=0x7f1b140d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:36.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.432+0000 7f1b137fe640 1 --2- 172.21.15.47:0/2083178725 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b1415f550 0x7f1b140d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:36.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.432+0000 7f1b137fe640 1 -- 172.21.15.47:0/2083178725 >> v2:172.21.15.203:3300/0 conn(0x7f1b1415f170 msgr2=0x7f1b140d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:36.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.432+0000 7f1b137fe640 1 --2- 172.21.15.47:0/2083178725 >> v2:172.21.15.203:3300/0 conn(0x7f1b1415f170 0x7f1b140d5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:36.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.432+0000 7f1b137fe640 1 -- 172.21.15.47:0/2083178725 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b140c9680 con 0x7f1b140d34b0 2024-04-26T23:20:36.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.432+0000 7f1b12ffd640 1 --2- 172.21.15.47:0/2083178725 >> v2:172.21.15.203:3300/0 conn(0x7f1b1415f170 0x7f1b140d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:36.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.433+0000 7f1b137fe640 1 --2- 172.21.15.47:0/2083178725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b140d34b0 0x7f1b140d38e0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f1b1408d450 tx=0x7f1b0c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=def2b710a5da1318 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:36.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.433+0000 7f1b11ffb640 1 -- 172.21.15.47:0/2083178725 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c036070 con 0x7f1b140d34b0 2024-04-26T23:20:36.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.433+0000 7f1b11ffb640 1 -- 172.21.15.47:0/2083178725 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b0c031600 con 0x7f1b140d34b0 2024-04-26T23:20:36.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.433+0000 7f1b11ffb640 1 -- 172.21.15.47:0/2083178725 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c0319d0 con 0x7f1b140d34b0 2024-04-26T23:20:36.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.433+0000 7f1b192dc640 1 -- 172.21.15.47:0/2083178725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b140d34b0 msgr2=0x7f1b140d38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:36.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.433+0000 7f1b192dc640 1 --2- 172.21.15.47:0/2083178725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b140d34b0 0x7f1b140d38e0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f1b1408d450 tx=0x7f1b0c02a6a0 comp rx=0 tx=0).stop 2024-04-26T23:20:36.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.433+0000 7f1b192dc640 1 -- 172.21.15.47:0/2083178725 shutdown_connections 2024-04-26T23:20:36.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.433+0000 7f1b192dc640 1 --2- 172.21.15.47:0/2083178725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b140d34b0 0x7f1b140d38e0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:36.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.433+0000 7f1b192dc640 1 --2- 172.21.15.47:0/2083178725 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b1415f550 0x7f1b140d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:36.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.433+0000 7f1b192dc640 1 --2- 172.21.15.47:0/2083178725 >> v2:172.21.15.203:3300/0 conn(0x7f1b1415f170 0x7f1b140d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:36.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.433+0000 7f1b192dc640 1 -- 172.21.15.47:0/2083178725 >> 172.21.15.47:0/2083178725 conn(0x7f1b140d4f60 msgr2=0x7f1b140d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:36.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.433+0000 7f1b192dc640 1 -- 172.21.15.47:0/2083178725 shutdown_connections 2024-04-26T23:20:36.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.433+0000 7f1b192dc640 1 -- 172.21.15.47:0/2083178725 wait complete. 2024-04-26T23:20:36.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.435+0000 7f1b192dc640 1 Processor -- start 2024-04-26T23:20:36.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.435+0000 7f1b192dc640 1 -- start start 2024-04-26T23:20:36.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.435+0000 7f1b192dc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b140d34b0 0x7f1b14152340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:36.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.435+0000 7f1b192dc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b1415f170 0x7f1b140ce6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:36.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.435+0000 7f1b192dc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b140cec30 0x7f1b140cc010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:36.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.435+0000 7f1b192dc640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1b140cfc50 con 0x7f1b140d34b0 2024-04-26T23:20:36.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.435+0000 7f1b192dc640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1b140c9c40 con 0x7f1b140cec30 2024-04-26T23:20:36.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.435+0000 7f1b192dc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f1b140cfdd0 con 0x7f1b1415f170 2024-04-26T23:20:36.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.435+0000 7f1b12ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b140d34b0 0x7f1b14152340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:36.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.436+0000 7f1b12ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b140d34b0 0x7f1b14152340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51532/0 (socket says 172.21.15.47:51532) 2024-04-26T23:20:36.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.436+0000 7f1b12ffd640 1 -- 172.21.15.47:0/2442570024 learned_addr learned my addr 172.21.15.47:0/2442570024 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:36.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.436+0000 7f1b127fc640 1 --2- 172.21.15.47:0/2442570024 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b1415f170 0x7f1b140ce6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:36.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.436+0000 7f1b137fe640 1 --2- 172.21.15.47:0/2442570024 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b140cec30 0x7f1b140cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:36.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.436+0000 7f1b12ffd640 1 -- 172.21.15.47:0/2442570024 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b1415f170 msgr2=0x7f1b140ce6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:36.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.436+0000 7f1b12ffd640 1 --2- 172.21.15.47:0/2442570024 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b1415f170 0x7f1b140ce6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:36.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.436+0000 7f1b12ffd640 1 -- 172.21.15.47:0/2442570024 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b140cec30 msgr2=0x7f1b140cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:36.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.436+0000 7f1b12ffd640 1 --2- 172.21.15.47:0/2442570024 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b140cec30 0x7f1b140cc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:36.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.436+0000 7f1b12ffd640 1 -- 172.21.15.47:0/2442570024 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b140ccd80 con 0x7f1b140d34b0 2024-04-26T23:20:36.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.436+0000 7f1b12ffd640 1 --2- 172.21.15.47:0/2442570024 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b140d34b0 0x7f1b14152340 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f1b0400c7c0 tx=0x7f1b0400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:36.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.436+0000 7f1afb7fe640 1 -- 172.21.15.47:0/2442570024 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b04015070 con 0x7f1b140d34b0 2024-04-26T23:20:36.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.436+0000 7f1afb7fe640 1 -- 172.21.15.47:0/2442570024 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b04007a30 con 0x7f1b140d34b0 2024-04-26T23:20:36.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.436+0000 7f1afb7fe640 1 -- 172.21.15.47:0/2442570024 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b04007e00 con 0x7f1b140d34b0 2024-04-26T23:20:36.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.436+0000 7f1b192dc640 1 -- 172.21.15.47:0/2442570024 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b140cd060 con 0x7f1b140d34b0 2024-04-26T23:20:36.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.436+0000 7f1b192dc640 1 -- 172.21.15.47:0/2442570024 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b140cd620 con 0x7f1b140d34b0 2024-04-26T23:20:36.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.438+0000 7f1b192dc640 1 -- 172.21.15.47:0/2442570024 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b140cf040 con 0x7f1b140d34b0 2024-04-26T23:20:36.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.438+0000 7f1afb7fe640 1 -- 172.21.15.47:0/2442570024 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1b04018070 con 0x7f1b140d34b0 2024-04-26T23:20:36.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.438+0000 7f1afb7fe640 1 --2- 172.21.15.47:0/2442570024 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1b0006c8f0 0x7f1b0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:36.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.438+0000 7f1afb7fe640 1 -- 172.21.15.47:0/2442570024 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f1b0408bc90 con 0x7f1b140d34b0 2024-04-26T23:20:36.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.439+0000 7f1b127fc640 1 --2- 172.21.15.47:0/2442570024 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1b0006c8f0 0x7f1b0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:36.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.439+0000 7f1b127fc640 1 --2- 172.21.15.47:0/2442570024 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1b0006c8f0 0x7f1b0006edb0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1b08006fd0 tx=0x7f1b08008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:36.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.443+0000 7f1afb7fe640 1 -- 172.21.15.47:0/2442570024 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1b140cf040 con 0x7f1b140d34b0 2024-04-26T23:20:36.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.699+0000 7f1b192dc640 1 -- 172.21.15.47:0/2442570024 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) -- 0x7f1b140694f0 con 0x7f1b140d34b0 2024-04-26T23:20:36.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.702+0000 7f1afb7fe640 1 -- 172.21.15.47:0/2442570024 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v17) ==== 184+0+141 (secure 0 0 0) 0x7f1b04059eb0 con 0x7f1b140d34b0 2024-04-26T23:20:36.704 INFO:teuthology.orchestra.run.smithi047.stdout:[client.0] 2024-04-26T23:20:36.704 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQAKNixmA5QFHRAAT89WPyzDs0acPluyvU4svw== 2024-04-26T23:20:36.704 INFO:teuthology.orchestra.run.smithi047.stdout: caps mds = "allow" 2024-04-26T23:20:36.704 INFO:teuthology.orchestra.run.smithi047.stdout: caps mon = "allow r" 2024-04-26T23:20:36.704 INFO:teuthology.orchestra.run.smithi047.stdout: caps osd = "allow rw pool=cephfs_data" 2024-04-26T23:20:36.704 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.0 2024-04-26T23:20:36.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.704+0000 7f1b192dc640 1 -- 172.21.15.47:0/2442570024 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1b0006c8f0 msgr2=0x7f1b0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:36.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.704+0000 7f1b192dc640 1 --2- 172.21.15.47:0/2442570024 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1b0006c8f0 0x7f1b0006edb0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1b08006fd0 tx=0x7f1b08008040 comp rx=0 tx=0).stop 2024-04-26T23:20:36.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.704+0000 7f1b192dc640 1 -- 172.21.15.47:0/2442570024 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b140d34b0 msgr2=0x7f1b14152340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:36.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.704+0000 7f1b192dc640 1 --2- 172.21.15.47:0/2442570024 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b140d34b0 0x7f1b14152340 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f1b0400c7c0 tx=0x7f1b0400cc80 comp rx=0 tx=0).stop 2024-04-26T23:20:36.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.704+0000 7f1b192dc640 1 -- 172.21.15.47:0/2442570024 shutdown_connections 2024-04-26T23:20:36.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.704+0000 7f1b192dc640 1 --2- 172.21.15.47:0/2442570024 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1b0006c8f0 0x7f1b0006edb0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:36.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.704+0000 7f1b192dc640 1 --2- 172.21.15.47:0/2442570024 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b140cec30 0x7f1b140cc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:36.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.704+0000 7f1b192dc640 1 --2- 172.21.15.47:0/2442570024 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b1415f170 0x7f1b140ce6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:36.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.704+0000 7f1b192dc640 1 --2- 172.21.15.47:0/2442570024 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b140d34b0 0x7f1b14152340 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:36.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.704+0000 7f1b192dc640 1 -- 172.21.15.47:0/2442570024 >> 172.21.15.47:0/2442570024 conn(0x7f1b140d4f60 msgr2=0x7f1b140c85f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:36.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.704+0000 7f1b192dc640 1 -- 172.21.15.47:0/2442570024 shutdown_connections 2024-04-26T23:20:36.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:36.704+0000 7f1b192dc640 1 -- 172.21.15.47:0/2442570024 wait complete. 2024-04-26T23:20:36.717 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T23:20:36.717 DEBUG:teuthology.orchestra.run.smithi047:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:20:36.760 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T23:20:36.804 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T23:20:36.804 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:20:36.804 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:20:36.804 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:20:36.821 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:20:36.821 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:20:36.876 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:20:37.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.015+0000 7fdb12a37640 1 Processor -- start 2024-04-26T23:20:37.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.015+0000 7fdb12a37640 1 -- start start 2024-04-26T23:20:37.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.016+0000 7fdb12a37640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb0c0cffa0 0x7fdb0c0d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:37.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.016+0000 7fdb12a37640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdb0c0d0a90 0x7fdb0c0b2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:37.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.016+0000 7fdb12a37640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb0c0d0f20 con 0x7fdb0c1608f0 2024-04-26T23:20:37.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.016+0000 7fdb12a37640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fdb0c0cea30 con 0x7fdb0c0d0a90 2024-04-26T23:20:37.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.016+0000 7fdb12a37640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fdb0c0cde20 con 0x7fdb0c0cffa0 2024-04-26T23:20:37.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.016+0000 7fdb10fd4640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdb0c0d0a90 0x7fdb0c0b2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:37.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.016+0000 7fdb0bfff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fdb0c1608f0 0x7fdb0c0cf890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37840/0 (socket says 172.21.15.47:37840) 2024-04-26T23:20:37.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.016+0000 7fdb0bfff640 1 -- 172.21.15.47:0/1801006892 learned_addr learned my addr 172.21.15.47:0/1801006892 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:37.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.016+0000 7fdb0b7fe640 1 --2- 172.21.15.47:0/1801006892 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb0c0cffa0 0x7fdb0c0d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:37.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.017+0000 7fdb10fd4640 1 -- 172.21.15.47:0/1801006892 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb0c0cffa0 msgr2=0x7fdb0c0d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:37.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.017+0000 7fdb10fd4640 1 --2- 172.21.15.47:0/1801006892 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb0c0cffa0 0x7fdb0c0d1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:37.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.017+0000 7fdb10fd4640 1 -- 172.21.15.47:0/1801006892 >> v1:172.21.15.47:6789/0 conn(0x7fdb0c1608f0 legacy=0x7fdb0c0cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:37.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.017+0000 7fdb10fd4640 1 -- 172.21.15.47:0/1801006892 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb0c0ce100 con 0x7fdb0c0d0a90 2024-04-26T23:20:37.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.017+0000 7fdb0affd640 1 -- 172.21.15.47:0/1801006892 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1112820099 0 0) 0x7fdb0c0d0f20 con 0x7fdb0c1608f0 2024-04-26T23:20:37.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.017+0000 7fdb0b7fe640 1 --2- 172.21.15.47:0/1801006892 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb0c0cffa0 0x7fdb0c0d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:20:37.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.017+0000 7fdb10fd4640 1 --2- 172.21.15.47:0/1801006892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb0c0d0a90 0x7fdb0c0b2300 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fdb04009d90 tx=0x7fdb0402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c0f6d99418d025f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:37.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.018+0000 7fdb0affd640 1 -- 172.21.15.47:0/1801006892 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb04007740 con 0x7fdb0c0d0a90 2024-04-26T23:20:37.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.018+0000 7fdb0affd640 1 -- 172.21.15.47:0/1801006892 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb040078e0 con 0x7fdb0c0d0a90 2024-04-26T23:20:37.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.018+0000 7fdb0affd640 1 -- 172.21.15.47:0/1801006892 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb04007cb0 con 0x7fdb0c0d0a90 2024-04-26T23:20:37.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.018+0000 7fdb12a37640 1 -- 172.21.15.47:0/1801006892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb0c0d0a90 msgr2=0x7fdb0c0b2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:37.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.018+0000 7fdb12a37640 1 --2- 172.21.15.47:0/1801006892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb0c0d0a90 0x7fdb0c0b2300 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fdb04009d90 tx=0x7fdb0402a660 comp rx=0 tx=0).stop 2024-04-26T23:20:37.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.018+0000 7fdb12a37640 1 -- 172.21.15.47:0/1801006892 shutdown_connections 2024-04-26T23:20:37.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.018+0000 7fdb12a37640 1 --2- 172.21.15.47:0/1801006892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb0c0d0a90 0x7fdb0c0b2300 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:37.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.018+0000 7fdb12a37640 1 --2- 172.21.15.47:0/1801006892 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb0c0cffa0 0x7fdb0c0d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:37.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.018+0000 7fdb12a37640 1 -- 172.21.15.47:0/1801006892 >> 172.21.15.47:0/1801006892 conn(0x7fdb0c0d2ec0 msgr2=0x7fdb0c0d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:37.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.018+0000 7fdb12a37640 1 -- 172.21.15.47:0/1801006892 shutdown_connections 2024-04-26T23:20:37.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.018+0000 7fdb12a37640 1 -- 172.21.15.47:0/1801006892 wait complete. 2024-04-26T23:20:37.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.019+0000 7fdb12a37640 1 Processor -- start 2024-04-26T23:20:37.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.019+0000 7fdb12a37640 1 -- start start 2024-04-26T23:20:37.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.020+0000 7fdb12a37640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb0c0cffa0 0x7fdb0c1f4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:37.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.020+0000 7fdb12a37640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb0c0d0a90 0x7fdb0c1583b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:37.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.020+0000 7fdb12a37640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb0c1608f0 0x7fdb0c0c6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:37.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.020+0000 7fdb12a37640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0c0ceec0 con 0x7fdb0c0d0a90 2024-04-26T23:20:37.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.020+0000 7fdb12a37640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0c0ce6c0 con 0x7fdb0c0cffa0 2024-04-26T23:20:37.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.020+0000 7fdb12a37640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fdb0c0cda40 con 0x7fdb0c1608f0 2024-04-26T23:20:37.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.020+0000 7fdb0b7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb0c0d0a90 0x7fdb0c1583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:37.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.020+0000 7fdb0b7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb0c0d0a90 0x7fdb0c1583b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51552/0 (socket says 172.21.15.47:51552) 2024-04-26T23:20:37.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.020+0000 7fdb0b7fe640 1 -- 172.21.15.47:0/3373816173 learned_addr learned my addr 172.21.15.47:0/3373816173 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:37.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.020+0000 7fdb0bfff640 1 --2- 172.21.15.47:0/3373816173 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb0c0cffa0 0x7fdb0c1f4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:37.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.020+0000 7fdb10fd4640 1 --2- 172.21.15.47:0/3373816173 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb0c1608f0 0x7fdb0c0c6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:37.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.021+0000 7fdb0b7fe640 1 -- 172.21.15.47:0/3373816173 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb0c1608f0 msgr2=0x7fdb0c0c6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:37.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.021+0000 7fdb0b7fe640 1 --2- 172.21.15.47:0/3373816173 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb0c1608f0 0x7fdb0c0c6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:37.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.021+0000 7fdb0b7fe640 1 -- 172.21.15.47:0/3373816173 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb0c0cffa0 msgr2=0x7fdb0c1f4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:37.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.021+0000 7fdb0b7fe640 1 --2- 172.21.15.47:0/3373816173 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb0c0cffa0 0x7fdb0c1f4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:37.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.021+0000 7fdb0b7fe640 1 -- 172.21.15.47:0/3373816173 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb0c0c6d60 con 0x7fdb0c0d0a90 2024-04-26T23:20:37.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.021+0000 7fdb0b7fe640 1 --2- 172.21.15.47:0/3373816173 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb0c0d0a90 0x7fdb0c1583b0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fdb0000d7d0 tx=0x7fdb0000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:37.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.021+0000 7fdb08ff9640 1 -- 172.21.15.47:0/3373816173 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb00004030 con 0x7fdb0c0d0a90 2024-04-26T23:20:37.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.021+0000 7fdb08ff9640 1 -- 172.21.15.47:0/3373816173 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb000041d0 con 0x7fdb0c0d0a90 2024-04-26T23:20:37.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.021+0000 7fdb12a37640 1 -- 172.21.15.47:0/3373816173 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb0c1f7080 con 0x7fdb0c0d0a90 2024-04-26T23:20:37.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.021+0000 7fdb08ff9640 1 -- 172.21.15.47:0/3373816173 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb000096a0 con 0x7fdb0c0d0a90 2024-04-26T23:20:37.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.021+0000 7fdb12a37640 1 -- 172.21.15.47:0/3373816173 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb0c1f7640 con 0x7fdb0c0d0a90 2024-04-26T23:20:37.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.022+0000 7fdb12a37640 1 -- 172.21.15.47:0/3373816173 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb0c0694f0 con 0x7fdb0c0d0a90 2024-04-26T23:20:37.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.023+0000 7fdb08ff9640 1 -- 172.21.15.47:0/3373816173 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdb00009840 con 0x7fdb0c0d0a90 2024-04-26T23:20:37.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.023+0000 7fdb08ff9640 1 --2- 172.21.15.47:0/3373816173 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdaf806c8f0 0x7fdaf806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:37.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.023+0000 7fdb08ff9640 1 -- 172.21.15.47:0/3373816173 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fdb0008bdb0 con 0x7fdb0c0d0a90 2024-04-26T23:20:37.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.023+0000 7fdb0bfff640 1 --2- 172.21.15.47:0/3373816173 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdaf806c8f0 0x7fdaf806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:37.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.024+0000 7fdb0bfff640 1 --2- 172.21.15.47:0/3373816173 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdaf806c8f0 0x7fdaf806edb0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fdafc000f60 tx=0x7fdafc007d50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:37.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.028+0000 7fdb08ff9640 1 -- 172.21.15.47:0/3373816173 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdb0c0c6d60 con 0x7fdb0c0d0a90 2024-04-26T23:20:37.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.212+0000 7fdb12a37640 1 -- 172.21.15.47:0/3373816173 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdb0c0c6600 con 0x7fdb0c0d0a90 2024-04-26T23:20:37.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.213+0000 7fdb08ff9640 1 -- 172.21.15.47:0/3373816173 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) ==== 74+0+16902 (secure 0 0 0) 0x7fdb00059a90 con 0x7fdb0c0d0a90 2024-04-26T23:20:37.215 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:20:37.216 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":54,"fsid":"399006d3-4db1-43e6-ba81-a79a276eabfd","created":"2024-04-26T23:18:06.731211+0000","modified":"2024-04-26T23:20:23.340537+0000","last_up_change":"2024-04-26T23:18:17.808736+0000","last_in_change":"2024-04-26T23:18:08.945859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:18:16.937492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:20:19.975691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-26T23:20:20.657778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ee091567-fea0-4a14-bf4e-0e16c20bb2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6819","nonce":326520779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6821","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6823","nonce":326520779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6829","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6831","nonce":326520779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6825","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6827","nonce":326520779}]},"public_addr":"172.21.15.47:6819/326520779","cluster_addr":"172.21.15.47:6823/326520779","heartbeat_back_addr":"172.21.15.47:6831/326520779","heartbeat_front_addr":"172.21.15.47:6827/326520779","state":["exists","up"]},{"osd":1,"uuid":"c14440a0-bba5-4931-a6db-b736c08e869b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6809","nonce":1912610931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6811","nonce":1912610931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6815","nonce":1912610931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6813","nonce":1912610931}]},"public_addr":"172.21.15.47:6809/1912610931","cluster_addr":"172.21.15.47:6811/1912610931","heartbeat_back_addr":"172.21.15.47:6815/1912610931","heartbeat_front_addr":"172.21.15.47:6813/1912610931","state":["exists","up"]},{"osd":2,"uuid":"3940f8de-c9b2-4cd3-a911-9c301729eb30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6801","nonce":2569869305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6803","nonce":2569869305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6807","nonce":2569869305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6805","nonce":2569869305}]},"public_addr":"172.21.15.47:6801/2569869305","cluster_addr":"172.21.15.47:6803/2569869305","heartbeat_back_addr":"172.21.15.47:6807/2569869305","heartbeat_front_addr":"172.21.15.47:6805/2569869305","state":["exists","up"]},{"osd":3,"uuid":"1286a67e-8958-426a-8c2e-bc85498c619c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6818","nonce":2791411683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6822","nonce":2791411683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6830","nonce":2791411683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6826","nonce":2791411683}]},"public_addr":"172.21.15.47:6818/2791411683","cluster_addr":"172.21.15.47:6822/2791411683","heartbeat_back_addr":"172.21.15.47:6830/2791411683","heartbeat_front_addr":"172.21.15.47:6826/2791411683","state":["exists","up"]},{"osd":4,"uuid":"d8ee2c20-e5b7-46c2-9999-b234070126cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6809","nonce":441313104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6811","nonce":441313104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6815","nonce":441313104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6813","nonce":441313104}]},"public_addr":"172.21.15.203:6809/441313104","cluster_addr":"172.21.15.203:6811/441313104","heartbeat_back_addr":"172.21.15.203:6815/441313104","heartbeat_front_addr":"172.21.15.203:6813/441313104","state":["exists","up"]},{"osd":5,"uuid":"b453a8f0-f324-4840-9ae0-d67bdadb9f8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6801","nonce":56838444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6803","nonce":56838444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6807","nonce":56838444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6805","nonce":56838444}]},"public_addr":"172.21.15.203:6801/56838444","cluster_addr":"172.21.15.203:6803/56838444","heartbeat_back_addr":"172.21.15.203:6807/56838444","heartbeat_front_addr":"172.21.15.203:6805/56838444","state":["exists","up"]},{"osd":6,"uuid":"a6d31951-2d4b-4e8d-a510-6283b01dc158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6825","nonce":2738517080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6827","nonce":2738517080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6831","nonce":2738517080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6829","nonce":2738517080}]},"public_addr":"172.21.15.203:6825/2738517080","cluster_addr":"172.21.15.203:6827/2738517080","heartbeat_back_addr":"172.21.15.203:6831/2738517080","heartbeat_front_addr":"172.21.15.203:6829/2738517080","state":["exists","up"]},{"osd":7,"uuid":"7119b8c0-fe96-402d-9e75-7b509d9d44d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6817","nonce":1315565058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6819","nonce":1315565058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6823","nonce":1315565058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6821","nonce":1315565058}]},"public_addr":"172.21.15.203:6817/1315565058","cluster_addr":"172.21.15.203:6819/1315565058","heartbeat_back_addr":"172.21.15.203:6823/1315565058","heartbeat_front_addr":"172.21.15.203:6821/1315565058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.121921+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.137494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.084814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.111593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.072430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.125066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.097867+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.057061+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:20:37.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.215+0000 7fdb12a37640 1 -- 172.21.15.47:0/3373816173 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdaf806c8f0 msgr2=0x7fdaf806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:37.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.215+0000 7fdb12a37640 1 --2- 172.21.15.47:0/3373816173 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdaf806c8f0 0x7fdaf806edb0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fdafc000f60 tx=0x7fdafc007d50 comp rx=0 tx=0).stop 2024-04-26T23:20:37.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.215+0000 7fdb12a37640 1 -- 172.21.15.47:0/3373816173 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb0c0d0a90 msgr2=0x7fdb0c1583b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:37.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.215+0000 7fdb12a37640 1 --2- 172.21.15.47:0/3373816173 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb0c0d0a90 0x7fdb0c1583b0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fdb0000d7d0 tx=0x7fdb0000dc90 comp rx=0 tx=0).stop 2024-04-26T23:20:37.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.215+0000 7fdb12a37640 1 -- 172.21.15.47:0/3373816173 shutdown_connections 2024-04-26T23:20:37.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.215+0000 7fdb12a37640 1 --2- 172.21.15.47:0/3373816173 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdaf806c8f0 0x7fdaf806edb0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:37.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.215+0000 7fdb12a37640 1 --2- 172.21.15.47:0/3373816173 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb0c1608f0 0x7fdb0c0c6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:37.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.215+0000 7fdb12a37640 1 --2- 172.21.15.47:0/3373816173 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb0c0d0a90 0x7fdb0c1583b0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:37.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.215+0000 7fdb12a37640 1 --2- 172.21.15.47:0/3373816173 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb0c0cffa0 0x7fdb0c1f4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:37.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.215+0000 7fdb12a37640 1 -- 172.21.15.47:0/3373816173 >> 172.21.15.47:0/3373816173 conn(0x7fdb0c0d2ec0 msgr2=0x7fdb0c071ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:37.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.216+0000 7fdb12a37640 1 -- 172.21.15.47:0/3373816173 shutdown_connections 2024-04-26T23:20:37.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.216+0000 7fdb12a37640 1 -- 172.21.15.47:0/3373816173 wait complete. 2024-04-26T23:20:37.229 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:18:16.937492+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:20:19.975691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:20:20.657778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:20:37.229 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:20:37.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.331+0000 7fd75a064640 1 Processor -- start 2024-04-26T23:20:37.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.331+0000 7fd75a064640 1 -- start start 2024-04-26T23:20:37.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.332+0000 7fd75a064640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd754070350 0x7fd754070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:37.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.332+0000 7fd75a064640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd754071090 0x7fd75415e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:37.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.332+0000 7fd75a064640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7540d2390 con 0x7fd7540cfde0 2024-04-26T23:20:37.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.332+0000 7fd75a064640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fd754071500 con 0x7fd754071090 2024-04-26T23:20:37.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.332+0000 7fd75a064640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd7540afab0 con 0x7fd754070350 2024-04-26T23:20:37.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.332+0000 7fd7537fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd7540cfde0 0x7fd7540d1b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37844/0 (socket says 172.21.15.47:37844) 2024-04-26T23:20:37.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.332+0000 7fd7537fe640 1 -- 172.21.15.47:0/3276384659 learned_addr learned my addr 172.21.15.47:0/3276384659 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:37.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.332+0000 7fd753fff640 1 --2- 172.21.15.47:0/3276384659 >> v2:172.21.15.47:3300/0 conn(0x7fd754071090 0x7fd75415e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:37.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.332+0000 7fd752ffd640 1 --2- 172.21.15.47:0/3276384659 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd754070350 0x7fd754070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:37.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.332+0000 7fd7527fc640 1 -- 172.21.15.47:0/3276384659 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2758067596 0 0) 0x7fd7540d2390 con 0x7fd7540cfde0 2024-04-26T23:20:37.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.332+0000 7fd7527fc640 1 -- 172.21.15.47:0/3276384659 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd73c003520 con 0x7fd7540cfde0 2024-04-26T23:20:37.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.332+0000 7fd753fff640 1 -- 172.21.15.47:0/3276384659 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd754070350 msgr2=0x7fd754070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:37.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.332+0000 7fd753fff640 1 --2- 172.21.15.47:0/3276384659 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd754070350 0x7fd754070b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:37.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.332+0000 7fd753fff640 1 -- 172.21.15.47:0/3276384659 >> v1:172.21.15.47:6789/0 conn(0x7fd7540cfde0 legacy=0x7fd7540d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:37.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.332+0000 7fd753fff640 1 -- 172.21.15.47:0/3276384659 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd75406d3f0 con 0x7fd754071090 2024-04-26T23:20:37.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.332+0000 7fd752ffd640 1 --2- 172.21.15.47:0/3276384659 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd754070350 0x7fd754070b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:20:37.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.333+0000 7fd753fff640 1 --2- 172.21.15.47:0/3276384659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd754071090 0x7fd75415e9f0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fd74c009d90 tx=0x7fd74c02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4d7337a658f5799 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:37.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.333+0000 7fd7527fc640 1 -- 172.21.15.47:0/3276384659 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74c031600 con 0x7fd754071090 2024-04-26T23:20:37.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.333+0000 7fd7527fc640 1 -- 172.21.15.47:0/3276384659 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd74c0317a0 con 0x7fd754071090 2024-04-26T23:20:37.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.333+0000 7fd7527fc640 1 -- 172.21.15.47:0/3276384659 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74c031b50 con 0x7fd754071090 2024-04-26T23:20:37.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.334+0000 7fd75a064640 1 -- 172.21.15.47:0/3276384659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd754071090 msgr2=0x7fd75415e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:37.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.334+0000 7fd75a064640 1 --2- 172.21.15.47:0/3276384659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd754071090 0x7fd75415e9f0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fd74c009d90 tx=0x7fd74c02a5d0 comp rx=0 tx=0).stop 2024-04-26T23:20:37.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.334+0000 7fd75a064640 1 -- 172.21.15.47:0/3276384659 shutdown_connections 2024-04-26T23:20:37.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.334+0000 7fd75a064640 1 --2- 172.21.15.47:0/3276384659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd754071090 0x7fd75415e9f0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:37.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.334+0000 7fd75a064640 1 --2- 172.21.15.47:0/3276384659 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd754070350 0x7fd754070b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:37.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.334+0000 7fd75a064640 1 -- 172.21.15.47:0/3276384659 >> 172.21.15.47:0/3276384659 conn(0x7fd7540d2d80 msgr2=0x7fd7540d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:37.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.334+0000 7fd75a064640 1 -- 172.21.15.47:0/3276384659 shutdown_connections 2024-04-26T23:20:37.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.334+0000 7fd75a064640 1 -- 172.21.15.47:0/3276384659 wait complete. 2024-04-26T23:20:37.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.335+0000 7fd75a064640 1 Processor -- start 2024-04-26T23:20:37.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.335+0000 7fd75a064640 1 -- start start 2024-04-26T23:20:37.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.336+0000 7fd75a064640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd754070350 0x7fd75415a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:37.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.336+0000 7fd75a064640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd754071090 0x7fd7540c9650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:37.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.336+0000 7fd75a064640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7540cfde0 0x7fd75415b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:37.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.336+0000 7fd75a064640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd75406d9b0 con 0x7fd754071090 2024-04-26T23:20:37.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.336+0000 7fd75a064640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd75406dd20 con 0x7fd7540cfde0 2024-04-26T23:20:37.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.336+0000 7fd75a064640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd75415ef30 con 0x7fd754070350 2024-04-26T23:20:37.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.336+0000 7fd752ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd754071090 0x7fd7540c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:37.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.336+0000 7fd752ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd754071090 0x7fd7540c9650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51572/0 (socket says 172.21.15.47:51572) 2024-04-26T23:20:37.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.336+0000 7fd752ffd640 1 -- 172.21.15.47:0/1212209941 learned_addr learned my addr 172.21.15.47:0/1212209941 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:37.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.336+0000 7fd7537fe640 1 --2- 172.21.15.47:0/1212209941 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd754070350 0x7fd75415a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:37.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.336+0000 7fd753fff640 1 --2- 172.21.15.47:0/1212209941 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7540cfde0 0x7fd75415b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:37.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.336+0000 7fd752ffd640 1 -- 172.21.15.47:0/1212209941 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd754070350 msgr2=0x7fd75415a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:37.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.336+0000 7fd752ffd640 1 --2- 172.21.15.47:0/1212209941 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd754070350 0x7fd75415a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:37.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.336+0000 7fd752ffd640 1 -- 172.21.15.47:0/1212209941 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7540cfde0 msgr2=0x7fd75415b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:37.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.336+0000 7fd752ffd640 1 --2- 172.21.15.47:0/1212209941 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7540cfde0 0x7fd75415b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:37.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.336+0000 7fd752ffd640 1 -- 172.21.15.47:0/1212209941 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd75415a690 con 0x7fd754071090 2024-04-26T23:20:37.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.336+0000 7fd753fff640 1 --2- 172.21.15.47:0/1212209941 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7540cfde0 0x7fd75415b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:20:37.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.337+0000 7fd752ffd640 1 --2- 172.21.15.47:0/1212209941 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd754071090 0x7fd7540c9650 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fd74800ebe0 tx=0x7fd74800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:37.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.337+0000 7fd73bfff640 1 -- 172.21.15.47:0/1212209941 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd748017070 con 0x7fd754071090 2024-04-26T23:20:37.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.337+0000 7fd73bfff640 1 -- 172.21.15.47:0/1212209941 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd74800cc30 con 0x7fd754071090 2024-04-26T23:20:37.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.337+0000 7fd75a064640 1 -- 172.21.15.47:0/1212209941 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd75415a970 con 0x7fd754071090 2024-04-26T23:20:37.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.337+0000 7fd73bfff640 1 -- 172.21.15.47:0/1212209941 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd748013470 con 0x7fd754071090 2024-04-26T23:20:37.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.337+0000 7fd75a064640 1 -- 172.21.15.47:0/1212209941 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd75415af30 con 0x7fd754071090 2024-04-26T23:20:37.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.338+0000 7fd75a064640 1 -- 172.21.15.47:0/1212209941 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7540694f0 con 0x7fd754071090 2024-04-26T23:20:37.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.338+0000 7fd73bfff640 1 -- 172.21.15.47:0/1212209941 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd74801a070 con 0x7fd754071090 2024-04-26T23:20:37.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.339+0000 7fd73bfff640 1 --2- 172.21.15.47:0/1212209941 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd74006c920 0x7fd74006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:37.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.339+0000 7fd73bfff640 1 -- 172.21.15.47:0/1212209941 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fd74808be50 con 0x7fd754071090 2024-04-26T23:20:37.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.339+0000 7fd7537fe640 1 --2- 172.21.15.47:0/1212209941 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd74006c920 0x7fd74006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:37.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.340+0000 7fd7537fe640 1 --2- 172.21.15.47:0/1212209941 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd74006c920 0x7fd74006ede0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd744005880 tx=0x7fd744009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:37.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.343+0000 7fd73bfff640 1 -- 172.21.15.47:0/1212209941 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd748059b30 con 0x7fd754071090 2024-04-26T23:20:37.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.528+0000 7fd75a064640 1 -- 172.21.15.47:0/1212209941 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd754071800 con 0x7fd754071090 2024-04-26T23:20:37.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.528+0000 7fd73bfff640 1 -- 172.21.15.47:0/1212209941 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) ==== 93+0+10 (secure 0 0 0) 0x7fd74805de30 con 0x7fd754071090 2024-04-26T23:20:37.532 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-26T23:20:37.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.530+0000 7fd75a064640 1 -- 172.21.15.47:0/1212209941 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd74006c920 msgr2=0x7fd74006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:37.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.530+0000 7fd75a064640 1 --2- 172.21.15.47:0/1212209941 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd74006c920 0x7fd74006ede0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd744005880 tx=0x7fd744009040 comp rx=0 tx=0).stop 2024-04-26T23:20:37.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.530+0000 7fd75a064640 1 -- 172.21.15.47:0/1212209941 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd754071090 msgr2=0x7fd7540c9650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:37.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.530+0000 7fd75a064640 1 --2- 172.21.15.47:0/1212209941 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd754071090 0x7fd7540c9650 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fd74800ebe0 tx=0x7fd74800c520 comp rx=0 tx=0).stop 2024-04-26T23:20:37.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.530+0000 7fd75a064640 1 -- 172.21.15.47:0/1212209941 shutdown_connections 2024-04-26T23:20:37.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.530+0000 7fd75a064640 1 --2- 172.21.15.47:0/1212209941 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd74006c920 0x7fd74006ede0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:37.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.530+0000 7fd75a064640 1 --2- 172.21.15.47:0/1212209941 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7540cfde0 0x7fd75415b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:37.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.530+0000 7fd75a064640 1 --2- 172.21.15.47:0/1212209941 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd754071090 0x7fd7540c9650 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:37.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.530+0000 7fd75a064640 1 --2- 172.21.15.47:0/1212209941 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd754070350 0x7fd75415a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:37.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.530+0000 7fd75a064640 1 -- 172.21.15.47:0/1212209941 >> 172.21.15.47:0/1212209941 conn(0x7fd7540d2d80 msgr2=0x7fd75415e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:37.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.530+0000 7fd75a064640 1 -- 172.21.15.47:0/1212209941 shutdown_connections 2024-04-26T23:20:37.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.530+0000 7fd75a064640 1 -- 172.21.15.47:0/1212209941 wait complete. 2024-04-26T23:20:37.547 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:20:37.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.649+0000 7fa9653af640 1 Processor -- start 2024-04-26T23:20:37.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.649+0000 7fa9653af640 1 -- start start 2024-04-26T23:20:37.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.650+0000 7fa9653af640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9600cfde0 0x7fa9600d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:37.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.650+0000 7fa9653af640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9600d2390 con 0x7fa960071260 2024-04-26T23:20:37.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.650+0000 7fa9653af640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa96006d3f0 con 0x7fa960070350 2024-04-26T23:20:37.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.650+0000 7fa9653af640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa9600afab0 con 0x7fa9600cfde0 2024-04-26T23:20:37.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.650+0000 7fa95f7fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa960071260 0x7fa96015e9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37860/0 (socket says 172.21.15.47:37860) 2024-04-26T23:20:37.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.650+0000 7fa95f7fe640 1 -- 172.21.15.47:0/2476686238 learned_addr learned my addr 172.21.15.47:0/2476686238 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:37.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.650+0000 7fa95effd640 1 --2- 172.21.15.47:0/2476686238 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9600cfde0 0x7fa9600d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:37.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.650+0000 7fa95e7fc640 1 -- 172.21.15.47:0/2476686238 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1877993114 0 0) 0x7fa9600d2390 con 0x7fa960071260 2024-04-26T23:20:37.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.650+0000 7fa95e7fc640 1 -- 172.21.15.47:0/2476686238 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa94c003520 con 0x7fa960071260 2024-04-26T23:20:37.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.650+0000 7fa95e7fc640 1 -- 172.21.15.47:0/2476686238 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4145238553 0 0) 0x7fa96006d3f0 con 0x7fa960070350 2024-04-26T23:20:37.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.650+0000 7fa95e7fc640 1 -- 172.21.15.47:0/2476686238 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa9600d2390 con 0x7fa960070350 2024-04-26T23:20:37.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.650+0000 7fa95e7fc640 1 -- 172.21.15.47:0/2476686238 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 855285994 0 0) 0x7fa94c003520 con 0x7fa960071260 2024-04-26T23:20:37.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.650+0000 7fa95e7fc640 1 -- 172.21.15.47:0/2476686238 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9600cfde0 msgr2=0x7fa9600d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:37.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.650+0000 7fa95e7fc640 1 --2- 172.21.15.47:0/2476686238 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9600cfde0 0x7fa9600d1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:37.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.650+0000 7fa95e7fc640 1 -- 172.21.15.47:0/2476686238 >> v1:172.21.15.203:6789/0 conn(0x7fa960070350 legacy=0x7fa960070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:37.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.650+0000 7fa95e7fc640 1 -- 172.21.15.47:0/2476686238 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa96006d6d0 con 0x7fa960071260 2024-04-26T23:20:37.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.650+0000 7fa95e7fc640 1 -- 172.21.15.47:0/2476686238 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7fa9580019f0 con 0x7fa960071260 2024-04-26T23:20:37.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.650+0000 7fa95effd640 1 --2- 172.21.15.47:0/2476686238 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9600cfde0 0x7fa9600d1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:20:37.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.650+0000 7fa95e7fc640 1 -- 172.21.15.47:0/2476686238 >> v1:172.21.15.47:6789/0 conn(0x7fa960071260 legacy=0x7fa96015e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:37.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.650+0000 7fa95e7fc640 1 --2- 172.21.15.47:0/2476686238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa94c003b70 0x7fa94c003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:37.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.650+0000 7fa95e7fc640 1 --2- 172.21.15.47:0/2476686238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa94c004800 0x7fa94c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:37.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.650+0000 7fa95e7fc640 1 --2- 172.21.15.47:0/2476686238 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9600cfde0 0x7fa94c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:37.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.650+0000 7fa95e7fc640 1 -- 172.21.15.47:0/2476686238 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c00ba70 con 0x7fa94c004800 2024-04-26T23:20:37.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.650+0000 7fa95e7fc640 1 -- 172.21.15.47:0/2476686238 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c00bc20 con 0x7fa94c003b70 2024-04-26T23:20:37.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.650+0000 7fa95e7fc640 1 -- 172.21.15.47:0/2476686238 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa94c00be20 con 0x7fa9600cfde0 2024-04-26T23:20:37.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.651+0000 7fa957fff640 1 --2- 172.21.15.47:0/2476686238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa94c004800 0x7fa94c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:37.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.651+0000 7fa95effd640 1 --2- 172.21.15.47:0/2476686238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa94c003b70 0x7fa94c003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:37.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.651+0000 7fa95f7fe640 1 --2- 172.21.15.47:0/2476686238 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9600cfde0 0x7fa94c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:37.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.651+0000 7fa957fff640 1 -- 172.21.15.47:0/2476686238 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9600cfde0 msgr2=0x7fa94c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:37.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.651+0000 7fa957fff640 1 --2- 172.21.15.47:0/2476686238 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9600cfde0 0x7fa94c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:37.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.651+0000 7fa957fff640 1 -- 172.21.15.47:0/2476686238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa94c003b70 msgr2=0x7fa94c003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:37.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.651+0000 7fa957fff640 1 --2- 172.21.15.47:0/2476686238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa94c003b70 0x7fa94c003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:37.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.651+0000 7fa957fff640 1 -- 172.21.15.47:0/2476686238 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa94c003520 con 0x7fa94c004800 2024-04-26T23:20:37.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.651+0000 7fa957fff640 1 --2- 172.21.15.47:0/2476686238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa94c004800 0x7fa94c004c80 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fa9480018a0 tx=0x7fa94c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e543550539fbdfbf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:37.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.652+0000 7fa95e7fc640 1 -- 172.21.15.47:0/2476686238 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9480014f0 con 0x7fa94c004800 2024-04-26T23:20:37.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.652+0000 7fa95e7fc640 1 -- 172.21.15.47:0/2476686238 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9480052d0 con 0x7fa94c004800 2024-04-26T23:20:37.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.652+0000 7fa95e7fc640 1 -- 172.21.15.47:0/2476686238 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94802ac40 con 0x7fa94c004800 2024-04-26T23:20:37.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.652+0000 7fa9653af640 1 -- 172.21.15.47:0/2476686238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa94c004800 msgr2=0x7fa94c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:37.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.652+0000 7fa9653af640 1 --2- 172.21.15.47:0/2476686238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa94c004800 0x7fa94c004c80 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fa9480018a0 tx=0x7fa94c005830 comp rx=0 tx=0).stop 2024-04-26T23:20:37.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.652+0000 7fa95f7fe640 1 -- 172.21.15.47:0/2476686238 reap_dead start 2024-04-26T23:20:37.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.652+0000 7fa9653af640 1 -- 172.21.15.47:0/2476686238 shutdown_connections 2024-04-26T23:20:37.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.652+0000 7fa9653af640 1 -- 172.21.15.47:0/2476686238 >> 172.21.15.47:0/2476686238 conn(0x7fa9600d2d80 msgr2=0x7fa9600d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:37.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.652+0000 7fa9653af640 1 -- 172.21.15.47:0/2476686238 shutdown_connections 2024-04-26T23:20:37.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.652+0000 7fa9653af640 1 -- 172.21.15.47:0/2476686238 wait complete. 2024-04-26T23:20:37.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.654+0000 7fa9653af640 1 Processor -- start 2024-04-26T23:20:37.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.654+0000 7fa9653af640 1 -- start start 2024-04-26T23:20:37.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.654+0000 7fa9653af640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa94c004800 0x7fa9601f7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:37.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.654+0000 7fa9653af640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9600b0b00 0x7fa9600b0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:37.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.654+0000 7fa9653af640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa960158f70 0x7fa9601593f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:37.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.654+0000 7fa9653af640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa96006f1c0 con 0x7fa960158f70 2024-04-26T23:20:37.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.654+0000 7fa9653af640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa96006dd20 con 0x7fa94c004800 2024-04-26T23:20:37.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.654+0000 7fa9653af640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa96006f340 con 0x7fa9600b0b00 2024-04-26T23:20:37.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.654+0000 7fa95f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa960158f70 0x7fa9601593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:37.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.654+0000 7fa95f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa960158f70 0x7fa9601593f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51596/0 (socket says 172.21.15.47:51596) 2024-04-26T23:20:37.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.654+0000 7fa95f7fe640 1 -- 172.21.15.47:0/609759794 learned_addr learned my addr 172.21.15.47:0/609759794 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:37.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.655+0000 7fa95effd640 1 --2- 172.21.15.47:0/609759794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa94c004800 0x7fa9601f7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:37.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.655+0000 7fa957fff640 1 --2- 172.21.15.47:0/609759794 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9600b0b00 0x7fa9600b0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:37.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.655+0000 7fa95f7fe640 1 -- 172.21.15.47:0/609759794 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9600b0b00 msgr2=0x7fa9600b0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:37.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.655+0000 7fa95f7fe640 1 --2- 172.21.15.47:0/609759794 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9600b0b00 0x7fa9600b0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:37.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.655+0000 7fa95f7fe640 1 -- 172.21.15.47:0/609759794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa94c004800 msgr2=0x7fa9601f7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:37.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.655+0000 7fa95f7fe640 1 --2- 172.21.15.47:0/609759794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa94c004800 0x7fa9601f7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:37.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.655+0000 7fa95f7fe640 1 -- 172.21.15.47:0/609759794 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa960159c60 con 0x7fa960158f70 2024-04-26T23:20:37.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.655+0000 7fa957fff640 1 --2- 172.21.15.47:0/609759794 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9600b0b00 0x7fa9600b0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:37.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.655+0000 7fa95effd640 1 --2- 172.21.15.47:0/609759794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa94c004800 0x7fa9601f7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:37.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.655+0000 7fa95f7fe640 1 --2- 172.21.15.47:0/609759794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa960158f70 0x7fa9601593f0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fa960074430 tx=0x7fa958007b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:37.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.655+0000 7fa9577fe640 1 -- 172.21.15.47:0/609759794 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95800ab70 con 0x7fa960158f70 2024-04-26T23:20:37.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.655+0000 7fa9577fe640 1 -- 172.21.15.47:0/609759794 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa958018070 con 0x7fa960158f70 2024-04-26T23:20:37.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.655+0000 7fa9577fe640 1 -- 172.21.15.47:0/609759794 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9580133f0 con 0x7fa960158f70 2024-04-26T23:20:37.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.655+0000 7fa9653af640 1 -- 172.21.15.47:0/609759794 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa960159f40 con 0x7fa960158f70 2024-04-26T23:20:37.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.655+0000 7fa9653af640 1 -- 172.21.15.47:0/609759794 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa96015a500 con 0x7fa960158f70 2024-04-26T23:20:37.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.656+0000 7fa9653af640 1 -- 172.21.15.47:0/609759794 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9600694f0 con 0x7fa960158f70 2024-04-26T23:20:37.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.657+0000 7fa9577fe640 1 -- 172.21.15.47:0/609759794 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa958013590 con 0x7fa960158f70 2024-04-26T23:20:37.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.657+0000 7fa9577fe640 1 --2- 172.21.15.47:0/609759794 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa94006c8f0 0x7fa94006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:37.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.657+0000 7fa9577fe640 1 -- 172.21.15.47:0/609759794 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fa95808c3a0 con 0x7fa960158f70 2024-04-26T23:20:37.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.657+0000 7fa95effd640 1 --2- 172.21.15.47:0/609759794 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa94006c8f0 0x7fa94006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:37.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.658+0000 7fa95effd640 1 --2- 172.21.15.47:0/609759794 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa94006c8f0 0x7fa94006edb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa9500099d0 tx=0x7fa950008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:37.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.661+0000 7fa9577fe640 1 -- 172.21.15.47:0/609759794 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa95805a5a0 con 0x7fa960158f70 2024-04-26T23:20:37.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.846+0000 7fa9653af640 1 -- 172.21.15.47:0/609759794 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa9601629b0 con 0x7fa960158f70 2024-04-26T23:20:37.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.846+0000 7fa9577fe640 1 -- 172.21.15.47:0/609759794 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v54) ==== 104+0+11 (secure 0 0 0) 0x7fa958059ea0 con 0x7fa960158f70 2024-04-26T23:20:37.848 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-26T23:20:37.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.848+0000 7fa9653af640 1 -- 172.21.15.47:0/609759794 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa94006c8f0 msgr2=0x7fa94006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:37.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.848+0000 7fa9653af640 1 --2- 172.21.15.47:0/609759794 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa94006c8f0 0x7fa94006edb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa9500099d0 tx=0x7fa950008040 comp rx=0 tx=0).stop 2024-04-26T23:20:37.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.848+0000 7fa9653af640 1 -- 172.21.15.47:0/609759794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa960158f70 msgr2=0x7fa9601593f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:37.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.848+0000 7fa9653af640 1 --2- 172.21.15.47:0/609759794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa960158f70 0x7fa9601593f0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fa960074430 tx=0x7fa958007b00 comp rx=0 tx=0).stop 2024-04-26T23:20:37.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.848+0000 7fa9653af640 1 -- 172.21.15.47:0/609759794 shutdown_connections 2024-04-26T23:20:37.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.848+0000 7fa9653af640 1 --2- 172.21.15.47:0/609759794 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa94006c8f0 0x7fa94006edb0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:37.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.848+0000 7fa9653af640 1 --2- 172.21.15.47:0/609759794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa960158f70 0x7fa9601593f0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:37.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.848+0000 7fa9653af640 1 --2- 172.21.15.47:0/609759794 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9600b0b00 0x7fa9600b0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:37.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.848+0000 7fa9653af640 1 --2- 172.21.15.47:0/609759794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa94c004800 0x7fa9601f7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:37.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.848+0000 7fa9653af640 1 -- 172.21.15.47:0/609759794 >> 172.21.15.47:0/609759794 conn(0x7fa9600d2d80 msgr2=0x7fa9600c9710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:37.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.848+0000 7fa9653af640 1 -- 172.21.15.47:0/609759794 shutdown_connections 2024-04-26T23:20:37.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.848+0000 7fa9653af640 1 -- 172.21.15.47:0/609759794 wait complete. 2024-04-26T23:20:37.861 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:20:37.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.964+0000 7f8e06d22640 1 Processor -- start 2024-04-26T23:20:37.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.964+0000 7f8e06d22640 1 -- start start 2024-04-26T23:20:37.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.965+0000 7f8e06d22640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8e000cfde0 0x7f8e000d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:37.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.965+0000 7f8e06d22640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8e00071260 0x7f8e0015e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:37.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.965+0000 7f8e06d22640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e000d2390 con 0x7f8e00070350 2024-04-26T23:20:37.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.965+0000 7f8e06d22640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f8e000afab0 con 0x7f8e000cfde0 2024-04-26T23:20:37.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.965+0000 7f8e06d22640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f8e0006dd20 con 0x7f8e00071260 2024-04-26T23:20:37.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.965+0000 7f8dfffff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f8e00070350 0x7f8e00070b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37876/0 (socket says 172.21.15.47:37876) 2024-04-26T23:20:37.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.965+0000 7f8dfffff640 1 -- 172.21.15.47:0/1463701581 learned_addr learned my addr 172.21.15.47:0/1463701581 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:37.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.965+0000 7f8e04abe640 1 --2- 172.21.15.47:0/1463701581 >> v2:172.21.15.203:3300/0 conn(0x7f8e000cfde0 0x7f8e000d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:37.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.965+0000 7f8e052bf640 1 --2- 172.21.15.47:0/1463701581 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8e00071260 0x7f8e0015e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:37.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.965+0000 7f8dff7fe640 1 -- 172.21.15.47:0/1463701581 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1624044245 0 0) 0x7f8e000d2390 con 0x7f8e00070350 2024-04-26T23:20:37.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.965+0000 7f8dff7fe640 1 -- 172.21.15.47:0/1463701581 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8df0003520 con 0x7f8e00070350 2024-04-26T23:20:37.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.965+0000 7f8dff7fe640 1 -- 172.21.15.47:0/1463701581 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1006701597 0 0) 0x7f8df0003520 con 0x7f8e00070350 2024-04-26T23:20:37.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.965+0000 7f8dff7fe640 1 -- 172.21.15.47:0/1463701581 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8e00071260 msgr2=0x7f8e0015e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:37.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.965+0000 7f8dff7fe640 1 --2- 172.21.15.47:0/1463701581 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8e00071260 0x7f8e0015e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:37.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.965+0000 7f8dff7fe640 1 -- 172.21.15.47:0/1463701581 >> v2:172.21.15.203:3300/0 conn(0x7f8e000cfde0 msgr2=0x7f8e000d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:37.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.965+0000 7f8dff7fe640 1 --2- 172.21.15.47:0/1463701581 >> v2:172.21.15.203:3300/0 conn(0x7f8e000cfde0 0x7f8e000d1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:37.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.965+0000 7f8dff7fe640 1 -- 172.21.15.47:0/1463701581 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e0006d3f0 con 0x7f8e00070350 2024-04-26T23:20:37.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.965+0000 7f8dff7fe640 1 -- 172.21.15.47:0/1463701581 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7f8dec001670 con 0x7f8e00070350 2024-04-26T23:20:37.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.965+0000 7f8dff7fe640 1 -- 172.21.15.47:0/1463701581 >> v1:172.21.15.47:6789/0 conn(0x7f8e00070350 legacy=0x7f8e00070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:37.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.965+0000 7f8dff7fe640 1 --2- 172.21.15.47:0/1463701581 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8df00039b0 0x7f8df0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:37.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.965+0000 7f8dff7fe640 1 --2- 172.21.15.47:0/1463701581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8df0004800 0x7f8df0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:37.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.965+0000 7f8dff7fe640 1 --2- 172.21.15.47:0/1463701581 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8e00071260 0x7f8df00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:37.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.965+0000 7f8dff7fe640 1 -- 172.21.15.47:0/1463701581 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0006dd20 con 0x7f8df0004800 2024-04-26T23:20:37.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.965+0000 7f8dff7fe640 1 -- 172.21.15.47:0/1463701581 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8df000b9c0 con 0x7f8df00039b0 2024-04-26T23:20:37.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.965+0000 7f8dff7fe640 1 -- 172.21.15.47:0/1463701581 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f8df000bb70 con 0x7f8e00071260 2024-04-26T23:20:37.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.966+0000 7f8dfffff640 1 --2- 172.21.15.47:0/1463701581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8df0004800 0x7f8df0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:37.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.966+0000 7f8e04abe640 1 --2- 172.21.15.47:0/1463701581 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8df00039b0 0x7f8df0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:37.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.966+0000 7f8e052bf640 1 --2- 172.21.15.47:0/1463701581 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8e00071260 0x7f8df00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:37.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.966+0000 7f8dfffff640 1 -- 172.21.15.47:0/1463701581 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8e00071260 msgr2=0x7f8df00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:37.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.966+0000 7f8dfffff640 1 --2- 172.21.15.47:0/1463701581 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8e00071260 0x7f8df00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:37.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.966+0000 7f8dfffff640 1 -- 172.21.15.47:0/1463701581 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8df00039b0 msgr2=0x7f8df0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:37.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.966+0000 7f8dfffff640 1 --2- 172.21.15.47:0/1463701581 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8df00039b0 0x7f8df0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:37.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.966+0000 7f8dfffff640 1 -- 172.21.15.47:0/1463701581 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8df0003520 con 0x7f8df0004800 2024-04-26T23:20:37.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.967+0000 7f8dfffff640 1 --2- 172.21.15.47:0/1463701581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8df0004800 0x7f8df0004c80 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f8dec001050 tx=0x7f8df0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5459c89f539ef9ba server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:37.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.967+0000 7f8dff7fe640 1 -- 172.21.15.47:0/1463701581 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dec007d70 con 0x7f8df0004800 2024-04-26T23:20:37.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.967+0000 7f8dff7fe640 1 -- 172.21.15.47:0/1463701581 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dec005af0 con 0x7f8df0004800 2024-04-26T23:20:37.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.967+0000 7f8dff7fe640 1 -- 172.21.15.47:0/1463701581 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dec007070 con 0x7f8df0004800 2024-04-26T23:20:37.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.967+0000 7f8e06d22640 1 -- 172.21.15.47:0/1463701581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8df0004800 msgr2=0x7f8df0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:37.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.967+0000 7f8e06d22640 1 --2- 172.21.15.47:0/1463701581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8df0004800 0x7f8df0004c80 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f8dec001050 tx=0x7f8df0005830 comp rx=0 tx=0).stop 2024-04-26T23:20:37.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.967+0000 7f8e052bf640 1 -- 172.21.15.47:0/1463701581 reap_dead start 2024-04-26T23:20:37.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.967+0000 7f8e06d22640 1 -- 172.21.15.47:0/1463701581 shutdown_connections 2024-04-26T23:20:37.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.967+0000 7f8e06d22640 1 -- 172.21.15.47:0/1463701581 >> 172.21.15.47:0/1463701581 conn(0x7f8e000d2d80 msgr2=0x7f8e000d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:37.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.967+0000 7f8e06d22640 1 -- 172.21.15.47:0/1463701581 shutdown_connections 2024-04-26T23:20:37.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.967+0000 7f8e06d22640 1 -- 172.21.15.47:0/1463701581 wait complete. 2024-04-26T23:20:37.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.968+0000 7f8e06d22640 1 Processor -- start 2024-04-26T23:20:37.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.969+0000 7f8e06d22640 1 -- start start 2024-04-26T23:20:37.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.969+0000 7f8e06d22640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8df0004800 0x7f8e0014b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:37.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.969+0000 7f8e06d22640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e0014bc80 0x7f8e000afab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:37.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.969+0000 7f8e06d22640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8e000afff0 0x7f8e001f4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:37.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.969+0000 7f8e06d22640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8e000d20a0 con 0x7f8e0014bc80 2024-04-26T23:20:37.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.969+0000 7f8e06d22640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0006d9b0 con 0x7f8df0004800 2024-04-26T23:20:37.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.969+0000 7f8e06d22640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f8e0015e9f0 con 0x7f8e000afff0 2024-04-26T23:20:37.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.969+0000 7f8dfffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e0014bc80 0x7f8e000afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:37.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.970+0000 7f8dfffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e0014bc80 0x7f8e000afab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51604/0 (socket says 172.21.15.47:51604) 2024-04-26T23:20:37.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.970+0000 7f8dfffff640 1 -- 172.21.15.47:0/2523835726 learned_addr learned my addr 172.21.15.47:0/2523835726 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:37.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.970+0000 7f8e04abe640 1 --2- 172.21.15.47:0/2523835726 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8df0004800 0x7f8e0014b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:37.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.970+0000 7f8e052bf640 1 --2- 172.21.15.47:0/2523835726 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8e000afff0 0x7f8e001f4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:37.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.970+0000 7f8dfffff640 1 -- 172.21.15.47:0/2523835726 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8e000afff0 msgr2=0x7f8e001f4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:37.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.970+0000 7f8dfffff640 1 --2- 172.21.15.47:0/2523835726 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8e000afff0 0x7f8e001f4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:37.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.970+0000 7f8dfffff640 1 -- 172.21.15.47:0/2523835726 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8df0004800 msgr2=0x7f8e0014b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:37.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.970+0000 7f8dfffff640 1 --2- 172.21.15.47:0/2523835726 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8df0004800 0x7f8e0014b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:37.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.970+0000 7f8dfffff640 1 -- 172.21.15.47:0/2523835726 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e0009f0a0 con 0x7f8e0014bc80 2024-04-26T23:20:37.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.970+0000 7f8dfffff640 1 --2- 172.21.15.47:0/2523835726 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e0014bc80 0x7f8e000afab0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f8dec00c300 tx=0x7f8dec000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:37.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.970+0000 7f8dfd7fa640 1 -- 172.21.15.47:0/2523835726 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dec042070 con 0x7f8e0014bc80 2024-04-26T23:20:37.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.970+0000 7f8dfd7fa640 1 -- 172.21.15.47:0/2523835726 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dec007480 con 0x7f8e0014bc80 2024-04-26T23:20:37.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.970+0000 7f8dfd7fa640 1 -- 172.21.15.47:0/2523835726 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dec007d50 con 0x7f8e0014bc80 2024-04-26T23:20:37.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.970+0000 7f8e06d22640 1 -- 172.21.15.47:0/2523835726 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e0009f310 con 0x7f8e0014bc80 2024-04-26T23:20:37.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.970+0000 7f8e06d22640 1 -- 172.21.15.47:0/2523835726 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e0014c100 con 0x7f8e0014bc80 2024-04-26T23:20:37.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.971+0000 7f8e06d22640 1 -- 172.21.15.47:0/2523835726 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e000694f0 con 0x7f8e0014bc80 2024-04-26T23:20:37.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.972+0000 7f8dfd7fa640 1 -- 172.21.15.47:0/2523835726 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8dec045070 con 0x7f8e0014bc80 2024-04-26T23:20:37.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.972+0000 7f8dfd7fa640 1 --2- 172.21.15.47:0/2523835726 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f8de406c920 0x7f8de406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:37.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.972+0000 7f8dfd7fa640 1 -- 172.21.15.47:0/2523835726 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f8dec0ac780 con 0x7f8e0014bc80 2024-04-26T23:20:37.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.972+0000 7f8e04abe640 1 --2- 172.21.15.47:0/2523835726 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f8de406c920 0x7f8de406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:37.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.973+0000 7f8e04abe640 1 --2- 172.21.15.47:0/2523835726 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f8de406c920 0x7f8de406ede0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8e000b0fe0 tx=0x7f8df4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:37.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:37.977+0000 7f8dfd7fa640 1 -- 172.21.15.47:0/2523835726 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8dec04c030 con 0x7f8e0014bc80 2024-04-26T23:20:38.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.160+0000 7f8e06d22640 1 -- 172.21.15.47:0/2523835726 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8e000b03d0 con 0x7f8e0014bc80 2024-04-26T23:20:38.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.161+0000 7f8dfd7fa640 1 -- 172.21.15.47:0/2523835726 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v54) ==== 100+0+11 (secure 0 0 0) 0x7f8dec03c340 con 0x7f8e0014bc80 2024-04-26T23:20:38.163 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-26T23:20:38.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.163+0000 7f8e06d22640 1 -- 172.21.15.47:0/2523835726 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f8de406c920 msgr2=0x7f8de406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:38.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.163+0000 7f8e06d22640 1 --2- 172.21.15.47:0/2523835726 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f8de406c920 0x7f8de406ede0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8e000b0fe0 tx=0x7f8df4008040 comp rx=0 tx=0).stop 2024-04-26T23:20:38.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.163+0000 7f8e06d22640 1 -- 172.21.15.47:0/2523835726 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e0014bc80 msgr2=0x7f8e000afab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:38.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.163+0000 7f8e06d22640 1 --2- 172.21.15.47:0/2523835726 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e0014bc80 0x7f8e000afab0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f8dec00c300 tx=0x7f8dec000fe0 comp rx=0 tx=0).stop 2024-04-26T23:20:38.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.163+0000 7f8e06d22640 1 -- 172.21.15.47:0/2523835726 shutdown_connections 2024-04-26T23:20:38.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.163+0000 7f8e06d22640 1 --2- 172.21.15.47:0/2523835726 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f8de406c920 0x7f8de406ede0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:38.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.163+0000 7f8e06d22640 1 --2- 172.21.15.47:0/2523835726 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8e000afff0 0x7f8e001f4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:38.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.163+0000 7f8e06d22640 1 --2- 172.21.15.47:0/2523835726 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e0014bc80 0x7f8e000afab0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:38.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.163+0000 7f8e06d22640 1 --2- 172.21.15.47:0/2523835726 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8df0004800 0x7f8e0014b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:38.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.163+0000 7f8e06d22640 1 -- 172.21.15.47:0/2523835726 >> 172.21.15.47:0/2523835726 conn(0x7f8e000d2d80 msgr2=0x7f8e000b1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:38.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.163+0000 7f8e06d22640 1 -- 172.21.15.47:0/2523835726 shutdown_connections 2024-04-26T23:20:38.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.163+0000 7f8e06d22640 1 -- 172.21.15.47:0/2523835726 wait complete. 2024-04-26T23:20:38.176 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T23:20:38.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.275+0000 7f3bb0b5f640 1 Processor -- start 2024-04-26T23:20:38.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.275+0000 7f3bb0b5f640 1 -- start start 2024-04-26T23:20:38.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.276+0000 7f3bb0b5f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bac0d1b30 0x7f3bac0d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:38.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.276+0000 7f3bb0b5f640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bac0d10d0 con 0x7f3bac0b2340 2024-04-26T23:20:38.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.276+0000 7f3bb0b5f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bac0d1370 con 0x7f3bac162810 2024-04-26T23:20:38.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.276+0000 7f3bb0b5f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3bac0d0750 con 0x7f3bac0d1b30 2024-04-26T23:20:38.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.276+0000 7f3ba3fff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f3bac0b2340 0x7f3bac0cd3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37884/0 (socket says 172.21.15.47:37884) 2024-04-26T23:20:38.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.276+0000 7f3ba3fff640 1 -- 172.21.15.47:0/2425608228 learned_addr learned my addr 172.21.15.47:0/2425608228 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:38.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.276+0000 7f3baad9d640 1 --2- 172.21.15.47:0/2425608228 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bac0d1b30 0x7f3bac0d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:38.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.276+0000 7f3ba9d9b640 1 -- 172.21.15.47:0/2425608228 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 924585370 0 0) 0x7f3bac0d10d0 con 0x7f3bac0b2340 2024-04-26T23:20:38.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.276+0000 7f3ba9d9b640 1 -- 172.21.15.47:0/2425608228 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b98003520 con 0x7f3bac0b2340 2024-04-26T23:20:38.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.276+0000 7f3ba9d9b640 1 -- 172.21.15.47:0/2425608228 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1996977622 0 0) 0x7f3bac0d1370 con 0x7f3bac162810 2024-04-26T23:20:38.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.276+0000 7f3baad9d640 1 -- 172.21.15.47:0/2425608228 >> v1:172.21.15.203:6789/0 conn(0x7f3bac162810 legacy=0x7f3bac070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:38.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.276+0000 7f3baad9d640 1 -- 172.21.15.47:0/2425608228 >> v1:172.21.15.47:6789/0 conn(0x7f3bac0b2340 legacy=0x7f3bac0cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:38.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.276+0000 7f3baad9d640 1 -- 172.21.15.47:0/2425608228 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bac0d01b0 con 0x7f3bac0d1b30 2024-04-26T23:20:38.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.277+0000 7f3baad9d640 1 --2- 172.21.15.47:0/2425608228 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bac0d1b30 0x7f3bac0d1fc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3ba4009dc0 tx=0x7f3ba402a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b81494ba3e11376c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:38.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.277+0000 7f3ba9d9b640 1 -- 172.21.15.47:0/2425608228 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba4007740 con 0x7f3bac0d1b30 2024-04-26T23:20:38.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.277+0000 7f3ba9d9b640 1 -- 172.21.15.47:0/2425608228 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ba40078e0 con 0x7f3bac0d1b30 2024-04-26T23:20:38.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.277+0000 7f3ba9d9b640 1 -- 172.21.15.47:0/2425608228 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba4007cb0 con 0x7f3bac0d1b30 2024-04-26T23:20:38.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.278+0000 7f3bb0b5f640 1 -- 172.21.15.47:0/2425608228 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bac0d1b30 msgr2=0x7f3bac0d1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:38.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.278+0000 7f3bb0b5f640 1 --2- 172.21.15.47:0/2425608228 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bac0d1b30 0x7f3bac0d1fc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3ba4009dc0 tx=0x7f3ba402a6d0 comp rx=0 tx=0).stop 2024-04-26T23:20:38.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.278+0000 7f3bb0b5f640 1 -- 172.21.15.47:0/2425608228 shutdown_connections 2024-04-26T23:20:38.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.278+0000 7f3bb0b5f640 1 --2- 172.21.15.47:0/2425608228 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bac0d1b30 0x7f3bac0d1fc0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:38.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.278+0000 7f3bb0b5f640 1 -- 172.21.15.47:0/2425608228 >> 172.21.15.47:0/2425608228 conn(0x7f3bac0d2da0 msgr2=0x7f3bac0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:38.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.278+0000 7f3bb0b5f640 1 -- 172.21.15.47:0/2425608228 shutdown_connections 2024-04-26T23:20:38.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.278+0000 7f3bb0b5f640 1 -- 172.21.15.47:0/2425608228 wait complete. 2024-04-26T23:20:38.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.279+0000 7f3bb0b5f640 1 Processor -- start 2024-04-26T23:20:38.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.279+0000 7f3bb0b5f640 1 -- start start 2024-04-26T23:20:38.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.280+0000 7f3bb0b5f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bac0b2340 0x7f3bac1f5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:38.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.280+0000 7f3bb0b5f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3bac0d1b30 0x7f3bac0c8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:38.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.280+0000 7f3bb0b5f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bac162810 0x7f3bac15bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:38.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.280+0000 7f3bb0b5f640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac0cdb00 con 0x7f3bac0d1b30 2024-04-26T23:20:38.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.280+0000 7f3bb0b5f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac0d16a0 con 0x7f3bac162810 2024-04-26T23:20:38.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.280+0000 7f3bb0b5f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3bac070b50 con 0x7f3bac0b2340 2024-04-26T23:20:38.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.280+0000 7f3ba3fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3bac0d1b30 0x7f3bac0c8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:38.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.280+0000 7f3ba3fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3bac0d1b30 0x7f3bac0c8650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51612/0 (socket says 172.21.15.47:51612) 2024-04-26T23:20:38.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.280+0000 7f3ba3fff640 1 -- 172.21.15.47:0/2316856818 learned_addr learned my addr 172.21.15.47:0/2316856818 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:38.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.280+0000 7f3baa59c640 1 --2- 172.21.15.47:0/2316856818 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bac0b2340 0x7f3bac1f5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:38.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.280+0000 7f3baad9d640 1 --2- 172.21.15.47:0/2316856818 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bac162810 0x7f3bac15bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:38.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.281+0000 7f3ba3fff640 1 -- 172.21.15.47:0/2316856818 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bac0b2340 msgr2=0x7f3bac1f5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:38.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.281+0000 7f3ba3fff640 1 --2- 172.21.15.47:0/2316856818 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bac0b2340 0x7f3bac1f5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:38.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.281+0000 7f3ba3fff640 1 -- 172.21.15.47:0/2316856818 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bac162810 msgr2=0x7f3bac15bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:38.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.281+0000 7f3ba3fff640 1 --2- 172.21.15.47:0/2316856818 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bac162810 0x7f3bac15bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:38.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.281+0000 7f3ba3fff640 1 -- 172.21.15.47:0/2316856818 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bac1f5c90 con 0x7f3bac0d1b30 2024-04-26T23:20:38.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.281+0000 7f3ba3fff640 1 --2- 172.21.15.47:0/2316856818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3bac0d1b30 0x7f3bac0c8650 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f3b9400c430 tx=0x7f3b9400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:38.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.281+0000 7f3ba2ffd640 1 -- 172.21.15.47:0/2316856818 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b94017070 con 0x7f3bac0d1b30 2024-04-26T23:20:38.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.281+0000 7f3ba2ffd640 1 -- 172.21.15.47:0/2316856818 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b94005ce0 con 0x7f3bac0d1b30 2024-04-26T23:20:38.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.281+0000 7f3bb0b5f640 1 -- 172.21.15.47:0/2316856818 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bac0c6de0 con 0x7f3bac0d1b30 2024-04-26T23:20:38.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.281+0000 7f3ba2ffd640 1 -- 172.21.15.47:0/2316856818 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b940072f0 con 0x7f3bac0d1b30 2024-04-26T23:20:38.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.281+0000 7f3bb0b5f640 1 -- 172.21.15.47:0/2316856818 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bac0c73a0 con 0x7f3bac0d1b30 2024-04-26T23:20:38.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.282+0000 7f3bb0b5f640 1 -- 172.21.15.47:0/2316856818 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bac0694f0 con 0x7f3bac0d1b30 2024-04-26T23:20:38.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.282+0000 7f3ba2ffd640 1 -- 172.21.15.47:0/2316856818 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3b9401a070 con 0x7f3bac0d1b30 2024-04-26T23:20:38.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.283+0000 7f3ba2ffd640 1 --2- 172.21.15.47:0/2316856818 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3b8c06c8f0 0x7f3b8c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:38.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.283+0000 7f3ba2ffd640 1 -- 172.21.15.47:0/2316856818 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f3b9408b110 con 0x7f3bac0d1b30 2024-04-26T23:20:38.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.283+0000 7f3baa59c640 1 --2- 172.21.15.47:0/2316856818 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3b8c06c8f0 0x7f3b8c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:38.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.283+0000 7f3baa59c640 1 --2- 172.21.15.47:0/2316856818 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3b8c06c8f0 0x7f3b8c06edb0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c007fd0 tx=0x7f3b9c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:38.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.287+0000 7f3ba2ffd640 1 -- 172.21.15.47:0/2316856818 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3bac1f5c90 con 0x7f3bac0d1b30 2024-04-26T23:20:38.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.490+0000 7f3bb0b5f640 1 -- 172.21.15.47:0/2316856818 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3bac0c9e50 con 0x7f3bac0d1b30 2024-04-26T23:20:38.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.490+0000 7f3ba2ffd640 1 -- 172.21.15.47:0/2316856818 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v54) ==== 80+0+0 (secure 0 0 0) 0x7f3b94059440 con 0x7f3bac0d1b30 2024-04-26T23:20:38.492 INFO:teuthology.orchestra.run.smithi047.stderr:listed 0 entries 2024-04-26T23:20:38.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.492+0000 7f3bb0b5f640 1 -- 172.21.15.47:0/2316856818 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3b8c06c8f0 msgr2=0x7f3b8c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:38.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.492+0000 7f3bb0b5f640 1 --2- 172.21.15.47:0/2316856818 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3b8c06c8f0 0x7f3b8c06edb0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c007fd0 tx=0x7f3b9c009040 comp rx=0 tx=0).stop 2024-04-26T23:20:38.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.492+0000 7f3bb0b5f640 1 -- 172.21.15.47:0/2316856818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3bac0d1b30 msgr2=0x7f3bac0c8650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:38.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.492+0000 7f3bb0b5f640 1 --2- 172.21.15.47:0/2316856818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3bac0d1b30 0x7f3bac0c8650 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f3b9400c430 tx=0x7f3b9400c8f0 comp rx=0 tx=0).stop 2024-04-26T23:20:38.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.492+0000 7f3bb0b5f640 1 -- 172.21.15.47:0/2316856818 shutdown_connections 2024-04-26T23:20:38.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.492+0000 7f3bb0b5f640 1 --2- 172.21.15.47:0/2316856818 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3b8c06c8f0 0x7f3b8c06edb0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:38.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.492+0000 7f3bb0b5f640 1 --2- 172.21.15.47:0/2316856818 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bac162810 0x7f3bac15bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:38.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.492+0000 7f3bb0b5f640 1 --2- 172.21.15.47:0/2316856818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3bac0d1b30 0x7f3bac0c8650 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:38.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.492+0000 7f3bb0b5f640 1 --2- 172.21.15.47:0/2316856818 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bac0b2340 0x7f3bac1f5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:38.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.492+0000 7f3bb0b5f640 1 -- 172.21.15.47:0/2316856818 >> 172.21.15.47:0/2316856818 conn(0x7f3bac0d2da0 msgr2=0x7f3bac0cd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:38.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.492+0000 7f3bb0b5f640 1 -- 172.21.15.47:0/2316856818 shutdown_connections 2024-04-26T23:20:38.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.493+0000 7f3bb0b5f640 1 -- 172.21.15.47:0/2316856818 wait complete. 2024-04-26T23:20:38.505 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T23:20:38.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:20:38.506 DEBUG:teuthology.orchestra.run.smithi047:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T23:20:38.534 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi047.front.sepia.ceph.com 2024-04-26T23:20:38.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:20:38.534 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T23:20:38.589 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:20:38.589 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-26T23:20:38.589 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:20:38.608 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056088,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:20:38.608 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:20:38.609 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:20:38.609 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:20:38.645 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=361 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T23:20:38.646 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T23:20:38.646 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-26T23:20:38.646 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:20:38.666 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056088,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:20:38.666 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:20:38.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.785+0000 7ff467146640 1 Processor -- start 2024-04-26T23:20:38.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.785+0000 7ff467146640 1 -- start start 2024-04-26T23:20:38.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.786+0000 7ff467146640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff4600d3bc0 0x7ff4600d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:38.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.786+0000 7ff467146640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff4600b5f20 0x7ff4600b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:38.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.786+0000 7ff467146640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4600b7e50 con 0x7ff460162880 2024-04-26T23:20:38.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.786+0000 7ff467146640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7ff4600d0580 con 0x7ff4600b5f20 2024-04-26T23:20:38.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.786+0000 7ff467146640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7ff4600d1800 con 0x7ff4600d3bc0 2024-04-26T23:20:38.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.786+0000 7ff4656e3640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff4600b5f20 0x7ff4600b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:38.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.786+0000 7ff464ee2640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff460162880 0x7ff4600d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37894/0 (socket says 172.21.15.47:37894) 2024-04-26T23:20:38.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.786+0000 7ff464ee2640 1 -- 172.21.15.47:0/3260723906 learned_addr learned my addr 172.21.15.47:0/3260723906 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:38.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.786+0000 7ff45ffff640 1 --2- 172.21.15.47:0/3260723906 >> v2:172.21.15.203:3300/0 conn(0x7ff4600d3bc0 0x7ff4600d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:38.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.786+0000 7ff4656e3640 1 -- 172.21.15.47:0/3260723906 >> v2:172.21.15.203:3300/0 conn(0x7ff4600d3bc0 msgr2=0x7ff4600d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:38.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.786+0000 7ff4656e3640 1 --2- 172.21.15.47:0/3260723906 >> v2:172.21.15.203:3300/0 conn(0x7ff4600d3bc0 0x7ff4600d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:38.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.786+0000 7ff4656e3640 1 -- 172.21.15.47:0/3260723906 >> v1:172.21.15.47:6789/0 conn(0x7ff460162880 legacy=0x7ff4600d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:38.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.786+0000 7ff4656e3640 1 -- 172.21.15.47:0/3260723906 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4600d07d0 con 0x7ff4600b5f20 2024-04-26T23:20:38.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.786+0000 7ff45f7fe640 1 -- 172.21.15.47:0/3260723906 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4188248384 0 0) 0x7ff4600b7e50 con 0x7ff460162880 2024-04-26T23:20:38.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.787+0000 7ff4656e3640 1 --2- 172.21.15.47:0/3260723906 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4600b5f20 0x7ff4600b8300 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff46008d3d0 tx=0x7ff45802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f23e8a6706ea8164 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:38.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.787+0000 7ff45f7fe640 1 -- 172.21.15.47:0/3260723906 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff458007740 con 0x7ff4600b5f20 2024-04-26T23:20:38.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.787+0000 7ff45f7fe640 1 -- 172.21.15.47:0/3260723906 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4580078e0 con 0x7ff4600b5f20 2024-04-26T23:20:38.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.787+0000 7ff45f7fe640 1 -- 172.21.15.47:0/3260723906 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff458007cb0 con 0x7ff4600b5f20 2024-04-26T23:20:38.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.788+0000 7ff467146640 1 -- 172.21.15.47:0/3260723906 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4600b5f20 msgr2=0x7ff4600b8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:38.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.788+0000 7ff467146640 1 --2- 172.21.15.47:0/3260723906 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4600b5f20 0x7ff4600b8300 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff46008d3d0 tx=0x7ff45802a680 comp rx=0 tx=0).stop 2024-04-26T23:20:38.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.788+0000 7ff467146640 1 -- 172.21.15.47:0/3260723906 shutdown_connections 2024-04-26T23:20:38.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.788+0000 7ff467146640 1 --2- 172.21.15.47:0/3260723906 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4600b5f20 0x7ff4600b8300 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:38.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.788+0000 7ff467146640 1 --2- 172.21.15.47:0/3260723906 >> v2:172.21.15.203:3300/0 conn(0x7ff4600d3bc0 0x7ff4600d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:38.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.788+0000 7ff467146640 1 -- 172.21.15.47:0/3260723906 >> 172.21.15.47:0/3260723906 conn(0x7ff4600d4f60 msgr2=0x7ff4600d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:38.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.788+0000 7ff467146640 1 -- 172.21.15.47:0/3260723906 shutdown_connections 2024-04-26T23:20:38.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.788+0000 7ff467146640 1 -- 172.21.15.47:0/3260723906 wait complete. 2024-04-26T23:20:38.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.789+0000 7ff467146640 1 Processor -- start 2024-04-26T23:20:38.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.789+0000 7ff467146640 1 -- start start 2024-04-26T23:20:38.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.790+0000 7ff467146640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4600b5f20 0x7ff4600b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:38.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.790+0000 7ff467146640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4600d3bc0 0x7ff4600b4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:38.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.790+0000 7ff467146640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff460162880 0x7ff4600b9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:38.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.790+0000 7ff467146640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff460162c60 con 0x7ff4600d3bc0 2024-04-26T23:20:38.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.790+0000 7ff467146640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff4600d0d90 con 0x7ff460162880 2024-04-26T23:20:38.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.790+0000 7ff467146640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7ff460162de0 con 0x7ff4600b5f20 2024-04-26T23:20:38.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.790+0000 7ff45ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4600d3bc0 0x7ff4600b4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:38.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.790+0000 7ff45ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4600d3bc0 0x7ff4600b4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51640/0 (socket says 172.21.15.47:51640) 2024-04-26T23:20:38.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.790+0000 7ff45ffff640 1 -- 172.21.15.47:0/3301401047 learned_addr learned my addr 172.21.15.47:0/3301401047 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:38.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.790+0000 7ff464ee2640 1 --2- 172.21.15.47:0/3301401047 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4600b5f20 0x7ff4600b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:38.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.790+0000 7ff4656e3640 1 --2- 172.21.15.47:0/3301401047 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff460162880 0x7ff4600b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:38.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.790+0000 7ff45ffff640 1 -- 172.21.15.47:0/3301401047 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4600b5f20 msgr2=0x7ff4600b3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:38.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.790+0000 7ff45ffff640 1 --2- 172.21.15.47:0/3301401047 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4600b5f20 0x7ff4600b3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:38.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.790+0000 7ff45ffff640 1 -- 172.21.15.47:0/3301401047 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff460162880 msgr2=0x7ff4600b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:38.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.790+0000 7ff45ffff640 1 --2- 172.21.15.47:0/3301401047 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff460162880 0x7ff4600b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:38.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.790+0000 7ff45ffff640 1 -- 172.21.15.47:0/3301401047 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4600b48a0 con 0x7ff4600d3bc0 2024-04-26T23:20:38.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.790+0000 7ff464ee2640 1 --2- 172.21.15.47:0/3301401047 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4600b5f20 0x7ff4600b3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:20:38.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.790+0000 7ff4656e3640 1 --2- 172.21.15.47:0/3301401047 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff460162880 0x7ff4600b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:20:38.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.790+0000 7ff45ffff640 1 --2- 172.21.15.47:0/3301401047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4600d3bc0 0x7ff4600b4070 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff44c00e9e0 tx=0x7ff44c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:38.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.790+0000 7ff45d7fa640 1 -- 172.21.15.47:0/3301401047 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff44c017070 con 0x7ff4600d3bc0 2024-04-26T23:20:38.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.790+0000 7ff45d7fa640 1 -- 172.21.15.47:0/3301401047 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff44c00c970 con 0x7ff4600d3bc0 2024-04-26T23:20:38.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.790+0000 7ff467146640 1 -- 172.21.15.47:0/3301401047 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4600b4b80 con 0x7ff4600d3bc0 2024-04-26T23:20:38.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.790+0000 7ff45d7fa640 1 -- 172.21.15.47:0/3301401047 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff44c00cd60 con 0x7ff4600d3bc0 2024-04-26T23:20:38.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.791+0000 7ff467146640 1 -- 172.21.15.47:0/3301401047 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4600b5140 con 0x7ff4600d3bc0 2024-04-26T23:20:38.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.791+0000 7ff467146640 1 -- 172.21.15.47:0/3301401047 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4600cd7f0 con 0x7ff4600d3bc0 2024-04-26T23:20:38.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.792+0000 7ff45d7fa640 1 -- 172.21.15.47:0/3301401047 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff44c01a070 con 0x7ff4600d3bc0 2024-04-26T23:20:38.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.792+0000 7ff45d7fa640 1 --2- 172.21.15.47:0/3301401047 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7ff44406c8f0 0x7ff44406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:38.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.793+0000 7ff45d7fa640 1 -- 172.21.15.47:0/3301401047 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7ff44c08bfe0 con 0x7ff4600d3bc0 2024-04-26T23:20:38.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.793+0000 7ff464ee2640 1 --2- 172.21.15.47:0/3301401047 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7ff44406c8f0 0x7ff44406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:38.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.793+0000 7ff464ee2640 1 --2- 172.21.15.47:0/3301401047 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7ff44406c8f0 0x7ff44406edb0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff454000f60 tx=0x7ff454007c10 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:38.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:38.797+0000 7ff45d7fa640 1 -- 172.21.15.47:0/3301401047 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff44c05a2d0 con 0x7ff4600d3bc0 2024-04-26T23:20:39.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.020+0000 7ff467146640 1 -- 172.21.15.47:0/3301401047 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4600b45b0 con 0x7ff4600d3bc0 2024-04-26T23:20:39.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.020+0000 7ff45d7fa640 1 -- 172.21.15.47:0/3301401047 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5190 (secure 0 0 0) 0x7ff44c059bd0 con 0x7ff4600d3bc0 2024-04-26T23:20:39.023 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:20:39.023 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5865,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/805147373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":805147373},{"type":"v1","addr":"172.21.15.47:6833","nonce":805147373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5868,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3515322868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3515322868},{"type":"v1","addr":"172.21.15.47:6835","nonce":3515322868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":15008,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/4223205269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":4223205269},{"type":"v1","addr":"172.21.15.203:6837","nonce":4223205269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:20:21.712604+0000","modified":"2024-04-26T23:20:24.725165+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":15011},"failed":[],"damaged":[],"stopped":[],"info":{"gid_15011":{"gid":15011,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6836/1168022146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1168022146},{"type":"v1","addr":"172.21.15.203:6836","nonce":1168022146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":15011,"qdb_cluster":[15011]},"id":3}]} 2024-04-26T23:20:39.023 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 27 2024-04-26T23:20:39.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.022+0000 7ff467146640 1 -- 172.21.15.47:0/3301401047 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7ff44406c8f0 msgr2=0x7ff44406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:39.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.022+0000 7ff467146640 1 --2- 172.21.15.47:0/3301401047 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7ff44406c8f0 0x7ff44406edb0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff454000f60 tx=0x7ff454007c10 comp rx=0 tx=0).stop 2024-04-26T23:20:39.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.022+0000 7ff467146640 1 -- 172.21.15.47:0/3301401047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4600d3bc0 msgr2=0x7ff4600b4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:39.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.022+0000 7ff467146640 1 --2- 172.21.15.47:0/3301401047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4600d3bc0 0x7ff4600b4070 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff44c00e9e0 tx=0x7ff44c00eea0 comp rx=0 tx=0).stop 2024-04-26T23:20:39.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.022+0000 7ff467146640 1 -- 172.21.15.47:0/3301401047 shutdown_connections 2024-04-26T23:20:39.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.022+0000 7ff467146640 1 --2- 172.21.15.47:0/3301401047 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7ff44406c8f0 0x7ff44406edb0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:39.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.022+0000 7ff467146640 1 --2- 172.21.15.47:0/3301401047 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff460162880 0x7ff4600b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:39.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.022+0000 7ff467146640 1 --2- 172.21.15.47:0/3301401047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4600d3bc0 0x7ff4600b4070 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:39.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.022+0000 7ff467146640 1 --2- 172.21.15.47:0/3301401047 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4600b5f20 0x7ff4600b3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:39.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.022+0000 7ff467146640 1 -- 172.21.15.47:0/3301401047 >> 172.21.15.47:0/3301401047 conn(0x7ff4600d4f60 msgr2=0x7ff4600d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:39.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.023+0000 7ff467146640 1 -- 172.21.15.47:0/3301401047 shutdown_connections 2024-04-26T23:20:39.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.023+0000 7ff467146640 1 -- 172.21.15.47:0/3301401047 wait complete. 2024-04-26T23:20:39.036 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:20:39.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.134+0000 7fee14fda640 1 Processor -- start 2024-04-26T23:20:39.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.134+0000 7fee14fda640 1 -- start start 2024-04-26T23:20:39.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.135+0000 7fee14fda640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fee100d06f0 0x7fee100d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:39.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.135+0000 7fee14fda640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee100d0ba0 con 0x7fee100b2850 2024-04-26T23:20:39.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.135+0000 7fee14fda640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee100d0e20 con 0x7fee1015f3a0 2024-04-26T23:20:39.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.135+0000 7fee14fda640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fee100b1d70 con 0x7fee100d06f0 2024-04-26T23:20:39.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.135+0000 7fee0dd9b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fee100d06f0 0x7fee100d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:39.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.135+0000 7fee0dd9b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fee100d06f0 0x7fee100d1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51656/0 (socket says 172.21.15.47:51656) 2024-04-26T23:20:39.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.135+0000 7fee0dd9b640 1 -- 172.21.15.47:0/2115682743 learned_addr learned my addr 172.21.15.47:0/2115682743 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:39.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.136+0000 7fee0dd9b640 1 -- 172.21.15.47:0/2115682743 >> v1:172.21.15.203:6789/0 conn(0x7fee1015f3a0 legacy=0x7fee100d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:39.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.136+0000 7fee0dd9b640 1 -- 172.21.15.47:0/2115682743 >> v1:172.21.15.47:6789/0 conn(0x7fee100b2850 legacy=0x7fee100b2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:39.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.136+0000 7fee0dd9b640 1 -- 172.21.15.47:0/2115682743 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee100b2050 con 0x7fee100d06f0 2024-04-26T23:20:39.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.136+0000 7fee0d59a640 1 -- 172.21.15.47:0/2115682743 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1688978016 0 0) 0x7fee00001490 con 0x7fee100b2850 2024-04-26T23:20:39.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.136+0000 7fee0dd9b640 1 --2- 172.21.15.47:0/2115682743 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee100d06f0 0x7fee100d1e80 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fedf8009870 tx=0x7fedf802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9a81d50919b65fc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:39.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.137+0000 7fee0d59a640 1 -- 172.21.15.47:0/2115682743 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf8037070 con 0x7fee100d06f0 2024-04-26T23:20:39.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.137+0000 7fee0d59a640 1 -- 172.21.15.47:0/2115682743 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedf802ae50 con 0x7fee100d06f0 2024-04-26T23:20:39.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.137+0000 7fee0d59a640 1 -- 172.21.15.47:0/2115682743 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf8032860 con 0x7fee100d06f0 2024-04-26T23:20:39.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.137+0000 7fee14fda640 1 -- 172.21.15.47:0/2115682743 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee100d06f0 msgr2=0x7fee100d1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:39.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.137+0000 7fee14fda640 1 --2- 172.21.15.47:0/2115682743 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee100d06f0 0x7fee100d1e80 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fedf8009870 tx=0x7fedf802a300 comp rx=0 tx=0).stop 2024-04-26T23:20:39.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.137+0000 7fee14fda640 1 -- 172.21.15.47:0/2115682743 shutdown_connections 2024-04-26T23:20:39.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.137+0000 7fee14fda640 1 --2- 172.21.15.47:0/2115682743 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee100d06f0 0x7fee100d1e80 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:39.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.137+0000 7fee14fda640 1 -- 172.21.15.47:0/2115682743 >> 172.21.15.47:0/2115682743 conn(0x7fee100d4f90 msgr2=0x7fee100d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:39.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.137+0000 7fee14fda640 1 -- 172.21.15.47:0/2115682743 shutdown_connections 2024-04-26T23:20:39.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.137+0000 7fee14fda640 1 -- 172.21.15.47:0/2115682743 wait complete. 2024-04-26T23:20:39.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.138+0000 7fee14fda640 1 Processor -- start 2024-04-26T23:20:39.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.139+0000 7fee14fda640 1 -- start start 2024-04-26T23:20:39.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.139+0000 7fee14fda640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee100b2850 0x7fee100c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:39.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.139+0000 7fee14fda640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee100d06f0 0x7fee101f8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:39.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.139+0000 7fee14fda640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee1015f3a0 0x7fee1014dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:39.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.139+0000 7fee14fda640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fee1015f900 con 0x7fee1015f3a0 2024-04-26T23:20:39.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.139+0000 7fee14fda640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fee1015f780 con 0x7fee100b2850 2024-04-26T23:20:39.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.139+0000 7fee14fda640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fee100d23c0 con 0x7fee100d06f0 2024-04-26T23:20:39.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.139+0000 7fee0ed9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee1015f3a0 0x7fee1014dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:39.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.139+0000 7fee0ed9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee1015f3a0 0x7fee1014dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51664/0 (socket says 172.21.15.47:51664) 2024-04-26T23:20:39.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.139+0000 7fee0ed9d640 1 -- 172.21.15.47:0/2476810467 learned_addr learned my addr 172.21.15.47:0/2476810467 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:39.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.140+0000 7fee0e59c640 1 --2- 172.21.15.47:0/2476810467 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee100b2850 0x7fee100c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:39.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.140+0000 7fee0dd9b640 1 --2- 172.21.15.47:0/2476810467 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee100d06f0 0x7fee101f8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:39.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.140+0000 7fee0ed9d640 1 -- 172.21.15.47:0/2476810467 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee100d06f0 msgr2=0x7fee101f8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:39.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.140+0000 7fee0ed9d640 1 --2- 172.21.15.47:0/2476810467 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee100d06f0 0x7fee101f8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:39.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.140+0000 7fee0ed9d640 1 -- 172.21.15.47:0/2476810467 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee100b2850 msgr2=0x7fee100c8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:39.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.140+0000 7fee0ed9d640 1 --2- 172.21.15.47:0/2476810467 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee100b2850 0x7fee100c8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:39.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.140+0000 7fee0ed9d640 1 -- 172.21.15.47:0/2476810467 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee100c9310 con 0x7fee1015f3a0 2024-04-26T23:20:39.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.140+0000 7fee0dd9b640 1 --2- 172.21.15.47:0/2476810467 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee100d06f0 0x7fee101f8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:20:39.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.140+0000 7fee0ed9d640 1 --2- 172.21.15.47:0/2476810467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee1015f3a0 0x7fee1014dbd0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fee0000c7c0 tx=0x7fee0000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:39.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.140+0000 7fee06ffd640 1 -- 172.21.15.47:0/2476810467 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee00003da0 con 0x7fee1015f3a0 2024-04-26T23:20:39.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.140+0000 7fee06ffd640 1 -- 172.21.15.47:0/2476810467 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee00005070 con 0x7fee1015f3a0 2024-04-26T23:20:39.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.140+0000 7fee06ffd640 1 -- 172.21.15.47:0/2476810467 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee00005460 con 0x7fee1015f3a0 2024-04-26T23:20:39.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.140+0000 7fee14fda640 1 -- 172.21.15.47:0/2476810467 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee100c95f0 con 0x7fee1015f3a0 2024-04-26T23:20:39.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.140+0000 7fee14fda640 1 -- 172.21.15.47:0/2476810467 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee100c9bb0 con 0x7fee1015f3a0 2024-04-26T23:20:39.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.141+0000 7fee14fda640 1 -- 172.21.15.47:0/2476810467 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee100694f0 con 0x7fee1015f3a0 2024-04-26T23:20:39.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.142+0000 7fee06ffd640 1 -- 172.21.15.47:0/2476810467 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fee00005600 con 0x7fee1015f3a0 2024-04-26T23:20:39.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.142+0000 7fee06ffd640 1 --2- 172.21.15.47:0/2476810467 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fedf006c8f0 0x7fedf006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:39.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.142+0000 7fee06ffd640 1 -- 172.21.15.47:0/2476810467 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fee0008bba0 con 0x7fee1015f3a0 2024-04-26T23:20:39.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.143+0000 7fee0e59c640 1 --2- 172.21.15.47:0/2476810467 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fedf006c8f0 0x7fedf006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:39.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.143+0000 7fee0e59c640 1 --2- 172.21.15.47:0/2476810467 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fedf006c8f0 0x7fedf006edb0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fee08000f60 tx=0x7fee08009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:39.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.147+0000 7fee06ffd640 1 -- 172.21.15.47:0/2476810467 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fee100d0ba0 con 0x7fee1015f3a0 2024-04-26T23:20:39.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.334+0000 7fee14fda640 1 -- 172.21.15.47:0/2476810467 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fee100c8410 con 0x7fee1015f3a0 2024-04-26T23:20:39.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.335+0000 7fee06ffd640 1 -- 172.21.15.47:0/2476810467 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) ==== 74+0+16902 (secure 0 0 0) 0x7fee00059e40 con 0x7fee1015f3a0 2024-04-26T23:20:39.337 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:20:39.337 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":54,"fsid":"399006d3-4db1-43e6-ba81-a79a276eabfd","created":"2024-04-26T23:18:06.731211+0000","modified":"2024-04-26T23:20:23.340537+0000","last_up_change":"2024-04-26T23:18:17.808736+0000","last_in_change":"2024-04-26T23:18:08.945859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:18:16.937492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:20:19.975691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-26T23:20:20.657778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ee091567-fea0-4a14-bf4e-0e16c20bb2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6819","nonce":326520779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6821","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6823","nonce":326520779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6829","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6831","nonce":326520779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6825","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6827","nonce":326520779}]},"public_addr":"172.21.15.47:6819/326520779","cluster_addr":"172.21.15.47:6823/326520779","heartbeat_back_addr":"172.21.15.47:6831/326520779","heartbeat_front_addr":"172.21.15.47:6827/326520779","state":["exists","up"]},{"osd":1,"uuid":"c14440a0-bba5-4931-a6db-b736c08e869b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6809","nonce":1912610931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6811","nonce":1912610931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6815","nonce":1912610931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6813","nonce":1912610931}]},"public_addr":"172.21.15.47:6809/1912610931","cluster_addr":"172.21.15.47:6811/1912610931","heartbeat_back_addr":"172.21.15.47:6815/1912610931","heartbeat_front_addr":"172.21.15.47:6813/1912610931","state":["exists","up"]},{"osd":2,"uuid":"3940f8de-c9b2-4cd3-a911-9c301729eb30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6801","nonce":2569869305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6803","nonce":2569869305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6807","nonce":2569869305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6805","nonce":2569869305}]},"public_addr":"172.21.15.47:6801/2569869305","cluster_addr":"172.21.15.47:6803/2569869305","heartbeat_back_addr":"172.21.15.47:6807/2569869305","heartbeat_front_addr":"172.21.15.47:6805/2569869305","state":["exists","up"]},{"osd":3,"uuid":"1286a67e-8958-426a-8c2e-bc85498c619c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6818","nonce":2791411683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6822","nonce":2791411683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6830","nonce":2791411683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6826","nonce":2791411683}]},"public_addr":"172.21.15.47:6818/2791411683","cluster_addr":"172.21.15.47:6822/2791411683","heartbeat_back_addr":"172.21.15.47:6830/2791411683","heartbeat_front_addr":"172.21.15.47:6826/2791411683","state":["exists","up"]},{"osd":4,"uuid":"d8ee2c20-e5b7-46c2-9999-b234070126cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6809","nonce":441313104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6811","nonce":441313104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6815","nonce":441313104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6813","nonce":441313104}]},"public_addr":"172.21.15.203:6809/441313104","cluster_addr":"172.21.15.203:6811/441313104","heartbeat_back_addr":"172.21.15.203:6815/441313104","heartbeat_front_addr":"172.21.15.203:6813/441313104","state":["exists","up"]},{"osd":5,"uuid":"b453a8f0-f324-4840-9ae0-d67bdadb9f8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6801","nonce":56838444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6803","nonce":56838444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6807","nonce":56838444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6805","nonce":56838444}]},"public_addr":"172.21.15.203:6801/56838444","cluster_addr":"172.21.15.203:6803/56838444","heartbeat_back_addr":"172.21.15.203:6807/56838444","heartbeat_front_addr":"172.21.15.203:6805/56838444","state":["exists","up"]},{"osd":6,"uuid":"a6d31951-2d4b-4e8d-a510-6283b01dc158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6825","nonce":2738517080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6827","nonce":2738517080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6831","nonce":2738517080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6829","nonce":2738517080}]},"public_addr":"172.21.15.203:6825/2738517080","cluster_addr":"172.21.15.203:6827/2738517080","heartbeat_back_addr":"172.21.15.203:6831/2738517080","heartbeat_front_addr":"172.21.15.203:6829/2738517080","state":["exists","up"]},{"osd":7,"uuid":"7119b8c0-fe96-402d-9e75-7b509d9d44d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6817","nonce":1315565058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6819","nonce":1315565058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6823","nonce":1315565058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6821","nonce":1315565058}]},"public_addr":"172.21.15.203:6817/1315565058","cluster_addr":"172.21.15.203:6819/1315565058","heartbeat_back_addr":"172.21.15.203:6823/1315565058","heartbeat_front_addr":"172.21.15.203:6821/1315565058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.121921+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.137494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.084814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.111593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.072430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.125066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.097867+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.057061+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:20:39.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.337+0000 7fee14fda640 1 -- 172.21.15.47:0/2476810467 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fedf006c8f0 msgr2=0x7fedf006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:39.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.337+0000 7fee14fda640 1 --2- 172.21.15.47:0/2476810467 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fedf006c8f0 0x7fedf006edb0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fee08000f60 tx=0x7fee08009040 comp rx=0 tx=0).stop 2024-04-26T23:20:39.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.337+0000 7fee14fda640 1 -- 172.21.15.47:0/2476810467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee1015f3a0 msgr2=0x7fee1014dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:39.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.337+0000 7fee14fda640 1 --2- 172.21.15.47:0/2476810467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee1015f3a0 0x7fee1014dbd0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fee0000c7c0 tx=0x7fee0000cc80 comp rx=0 tx=0).stop 2024-04-26T23:20:39.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.337+0000 7fee14fda640 1 -- 172.21.15.47:0/2476810467 shutdown_connections 2024-04-26T23:20:39.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.337+0000 7fee14fda640 1 --2- 172.21.15.47:0/2476810467 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fedf006c8f0 0x7fedf006edb0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:39.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.337+0000 7fee14fda640 1 --2- 172.21.15.47:0/2476810467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee1015f3a0 0x7fee1014dbd0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:39.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.337+0000 7fee14fda640 1 --2- 172.21.15.47:0/2476810467 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee100d06f0 0x7fee101f8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:39.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.337+0000 7fee14fda640 1 --2- 172.21.15.47:0/2476810467 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee100b2850 0x7fee100c8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:39.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.337+0000 7fee14fda640 1 -- 172.21.15.47:0/2476810467 >> 172.21.15.47:0/2476810467 conn(0x7fee100d4f90 msgr2=0x7fee100b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:39.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.337+0000 7fee14fda640 1 -- 172.21.15.47:0/2476810467 shutdown_connections 2024-04-26T23:20:39.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.337+0000 7fee14fda640 1 -- 172.21.15.47:0/2476810467 wait complete. 2024-04-26T23:20:39.350 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:18:16.937492+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:20:19.975691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:20:20.657778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:20:39.351 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:20:39.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.453+0000 7f1cb30fc640 1 Processor -- start 2024-04-26T23:20:39.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.453+0000 7f1cb30fc640 1 -- start start 2024-04-26T23:20:39.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.453+0000 7f1cb30fc640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1cac0d1720 0x7f1cac0d0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:39.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.453+0000 7f1cb30fc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1cac0d39f0 0x7f1cac070970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:39.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.453+0000 7f1cb30fc640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cac0701e0 con 0x7f1cac15d250 2024-04-26T23:20:39.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.453+0000 7f1cb30fc640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f1cac15d630 con 0x7f1cac0d1720 2024-04-26T23:20:39.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.453+0000 7f1cb30fc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f1cac0704e0 con 0x7f1cac0d39f0 2024-04-26T23:20:39.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.454+0000 7f1cb0e98640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1cac15d250 0x7f1cac0d1010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37916/0 (socket says 172.21.15.47:37916) 2024-04-26T23:20:39.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.454+0000 7f1cb0e98640 1 -- 172.21.15.47:0/4253036020 learned_addr learned my addr 172.21.15.47:0/4253036020 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:39.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.454+0000 7f1cabfff640 1 --2- 172.21.15.47:0/4253036020 >> v2:172.21.15.203:3300/0 conn(0x7f1cac0d1720 0x7f1cac0d0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:39.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.454+0000 7f1cb1699640 1 --2- 172.21.15.47:0/4253036020 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1cac0d39f0 0x7f1cac070970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:39.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.454+0000 7f1cab7fe640 1 -- 172.21.15.47:0/4253036020 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3034696040 0 0) 0x7f1cac0701e0 con 0x7f1cac15d250 2024-04-26T23:20:39.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.454+0000 7f1cab7fe640 1 -- 172.21.15.47:0/4253036020 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c94003520 con 0x7f1cac15d250 2024-04-26T23:20:39.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.454+0000 7f1cab7fe640 1 -- 172.21.15.47:0/4253036020 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4132392141 0 0) 0x7f1c94003520 con 0x7f1cac15d250 2024-04-26T23:20:39.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.454+0000 7f1cab7fe640 1 -- 172.21.15.47:0/4253036020 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1cac0d39f0 msgr2=0x7f1cac070970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:39.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.454+0000 7f1cab7fe640 1 --2- 172.21.15.47:0/4253036020 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1cac0d39f0 0x7f1cac070970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:39.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.454+0000 7f1cab7fe640 1 -- 172.21.15.47:0/4253036020 >> v2:172.21.15.203:3300/0 conn(0x7f1cac0d1720 msgr2=0x7f1cac0d0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:39.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.454+0000 7f1cab7fe640 1 --2- 172.21.15.47:0/4253036020 >> v2:172.21.15.203:3300/0 conn(0x7f1cac0d1720 0x7f1cac0d0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:39.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.454+0000 7f1cab7fe640 1 -- 172.21.15.47:0/4253036020 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cac0ce370 con 0x7f1cac15d250 2024-04-26T23:20:39.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.454+0000 7f1cab7fe640 1 -- 172.21.15.47:0/4253036020 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7f1c9c001670 con 0x7f1cac15d250 2024-04-26T23:20:39.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.454+0000 7f1cab7fe640 1 -- 172.21.15.47:0/4253036020 >> v1:172.21.15.47:6789/0 conn(0x7f1cac15d250 legacy=0x7f1cac0d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:39.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.454+0000 7f1cab7fe640 1 --2- 172.21.15.47:0/4253036020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c940039b0 0x7f1c94003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:39.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.454+0000 7f1cab7fe640 1 --2- 172.21.15.47:0/4253036020 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c94004800 0x7f1c94004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:39.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.454+0000 7f1cab7fe640 1 --2- 172.21.15.47:0/4253036020 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1cac0d39f0 0x7f1c940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:39.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.454+0000 7f1cab7fe640 1 -- 172.21.15.47:0/4253036020 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1cac15d630 con 0x7f1c940039b0 2024-04-26T23:20:39.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.454+0000 7f1cab7fe640 1 -- 172.21.15.47:0/4253036020 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1cac0704e0 con 0x7f1c94004800 2024-04-26T23:20:39.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.454+0000 7f1cab7fe640 1 -- 172.21.15.47:0/4253036020 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f1c9400b9c0 con 0x7f1cac0d39f0 2024-04-26T23:20:39.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.454+0000 7f1cb0e98640 1 --2- 172.21.15.47:0/4253036020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c940039b0 0x7f1c94003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:39.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.454+0000 7f1cabfff640 1 --2- 172.21.15.47:0/4253036020 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c94004800 0x7f1c94004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:39.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.454+0000 7f1cb1699640 1 --2- 172.21.15.47:0/4253036020 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1cac0d39f0 0x7f1c940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:39.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.455+0000 7f1cb0e98640 1 -- 172.21.15.47:0/4253036020 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1cac0d39f0 msgr2=0x7f1c940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:39.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.455+0000 7f1cb0e98640 1 --2- 172.21.15.47:0/4253036020 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1cac0d39f0 0x7f1c940051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:39.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.455+0000 7f1cb0e98640 1 -- 172.21.15.47:0/4253036020 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c94004800 msgr2=0x7f1c94004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:39.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.455+0000 7f1cb0e98640 1 --2- 172.21.15.47:0/4253036020 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c94004800 0x7f1c94004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:39.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.455+0000 7f1cb0e98640 1 -- 172.21.15.47:0/4253036020 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c94003520 con 0x7f1c940039b0 2024-04-26T23:20:39.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.455+0000 7f1cb0e98640 1 --2- 172.21.15.47:0/4253036020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c940039b0 0x7f1c94003dd0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c00c270 tx=0x7f1c94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fe20a7d62b5a322 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:39.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.455+0000 7f1cab7fe640 1 -- 172.21.15.47:0/4253036020 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c007d60 con 0x7f1c940039b0 2024-04-26T23:20:39.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.455+0000 7f1cab7fe640 1 -- 172.21.15.47:0/4253036020 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c9c005ad0 con 0x7f1c940039b0 2024-04-26T23:20:39.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.455+0000 7f1cab7fe640 1 -- 172.21.15.47:0/4253036020 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c007070 con 0x7f1c940039b0 2024-04-26T23:20:39.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.456+0000 7f1cb30fc640 1 -- 172.21.15.47:0/4253036020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c940039b0 msgr2=0x7f1c94003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:39.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.456+0000 7f1cb30fc640 1 --2- 172.21.15.47:0/4253036020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c940039b0 0x7f1c94003dd0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c00c270 tx=0x7f1c94005830 comp rx=0 tx=0).stop 2024-04-26T23:20:39.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.456+0000 7f1cb1699640 1 -- 172.21.15.47:0/4253036020 reap_dead start 2024-04-26T23:20:39.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.456+0000 7f1cb30fc640 1 -- 172.21.15.47:0/4253036020 shutdown_connections 2024-04-26T23:20:39.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.456+0000 7f1cb30fc640 1 -- 172.21.15.47:0/4253036020 >> 172.21.15.47:0/4253036020 conn(0x7f1cac0d2d80 msgr2=0x7f1cac0d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:39.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.456+0000 7f1cb30fc640 1 -- 172.21.15.47:0/4253036020 shutdown_connections 2024-04-26T23:20:39.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.456+0000 7f1cb30fc640 1 -- 172.21.15.47:0/4253036020 wait complete. 2024-04-26T23:20:39.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.457+0000 7f1cb30fc640 1 Processor -- start 2024-04-26T23:20:39.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.457+0000 7f1cb30fc640 1 -- start start 2024-04-26T23:20:39.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.458+0000 7f1cb30fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c940039b0 0x7f1cac0b86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:39.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.458+0000 7f1cb30fc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1cac09f750 0x7f1cac09fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:39.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.458+0000 7f1cb30fc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cac0b27b0 0x7f1cac0b2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:39.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.458+0000 7f1cb30fc640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1cac0cebb0 con 0x7f1c940039b0 2024-04-26T23:20:39.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.458+0000 7f1cb30fc640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1cac0ce8d0 con 0x7f1cac0b27b0 2024-04-26T23:20:39.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.458+0000 7f1cb30fc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f1cac070970 con 0x7f1cac09f750 2024-04-26T23:20:39.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.458+0000 7f1cb0e98640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c940039b0 0x7f1cac0b86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:39.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.458+0000 7f1cb0e98640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c940039b0 0x7f1cac0b86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51686/0 (socket says 172.21.15.47:51686) 2024-04-26T23:20:39.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.458+0000 7f1cb0e98640 1 -- 172.21.15.47:0/2527073920 learned_addr learned my addr 172.21.15.47:0/2527073920 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:39.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.458+0000 7f1cabfff640 1 --2- 172.21.15.47:0/2527073920 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1cac09f750 0x7f1cac09fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:39.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.458+0000 7f1cb1699640 1 --2- 172.21.15.47:0/2527073920 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cac0b27b0 0x7f1cac0b2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:39.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.458+0000 7f1cb0e98640 1 -- 172.21.15.47:0/2527073920 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1cac09f750 msgr2=0x7f1cac09fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:39.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.458+0000 7f1cb0e98640 1 --2- 172.21.15.47:0/2527073920 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1cac09f750 0x7f1cac09fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:39.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.458+0000 7f1cb0e98640 1 -- 172.21.15.47:0/2527073920 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cac0b27b0 msgr2=0x7f1cac0b2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:39.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.458+0000 7f1cb0e98640 1 --2- 172.21.15.47:0/2527073920 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cac0b27b0 0x7f1cac0b2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:39.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.459+0000 7f1cb0e98640 1 -- 172.21.15.47:0/2527073920 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cac15b2f0 con 0x7f1c940039b0 2024-04-26T23:20:39.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.459+0000 7f1cb0e98640 1 --2- 172.21.15.47:0/2527073920 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c940039b0 0x7f1cac0b86a0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c00c390 tx=0x7f1c9c001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:39.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.459+0000 7f1ca97fa640 1 -- 172.21.15.47:0/2527073920 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c042070 con 0x7f1c940039b0 2024-04-26T23:20:39.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.459+0000 7f1ca97fa640 1 -- 172.21.15.47:0/2527073920 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c9c007480 con 0x7f1c940039b0 2024-04-26T23:20:39.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.459+0000 7f1cb30fc640 1 -- 172.21.15.47:0/2527073920 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cac0a0310 con 0x7f1c940039b0 2024-04-26T23:20:39.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.459+0000 7f1ca97fa640 1 -- 172.21.15.47:0/2527073920 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c007d60 con 0x7f1c940039b0 2024-04-26T23:20:39.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.459+0000 7f1cb30fc640 1 -- 172.21.15.47:0/2527073920 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cac14bf30 con 0x7f1c940039b0 2024-04-26T23:20:39.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.460+0000 7f1cb30fc640 1 -- 172.21.15.47:0/2527073920 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cac0694f0 con 0x7f1c940039b0 2024-04-26T23:20:39.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.460+0000 7f1ca97fa640 1 -- 172.21.15.47:0/2527073920 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1c9c045070 con 0x7f1c940039b0 2024-04-26T23:20:39.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.461+0000 7f1ca97fa640 1 --2- 172.21.15.47:0/2527073920 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1c9806c8f0 0x7f1c9806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:39.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.461+0000 7f1ca97fa640 1 -- 172.21.15.47:0/2527073920 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f1c9c0ac960 con 0x7f1c940039b0 2024-04-26T23:20:39.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.461+0000 7f1cabfff640 1 --2- 172.21.15.47:0/2527073920 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1c9806c8f0 0x7f1c9806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:39.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.462+0000 7f1cabfff640 1 --2- 172.21.15.47:0/2527073920 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1c9806c8f0 0x7f1c9806edb0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0009a60 tx=0x7f1ca0009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:39.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.465+0000 7f1ca97fa640 1 -- 172.21.15.47:0/2527073920 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1c9c04c030 con 0x7f1c940039b0 2024-04-26T23:20:39.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.648+0000 7f1cb30fc640 1 -- 172.21.15.47:0/2527073920 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1cac0cf490 con 0x7f1c940039b0 2024-04-26T23:20:39.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.649+0000 7f1ca97fa640 1 -- 172.21.15.47:0/2527073920 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) ==== 93+0+10 (secure 0 0 0) 0x7f1cac0ce370 con 0x7f1c940039b0 2024-04-26T23:20:39.651 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-26T23:20:39.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.651+0000 7f1cb30fc640 1 -- 172.21.15.47:0/2527073920 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1c9806c8f0 msgr2=0x7f1c9806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:39.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.651+0000 7f1cb30fc640 1 --2- 172.21.15.47:0/2527073920 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1c9806c8f0 0x7f1c9806edb0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0009a60 tx=0x7f1ca0009190 comp rx=0 tx=0).stop 2024-04-26T23:20:39.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.651+0000 7f1cb30fc640 1 -- 172.21.15.47:0/2527073920 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c940039b0 msgr2=0x7f1cac0b86a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:39.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.651+0000 7f1cb30fc640 1 --2- 172.21.15.47:0/2527073920 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c940039b0 0x7f1cac0b86a0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c00c390 tx=0x7f1c9c001a40 comp rx=0 tx=0).stop 2024-04-26T23:20:39.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.651+0000 7f1cb30fc640 1 -- 172.21.15.47:0/2527073920 shutdown_connections 2024-04-26T23:20:39.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.651+0000 7f1cb30fc640 1 --2- 172.21.15.47:0/2527073920 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1c9806c8f0 0x7f1c9806edb0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:39.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.651+0000 7f1cb30fc640 1 --2- 172.21.15.47:0/2527073920 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cac0b27b0 0x7f1cac0b2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:39.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.651+0000 7f1cb30fc640 1 --2- 172.21.15.47:0/2527073920 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1cac09f750 0x7f1cac09fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:39.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.651+0000 7f1cb30fc640 1 --2- 172.21.15.47:0/2527073920 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c940039b0 0x7f1cac0b86a0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:39.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.651+0000 7f1cb30fc640 1 -- 172.21.15.47:0/2527073920 >> 172.21.15.47:0/2527073920 conn(0x7f1cac0d2d80 msgr2=0x7f1cac0cf9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:39.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.651+0000 7f1cb30fc640 1 -- 172.21.15.47:0/2527073920 shutdown_connections 2024-04-26T23:20:39.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.651+0000 7f1cb30fc640 1 -- 172.21.15.47:0/2527073920 wait complete. 2024-04-26T23:20:39.664 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:20:39.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.765+0000 7f79d007e640 1 Processor -- start 2024-04-26T23:20:39.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.766+0000 7f79d007e640 1 -- start start 2024-04-26T23:20:39.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.766+0000 7f79d007e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f79c80d1a00 0x7f79c80d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:39.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.766+0000 7f79d007e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79c80b40d0 0x7f79c80b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:39.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.766+0000 7f79d007e640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79c80d0a10 con 0x7f79c80d2320 2024-04-26T23:20:39.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.766+0000 7f79d007e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f79c80d0cf0 con 0x7f79c80d1a00 2024-04-26T23:20:39.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.766+0000 7f79d007e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f79c80d0fd0 con 0x7f79c80b40d0 2024-04-26T23:20:39.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.767+0000 7f79cd619640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f79c80d2320 0x7f79c81625f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47104/0 (socket says 172.21.15.47:47104) 2024-04-26T23:20:39.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.767+0000 7f79cd619640 1 -- 172.21.15.47:0/3326462511 learned_addr learned my addr 172.21.15.47:0/3326462511 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:39.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.767+0000 7f79ce61b640 1 --2- 172.21.15.47:0/3326462511 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79c80b40d0 0x7f79c80b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:39.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.767+0000 7f79cde1a640 1 --2- 172.21.15.47:0/3326462511 >> v2:172.21.15.203:3300/0 conn(0x7f79c80d1a00 0x7f79c80d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:39.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.767+0000 7f79cce18640 1 -- 172.21.15.47:0/3326462511 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2290331246 0 0) 0x7f79c80d0a10 con 0x7f79c80d2320 2024-04-26T23:20:39.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.767+0000 7f79cce18640 1 -- 172.21.15.47:0/3326462511 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f79b4003520 con 0x7f79c80d2320 2024-04-26T23:20:39.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.767+0000 7f79cce18640 1 -- 172.21.15.47:0/3326462511 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 783157435 0 0) 0x7f79b4003520 con 0x7f79c80d2320 2024-04-26T23:20:39.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.767+0000 7f79cce18640 1 -- 172.21.15.47:0/3326462511 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79c80b40d0 msgr2=0x7f79c80b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:39.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.767+0000 7f79cce18640 1 --2- 172.21.15.47:0/3326462511 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79c80b40d0 0x7f79c80b4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:39.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.767+0000 7f79cce18640 1 -- 172.21.15.47:0/3326462511 >> v2:172.21.15.203:3300/0 conn(0x7f79c80d1a00 msgr2=0x7f79c80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:39.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.767+0000 7f79cce18640 1 --2- 172.21.15.47:0/3326462511 >> v2:172.21.15.203:3300/0 conn(0x7f79c80d1a00 0x7f79c80d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:39.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.767+0000 7f79cce18640 1 -- 172.21.15.47:0/3326462511 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79c80d40a0 con 0x7f79c80d2320 2024-04-26T23:20:39.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.767+0000 7f79cce18640 1 -- 172.21.15.47:0/3326462511 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7f79b8001670 con 0x7f79c80d2320 2024-04-26T23:20:39.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.767+0000 7f79cce18640 1 -- 172.21.15.47:0/3326462511 >> v1:172.21.15.47:6789/0 conn(0x7f79c80d2320 legacy=0x7f79c81625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:39.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.767+0000 7f79cce18640 1 --2- 172.21.15.47:0/3326462511 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79b40039b0 0x7f79b4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:39.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.767+0000 7f79cce18640 1 --2- 172.21.15.47:0/3326462511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79b4004800 0x7f79b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:39.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.767+0000 7f79cce18640 1 --2- 172.21.15.47:0/3326462511 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79c80b40d0 0x7f79b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:39.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.767+0000 7f79cce18640 1 -- 172.21.15.47:0/3326462511 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f79c80d0cf0 con 0x7f79b4004800 2024-04-26T23:20:39.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.767+0000 7f79cce18640 1 -- 172.21.15.47:0/3326462511 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f79c80d0fd0 con 0x7f79b40039b0 2024-04-26T23:20:39.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.767+0000 7f79cce18640 1 -- 172.21.15.47:0/3326462511 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f79b400ba70 con 0x7f79c80b40d0 2024-04-26T23:20:39.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.767+0000 7f79cd619640 1 --2- 172.21.15.47:0/3326462511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79b4004800 0x7f79b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:39.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.767+0000 7f79cde1a640 1 --2- 172.21.15.47:0/3326462511 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79b40039b0 0x7f79b4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:39.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.768+0000 7f79ce61b640 1 --2- 172.21.15.47:0/3326462511 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79c80b40d0 0x7f79b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:39.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.768+0000 7f79cd619640 1 -- 172.21.15.47:0/3326462511 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79c80b40d0 msgr2=0x7f79b40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:39.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.768+0000 7f79cd619640 1 --2- 172.21.15.47:0/3326462511 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79c80b40d0 0x7f79b40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:39.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.768+0000 7f79cd619640 1 -- 172.21.15.47:0/3326462511 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79b40039b0 msgr2=0x7f79b4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:39.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.768+0000 7f79cd619640 1 --2- 172.21.15.47:0/3326462511 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79b40039b0 0x7f79b4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:39.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.768+0000 7f79cd619640 1 -- 172.21.15.47:0/3326462511 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79b4003520 con 0x7f79b4004800 2024-04-26T23:20:39.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.768+0000 7f79cde1a640 1 --2- 172.21.15.47:0/3326462511 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79b40039b0 0x7f79b4003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:20:39.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.768+0000 7f79cd619640 1 --2- 172.21.15.47:0/3326462511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79b4004800 0x7f79b4004c80 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f79b800c210 tx=0x7f79b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ea1393e7ae63ab2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:39.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.768+0000 7f79cce18640 1 -- 172.21.15.47:0/3326462511 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b8007d50 con 0x7f79b4004800 2024-04-26T23:20:39.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.768+0000 7f79cce18640 1 -- 172.21.15.47:0/3326462511 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79b8005ad0 con 0x7f79b4004800 2024-04-26T23:20:39.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.768+0000 7f79cce18640 1 -- 172.21.15.47:0/3326462511 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b8007070 con 0x7f79b4004800 2024-04-26T23:20:39.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.769+0000 7f79d007e640 1 -- 172.21.15.47:0/3326462511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79b4004800 msgr2=0x7f79b4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:39.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.769+0000 7f79d007e640 1 --2- 172.21.15.47:0/3326462511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79b4004800 0x7f79b4004c80 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f79b800c210 tx=0x7f79b4005830 comp rx=0 tx=0).stop 2024-04-26T23:20:39.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.769+0000 7f79ce61b640 1 -- 172.21.15.47:0/3326462511 reap_dead start 2024-04-26T23:20:39.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.769+0000 7f79d007e640 1 -- 172.21.15.47:0/3326462511 shutdown_connections 2024-04-26T23:20:39.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.769+0000 7f79d007e640 1 -- 172.21.15.47:0/3326462511 >> 172.21.15.47:0/3326462511 conn(0x7f79c80d4fd0 msgr2=0x7f79c80d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:39.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.769+0000 7f79d007e640 1 -- 172.21.15.47:0/3326462511 shutdown_connections 2024-04-26T23:20:39.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.769+0000 7f79d007e640 1 -- 172.21.15.47:0/3326462511 wait complete. 2024-04-26T23:20:39.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.770+0000 7f79d007e640 1 Processor -- start 2024-04-26T23:20:39.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.770+0000 7f79d007e640 1 -- start start 2024-04-26T23:20:39.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.771+0000 7f79d007e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79b4004800 0x7f79c80c86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:39.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.771+0000 7f79d007e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79c814f220 0x7f79c8151cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:39.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.771+0000 7f79d007e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79c814e8c0 0x7f79c80b2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:39.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.771+0000 7f79d007e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f79c80b4b00 con 0x7f79b4004800 2024-04-26T23:20:39.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.771+0000 7f79d007e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f79c8162f50 con 0x7f79c814f220 2024-04-26T23:20:39.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.771+0000 7f79d007e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f79c80b4c80 con 0x7f79c814e8c0 2024-04-26T23:20:39.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.771+0000 7f79cde1a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79b4004800 0x7f79c80c86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:39.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.771+0000 7f79cde1a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79b4004800 0x7f79c80c86f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39482/0 (socket says 172.21.15.47:39482) 2024-04-26T23:20:39.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.771+0000 7f79cde1a640 1 -- 172.21.15.47:0/1751014681 learned_addr learned my addr 172.21.15.47:0/1751014681 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:39.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.771+0000 7f79ce61b640 1 --2- 172.21.15.47:0/1751014681 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79c814e8c0 0x7f79c80b2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:39.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.771+0000 7f79cd619640 1 --2- 172.21.15.47:0/1751014681 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79c814f220 0x7f79c8151cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:39.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.771+0000 7f79cde1a640 1 -- 172.21.15.47:0/1751014681 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79c814e8c0 msgr2=0x7f79c80b2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:39.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.771+0000 7f79cde1a640 1 --2- 172.21.15.47:0/1751014681 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79c814e8c0 0x7f79c80b2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:39.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.771+0000 7f79cde1a640 1 -- 172.21.15.47:0/1751014681 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79c814f220 msgr2=0x7f79c8151cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:39.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.771+0000 7f79cde1a640 1 --2- 172.21.15.47:0/1751014681 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79c814f220 0x7f79c8151cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:39.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.771+0000 7f79cde1a640 1 -- 172.21.15.47:0/1751014681 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79c814f6a0 con 0x7f79b4004800 2024-04-26T23:20:39.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.772+0000 7f79cde1a640 1 --2- 172.21.15.47:0/1751014681 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79b4004800 0x7f79c80c86f0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f79c40115d0 tx=0x7f79c4011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:39.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.772+0000 7f79be7fc640 1 -- 172.21.15.47:0/1751014681 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c400f8f0 con 0x7f79b4004800 2024-04-26T23:20:39.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.772+0000 7f79be7fc640 1 -- 172.21.15.47:0/1751014681 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79c400fa90 con 0x7f79b4004800 2024-04-26T23:20:39.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.772+0000 7f79d007e640 1 -- 172.21.15.47:0/1751014681 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79c80c8d80 con 0x7f79b4004800 2024-04-26T23:20:39.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.772+0000 7f79be7fc640 1 -- 172.21.15.47:0/1751014681 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c400fe60 con 0x7f79b4004800 2024-04-26T23:20:39.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.772+0000 7f79d007e640 1 -- 172.21.15.47:0/1751014681 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79c80caa40 con 0x7f79b4004800 2024-04-26T23:20:39.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.773+0000 7f79d007e640 1 -- 172.21.15.47:0/1751014681 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79c80694f0 con 0x7f79b4004800 2024-04-26T23:20:39.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.773+0000 7f79be7fc640 1 -- 172.21.15.47:0/1751014681 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f79c4015460 con 0x7f79b4004800 2024-04-26T23:20:39.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.774+0000 7f79be7fc640 1 --2- 172.21.15.47:0/1751014681 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f79ac06c920 0x7f79ac06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:39.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.774+0000 7f79be7fc640 1 -- 172.21.15.47:0/1751014681 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f79c408c5e0 con 0x7f79b4004800 2024-04-26T23:20:39.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.774+0000 7f79cd619640 1 --2- 172.21.15.47:0/1751014681 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f79ac06c920 0x7f79ac06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:39.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.775+0000 7f79cd619640 1 --2- 172.21.15.47:0/1751014681 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f79ac06c920 0x7f79ac06ede0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f79b8033000 tx=0x7f79b8009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:39.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.778+0000 7f79be7fc640 1 -- 172.21.15.47:0/1751014681 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f79c814f6a0 con 0x7f79b4004800 2024-04-26T23:20:39.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.961+0000 7f79d007e640 1 -- 172.21.15.47:0/1751014681 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f79c80d1e70 con 0x7f79b4004800 2024-04-26T23:20:39.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.962+0000 7f79be7fc640 1 -- 172.21.15.47:0/1751014681 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v54) ==== 104+0+11 (secure 0 0 0) 0x7f79c4019030 con 0x7f79b4004800 2024-04-26T23:20:39.964 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-26T23:20:39.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.963+0000 7f79d007e640 1 -- 172.21.15.47:0/1751014681 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f79ac06c920 msgr2=0x7f79ac06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:39.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.963+0000 7f79d007e640 1 --2- 172.21.15.47:0/1751014681 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f79ac06c920 0x7f79ac06ede0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f79b8033000 tx=0x7f79b8009030 comp rx=0 tx=0).stop 2024-04-26T23:20:39.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.963+0000 7f79d007e640 1 -- 172.21.15.47:0/1751014681 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79b4004800 msgr2=0x7f79c80c86f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:39.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.963+0000 7f79d007e640 1 --2- 172.21.15.47:0/1751014681 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79b4004800 0x7f79c80c86f0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f79c40115d0 tx=0x7f79c4011a90 comp rx=0 tx=0).stop 2024-04-26T23:20:39.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.963+0000 7f79d007e640 1 -- 172.21.15.47:0/1751014681 shutdown_connections 2024-04-26T23:20:39.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.963+0000 7f79d007e640 1 --2- 172.21.15.47:0/1751014681 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f79ac06c920 0x7f79ac06ede0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:39.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.963+0000 7f79d007e640 1 --2- 172.21.15.47:0/1751014681 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79c814e8c0 0x7f79c80b2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:39.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.964+0000 7f79d007e640 1 --2- 172.21.15.47:0/1751014681 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79c814f220 0x7f79c8151cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:39.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.964+0000 7f79d007e640 1 --2- 172.21.15.47:0/1751014681 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79b4004800 0x7f79c80c86f0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:39.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.964+0000 7f79d007e640 1 -- 172.21.15.47:0/1751014681 >> 172.21.15.47:0/1751014681 conn(0x7f79c80d4fd0 msgr2=0x7f79c80b9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:39.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.964+0000 7f79d007e640 1 -- 172.21.15.47:0/1751014681 shutdown_connections 2024-04-26T23:20:39.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:39.964+0000 7f79d007e640 1 -- 172.21.15.47:0/1751014681 wait complete. 2024-04-26T23:20:39.976 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:20:40.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.078+0000 7fa1a8966640 1 Processor -- start 2024-04-26T23:20:40.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.078+0000 7fa1a8966640 1 -- start start 2024-04-26T23:20:40.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.079+0000 7fa1a8966640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa1a40cfde0 0x7fa1a40d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:40.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.079+0000 7fa1a8966640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1a4070350 0x7fa1a4070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:40.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.079+0000 7fa1a8966640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1a40d2390 con 0x7fa1a4071090 2024-04-26T23:20:40.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.079+0000 7fa1a8966640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fa1a4071500 con 0x7fa1a40cfde0 2024-04-26T23:20:40.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.079+0000 7fa1a8966640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa1a40afab0 con 0x7fa1a4070350 2024-04-26T23:20:40.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.079+0000 7fa1a259c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa1a40cfde0 0x7fa1a40d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:40.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.079+0000 7fa1a2d9d640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa1a4071090 0x7fa1a415e9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47112/0 (socket says 172.21.15.47:47112) 2024-04-26T23:20:40.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.079+0000 7fa1a2d9d640 1 -- 172.21.15.47:0/1532872908 learned_addr learned my addr 172.21.15.47:0/1532872908 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:40.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.079+0000 7fa1a1d9b640 1 --2- 172.21.15.47:0/1532872908 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1a4070350 0x7fa1a4070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:40.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.080+0000 7fa1a259c640 1 -- 172.21.15.47:0/1532872908 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1a4070350 msgr2=0x7fa1a4070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:40.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.080+0000 7fa1a259c640 1 --2- 172.21.15.47:0/1532872908 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1a4070350 0x7fa1a4070b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:40.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.080+0000 7fa1a259c640 1 -- 172.21.15.47:0/1532872908 >> v1:172.21.15.47:6789/0 conn(0x7fa1a4071090 legacy=0x7fa1a415e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:40.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.080+0000 7fa1a259c640 1 -- 172.21.15.47:0/1532872908 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a406d3f0 con 0x7fa1a40cfde0 2024-04-26T23:20:40.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.080+0000 7fa1a159a640 1 -- 172.21.15.47:0/1532872908 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1298687947 0 0) 0x7fa1a40d2390 con 0x7fa1a4071090 2024-04-26T23:20:40.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.080+0000 7fa1a259c640 1 --2- 172.21.15.47:0/1532872908 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1a40cfde0 0x7fa1a40d1b60 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fa19c009d90 tx=0x7fa19c02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=343bba84a9b670ad server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:40.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.080+0000 7fa1a159a640 1 -- 172.21.15.47:0/1532872908 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c031600 con 0x7fa1a40cfde0 2024-04-26T23:20:40.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.081+0000 7fa1a159a640 1 -- 172.21.15.47:0/1532872908 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa19c0317a0 con 0x7fa1a40cfde0 2024-04-26T23:20:40.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.081+0000 7fa1a159a640 1 -- 172.21.15.47:0/1532872908 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c031b70 con 0x7fa1a40cfde0 2024-04-26T23:20:40.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.081+0000 7fa1a8966640 1 -- 172.21.15.47:0/1532872908 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1a40cfde0 msgr2=0x7fa1a40d1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:40.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.081+0000 7fa1a8966640 1 --2- 172.21.15.47:0/1532872908 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1a40cfde0 0x7fa1a40d1b60 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fa19c009d90 tx=0x7fa19c02a5d0 comp rx=0 tx=0).stop 2024-04-26T23:20:40.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.081+0000 7fa1a8966640 1 -- 172.21.15.47:0/1532872908 shutdown_connections 2024-04-26T23:20:40.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.081+0000 7fa1a8966640 1 --2- 172.21.15.47:0/1532872908 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1a4070350 0x7fa1a4070b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:40.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.081+0000 7fa1a8966640 1 --2- 172.21.15.47:0/1532872908 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1a40cfde0 0x7fa1a40d1b60 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:40.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.081+0000 7fa1a8966640 1 -- 172.21.15.47:0/1532872908 >> 172.21.15.47:0/1532872908 conn(0x7fa1a40d2d80 msgr2=0x7fa1a40d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:40.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.081+0000 7fa1a8966640 1 -- 172.21.15.47:0/1532872908 shutdown_connections 2024-04-26T23:20:40.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.081+0000 7fa1a8966640 1 -- 172.21.15.47:0/1532872908 wait complete. 2024-04-26T23:20:40.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.082+0000 7fa1a8966640 1 Processor -- start 2024-04-26T23:20:40.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.083+0000 7fa1a8966640 1 -- start start 2024-04-26T23:20:40.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.083+0000 7fa1a8966640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1a4070350 0x7fa1a415fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:40.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.083+0000 7fa1a8966640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1a4071090 0x7fa1a409cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:40.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.083+0000 7fa1a8966640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1a40cfde0 0x7fa1a409d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:40.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.083+0000 7fa1a8966640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a406d9b0 con 0x7fa1a4071090 2024-04-26T23:20:40.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.083+0000 7fa1a8966640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a406dd20 con 0x7fa1a40cfde0 2024-04-26T23:20:40.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.083+0000 7fa1a8966640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa1a40d20a0 con 0x7fa1a4070350 2024-04-26T23:20:40.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.083+0000 7fa1a1d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1a4071090 0x7fa1a409cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:40.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.083+0000 7fa1a1d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1a4071090 0x7fa1a409cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39496/0 (socket says 172.21.15.47:39496) 2024-04-26T23:20:40.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.083+0000 7fa1a1d9b640 1 -- 172.21.15.47:0/4198122265 learned_addr learned my addr 172.21.15.47:0/4198122265 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:40.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.083+0000 7fa1a2d9d640 1 --2- 172.21.15.47:0/4198122265 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1a40cfde0 0x7fa1a409d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:40.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.083+0000 7fa1a259c640 1 --2- 172.21.15.47:0/4198122265 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1a4070350 0x7fa1a415fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:40.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.084+0000 7fa1a1d9b640 1 -- 172.21.15.47:0/4198122265 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1a4070350 msgr2=0x7fa1a415fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:40.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.084+0000 7fa1a1d9b640 1 --2- 172.21.15.47:0/4198122265 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1a4070350 0x7fa1a415fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:40.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.084+0000 7fa1a1d9b640 1 -- 172.21.15.47:0/4198122265 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1a40cfde0 msgr2=0x7fa1a409d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:40.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.084+0000 7fa1a1d9b640 1 --2- 172.21.15.47:0/4198122265 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1a40cfde0 0x7fa1a409d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:40.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.084+0000 7fa1a1d9b640 1 -- 172.21.15.47:0/4198122265 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a4160290 con 0x7fa1a4071090 2024-04-26T23:20:40.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.084+0000 7fa1a1d9b640 1 --2- 172.21.15.47:0/4198122265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1a4071090 0x7fa1a409cbc0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fa18c00d620 tx=0x7fa18c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:40.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.084+0000 7fa19affd640 1 -- 172.21.15.47:0/4198122265 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18c00b620 con 0x7fa1a4071090 2024-04-26T23:20:40.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.084+0000 7fa19affd640 1 -- 172.21.15.47:0/4198122265 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa18c019070 con 0x7fa1a4071090 2024-04-26T23:20:40.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.084+0000 7fa1a8966640 1 -- 172.21.15.47:0/4198122265 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a4160570 con 0x7fa1a4071090 2024-04-26T23:20:40.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.084+0000 7fa19affd640 1 -- 172.21.15.47:0/4198122265 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18c00ba00 con 0x7fa1a4071090 2024-04-26T23:20:40.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.084+0000 7fa1a8966640 1 -- 172.21.15.47:0/4198122265 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a4160b30 con 0x7fa1a4071090 2024-04-26T23:20:40.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.085+0000 7fa1a8966640 1 -- 172.21.15.47:0/4198122265 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1a40694f0 con 0x7fa1a4071090 2024-04-26T23:20:40.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.086+0000 7fa19affd640 1 -- 172.21.15.47:0/4198122265 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa18c00bba0 con 0x7fa1a4071090 2024-04-26T23:20:40.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.086+0000 7fa19affd640 1 --2- 172.21.15.47:0/4198122265 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa18406c8f0 0x7fa18406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:40.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.086+0000 7fa19affd640 1 -- 172.21.15.47:0/4198122265 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fa18c08bf30 con 0x7fa1a4071090 2024-04-26T23:20:40.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.086+0000 7fa1a259c640 1 --2- 172.21.15.47:0/4198122265 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa18406c8f0 0x7fa18406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:40.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.087+0000 7fa1a259c640 1 --2- 172.21.15.47:0/4198122265 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa18406c8f0 0x7fa18406edb0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa19c034000 tx=0x7fa19c02f000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:40.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.090+0000 7fa19affd640 1 -- 172.21.15.47:0/4198122265 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa18c059c10 con 0x7fa1a4071090 2024-04-26T23:20:40.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.278+0000 7fa1a8966640 1 -- 172.21.15.47:0/4198122265 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa1a40b2150 con 0x7fa1a4071090 2024-04-26T23:20:40.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.278+0000 7fa19affd640 1 -- 172.21.15.47:0/4198122265 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v54) ==== 100+0+11 (secure 0 0 0) 0x7fa18c05df10 con 0x7fa1a4071090 2024-04-26T23:20:40.280 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-26T23:20:40.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.280+0000 7fa1a8966640 1 -- 172.21.15.47:0/4198122265 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa18406c8f0 msgr2=0x7fa18406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:40.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.280+0000 7fa1a8966640 1 --2- 172.21.15.47:0/4198122265 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa18406c8f0 0x7fa18406edb0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa19c034000 tx=0x7fa19c02f000 comp rx=0 tx=0).stop 2024-04-26T23:20:40.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.280+0000 7fa1a8966640 1 -- 172.21.15.47:0/4198122265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1a4071090 msgr2=0x7fa1a409cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:40.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.280+0000 7fa1a8966640 1 --2- 172.21.15.47:0/4198122265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1a4071090 0x7fa1a409cbc0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fa18c00d620 tx=0x7fa18c00dae0 comp rx=0 tx=0).stop 2024-04-26T23:20:40.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.280+0000 7fa1a8966640 1 -- 172.21.15.47:0/4198122265 shutdown_connections 2024-04-26T23:20:40.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.280+0000 7fa1a8966640 1 --2- 172.21.15.47:0/4198122265 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa18406c8f0 0x7fa18406edb0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:40.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.280+0000 7fa1a8966640 1 --2- 172.21.15.47:0/4198122265 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1a40cfde0 0x7fa1a409d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:40.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.280+0000 7fa1a8966640 1 --2- 172.21.15.47:0/4198122265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1a4071090 0x7fa1a409cbc0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:40.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.280+0000 7fa1a8966640 1 --2- 172.21.15.47:0/4198122265 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1a4070350 0x7fa1a415fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:40.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.280+0000 7fa1a8966640 1 -- 172.21.15.47:0/4198122265 >> 172.21.15.47:0/4198122265 conn(0x7fa1a40d2d80 msgr2=0x7fa1a406e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:40.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.280+0000 7fa1a8966640 1 -- 172.21.15.47:0/4198122265 shutdown_connections 2024-04-26T23:20:40.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.280+0000 7fa1a8966640 1 -- 172.21.15.47:0/4198122265 wait complete. 2024-04-26T23:20:40.293 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:20:40.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.391+0000 7ff43fbda640 1 Processor -- start 2024-04-26T23:20:40.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.391+0000 7ff43fbda640 1 -- start start 2024-04-26T23:20:40.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.392+0000 7ff43fbda640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff438162810 0x7ff438070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:40.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.392+0000 7ff43fbda640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff4380b2340 0x7ff4380cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:40.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.392+0000 7ff43fbda640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff4380cd930 0x7ff4380d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:40.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.392+0000 7ff43fbda640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7ff4380d0750 con 0x7ff4380b2340 2024-04-26T23:20:40.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.392+0000 7ff43fbda640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7ff4380d16a0 con 0x7ff4380cd930 2024-04-26T23:20:40.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.392+0000 7ff43fbda640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7ff4380d10d0 con 0x7ff438162810 2024-04-26T23:20:40.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.392+0000 7ff43d175640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff4380b2340 0x7ff4380cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:40.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.392+0000 7ff43d175640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff4380b2340 0x7ff4380cd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39498/0 (socket says 172.21.15.47:39498) 2024-04-26T23:20:40.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.392+0000 7ff43d175640 1 -- 172.21.15.47:0/1871048951 learned_addr learned my addr 172.21.15.47:0/1871048951 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:40.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.392+0000 7ff43d976640 1 --2- 172.21.15.47:0/1871048951 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff438162810 0x7ff438070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:40.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.392+0000 7ff43e177640 1 --2- 172.21.15.47:0/1871048951 >> v2:172.21.15.203:3300/0 conn(0x7ff4380cd930 0x7ff4380d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:40.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.393+0000 7ff43d175640 1 -- 172.21.15.47:0/1871048951 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff438162810 msgr2=0x7ff438070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:40.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.393+0000 7ff43d175640 1 --2- 172.21.15.47:0/1871048951 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff438162810 0x7ff438070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:40.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.393+0000 7ff43d175640 1 -- 172.21.15.47:0/1871048951 >> v2:172.21.15.203:3300/0 conn(0x7ff4380cd930 msgr2=0x7ff4380d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:40.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.393+0000 7ff43d175640 1 --2- 172.21.15.47:0/1871048951 >> v2:172.21.15.203:3300/0 conn(0x7ff4380cd930 0x7ff4380d1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:40.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.393+0000 7ff43d175640 1 -- 172.21.15.47:0/1871048951 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4380d1370 con 0x7ff4380b2340 2024-04-26T23:20:40.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.393+0000 7ff43d175640 1 --2- 172.21.15.47:0/1871048951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4380b2340 0x7ff4380cd3f0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7ff4380b2250 tx=0x7ff42802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=910c964ade8e3302 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:40.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.393+0000 7ff43c974640 1 -- 172.21.15.47:0/1871048951 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff428037070 con 0x7ff4380b2340 2024-04-26T23:20:40.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.394+0000 7ff43c974640 1 -- 172.21.15.47:0/1871048951 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff428032440 con 0x7ff4380b2340 2024-04-26T23:20:40.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.394+0000 7ff43c974640 1 -- 172.21.15.47:0/1871048951 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff428032810 con 0x7ff4380b2340 2024-04-26T23:20:40.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.394+0000 7ff43fbda640 1 -- 172.21.15.47:0/1871048951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4380b2340 msgr2=0x7ff4380cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:40.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.394+0000 7ff43fbda640 1 --2- 172.21.15.47:0/1871048951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4380b2340 0x7ff4380cd3f0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7ff4380b2250 tx=0x7ff42802a1e0 comp rx=0 tx=0).stop 2024-04-26T23:20:40.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.394+0000 7ff43fbda640 1 -- 172.21.15.47:0/1871048951 shutdown_connections 2024-04-26T23:20:40.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.394+0000 7ff43fbda640 1 --2- 172.21.15.47:0/1871048951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4380b2340 0x7ff4380cd3f0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:40.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.394+0000 7ff43fbda640 1 --2- 172.21.15.47:0/1871048951 >> v2:172.21.15.203:3300/0 conn(0x7ff4380cd930 0x7ff4380d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:40.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.394+0000 7ff43fbda640 1 --2- 172.21.15.47:0/1871048951 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff438162810 0x7ff438070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:40.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.394+0000 7ff43fbda640 1 -- 172.21.15.47:0/1871048951 >> 172.21.15.47:0/1871048951 conn(0x7ff4380d2da0 msgr2=0x7ff4380d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:40.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.394+0000 7ff43fbda640 1 -- 172.21.15.47:0/1871048951 shutdown_connections 2024-04-26T23:20:40.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.394+0000 7ff43fbda640 1 -- 172.21.15.47:0/1871048951 wait complete. 2024-04-26T23:20:40.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.395+0000 7ff43fbda640 1 Processor -- start 2024-04-26T23:20:40.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.395+0000 7ff43fbda640 1 -- start start 2024-04-26T23:20:40.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.396+0000 7ff43fbda640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4380b2340 0x7ff4381f8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:40.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.396+0000 7ff43fbda640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4380cd930 0x7ff4381f8750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:40.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.396+0000 7ff43fbda640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff438162810 0x7ff4381f8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:40.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.396+0000 7ff43fbda640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff438070890 con 0x7ff4380b2340 2024-04-26T23:20:40.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.396+0000 7ff43fbda640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff4380d0470 con 0x7ff438162810 2024-04-26T23:20:40.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.396+0000 7ff43fbda640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7ff4380d2070 con 0x7ff4380cd930 2024-04-26T23:20:40.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.396+0000 7ff43d976640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4380b2340 0x7ff4381f8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:40.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.396+0000 7ff43d976640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4380b2340 0x7ff4381f8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39514/0 (socket says 172.21.15.47:39514) 2024-04-26T23:20:40.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.396+0000 7ff43d976640 1 -- 172.21.15.47:0/2577132159 learned_addr learned my addr 172.21.15.47:0/2577132159 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:40.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.396+0000 7ff43d175640 1 --2- 172.21.15.47:0/2577132159 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4380cd930 0x7ff4381f8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:40.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.396+0000 7ff43e177640 1 --2- 172.21.15.47:0/2577132159 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff438162810 0x7ff4381f8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:40.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.396+0000 7ff43d976640 1 -- 172.21.15.47:0/2577132159 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4380cd930 msgr2=0x7ff4381f8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:40.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.396+0000 7ff43d976640 1 --2- 172.21.15.47:0/2577132159 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4380cd930 0x7ff4381f8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:40.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.396+0000 7ff43d976640 1 -- 172.21.15.47:0/2577132159 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff438162810 msgr2=0x7ff4381f8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:40.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.397+0000 7ff43d976640 1 --2- 172.21.15.47:0/2577132159 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff438162810 0x7ff4381f8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:40.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.397+0000 7ff43d976640 1 -- 172.21.15.47:0/2577132159 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff43815d860 con 0x7ff4380b2340 2024-04-26T23:20:40.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.397+0000 7ff43d976640 1 --2- 172.21.15.47:0/2577132159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4380b2340 0x7ff4381f8100 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7ff43400d5f0 tx=0x7ff43400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:40.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.397+0000 7ff42e7fc640 1 -- 172.21.15.47:0/2577132159 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4340125f0 con 0x7ff4380b2340 2024-04-26T23:20:40.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.397+0000 7ff42e7fc640 1 -- 172.21.15.47:0/2577132159 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff434019070 con 0x7ff4380b2340 2024-04-26T23:20:40.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.397+0000 7ff43fbda640 1 -- 172.21.15.47:0/2577132159 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff43815db40 con 0x7ff4380b2340 2024-04-26T23:20:40.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.397+0000 7ff42e7fc640 1 -- 172.21.15.47:0/2577132159 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4340129d0 con 0x7ff4380b2340 2024-04-26T23:20:40.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.397+0000 7ff43fbda640 1 -- 172.21.15.47:0/2577132159 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4380caf60 con 0x7ff4380b2340 2024-04-26T23:20:40.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.398+0000 7ff43fbda640 1 -- 172.21.15.47:0/2577132159 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4380694f0 con 0x7ff4380b2340 2024-04-26T23:20:40.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.398+0000 7ff42e7fc640 1 -- 172.21.15.47:0/2577132159 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff434012b70 con 0x7ff4380b2340 2024-04-26T23:20:40.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.399+0000 7ff42e7fc640 1 --2- 172.21.15.47:0/2577132159 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7ff41c06c8f0 0x7ff41c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:40.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.399+0000 7ff42e7fc640 1 -- 172.21.15.47:0/2577132159 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7ff43408bf00 con 0x7ff4380b2340 2024-04-26T23:20:40.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.399+0000 7ff43d175640 1 --2- 172.21.15.47:0/2577132159 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7ff41c06c8f0 0x7ff41c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:40.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.400+0000 7ff43d175640 1 --2- 172.21.15.47:0/2577132159 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7ff41c06c8f0 0x7ff41c06edb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff428034000 tx=0x7ff42802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:40.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.403+0000 7ff42e7fc640 1 -- 172.21.15.47:0/2577132159 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff434059be0 con 0x7ff4380b2340 2024-04-26T23:20:40.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.628+0000 7ff43fbda640 1 -- 172.21.15.47:0/2577132159 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4380b2970 con 0x7ff4380b2340 2024-04-26T23:20:40.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.628+0000 7ff42e7fc640 1 -- 172.21.15.47:0/2577132159 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5190 (secure 0 0 0) 0x7ff43405dee0 con 0x7ff4380b2340 2024-04-26T23:20:40.631 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:20:40.631 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5865,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/805147373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":805147373},{"type":"v1","addr":"172.21.15.47:6833","nonce":805147373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5868,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3515322868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3515322868},{"type":"v1","addr":"172.21.15.47:6835","nonce":3515322868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":15008,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/4223205269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":4223205269},{"type":"v1","addr":"172.21.15.203:6837","nonce":4223205269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:20:21.712604+0000","modified":"2024-04-26T23:20:24.725165+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":15011},"failed":[],"damaged":[],"stopped":[],"info":{"gid_15011":{"gid":15011,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6836/1168022146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1168022146},{"type":"v1","addr":"172.21.15.203:6836","nonce":1168022146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":15011,"qdb_cluster":[15011]},"id":3}]} 2024-04-26T23:20:40.631 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 27 2024-04-26T23:20:40.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.630+0000 7ff43fbda640 1 -- 172.21.15.47:0/2577132159 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7ff41c06c8f0 msgr2=0x7ff41c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:40.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.630+0000 7ff43fbda640 1 --2- 172.21.15.47:0/2577132159 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7ff41c06c8f0 0x7ff41c06edb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff428034000 tx=0x7ff42802e000 comp rx=0 tx=0).stop 2024-04-26T23:20:40.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.630+0000 7ff43fbda640 1 -- 172.21.15.47:0/2577132159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4380b2340 msgr2=0x7ff4381f8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:40.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.630+0000 7ff43fbda640 1 --2- 172.21.15.47:0/2577132159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4380b2340 0x7ff4381f8100 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7ff43400d5f0 tx=0x7ff43400dab0 comp rx=0 tx=0).stop 2024-04-26T23:20:40.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.630+0000 7ff43fbda640 1 -- 172.21.15.47:0/2577132159 shutdown_connections 2024-04-26T23:20:40.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.630+0000 7ff43fbda640 1 --2- 172.21.15.47:0/2577132159 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7ff41c06c8f0 0x7ff41c06edb0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:40.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.630+0000 7ff43fbda640 1 --2- 172.21.15.47:0/2577132159 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff438162810 0x7ff4381f8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:40.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.630+0000 7ff43fbda640 1 --2- 172.21.15.47:0/2577132159 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4380cd930 0x7ff4381f8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:40.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.630+0000 7ff43fbda640 1 --2- 172.21.15.47:0/2577132159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4380b2340 0x7ff4381f8100 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:40.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.630+0000 7ff43fbda640 1 -- 172.21.15.47:0/2577132159 >> 172.21.15.47:0/2577132159 conn(0x7ff4380d2da0 msgr2=0x7ff43806e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:40.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.631+0000 7ff43fbda640 1 -- 172.21.15.47:0/2577132159 shutdown_connections 2024-04-26T23:20:40.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.631+0000 7ff43fbda640 1 -- 172.21.15.47:0/2577132159 wait complete. 2024-04-26T23:20:40.643 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:20:40.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.742+0000 7f560f95f640 1 Processor -- start 2024-04-26T23:20:40.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.742+0000 7f560f95f640 1 -- start start 2024-04-26T23:20:40.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.743+0000 7f560f95f640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5608162880 0x7f56080d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:40.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.743+0000 7f560f95f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f56080d3bc0 0x7f56080b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:40.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.743+0000 7f560f95f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56080b7e50 con 0x7f56080b5f20 2024-04-26T23:20:40.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.743+0000 7f560f95f640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f56080d0580 con 0x7f5608162880 2024-04-26T23:20:40.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.743+0000 7f560f95f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f56080d1800 con 0x7f56080d3bc0 2024-04-26T23:20:40.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.743+0000 7f560d6fb640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5608162880 0x7f56080d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:40.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.743+0000 7f560d6fb640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5608162880 0x7f56080d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39522/0 (socket says 172.21.15.47:39522) 2024-04-26T23:20:40.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.743+0000 7f560d6fb640 1 -- 172.21.15.47:0/3412734271 learned_addr learned my addr 172.21.15.47:0/3412734271 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:40.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.743+0000 7f560d6fb640 1 -- 172.21.15.47:0/3412734271 >> v2:172.21.15.203:3300/0 conn(0x7f56080d3bc0 msgr2=0x7f56080b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:40.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.743+0000 7f560defc640 1 --2- 172.21.15.47:0/3412734271 >> v2:172.21.15.203:3300/0 conn(0x7f56080d3bc0 0x7f56080b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:40.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.743+0000 7f560d6fb640 1 --2- 172.21.15.47:0/3412734271 >> v2:172.21.15.203:3300/0 conn(0x7f56080d3bc0 0x7f56080b8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:40.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.743+0000 7f560d6fb640 1 -- 172.21.15.47:0/3412734271 >> v1:172.21.15.203:6789/0 conn(0x7f56080b5f20 legacy=0x7f56080d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:40.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.743+0000 7f560d6fb640 1 -- 172.21.15.47:0/3412734271 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56080d07d0 con 0x7f5608162880 2024-04-26T23:20:40.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.744+0000 7f560d6fb640 1 --2- 172.21.15.47:0/3412734271 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5608162880 0x7f56080d2cb0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f56080d11d0 tx=0x7f55f802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3edb1b874a6d0f71 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:40.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.744+0000 7f5607fff640 1 -- 172.21.15.47:0/3412734271 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f8036070 con 0x7f5608162880 2024-04-26T23:20:40.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.744+0000 7f5607fff640 1 -- 172.21.15.47:0/3412734271 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55f8031600 con 0x7f5608162880 2024-04-26T23:20:40.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.744+0000 7f5607fff640 1 -- 172.21.15.47:0/3412734271 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f80319d0 con 0x7f5608162880 2024-04-26T23:20:40.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.745+0000 7f560f95f640 1 -- 172.21.15.47:0/3412734271 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5608162880 msgr2=0x7f56080d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:40.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.745+0000 7f560f95f640 1 --2- 172.21.15.47:0/3412734271 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5608162880 0x7f56080d2cb0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f56080d11d0 tx=0x7f55f802a6f0 comp rx=0 tx=0).stop 2024-04-26T23:20:40.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.745+0000 7f560f95f640 1 -- 172.21.15.47:0/3412734271 shutdown_connections 2024-04-26T23:20:40.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.745+0000 7f560f95f640 1 --2- 172.21.15.47:0/3412734271 >> v2:172.21.15.203:3300/0 conn(0x7f56080d3bc0 0x7f56080b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:40.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.745+0000 7f560f95f640 1 --2- 172.21.15.47:0/3412734271 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5608162880 0x7f56080d2cb0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:40.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.745+0000 7f560f95f640 1 -- 172.21.15.47:0/3412734271 >> 172.21.15.47:0/3412734271 conn(0x7f56080d4f60 msgr2=0x7f56080d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:40.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.745+0000 7f560f95f640 1 -- 172.21.15.47:0/3412734271 shutdown_connections 2024-04-26T23:20:40.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.745+0000 7f560f95f640 1 -- 172.21.15.47:0/3412734271 wait complete. 2024-04-26T23:20:40.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.746+0000 7f560f95f640 1 Processor -- start 2024-04-26T23:20:40.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.746+0000 7f560f95f640 1 -- start start 2024-04-26T23:20:40.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.747+0000 7f560f95f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56080b5f20 0x7f5608155500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:40.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.747+0000 7f560f95f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56080d3bc0 0x7f5608159880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:40.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.747+0000 7f560f95f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5608162880 0x7f560815df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:40.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.747+0000 7f560f95f640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5608162c60 con 0x7f56080d3bc0 2024-04-26T23:20:40.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.747+0000 7f560f95f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f56080d0d90 con 0x7f56080b5f20 2024-04-26T23:20:40.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.747+0000 7f560f95f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f5608162de0 con 0x7f5608162880 2024-04-26T23:20:40.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.747+0000 7f560cefa640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56080d3bc0 0x7f5608159880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:40.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.747+0000 7f560cefa640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56080d3bc0 0x7f5608159880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39530/0 (socket says 172.21.15.47:39530) 2024-04-26T23:20:40.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.747+0000 7f560cefa640 1 -- 172.21.15.47:0/1832088287 learned_addr learned my addr 172.21.15.47:0/1832088287 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:40.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.747+0000 7f560d6fb640 1 --2- 172.21.15.47:0/1832088287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56080b5f20 0x7f5608155500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:40.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.747+0000 7f560defc640 1 --2- 172.21.15.47:0/1832088287 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5608162880 0x7f560815df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:40.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.747+0000 7f560cefa640 1 -- 172.21.15.47:0/1832088287 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5608162880 msgr2=0x7f560815df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:40.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.747+0000 7f560cefa640 1 --2- 172.21.15.47:0/1832088287 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5608162880 0x7f560815df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:40.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.747+0000 7f560cefa640 1 -- 172.21.15.47:0/1832088287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56080b5f20 msgr2=0x7f5608155500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:40.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.747+0000 7f560cefa640 1 --2- 172.21.15.47:0/1832088287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56080b5f20 0x7f5608155500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:40.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.747+0000 7f560cefa640 1 -- 172.21.15.47:0/1832088287 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56081f5f70 con 0x7f56080d3bc0 2024-04-26T23:20:40.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.747+0000 7f560d6fb640 1 --2- 172.21.15.47:0/1832088287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56080b5f20 0x7f5608155500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:40.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.747+0000 7f560cefa640 1 --2- 172.21.15.47:0/1832088287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56080d3bc0 0x7f5608159880 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f55fc00b3f0 tx=0x7f55fc00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:40.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.748+0000 7f5605ffb640 1 -- 172.21.15.47:0/1832088287 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc015020 con 0x7f56080d3bc0 2024-04-26T23:20:40.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.748+0000 7f5605ffb640 1 -- 172.21.15.47:0/1832088287 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55fc011830 con 0x7f56080d3bc0 2024-04-26T23:20:40.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.748+0000 7f560f95f640 1 -- 172.21.15.47:0/1832088287 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56081f53f0 con 0x7f56080d3bc0 2024-04-26T23:20:40.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.748+0000 7f5605ffb640 1 -- 172.21.15.47:0/1832088287 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc011c20 con 0x7f56080d3bc0 2024-04-26T23:20:40.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.748+0000 7f560f95f640 1 -- 172.21.15.47:0/1832088287 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56081f59b0 con 0x7f56080d3bc0 2024-04-26T23:20:40.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.749+0000 7f560f95f640 1 -- 172.21.15.47:0/1832088287 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f560808e6c0 con 0x7f56080d3bc0 2024-04-26T23:20:40.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.749+0000 7f5605ffb640 1 -- 172.21.15.47:0/1832088287 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f55fc018020 con 0x7f56080d3bc0 2024-04-26T23:20:40.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.750+0000 7f5605ffb640 1 --2- 172.21.15.47:0/1832088287 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f55f406c920 0x7f55f406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:40.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.750+0000 7f5605ffb640 1 -- 172.21.15.47:0/1832088287 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f55fc08bbf0 con 0x7f56080d3bc0 2024-04-26T23:20:40.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.750+0000 7f560d6fb640 1 --2- 172.21.15.47:0/1832088287 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f55f406c920 0x7f55f406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:40.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.751+0000 7f560d6fb640 1 --2- 172.21.15.47:0/1832088287 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f55f406c920 0x7f55f406ede0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f55f8033000 tx=0x7f55f8009000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:40.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.754+0000 7f5605ffb640 1 -- 172.21.15.47:0/1832088287 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f560808e6c0 con 0x7f56080d3bc0 2024-04-26T23:20:40.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.946+0000 7f560f95f640 1 -- 172.21.15.47:0/1832088287 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f560815efc0 con 0x7f56080d3bc0 2024-04-26T23:20:40.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.947+0000 7f5605ffb640 1 -- 172.21.15.47:0/1832088287 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) ==== 74+0+16902 (secure 0 0 0) 0x7f55fc01f020 con 0x7f56080d3bc0 2024-04-26T23:20:40.949 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:20:40.949 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":54,"fsid":"399006d3-4db1-43e6-ba81-a79a276eabfd","created":"2024-04-26T23:18:06.731211+0000","modified":"2024-04-26T23:20:23.340537+0000","last_up_change":"2024-04-26T23:18:17.808736+0000","last_in_change":"2024-04-26T23:18:08.945859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:18:16.937492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:20:19.975691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-26T23:20:20.657778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ee091567-fea0-4a14-bf4e-0e16c20bb2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6819","nonce":326520779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6821","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6823","nonce":326520779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6829","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6831","nonce":326520779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6825","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6827","nonce":326520779}]},"public_addr":"172.21.15.47:6819/326520779","cluster_addr":"172.21.15.47:6823/326520779","heartbeat_back_addr":"172.21.15.47:6831/326520779","heartbeat_front_addr":"172.21.15.47:6827/326520779","state":["exists","up"]},{"osd":1,"uuid":"c14440a0-bba5-4931-a6db-b736c08e869b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6809","nonce":1912610931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6811","nonce":1912610931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6815","nonce":1912610931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6813","nonce":1912610931}]},"public_addr":"172.21.15.47:6809/1912610931","cluster_addr":"172.21.15.47:6811/1912610931","heartbeat_back_addr":"172.21.15.47:6815/1912610931","heartbeat_front_addr":"172.21.15.47:6813/1912610931","state":["exists","up"]},{"osd":2,"uuid":"3940f8de-c9b2-4cd3-a911-9c301729eb30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6801","nonce":2569869305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6803","nonce":2569869305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6807","nonce":2569869305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6805","nonce":2569869305}]},"public_addr":"172.21.15.47:6801/2569869305","cluster_addr":"172.21.15.47:6803/2569869305","heartbeat_back_addr":"172.21.15.47:6807/2569869305","heartbeat_front_addr":"172.21.15.47:6805/2569869305","state":["exists","up"]},{"osd":3,"uuid":"1286a67e-8958-426a-8c2e-bc85498c619c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6818","nonce":2791411683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6822","nonce":2791411683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6830","nonce":2791411683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6826","nonce":2791411683}]},"public_addr":"172.21.15.47:6818/2791411683","cluster_addr":"172.21.15.47:6822/2791411683","heartbeat_back_addr":"172.21.15.47:6830/2791411683","heartbeat_front_addr":"172.21.15.47:6826/2791411683","state":["exists","up"]},{"osd":4,"uuid":"d8ee2c20-e5b7-46c2-9999-b234070126cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6809","nonce":441313104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6811","nonce":441313104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6815","nonce":441313104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6813","nonce":441313104}]},"public_addr":"172.21.15.203:6809/441313104","cluster_addr":"172.21.15.203:6811/441313104","heartbeat_back_addr":"172.21.15.203:6815/441313104","heartbeat_front_addr":"172.21.15.203:6813/441313104","state":["exists","up"]},{"osd":5,"uuid":"b453a8f0-f324-4840-9ae0-d67bdadb9f8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6801","nonce":56838444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6803","nonce":56838444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6807","nonce":56838444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6805","nonce":56838444}]},"public_addr":"172.21.15.203:6801/56838444","cluster_addr":"172.21.15.203:6803/56838444","heartbeat_back_addr":"172.21.15.203:6807/56838444","heartbeat_front_addr":"172.21.15.203:6805/56838444","state":["exists","up"]},{"osd":6,"uuid":"a6d31951-2d4b-4e8d-a510-6283b01dc158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6825","nonce":2738517080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6827","nonce":2738517080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6831","nonce":2738517080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6829","nonce":2738517080}]},"public_addr":"172.21.15.203:6825/2738517080","cluster_addr":"172.21.15.203:6827/2738517080","heartbeat_back_addr":"172.21.15.203:6831/2738517080","heartbeat_front_addr":"172.21.15.203:6829/2738517080","state":["exists","up"]},{"osd":7,"uuid":"7119b8c0-fe96-402d-9e75-7b509d9d44d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6817","nonce":1315565058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6819","nonce":1315565058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6823","nonce":1315565058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6821","nonce":1315565058}]},"public_addr":"172.21.15.203:6817/1315565058","cluster_addr":"172.21.15.203:6819/1315565058","heartbeat_back_addr":"172.21.15.203:6823/1315565058","heartbeat_front_addr":"172.21.15.203:6821/1315565058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.121921+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.137494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.084814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.111593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.072430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.125066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.097867+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.057061+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:20:40.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.949+0000 7f560f95f640 1 -- 172.21.15.47:0/1832088287 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f55f406c920 msgr2=0x7f55f406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:40.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.949+0000 7f560f95f640 1 --2- 172.21.15.47:0/1832088287 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f55f406c920 0x7f55f406ede0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f55f8033000 tx=0x7f55f8009000 comp rx=0 tx=0).stop 2024-04-26T23:20:40.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.949+0000 7f560f95f640 1 -- 172.21.15.47:0/1832088287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56080d3bc0 msgr2=0x7f5608159880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:40.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.949+0000 7f560f95f640 1 --2- 172.21.15.47:0/1832088287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56080d3bc0 0x7f5608159880 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f55fc00b3f0 tx=0x7f55fc00b8b0 comp rx=0 tx=0).stop 2024-04-26T23:20:40.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.949+0000 7f560f95f640 1 -- 172.21.15.47:0/1832088287 shutdown_connections 2024-04-26T23:20:40.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.949+0000 7f560f95f640 1 --2- 172.21.15.47:0/1832088287 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f55f406c920 0x7f55f406ede0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:40.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.949+0000 7f560f95f640 1 --2- 172.21.15.47:0/1832088287 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5608162880 0x7f560815df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:40.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.949+0000 7f560f95f640 1 --2- 172.21.15.47:0/1832088287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56080d3bc0 0x7f5608159880 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:40.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.949+0000 7f560f95f640 1 --2- 172.21.15.47:0/1832088287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56080b5f20 0x7f5608155500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:40.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.949+0000 7f560f95f640 1 -- 172.21.15.47:0/1832088287 >> 172.21.15.47:0/1832088287 conn(0x7f56080d4f60 msgr2=0x7f560806cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:40.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.949+0000 7f560f95f640 1 -- 172.21.15.47:0/1832088287 shutdown_connections 2024-04-26T23:20:40.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:40.949+0000 7f560f95f640 1 -- 172.21.15.47:0/1832088287 wait complete. 2024-04-26T23:20:40.962 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T23:20:40.962 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:20:41.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.061+0000 7f1ac3edc640 1 Processor -- start 2024-04-26T23:20:41.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.061+0000 7f1ac3edc640 1 -- start start 2024-04-26T23:20:41.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.062+0000 7f1ac3edc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1abc06fc40 0x7f1abc073360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:41.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.062+0000 7f1ac3edc640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1abc156840 con 0x7f1abc156460 2024-04-26T23:20:41.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.062+0000 7f1ac3edc640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1abc070090 con 0x7f1abc06f790 2024-04-26T23:20:41.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.062+0000 7f1ac3edc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f1abc0cbf20 con 0x7f1abc06fc40 2024-04-26T23:20:41.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.062+0000 7f1ac1c78640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1abc156460 0x7f1abc0cb4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47128/0 (socket says 172.21.15.47:47128) 2024-04-26T23:20:41.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.062+0000 7f1ac1c78640 1 -- 172.21.15.47:0/890712632 learned_addr learned my addr 172.21.15.47:0/890712632 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:41.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.063+0000 7f1ac2479640 1 --2- 172.21.15.47:0/890712632 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1abc06fc40 0x7f1abc073360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:41.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.063+0000 7f1ac0c76640 1 -- 172.21.15.47:0/890712632 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1918832353 0 0) 0x7f1abc156840 con 0x7f1abc156460 2024-04-26T23:20:41.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.063+0000 7f1ac0c76640 1 -- 172.21.15.47:0/890712632 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1aa4003520 con 0x7f1abc156460 2024-04-26T23:20:41.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.063+0000 7f1ac0c76640 1 -- 172.21.15.47:0/890712632 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2895387314 0 0) 0x7f1abc070090 con 0x7f1abc06f790 2024-04-26T23:20:41.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.063+0000 7f1ac0c76640 1 -- 172.21.15.47:0/890712632 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1abc156840 con 0x7f1abc06f790 2024-04-26T23:20:41.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.063+0000 7f1ac0c76640 1 -- 172.21.15.47:0/890712632 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4118398819 0 0) 0x7f1aa4003520 con 0x7f1abc156460 2024-04-26T23:20:41.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.063+0000 7f1ac0c76640 1 -- 172.21.15.47:0/890712632 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1abc06fc40 msgr2=0x7f1abc073360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:41.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.063+0000 7f1ac0c76640 1 --2- 172.21.15.47:0/890712632 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1abc06fc40 0x7f1abc073360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:41.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.063+0000 7f1ac0c76640 1 -- 172.21.15.47:0/890712632 >> v1:172.21.15.203:6789/0 conn(0x7f1abc06f790 legacy=0x7f1abc072c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:41.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.063+0000 7f1ac0c76640 1 -- 172.21.15.47:0/890712632 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1abc0cc200 con 0x7f1abc156460 2024-04-26T23:20:41.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.063+0000 7f1ac0c76640 1 -- 172.21.15.47:0/890712632 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7f1aac0019f0 con 0x7f1abc156460 2024-04-26T23:20:41.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.063+0000 7f1ac0c76640 1 -- 172.21.15.47:0/890712632 >> v1:172.21.15.47:6789/0 conn(0x7f1abc156460 legacy=0x7f1abc0cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:41.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.063+0000 7f1ac0c76640 1 --2- 172.21.15.47:0/890712632 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1abc06fc40 0x7f1aa4003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:41.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.063+0000 7f1ac0c76640 1 --2- 172.21.15.47:0/890712632 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1aa40040d0 0x7f1aa4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:41.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.063+0000 7f1ac0c76640 1 --2- 172.21.15.47:0/890712632 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1aa4004d40 0x7f1aa40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:41.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.063+0000 7f1ac0c76640 1 -- 172.21.15.47:0/890712632 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc0cbf20 con 0x7f1aa40040d0 2024-04-26T23:20:41.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.063+0000 7f1ac0c76640 1 -- 172.21.15.47:0/890712632 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa400baa0 con 0x7f1aa4004d40 2024-04-26T23:20:41.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.063+0000 7f1ac0c76640 1 -- 172.21.15.47:0/890712632 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f1aa400bca0 con 0x7f1abc06fc40 2024-04-26T23:20:41.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.063+0000 7f1ac1477640 1 --2- 172.21.15.47:0/890712632 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1aa40040d0 0x7f1aa4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:41.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.063+0000 7f1ac1c78640 1 --2- 172.21.15.47:0/890712632 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1abc06fc40 0x7f1aa4003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:41.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.063+0000 7f1ac2479640 1 --2- 172.21.15.47:0/890712632 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1aa4004d40 0x7f1aa40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:41.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.063+0000 7f1ac1477640 1 -- 172.21.15.47:0/890712632 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1abc06fc40 msgr2=0x7f1aa4003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:41.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.063+0000 7f1ac1477640 1 --2- 172.21.15.47:0/890712632 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1abc06fc40 0x7f1aa4003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:41.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.063+0000 7f1ac1477640 1 -- 172.21.15.47:0/890712632 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1aa4004d40 msgr2=0x7f1aa40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:41.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.063+0000 7f1ac1477640 1 --2- 172.21.15.47:0/890712632 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1aa4004d40 0x7f1aa40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:41.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.063+0000 7f1ac1477640 1 -- 172.21.15.47:0/890712632 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aa4003520 con 0x7f1aa40040d0 2024-04-26T23:20:41.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.063+0000 7f1ac2479640 1 --2- 172.21.15.47:0/890712632 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1aa4004d40 0x7f1aa40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:20:41.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.063+0000 7f1ac1c78640 1 --2- 172.21.15.47:0/890712632 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1abc06fc40 0x7f1aa4003b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:20:41.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.064+0000 7f1ac1477640 1 --2- 172.21.15.47:0/890712632 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1aa40040d0 0x7f1aa4004800 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f1ab00018d0 tx=0x7f1aa4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f22c4672a8dcbf24 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:41.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.064+0000 7f1ac0c76640 1 -- 172.21.15.47:0/890712632 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab00014f0 con 0x7f1aa40040d0 2024-04-26T23:20:41.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.064+0000 7f1ac0c76640 1 -- 172.21.15.47:0/890712632 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ab00052d0 con 0x7f1aa40040d0 2024-04-26T23:20:41.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.064+0000 7f1ac0c76640 1 -- 172.21.15.47:0/890712632 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab002aca0 con 0x7f1aa40040d0 2024-04-26T23:20:41.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.065+0000 7f1ac3edc640 1 -- 172.21.15.47:0/890712632 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1aa40040d0 msgr2=0x7f1aa4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:41.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.065+0000 7f1ac3edc640 1 --2- 172.21.15.47:0/890712632 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1aa40040d0 0x7f1aa4004800 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f1ab00018d0 tx=0x7f1aa4005830 comp rx=0 tx=0).stop 2024-04-26T23:20:41.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.065+0000 7f1ac2479640 1 -- 172.21.15.47:0/890712632 reap_dead start 2024-04-26T23:20:41.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.065+0000 7f1ac3edc640 1 -- 172.21.15.47:0/890712632 shutdown_connections 2024-04-26T23:20:41.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.065+0000 7f1ac3edc640 1 -- 172.21.15.47:0/890712632 >> 172.21.15.47:0/890712632 conn(0x7f1abc0d0be0 msgr2=0x7f1abc0d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:41.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.065+0000 7f1ac3edc640 1 -- 172.21.15.47:0/890712632 shutdown_connections 2024-04-26T23:20:41.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.065+0000 7f1ac3edc640 1 -- 172.21.15.47:0/890712632 wait complete. 2024-04-26T23:20:41.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.066+0000 7f1ac3edc640 1 Processor -- start 2024-04-26T23:20:41.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.066+0000 7f1ac3edc640 1 -- start start 2024-04-26T23:20:41.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.067+0000 7f1ac3edc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1aa40040d0 0x7f1abc14ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:41.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.067+0000 7f1ac3edc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1abc0b4590 0x7f1abc0ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:41.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.067+0000 7f1ac3edc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1abc0ca5f0 0x7f1abc1f5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:41.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.067+0000 7f1ac3edc640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc072700 con 0x7f1abc0ca5f0 2024-04-26T23:20:41.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.067+0000 7f1ac3edc640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc0cc7c0 con 0x7f1abc0b4590 2024-04-26T23:20:41.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.067+0000 7f1ac3edc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f1abc0cbbe0 con 0x7f1aa40040d0 2024-04-26T23:20:41.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.067+0000 7f1ac2479640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1abc0ca5f0 0x7f1abc1f5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:41.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.067+0000 7f1ac2479640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1abc0ca5f0 0x7f1abc1f5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39556/0 (socket says 172.21.15.47:39556) 2024-04-26T23:20:41.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.067+0000 7f1ac2479640 1 -- 172.21.15.47:0/1614723312 learned_addr learned my addr 172.21.15.47:0/1614723312 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:41.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.067+0000 7f1ac1c78640 1 --2- 172.21.15.47:0/1614723312 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1aa40040d0 0x7f1abc14ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:41.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.067+0000 7f1ac1477640 1 --2- 172.21.15.47:0/1614723312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1abc0b4590 0x7f1abc0ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:41.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.067+0000 7f1ac2479640 1 -- 172.21.15.47:0/1614723312 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1aa40040d0 msgr2=0x7f1abc14ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:41.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.067+0000 7f1ac2479640 1 --2- 172.21.15.47:0/1614723312 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1aa40040d0 0x7f1abc14ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:41.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.067+0000 7f1ac2479640 1 -- 172.21.15.47:0/1614723312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1abc0b4590 msgr2=0x7f1abc0ca0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:41.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.067+0000 7f1ac2479640 1 --2- 172.21.15.47:0/1614723312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1abc0b4590 0x7f1abc0ca0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:41.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.067+0000 7f1ac2479640 1 -- 172.21.15.47:0/1614723312 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1abc1f5f50 con 0x7f1abc0ca5f0 2024-04-26T23:20:41.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.067+0000 7f1ac2479640 1 --2- 172.21.15.47:0/1614723312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1abc0ca5f0 0x7f1abc1f5220 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f1abc0ce700 tx=0x7f1ab4011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:41.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.068+0000 7f1aba7fc640 1 -- 172.21.15.47:0/1614723312 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab4003610 con 0x7f1abc0ca5f0 2024-04-26T23:20:41.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.068+0000 7f1aba7fc640 1 -- 172.21.15.47:0/1614723312 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ab401b070 con 0x7f1abc0ca5f0 2024-04-26T23:20:41.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.068+0000 7f1aba7fc640 1 -- 172.21.15.47:0/1614723312 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab40039f0 con 0x7f1abc0ca5f0 2024-04-26T23:20:41.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.068+0000 7f1ac3edc640 1 -- 172.21.15.47:0/1614723312 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1abc0b3ab0 con 0x7f1abc0ca5f0 2024-04-26T23:20:41.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.068+0000 7f1ac3edc640 1 -- 172.21.15.47:0/1614723312 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1abc0b4070 con 0x7f1abc0ca5f0 2024-04-26T23:20:41.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.069+0000 7f1ac3edc640 1 -- 172.21.15.47:0/1614723312 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1abc0694f0 con 0x7f1abc0ca5f0 2024-04-26T23:20:41.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.069+0000 7f1aba7fc640 1 -- 172.21.15.47:0/1614723312 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1ab4003b90 con 0x7f1abc0ca5f0 2024-04-26T23:20:41.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.069+0000 7f1aba7fc640 1 --2- 172.21.15.47:0/1614723312 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1aa806c920 0x7f1aa806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:41.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.070+0000 7f1aba7fc640 1 -- 172.21.15.47:0/1614723312 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f1ab408bbd0 con 0x7f1abc0ca5f0 2024-04-26T23:20:41.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.070+0000 7f1ac1c78640 1 --2- 172.21.15.47:0/1614723312 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1aa806c920 0x7f1aa806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:41.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.070+0000 7f1ac1c78640 1 --2- 172.21.15.47:0/1614723312 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1aa806c920 0x7f1aa806ede0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1aac002230 tx=0x7f1aac0021c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:41.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.074+0000 7f1aba7fc640 1 -- 172.21.15.47:0/1614723312 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1ab4025020 con 0x7f1abc0ca5f0 2024-04-26T23:20:41.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.297+0000 7f1ac3edc640 1 -- 172.21.15.47:0/1614723312 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1abc06de80 con 0x7f1abc0ca5f0 2024-04-26T23:20:41.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.297+0000 7f1aba7fc640 1 -- 172.21.15.47:0/1614723312 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5190 (secure 0 0 0) 0x7f1ab4059780 con 0x7f1abc0ca5f0 2024-04-26T23:20:41.299 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:20:41.300 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5865,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/805147373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":805147373},{"type":"v1","addr":"172.21.15.47:6833","nonce":805147373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5868,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3515322868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3515322868},{"type":"v1","addr":"172.21.15.47:6835","nonce":3515322868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":15008,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/4223205269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":4223205269},{"type":"v1","addr":"172.21.15.203:6837","nonce":4223205269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:20:21.712604+0000","modified":"2024-04-26T23:20:24.725165+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":15011},"failed":[],"damaged":[],"stopped":[],"info":{"gid_15011":{"gid":15011,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6836/1168022146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1168022146},{"type":"v1","addr":"172.21.15.203:6836","nonce":1168022146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":15011,"qdb_cluster":[15011]},"id":3}]} 2024-04-26T23:20:41.300 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 27 2024-04-26T23:20:41.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.299+0000 7f1ac3edc640 1 -- 172.21.15.47:0/1614723312 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1aa806c920 msgr2=0x7f1aa806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:41.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.299+0000 7f1ac3edc640 1 --2- 172.21.15.47:0/1614723312 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1aa806c920 0x7f1aa806ede0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1aac002230 tx=0x7f1aac0021c0 comp rx=0 tx=0).stop 2024-04-26T23:20:41.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.299+0000 7f1ac3edc640 1 -- 172.21.15.47:0/1614723312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1abc0ca5f0 msgr2=0x7f1abc1f5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:41.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.299+0000 7f1ac3edc640 1 --2- 172.21.15.47:0/1614723312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1abc0ca5f0 0x7f1abc1f5220 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f1abc0ce700 tx=0x7f1ab4011920 comp rx=0 tx=0).stop 2024-04-26T23:20:41.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.299+0000 7f1ac3edc640 1 -- 172.21.15.47:0/1614723312 shutdown_connections 2024-04-26T23:20:41.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.299+0000 7f1ac3edc640 1 --2- 172.21.15.47:0/1614723312 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f1aa806c920 0x7f1aa806ede0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:41.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.299+0000 7f1ac3edc640 1 --2- 172.21.15.47:0/1614723312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1abc0ca5f0 0x7f1abc1f5220 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:41.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.299+0000 7f1ac3edc640 1 --2- 172.21.15.47:0/1614723312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1abc0b4590 0x7f1abc0ca0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:41.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.299+0000 7f1ac3edc640 1 --2- 172.21.15.47:0/1614723312 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1aa40040d0 0x7f1abc14ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:41.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.299+0000 7f1ac3edc640 1 -- 172.21.15.47:0/1614723312 >> 172.21.15.47:0/1614723312 conn(0x7f1abc0d0be0 msgr2=0x7f1abc0730d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:41.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.299+0000 7f1ac3edc640 1 -- 172.21.15.47:0/1614723312 shutdown_connections 2024-04-26T23:20:41.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.299+0000 7f1ac3edc640 1 -- 172.21.15.47:0/1614723312 wait complete. 2024-04-26T23:20:41.312 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:20:41.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.411+0000 7fd6309a4640 1 Processor -- start 2024-04-26T23:20:41.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.411+0000 7fd6309a4640 1 -- start start 2024-04-26T23:20:41.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.412+0000 7fd6309a4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd62c1608f0 0x7fd62c0d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:41.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.412+0000 7fd6309a4640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd62c0cde20 con 0x7fd62c0d0a90 2024-04-26T23:20:41.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.412+0000 7fd6309a4640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd62c0ce100 con 0x7fd62c0cffa0 2024-04-26T23:20:41.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.412+0000 7fd6309a4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd62c0cea30 con 0x7fd62c1608f0 2024-04-26T23:20:41.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.412+0000 7fd623fff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd62c0d0a90 0x7fd62c0cf890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47136/0 (socket says 172.21.15.47:47136) 2024-04-26T23:20:41.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.412+0000 7fd623fff640 1 -- 172.21.15.47:0/4032809764 learned_addr learned my addr 172.21.15.47:0/4032809764 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:41.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.412+0000 7fd62a59c640 1 --2- 172.21.15.47:0/4032809764 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd62c1608f0 0x7fd62c0d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:41.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.412+0000 7fd629d9b640 1 -- 172.21.15.47:0/4032809764 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3146783477 0 0) 0x7fd62c0cde20 con 0x7fd62c0d0a90 2024-04-26T23:20:41.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.412+0000 7fd629d9b640 1 -- 172.21.15.47:0/4032809764 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd618003520 con 0x7fd62c0d0a90 2024-04-26T23:20:41.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.412+0000 7fd629d9b640 1 -- 172.21.15.47:0/4032809764 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3014472480 0 0) 0x7fd62c0ce100 con 0x7fd62c0cffa0 2024-04-26T23:20:41.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.412+0000 7fd629d9b640 1 -- 172.21.15.47:0/4032809764 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd62c0cde20 con 0x7fd62c0cffa0 2024-04-26T23:20:41.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.412+0000 7fd62a59c640 1 -- 172.21.15.47:0/4032809764 >> v1:172.21.15.203:6789/0 conn(0x7fd62c0cffa0 legacy=0x7fd62c0b2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:41.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.412+0000 7fd62a59c640 1 -- 172.21.15.47:0/4032809764 >> v1:172.21.15.47:6789/0 conn(0x7fd62c0d0a90 legacy=0x7fd62c0cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:41.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.412+0000 7fd62a59c640 1 -- 172.21.15.47:0/4032809764 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd62c0ce3e0 con 0x7fd62c1608f0 2024-04-26T23:20:41.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.413+0000 7fd62a59c640 1 --2- 172.21.15.47:0/4032809764 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd62c1608f0 0x7fd62c0d1bf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd61c0097d0 tx=0x7fd61c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d48568d9b33b4780 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:41.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.413+0000 7fd629d9b640 1 -- 172.21.15.47:0/4032809764 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61c037070 con 0x7fd62c1608f0 2024-04-26T23:20:41.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.413+0000 7fd629d9b640 1 -- 172.21.15.47:0/4032809764 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd61c032440 con 0x7fd62c1608f0 2024-04-26T23:20:41.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.413+0000 7fd629d9b640 1 -- 172.21.15.47:0/4032809764 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61c032810 con 0x7fd62c1608f0 2024-04-26T23:20:41.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.414+0000 7fd6309a4640 1 -- 172.21.15.47:0/4032809764 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd62c1608f0 msgr2=0x7fd62c0d1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:41.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.414+0000 7fd6309a4640 1 --2- 172.21.15.47:0/4032809764 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd62c1608f0 0x7fd62c0d1bf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd61c0097d0 tx=0x7fd61c02a210 comp rx=0 tx=0).stop 2024-04-26T23:20:41.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.414+0000 7fd6309a4640 1 -- 172.21.15.47:0/4032809764 shutdown_connections 2024-04-26T23:20:41.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.414+0000 7fd6309a4640 1 --2- 172.21.15.47:0/4032809764 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd62c1608f0 0x7fd62c0d1bf0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:41.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.414+0000 7fd6309a4640 1 -- 172.21.15.47:0/4032809764 >> 172.21.15.47:0/4032809764 conn(0x7fd62c0d2ec0 msgr2=0x7fd62c0d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:41.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.414+0000 7fd6309a4640 1 -- 172.21.15.47:0/4032809764 shutdown_connections 2024-04-26T23:20:41.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.414+0000 7fd6309a4640 1 -- 172.21.15.47:0/4032809764 wait complete. 2024-04-26T23:20:41.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.415+0000 7fd6309a4640 1 Processor -- start 2024-04-26T23:20:41.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.415+0000 7fd6309a4640 1 -- start start 2024-04-26T23:20:41.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.416+0000 7fd6309a4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd62c0cffa0 0x7fd62c15e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:41.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.416+0000 7fd6309a4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd62c0d0a90 0x7fd62c0cbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:41.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.416+0000 7fd6309a4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd62c1608f0 0x7fd62c0cc520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:41.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.416+0000 7fd6309a4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd62c0ceec0 con 0x7fd62c0d0a90 2024-04-26T23:20:41.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.416+0000 7fd6309a4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd62c0d0ef0 con 0x7fd62c0cffa0 2024-04-26T23:20:41.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.416+0000 7fd6309a4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd62c0cda40 con 0x7fd62c1608f0 2024-04-26T23:20:41.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.416+0000 7fd623fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd62c0d0a90 0x7fd62c0cbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:41.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.416+0000 7fd623fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd62c0d0a90 0x7fd62c0cbfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39558/0 (socket says 172.21.15.47:39558) 2024-04-26T23:20:41.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.416+0000 7fd623fff640 1 -- 172.21.15.47:0/246235880 learned_addr learned my addr 172.21.15.47:0/246235880 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:41.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.416+0000 7fd62ad9d640 1 --2- 172.21.15.47:0/246235880 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd62c1608f0 0x7fd62c0cc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:41.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.416+0000 7fd62a59c640 1 --2- 172.21.15.47:0/246235880 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd62c0cffa0 0x7fd62c15e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:41.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.417+0000 7fd623fff640 1 -- 172.21.15.47:0/246235880 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd62c1608f0 msgr2=0x7fd62c0cc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:41.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.417+0000 7fd623fff640 1 --2- 172.21.15.47:0/246235880 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd62c1608f0 0x7fd62c0cc520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:41.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.417+0000 7fd623fff640 1 -- 172.21.15.47:0/246235880 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd62c0cffa0 msgr2=0x7fd62c15e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:41.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.417+0000 7fd623fff640 1 --2- 172.21.15.47:0/246235880 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd62c0cffa0 0x7fd62c15e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:41.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.417+0000 7fd623fff640 1 -- 172.21.15.47:0/246235880 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd62c15ebb0 con 0x7fd62c0d0a90 2024-04-26T23:20:41.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.417+0000 7fd62ad9d640 1 --2- 172.21.15.47:0/246235880 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd62c1608f0 0x7fd62c0cc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:41.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.417+0000 7fd62a59c640 1 --2- 172.21.15.47:0/246235880 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd62c0cffa0 0x7fd62c15e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:41.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.417+0000 7fd623fff640 1 --2- 172.21.15.47:0/246235880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd62c0d0a90 0x7fd62c0cbfe0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fd61400c460 tx=0x7fd61400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:41.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.417+0000 7fd622ffd640 1 -- 172.21.15.47:0/246235880 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd614017070 con 0x7fd62c0d0a90 2024-04-26T23:20:41.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.417+0000 7fd622ffd640 1 -- 172.21.15.47:0/246235880 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd614005ce0 con 0x7fd62c0d0a90 2024-04-26T23:20:41.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.417+0000 7fd6309a4640 1 -- 172.21.15.47:0/246235880 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd62c15ee90 con 0x7fd62c0d0a90 2024-04-26T23:20:41.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.417+0000 7fd622ffd640 1 -- 172.21.15.47:0/246235880 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd614007290 con 0x7fd62c0d0a90 2024-04-26T23:20:41.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.417+0000 7fd6309a4640 1 -- 172.21.15.47:0/246235880 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd62c15f450 con 0x7fd62c0d0a90 2024-04-26T23:20:41.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.418+0000 7fd6309a4640 1 -- 172.21.15.47:0/246235880 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd62c0694f0 con 0x7fd62c0d0a90 2024-04-26T23:20:41.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.419+0000 7fd622ffd640 1 -- 172.21.15.47:0/246235880 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd61401a070 con 0x7fd62c0d0a90 2024-04-26T23:20:41.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.419+0000 7fd622ffd640 1 --2- 172.21.15.47:0/246235880 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd60c06c920 0x7fd60c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:41.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.419+0000 7fd622ffd640 1 -- 172.21.15.47:0/246235880 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fd61408afe0 con 0x7fd62c0d0a90 2024-04-26T23:20:41.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.419+0000 7fd62a59c640 1 --2- 172.21.15.47:0/246235880 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd60c06c920 0x7fd60c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:41.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.420+0000 7fd62a59c640 1 --2- 172.21.15.47:0/246235880 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd60c06c920 0x7fd60c06ede0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd61c034000 tx=0x7fd61c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:41.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.423+0000 7fd622ffd640 1 -- 172.21.15.47:0/246235880 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd614059280 con 0x7fd62c0d0a90 2024-04-26T23:20:41.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.645+0000 7fd6309a4640 1 -- 172.21.15.47:0/246235880 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd62c0c9200 con 0x7fd62c0d0a90 2024-04-26T23:20:41.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.646+0000 7fd622ffd640 1 -- 172.21.15.47:0/246235880 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5190 (secure 0 0 0) 0x7fd614058b80 con 0x7fd62c0d0a90 2024-04-26T23:20:41.648 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:20:41.648 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5865,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/805147373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":805147373},{"type":"v1","addr":"172.21.15.47:6833","nonce":805147373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5868,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3515322868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3515322868},{"type":"v1","addr":"172.21.15.47:6835","nonce":3515322868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":15008,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/4223205269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":4223205269},{"type":"v1","addr":"172.21.15.203:6837","nonce":4223205269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:20:21.712604+0000","modified":"2024-04-26T23:20:24.725165+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":15011},"failed":[],"damaged":[],"stopped":[],"info":{"gid_15011":{"gid":15011,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6836/1168022146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1168022146},{"type":"v1","addr":"172.21.15.203:6836","nonce":1168022146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":15011,"qdb_cluster":[15011]},"id":3}]} 2024-04-26T23:20:41.648 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 27 2024-04-26T23:20:41.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.648+0000 7fd6309a4640 1 -- 172.21.15.47:0/246235880 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd60c06c920 msgr2=0x7fd60c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:41.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.648+0000 7fd6309a4640 1 --2- 172.21.15.47:0/246235880 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd60c06c920 0x7fd60c06ede0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd61c034000 tx=0x7fd61c02e000 comp rx=0 tx=0).stop 2024-04-26T23:20:41.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.648+0000 7fd6309a4640 1 -- 172.21.15.47:0/246235880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd62c0d0a90 msgr2=0x7fd62c0cbfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:41.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.648+0000 7fd6309a4640 1 --2- 172.21.15.47:0/246235880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd62c0d0a90 0x7fd62c0cbfe0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fd61400c460 tx=0x7fd61400c920 comp rx=0 tx=0).stop 2024-04-26T23:20:41.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.648+0000 7fd6309a4640 1 -- 172.21.15.47:0/246235880 shutdown_connections 2024-04-26T23:20:41.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.648+0000 7fd6309a4640 1 --2- 172.21.15.47:0/246235880 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fd60c06c920 0x7fd60c06ede0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:41.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.648+0000 7fd6309a4640 1 --2- 172.21.15.47:0/246235880 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd62c1608f0 0x7fd62c0cc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:41.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.648+0000 7fd6309a4640 1 --2- 172.21.15.47:0/246235880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd62c0d0a90 0x7fd62c0cbfe0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:41.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.648+0000 7fd6309a4640 1 --2- 172.21.15.47:0/246235880 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd62c0cffa0 0x7fd62c15e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:41.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.648+0000 7fd6309a4640 1 -- 172.21.15.47:0/246235880 >> 172.21.15.47:0/246235880 conn(0x7fd62c0d2ec0 msgr2=0x7fd62c0b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:41.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.648+0000 7fd6309a4640 1 -- 172.21.15.47:0/246235880 shutdown_connections 2024-04-26T23:20:41.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.648+0000 7fd6309a4640 1 -- 172.21.15.47:0/246235880 wait complete. 2024-04-26T23:20:41.661 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:20:41.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.760+0000 7f0015986640 1 Processor -- start 2024-04-26T23:20:41.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.760+0000 7f0015986640 1 -- start start 2024-04-26T23:20:41.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.761+0000 7f0015986640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f00100d3bc0 0x7f00100d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:41.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.761+0000 7f0015986640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00100b7e50 con 0x7f00100b5f20 2024-04-26T23:20:41.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.761+0000 7f0015986640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00100d1800 con 0x7f0010162880 2024-04-26T23:20:41.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.761+0000 7f0015986640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f00100d0580 con 0x7f00100d3bc0 2024-04-26T23:20:41.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.761+0000 7f000e7fc640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f00100d3bc0 0x7f00100d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:41.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.761+0000 7f000f7fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f00100b5f20 0x7f00100b8300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47138/0 (socket says 172.21.15.47:47138) 2024-04-26T23:20:41.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.761+0000 7f000f7fe640 1 -- 172.21.15.47:0/1816473249 learned_addr learned my addr 172.21.15.47:0/1816473249 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:41.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.761+0000 7f000e7fc640 1 -- 172.21.15.47:0/1816473249 >> v1:172.21.15.203:6789/0 conn(0x7f0010162880 legacy=0x7f00100d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:41.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.761+0000 7f000e7fc640 1 -- 172.21.15.47:0/1816473249 >> v1:172.21.15.47:6789/0 conn(0x7f00100b5f20 legacy=0x7f00100b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:41.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.761+0000 7f000e7fc640 1 -- 172.21.15.47:0/1816473249 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00100d07d0 con 0x7f00100d3bc0 2024-04-26T23:20:41.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.761+0000 7f000dffb640 1 -- 172.21.15.47:0/1816473249 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3296665985 0 0) 0x7f00100b7e50 con 0x7f00100b5f20 2024-04-26T23:20:41.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.762+0000 7f000e7fc640 1 --2- 172.21.15.47:0/1816473249 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00100d3bc0 0x7f00100d2cb0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7efff80097a0 tx=0x7efff802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c9f2783573b6e2d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:41.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.762+0000 7f000dffb640 1 -- 172.21.15.47:0/1816473249 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff8037070 con 0x7f00100d3bc0 2024-04-26T23:20:41.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.762+0000 7f000dffb640 1 -- 172.21.15.47:0/1816473249 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efff8032440 con 0x7f00100d3bc0 2024-04-26T23:20:41.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.762+0000 7f000dffb640 1 -- 172.21.15.47:0/1816473249 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff8032810 con 0x7f00100d3bc0 2024-04-26T23:20:41.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.763+0000 7f0015986640 1 -- 172.21.15.47:0/1816473249 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00100d3bc0 msgr2=0x7f00100d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:41.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.763+0000 7f0015986640 1 --2- 172.21.15.47:0/1816473249 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00100d3bc0 0x7f00100d2cb0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7efff80097a0 tx=0x7efff802a1e0 comp rx=0 tx=0).stop 2024-04-26T23:20:41.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.763+0000 7f0015986640 1 -- 172.21.15.47:0/1816473249 shutdown_connections 2024-04-26T23:20:41.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.763+0000 7f0015986640 1 --2- 172.21.15.47:0/1816473249 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00100d3bc0 0x7f00100d2cb0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:41.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.763+0000 7f0015986640 1 -- 172.21.15.47:0/1816473249 >> 172.21.15.47:0/1816473249 conn(0x7f00100d4f60 msgr2=0x7f00100d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:41.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.763+0000 7f0015986640 1 -- 172.21.15.47:0/1816473249 shutdown_connections 2024-04-26T23:20:41.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.763+0000 7f0015986640 1 -- 172.21.15.47:0/1816473249 wait complete. 2024-04-26T23:20:41.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.764+0000 7f0015986640 1 Processor -- start 2024-04-26T23:20:41.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.764+0000 7f0015986640 1 -- start start 2024-04-26T23:20:41.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.765+0000 7f0015986640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00100b5f20 0x7f00100cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:41.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.765+0000 7f0015986640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00100d3bc0 0x7f00101f5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:41.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.765+0000 7f0015986640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0010162880 0x7f001015c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:41.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.765+0000 7f0015986640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0010162c60 con 0x7f0010162880 2024-04-26T23:20:41.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.765+0000 7f0015986640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f00100d0d90 con 0x7f00100d3bc0 2024-04-26T23:20:41.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.765+0000 7f0015986640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f0010162de0 con 0x7f00100b5f20 2024-04-26T23:20:41.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.765+0000 7f000f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0010162880 0x7f001015c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:41.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.765+0000 7f000f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0010162880 0x7f001015c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39574/0 (socket says 172.21.15.47:39574) 2024-04-26T23:20:41.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.765+0000 7f000f7fe640 1 -- 172.21.15.47:0/76604799 learned_addr learned my addr 172.21.15.47:0/76604799 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:41.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.765+0000 7f000effd640 1 --2- 172.21.15.47:0/76604799 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00100b5f20 0x7f00100cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:41.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.765+0000 7f000e7fc640 1 --2- 172.21.15.47:0/76604799 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00100d3bc0 0x7f00101f5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:41.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.765+0000 7f000f7fe640 1 -- 172.21.15.47:0/76604799 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00100b5f20 msgr2=0x7f00100cbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:41.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.765+0000 7f000f7fe640 1 --2- 172.21.15.47:0/76604799 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00100b5f20 0x7f00100cbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:41.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.765+0000 7f000f7fe640 1 -- 172.21.15.47:0/76604799 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00100d3bc0 msgr2=0x7f00101f5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:41.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.765+0000 7f000f7fe640 1 --2- 172.21.15.47:0/76604799 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00100d3bc0 0x7f00101f5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:41.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.765+0000 7f000f7fe640 1 -- 172.21.15.47:0/76604799 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f001009ee00 con 0x7f0010162880 2024-04-26T23:20:41.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.765+0000 7f000f7fe640 1 --2- 172.21.15.47:0/76604799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0010162880 0x7f001015c700 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f000000c7a0 tx=0x7f000000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:41.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.766+0000 7f0007fff640 1 -- 172.21.15.47:0/76604799 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0000018070 con 0x7f0010162880 2024-04-26T23:20:41.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.766+0000 7f0007fff640 1 -- 172.21.15.47:0/76604799 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f000000a640 con 0x7f0010162880 2024-04-26T23:20:41.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.766+0000 7f0015986640 1 -- 172.21.15.47:0/76604799 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f001009f0e0 con 0x7f0010162880 2024-04-26T23:20:41.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.766+0000 7f0007fff640 1 -- 172.21.15.47:0/76604799 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000000aab0 con 0x7f0010162880 2024-04-26T23:20:41.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.766+0000 7f0015986640 1 -- 172.21.15.47:0/76604799 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00100a2bd0 con 0x7f0010162880 2024-04-26T23:20:41.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.767+0000 7f0015986640 1 -- 172.21.15.47:0/76604799 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00100694f0 con 0x7f0010162880 2024-04-26T23:20:41.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.767+0000 7f0007fff640 1 -- 172.21.15.47:0/76604799 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f000001b070 con 0x7f0010162880 2024-04-26T23:20:41.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.768+0000 7f0007fff640 1 --2- 172.21.15.47:0/76604799 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7efff006c8f0 0x7efff006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:41.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.768+0000 7f0007fff640 1 -- 172.21.15.47:0/76604799 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f000008b1d0 con 0x7f0010162880 2024-04-26T23:20:41.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.768+0000 7f000effd640 1 --2- 172.21.15.47:0/76604799 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7efff006c8f0 0x7efff006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:41.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.768+0000 7f000effd640 1 --2- 172.21.15.47:0/76604799 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7efff006c8f0 0x7efff006edb0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f00100cb800 tx=0x7f0008009060 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:41.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.772+0000 7f0007fff640 1 -- 172.21.15.47:0/76604799 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0000058eb0 con 0x7f0010162880 2024-04-26T23:20:41.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.963+0000 7f0015986640 1 -- 172.21.15.47:0/76604799 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f00100d38c0 con 0x7f0010162880 2024-04-26T23:20:41.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.964+0000 7f0007fff640 1 -- 172.21.15.47:0/76604799 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) ==== 74+0+16902 (secure 0 0 0) 0x7f000005d1b0 con 0x7f0010162880 2024-04-26T23:20:41.966 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:20:41.966 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":54,"fsid":"399006d3-4db1-43e6-ba81-a79a276eabfd","created":"2024-04-26T23:18:06.731211+0000","modified":"2024-04-26T23:20:23.340537+0000","last_up_change":"2024-04-26T23:18:17.808736+0000","last_in_change":"2024-04-26T23:18:08.945859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:18:16.937492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:20:19.975691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-26T23:20:20.657778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ee091567-fea0-4a14-bf4e-0e16c20bb2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6819","nonce":326520779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6821","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6823","nonce":326520779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6829","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6831","nonce":326520779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6825","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6827","nonce":326520779}]},"public_addr":"172.21.15.47:6819/326520779","cluster_addr":"172.21.15.47:6823/326520779","heartbeat_back_addr":"172.21.15.47:6831/326520779","heartbeat_front_addr":"172.21.15.47:6827/326520779","state":["exists","up"]},{"osd":1,"uuid":"c14440a0-bba5-4931-a6db-b736c08e869b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6809","nonce":1912610931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6811","nonce":1912610931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6815","nonce":1912610931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6813","nonce":1912610931}]},"public_addr":"172.21.15.47:6809/1912610931","cluster_addr":"172.21.15.47:6811/1912610931","heartbeat_back_addr":"172.21.15.47:6815/1912610931","heartbeat_front_addr":"172.21.15.47:6813/1912610931","state":["exists","up"]},{"osd":2,"uuid":"3940f8de-c9b2-4cd3-a911-9c301729eb30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6801","nonce":2569869305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6803","nonce":2569869305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6807","nonce":2569869305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6805","nonce":2569869305}]},"public_addr":"172.21.15.47:6801/2569869305","cluster_addr":"172.21.15.47:6803/2569869305","heartbeat_back_addr":"172.21.15.47:6807/2569869305","heartbeat_front_addr":"172.21.15.47:6805/2569869305","state":["exists","up"]},{"osd":3,"uuid":"1286a67e-8958-426a-8c2e-bc85498c619c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6818","nonce":2791411683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6822","nonce":2791411683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6830","nonce":2791411683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6826","nonce":2791411683}]},"public_addr":"172.21.15.47:6818/2791411683","cluster_addr":"172.21.15.47:6822/2791411683","heartbeat_back_addr":"172.21.15.47:6830/2791411683","heartbeat_front_addr":"172.21.15.47:6826/2791411683","state":["exists","up"]},{"osd":4,"uuid":"d8ee2c20-e5b7-46c2-9999-b234070126cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6809","nonce":441313104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6811","nonce":441313104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6815","nonce":441313104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6813","nonce":441313104}]},"public_addr":"172.21.15.203:6809/441313104","cluster_addr":"172.21.15.203:6811/441313104","heartbeat_back_addr":"172.21.15.203:6815/441313104","heartbeat_front_addr":"172.21.15.203:6813/441313104","state":["exists","up"]},{"osd":5,"uuid":"b453a8f0-f324-4840-9ae0-d67bdadb9f8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6801","nonce":56838444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6803","nonce":56838444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6807","nonce":56838444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6805","nonce":56838444}]},"public_addr":"172.21.15.203:6801/56838444","cluster_addr":"172.21.15.203:6803/56838444","heartbeat_back_addr":"172.21.15.203:6807/56838444","heartbeat_front_addr":"172.21.15.203:6805/56838444","state":["exists","up"]},{"osd":6,"uuid":"a6d31951-2d4b-4e8d-a510-6283b01dc158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6825","nonce":2738517080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6827","nonce":2738517080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6831","nonce":2738517080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6829","nonce":2738517080}]},"public_addr":"172.21.15.203:6825/2738517080","cluster_addr":"172.21.15.203:6827/2738517080","heartbeat_back_addr":"172.21.15.203:6831/2738517080","heartbeat_front_addr":"172.21.15.203:6829/2738517080","state":["exists","up"]},{"osd":7,"uuid":"7119b8c0-fe96-402d-9e75-7b509d9d44d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6817","nonce":1315565058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6819","nonce":1315565058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6823","nonce":1315565058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6821","nonce":1315565058}]},"public_addr":"172.21.15.203:6817/1315565058","cluster_addr":"172.21.15.203:6819/1315565058","heartbeat_back_addr":"172.21.15.203:6823/1315565058","heartbeat_front_addr":"172.21.15.203:6821/1315565058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.121921+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.137494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.084814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.111593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.072430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.125066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.097867+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.057061+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:20:41.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.966+0000 7f0015986640 1 -- 172.21.15.47:0/76604799 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7efff006c8f0 msgr2=0x7efff006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:41.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.966+0000 7f0015986640 1 --2- 172.21.15.47:0/76604799 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7efff006c8f0 0x7efff006edb0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f00100cb800 tx=0x7f0008009060 comp rx=0 tx=0).stop 2024-04-26T23:20:41.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.966+0000 7f0015986640 1 -- 172.21.15.47:0/76604799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0010162880 msgr2=0x7f001015c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:41.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.966+0000 7f0015986640 1 --2- 172.21.15.47:0/76604799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0010162880 0x7f001015c700 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f000000c7a0 tx=0x7f000000cc60 comp rx=0 tx=0).stop 2024-04-26T23:20:41.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.966+0000 7f0015986640 1 -- 172.21.15.47:0/76604799 shutdown_connections 2024-04-26T23:20:41.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.966+0000 7f0015986640 1 --2- 172.21.15.47:0/76604799 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7efff006c8f0 0x7efff006edb0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:41.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.966+0000 7f0015986640 1 --2- 172.21.15.47:0/76604799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0010162880 0x7f001015c700 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:41.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.966+0000 7f0015986640 1 --2- 172.21.15.47:0/76604799 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00100d3bc0 0x7f00101f5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:41.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.966+0000 7f0015986640 1 --2- 172.21.15.47:0/76604799 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00100b5f20 0x7f00100cbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:41.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.966+0000 7f0015986640 1 -- 172.21.15.47:0/76604799 >> 172.21.15.47:0/76604799 conn(0x7f00100d4f60 msgr2=0x7f001006cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:41.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.966+0000 7f0015986640 1 -- 172.21.15.47:0/76604799 shutdown_connections 2024-04-26T23:20:41.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:41.966+0000 7f0015986640 1 -- 172.21.15.47:0/76604799 wait complete. 2024-04-26T23:20:41.979 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T23:20:42.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.080+0000 7f4d3a4da640 1 Processor -- start 2024-04-26T23:20:42.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.080+0000 7f4d3a4da640 1 -- start start 2024-04-26T23:20:42.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.081+0000 7f4d3a4da640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4d340d3bc0 0x7f4d340d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:42.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.081+0000 7f4d3a4da640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4d340b5f20 0x7f4d340b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:42.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.081+0000 7f4d3a4da640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d340b7e50 con 0x7f4d34162880 2024-04-26T23:20:42.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.081+0000 7f4d3a4da640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f4d340d0580 con 0x7f4d340b5f20 2024-04-26T23:20:42.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.081+0000 7f4d3a4da640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f4d340d1800 con 0x7f4d340d3bc0 2024-04-26T23:20:42.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.081+0000 7f4d38a77640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4d340b5f20 0x7f4d340b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:42.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.081+0000 7f4d38a77640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4d340b5f20 0x7f4d340b8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39578/0 (socket says 172.21.15.47:39578) 2024-04-26T23:20:42.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.081+0000 7f4d38a77640 1 -- 172.21.15.47:0/1806375139 learned_addr learned my addr 172.21.15.47:0/1806375139 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:42.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.081+0000 7f4d337fe640 1 --2- 172.21.15.47:0/1806375139 >> v2:172.21.15.203:3300/0 conn(0x7f4d340d3bc0 0x7f4d340d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:42.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.081+0000 7f4d38a77640 1 -- 172.21.15.47:0/1806375139 >> v2:172.21.15.203:3300/0 conn(0x7f4d340d3bc0 msgr2=0x7f4d340d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:42.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.081+0000 7f4d38a77640 1 --2- 172.21.15.47:0/1806375139 >> v2:172.21.15.203:3300/0 conn(0x7f4d340d3bc0 0x7f4d340d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:42.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.081+0000 7f4d38a77640 1 -- 172.21.15.47:0/1806375139 >> v1:172.21.15.203:6789/0 conn(0x7f4d34162880 legacy=0x7f4d340d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:42.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.081+0000 7f4d38a77640 1 -- 172.21.15.47:0/1806375139 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d340d07d0 con 0x7f4d340b5f20 2024-04-26T23:20:42.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.082+0000 7f4d38a77640 1 --2- 172.21.15.47:0/1806375139 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d340b5f20 0x7f4d340b8300 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f4d3408d3d0 tx=0x7f4d2c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bc436d08ec6408e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:42.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.082+0000 7f4d32ffd640 1 -- 172.21.15.47:0/1806375139 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c036070 con 0x7f4d340b5f20 2024-04-26T23:20:42.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.082+0000 7f4d32ffd640 1 -- 172.21.15.47:0/1806375139 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d2c031600 con 0x7f4d340b5f20 2024-04-26T23:20:42.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.082+0000 7f4d32ffd640 1 -- 172.21.15.47:0/1806375139 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c0319d0 con 0x7f4d340b5f20 2024-04-26T23:20:42.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.082+0000 7f4d3a4da640 1 -- 172.21.15.47:0/1806375139 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d340b5f20 msgr2=0x7f4d340b8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:42.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.082+0000 7f4d3a4da640 1 --2- 172.21.15.47:0/1806375139 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d340b5f20 0x7f4d340b8300 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f4d3408d3d0 tx=0x7f4d2c02a730 comp rx=0 tx=0).stop 2024-04-26T23:20:42.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.083+0000 7f4d3a4da640 1 -- 172.21.15.47:0/1806375139 shutdown_connections 2024-04-26T23:20:42.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.083+0000 7f4d3a4da640 1 --2- 172.21.15.47:0/1806375139 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d340b5f20 0x7f4d340b8300 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:42.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.083+0000 7f4d3a4da640 1 --2- 172.21.15.47:0/1806375139 >> v2:172.21.15.203:3300/0 conn(0x7f4d340d3bc0 0x7f4d340d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:42.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.083+0000 7f4d3a4da640 1 -- 172.21.15.47:0/1806375139 >> 172.21.15.47:0/1806375139 conn(0x7f4d340d4f60 msgr2=0x7f4d340d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:42.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.083+0000 7f4d3a4da640 1 -- 172.21.15.47:0/1806375139 shutdown_connections 2024-04-26T23:20:42.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.083+0000 7f4d3a4da640 1 -- 172.21.15.47:0/1806375139 wait complete. 2024-04-26T23:20:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.084+0000 7f4d3a4da640 1 Processor -- start 2024-04-26T23:20:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.084+0000 7f4d3a4da640 1 -- start start 2024-04-26T23:20:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.085+0000 7f4d3a4da640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d340b5f20 0x7f4d34159490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.085+0000 7f4d3a4da640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4d340d3bc0 0x7f4d341f6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.085+0000 7f4d3a4da640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d34162880 0x7f4d340ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.085+0000 7f4d3a4da640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4d34162c60 con 0x7f4d340b5f20 2024-04-26T23:20:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.085+0000 7f4d3a4da640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4d340d0d90 con 0x7f4d34162880 2024-04-26T23:20:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.085+0000 7f4d3a4da640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f4d34162de0 con 0x7f4d340d3bc0 2024-04-26T23:20:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.085+0000 7f4d33fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d340b5f20 0x7f4d34159490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.085+0000 7f4d33fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d340b5f20 0x7f4d34159490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39592/0 (socket says 172.21.15.47:39592) 2024-04-26T23:20:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.085+0000 7f4d33fff640 1 -- 172.21.15.47:0/1379494039 learned_addr learned my addr 172.21.15.47:0/1379494039 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:42.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.085+0000 7f4d33fff640 1 -- 172.21.15.47:0/1379494039 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4d340d3bc0 msgr2=0x7f4d341f6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:42.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.085+0000 7f4d337fe640 1 --2- 172.21.15.47:0/1379494039 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4d340d3bc0 0x7f4d341f6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:42.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.085+0000 7f4d38a77640 1 --2- 172.21.15.47:0/1379494039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d34162880 0x7f4d340ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:42.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.085+0000 7f4d33fff640 1 --2- 172.21.15.47:0/1379494039 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4d340d3bc0 0x7f4d341f6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:42.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.085+0000 7f4d33fff640 1 -- 172.21.15.47:0/1379494039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d34162880 msgr2=0x7f4d340ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:42.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.085+0000 7f4d33fff640 1 --2- 172.21.15.47:0/1379494039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d34162880 0x7f4d340ce4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:42.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.085+0000 7f4d33fff640 1 -- 172.21.15.47:0/1379494039 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d341599d0 con 0x7f4d340b5f20 2024-04-26T23:20:42.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.085+0000 7f4d33fff640 1 --2- 172.21.15.47:0/1379494039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d340b5f20 0x7f4d34159490 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f4d2800c400 tx=0x7f4d2800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:42.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.085+0000 7f4d30ff9640 1 -- 172.21.15.47:0/1379494039 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d280052d0 con 0x7f4d340b5f20 2024-04-26T23:20:42.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.085+0000 7f4d30ff9640 1 -- 172.21.15.47:0/1379494039 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d28005490 con 0x7f4d340b5f20 2024-04-26T23:20:42.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.085+0000 7f4d3a4da640 1 -- 172.21.15.47:0/1379494039 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d340cc160 con 0x7f4d340b5f20 2024-04-26T23:20:42.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.085+0000 7f4d30ff9640 1 -- 172.21.15.47:0/1379494039 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d28007260 con 0x7f4d340b5f20 2024-04-26T23:20:42.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.085+0000 7f4d3a4da640 1 -- 172.21.15.47:0/1379494039 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d340cc720 con 0x7f4d340b5f20 2024-04-26T23:20:42.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.086+0000 7f4d3a4da640 1 -- 172.21.15.47:0/1379494039 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d340694f0 con 0x7f4d340b5f20 2024-04-26T23:20:42.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.087+0000 7f4d30ff9640 1 -- 172.21.15.47:0/1379494039 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4d28007400 con 0x7f4d340b5f20 2024-04-26T23:20:42.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.087+0000 7f4d30ff9640 1 --2- 172.21.15.47:0/1379494039 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4d2006c8f0 0x7f4d2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:42.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.088+0000 7f4d30ff9640 1 -- 172.21.15.47:0/1379494039 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f4d2808b160 con 0x7f4d340b5f20 2024-04-26T23:20:42.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.088+0000 7f4d337fe640 1 --2- 172.21.15.47:0/1379494039 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4d2006c8f0 0x7f4d2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:42.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.088+0000 7f4d337fe640 1 --2- 172.21.15.47:0/1379494039 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4d2006c8f0 0x7f4d2006edb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4d340b1f70 tx=0x7f4d24008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:42.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.092+0000 7f4d30ff9640 1 -- 172.21.15.47:0/1379494039 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4d28059490 con 0x7f4d340b5f20 2024-04-26T23:20:42.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.317+0000 7f4d3a4da640 1 -- 172.21.15.47:0/1379494039 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f4d340d15e0 con 0x7f4d340b5f20 2024-04-26T23:20:42.319 INFO:tasks.ceph.mon.a.smithi047.stderr:2024-04-26T23:20:42.317+0000 7fb226ccb640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T23:20:42.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.319+0000 7f4d30ff9640 1 -- 172.21.15.47:0/1379494039 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v28) ==== 161+0+0 (secure 0 0 0) 0x7f4d28058d90 con 0x7f4d340b5f20 2024-04-26T23:20:42.321 INFO:teuthology.orchestra.run.smithi047.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T23:20:42.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.321+0000 7f4d3a4da640 1 -- 172.21.15.47:0/1379494039 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4d2006c8f0 msgr2=0x7f4d2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:42.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.321+0000 7f4d3a4da640 1 --2- 172.21.15.47:0/1379494039 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4d2006c8f0 0x7f4d2006edb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4d340b1f70 tx=0x7f4d24008040 comp rx=0 tx=0).stop 2024-04-26T23:20:42.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.321+0000 7f4d3a4da640 1 -- 172.21.15.47:0/1379494039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d340b5f20 msgr2=0x7f4d34159490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:42.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.321+0000 7f4d3a4da640 1 --2- 172.21.15.47:0/1379494039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d340b5f20 0x7f4d34159490 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f4d2800c400 tx=0x7f4d2800c8c0 comp rx=0 tx=0).stop 2024-04-26T23:20:42.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.322+0000 7f4d3a4da640 1 -- 172.21.15.47:0/1379494039 shutdown_connections 2024-04-26T23:20:42.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.322+0000 7f4d3a4da640 1 --2- 172.21.15.47:0/1379494039 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4d2006c8f0 0x7f4d2006edb0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:42.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.322+0000 7f4d3a4da640 1 --2- 172.21.15.47:0/1379494039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d34162880 0x7f4d340ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:42.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.322+0000 7f4d3a4da640 1 --2- 172.21.15.47:0/1379494039 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4d340d3bc0 0x7f4d341f6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:42.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.322+0000 7f4d3a4da640 1 --2- 172.21.15.47:0/1379494039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d340b5f20 0x7f4d34159490 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:42.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.322+0000 7f4d3a4da640 1 -- 172.21.15.47:0/1379494039 >> 172.21.15.47:0/1379494039 conn(0x7f4d340d4f60 msgr2=0x7f4d340d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:42.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.322+0000 7f4d3a4da640 1 -- 172.21.15.47:0/1379494039 shutdown_connections 2024-04-26T23:20:42.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.322+0000 7f4d3a4da640 1 -- 172.21.15.47:0/1379494039 wait complete. 2024-04-26T23:20:42.335 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T23:20:42.358 INFO:tasks.ceph.mds.b.smithi203.stdout:starting mds.b at 2024-04-26T23:20:42.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.435+0000 7f9ee0899640 1 Processor -- start 2024-04-26T23:20:42.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.436+0000 7f9ee0899640 1 -- start start 2024-04-26T23:20:42.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.436+0000 7f9ee0899640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9edc1608f0 0x7f9edc0d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:42.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.436+0000 7f9ee0899640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9edc0d0a90 0x7f9edc0cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:42.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.436+0000 7f9ee0899640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9edc0cde20 con 0x7f9edc0cfdd0 2024-04-26T23:20:42.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.436+0000 7f9ee0899640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f9edc0cea30 con 0x7f9edc0d0a90 2024-04-26T23:20:42.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.436+0000 7f9ee0899640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f9edc0ce100 con 0x7f9edc1608f0 2024-04-26T23:20:42.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.437+0000 7f9ed9d9b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9edc0d0a90 0x7f9edc0cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:42.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.437+0000 7f9edad9d640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9edc0cfdd0 0x7f9edc160e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47154/0 (socket says 172.21.15.47:47154) 2024-04-26T23:20:42.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.437+0000 7f9edad9d640 1 -- 172.21.15.47:0/744734405 learned_addr learned my addr 172.21.15.47:0/744734405 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:42.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.437+0000 7f9ed9d9b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9edc0d0a90 0x7f9edc0cf890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39604/0 (socket says 172.21.15.47:39604) 2024-04-26T23:20:42.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.437+0000 7f9eda59c640 1 --2- 172.21.15.47:0/744734405 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9edc1608f0 0x7f9edc0d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:42.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.437+0000 7f9ed9d9b640 1 -- 172.21.15.47:0/744734405 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9edc1608f0 msgr2=0x7f9edc0d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:42.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.437+0000 7f9ed9d9b640 1 --2- 172.21.15.47:0/744734405 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9edc1608f0 0x7f9edc0d1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:42.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.437+0000 7f9ed9d9b640 1 -- 172.21.15.47:0/744734405 >> v1:172.21.15.47:6789/0 conn(0x7f9edc0cfdd0 legacy=0x7f9edc160e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:42.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.437+0000 7f9ed959a640 1 -- 172.21.15.47:0/744734405 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 812874104 0 0) 0x7f9edc0cde20 con 0x7f9edc0cfdd0 2024-04-26T23:20:42.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.437+0000 7f9ed9d9b640 1 -- 172.21.15.47:0/744734405 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9edc0ce3e0 con 0x7f9edc0d0a90 2024-04-26T23:20:42.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.437+0000 7f9eda59c640 1 --2- 172.21.15.47:0/744734405 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9edc1608f0 0x7f9edc0d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:42.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.438+0000 7f9ed9d9b640 1 --2- 172.21.15.47:0/744734405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9edc0d0a90 0x7f9edc0cf890 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f9ed00097a0 tx=0x7f9ed002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7981c970899a90e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:42.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.438+0000 7f9ed959a640 1 -- 172.21.15.47:0/744734405 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0037070 con 0x7f9edc0d0a90 2024-04-26T23:20:42.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.438+0000 7f9ed959a640 1 -- 172.21.15.47:0/744734405 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ed0032440 con 0x7f9edc0d0a90 2024-04-26T23:20:42.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.438+0000 7f9ed959a640 1 -- 172.21.15.47:0/744734405 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0032810 con 0x7f9edc0d0a90 2024-04-26T23:20:42.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.438+0000 7f9ee0899640 1 -- 172.21.15.47:0/744734405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9edc0d0a90 msgr2=0x7f9edc0cf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:42.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.438+0000 7f9ee0899640 1 --2- 172.21.15.47:0/744734405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9edc0d0a90 0x7f9edc0cf890 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f9ed00097a0 tx=0x7f9ed002a180 comp rx=0 tx=0).stop 2024-04-26T23:20:42.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.438+0000 7f9ee0899640 1 -- 172.21.15.47:0/744734405 shutdown_connections 2024-04-26T23:20:42.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.438+0000 7f9ee0899640 1 --2- 172.21.15.47:0/744734405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9edc0d0a90 0x7f9edc0cf890 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:42.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.438+0000 7f9ee0899640 1 --2- 172.21.15.47:0/744734405 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9edc1608f0 0x7f9edc0d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:42.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.438+0000 7f9ee0899640 1 -- 172.21.15.47:0/744734405 >> 172.21.15.47:0/744734405 conn(0x7f9edc0d2ec0 msgr2=0x7f9edc0d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:42.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.438+0000 7f9ee0899640 1 -- 172.21.15.47:0/744734405 shutdown_connections 2024-04-26T23:20:42.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.438+0000 7f9ee0899640 1 -- 172.21.15.47:0/744734405 wait complete. 2024-04-26T23:20:42.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.440+0000 7f9ee0899640 1 Processor -- start 2024-04-26T23:20:42.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.440+0000 7f9ee0899640 1 -- start start 2024-04-26T23:20:42.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.440+0000 7f9ee0899640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9edc0cfdd0 0x7f9edc158420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:42.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.441+0000 7f9ee0899640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9edc0d0a90 0x7f9edc0cb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:42.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.441+0000 7f9ee0899640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9edc1608f0 0x7f9edc15b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:42.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.441+0000 7f9ee0899640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9edc0b3310 con 0x7f9edc1608f0 2024-04-26T23:20:42.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.441+0000 7f9ee0899640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9edc0d0ef0 con 0x7f9edc0d0a90 2024-04-26T23:20:42.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.441+0000 7f9ee0899640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f9edc0ceec0 con 0x7f9edc0cfdd0 2024-04-26T23:20:42.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.441+0000 7f9edad9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9edc1608f0 0x7f9edc15b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:42.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.441+0000 7f9edad9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9edc1608f0 0x7f9edc15b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39608/0 (socket says 172.21.15.47:39608) 2024-04-26T23:20:42.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.441+0000 7f9edad9d640 1 -- 172.21.15.47:0/3759200426 learned_addr learned my addr 172.21.15.47:0/3759200426 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:42.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.441+0000 7f9ed9d9b640 1 --2- 172.21.15.47:0/3759200426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9edc0d0a90 0x7f9edc0cb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:42.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.441+0000 7f9eda59c640 1 --2- 172.21.15.47:0/3759200426 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9edc0cfdd0 0x7f9edc158420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:42.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.441+0000 7f9edad9d640 1 -- 172.21.15.47:0/3759200426 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9edc0cfdd0 msgr2=0x7f9edc158420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:42.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.441+0000 7f9edad9d640 1 --2- 172.21.15.47:0/3759200426 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9edc0cfdd0 0x7f9edc158420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:42.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.441+0000 7f9edad9d640 1 -- 172.21.15.47:0/3759200426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9edc0d0a90 msgr2=0x7f9edc0cb700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:42.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.441+0000 7f9edad9d640 1 --2- 172.21.15.47:0/3759200426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9edc0d0a90 0x7f9edc0cb700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:42.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.441+0000 7f9edad9d640 1 -- 172.21.15.47:0/3759200426 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9edc158960 con 0x7f9edc1608f0 2024-04-26T23:20:42.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.441+0000 7f9edad9d640 1 --2- 172.21.15.47:0/3759200426 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9edc1608f0 0x7f9edc15b770 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f9ed400c730 tx=0x7f9ed400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:42.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.441+0000 7f9ecaffd640 1 -- 172.21.15.47:0/3759200426 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed4018070 con 0x7f9edc1608f0 2024-04-26T23:20:42.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.441+0000 7f9ecaffd640 1 -- 172.21.15.47:0/3759200426 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ed400ce40 con 0x7f9edc1608f0 2024-04-26T23:20:42.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.441+0000 7f9ee0899640 1 -- 172.21.15.47:0/3759200426 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9edc0cc0c0 con 0x7f9edc1608f0 2024-04-26T23:20:42.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.441+0000 7f9ecaffd640 1 -- 172.21.15.47:0/3759200426 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed400a860 con 0x7f9edc1608f0 2024-04-26T23:20:42.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.441+0000 7f9ee0899640 1 -- 172.21.15.47:0/3759200426 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9edc0cc960 con 0x7f9edc1608f0 2024-04-26T23:20:42.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.442+0000 7f9ee0899640 1 -- 172.21.15.47:0/3759200426 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9edc0694f0 con 0x7f9edc1608f0 2024-04-26T23:20:42.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.443+0000 7f9ecaffd640 1 -- 172.21.15.47:0/3759200426 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9ed401b070 con 0x7f9edc1608f0 2024-04-26T23:20:42.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.443+0000 7f9ecaffd640 1 --2- 172.21.15.47:0/3759200426 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9ebc06c8f0 0x7f9ebc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:42.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.443+0000 7f9ecaffd640 1 -- 172.21.15.47:0/3759200426 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6187+0+0 (secure 0 0 0) 0x7f9ed408b220 con 0x7f9edc1608f0 2024-04-26T23:20:42.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.444+0000 7f9eda59c640 1 --2- 172.21.15.47:0/3759200426 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9ebc06c8f0 0x7f9ebc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:42.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.444+0000 7f9eda59c640 1 --2- 172.21.15.47:0/3759200426 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9ebc06c8f0 0x7f9ebc06edb0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9edc06ee40 tx=0x7f9ecc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:42.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.448+0000 7f9ecaffd640 1 -- 172.21.15.47:0/3759200426 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9edc158960 con 0x7f9edc1608f0 2024-04-26T23:20:42.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:42.674+0000 7f9ee0899640 1 -- 172.21.15.47:0/3759200426 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f9edc0cc450 con 0x7f9edc1608f0 2024-04-26T23:20:43.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.321+0000 7f9ecaffd640 1 -- 172.21.15.47:0/3759200426 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v29) ==== 104+0+0 (secure 0 0 0) 0x7f9ed4058e20 con 0x7f9edc1608f0 2024-04-26T23:20:43.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.324+0000 7f9ee0899640 1 -- 172.21.15.47:0/3759200426 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9ebc06c8f0 msgr2=0x7f9ebc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:43.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.324+0000 7f9ee0899640 1 --2- 172.21.15.47:0/3759200426 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9ebc06c8f0 0x7f9ebc06edb0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9edc06ee40 tx=0x7f9ecc008040 comp rx=0 tx=0).stop 2024-04-26T23:20:43.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.324+0000 7f9ee0899640 1 -- 172.21.15.47:0/3759200426 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9edc1608f0 msgr2=0x7f9edc15b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:43.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.324+0000 7f9ee0899640 1 --2- 172.21.15.47:0/3759200426 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9edc1608f0 0x7f9edc15b770 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f9ed400c730 tx=0x7f9ed400cbf0 comp rx=0 tx=0).stop 2024-04-26T23:20:43.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.324+0000 7f9ee0899640 1 -- 172.21.15.47:0/3759200426 shutdown_connections 2024-04-26T23:20:43.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.324+0000 7f9ee0899640 1 --2- 172.21.15.47:0/3759200426 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9ebc06c8f0 0x7f9ebc06edb0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:43.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.324+0000 7f9ee0899640 1 --2- 172.21.15.47:0/3759200426 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9edc1608f0 0x7f9edc15b770 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:43.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.324+0000 7f9ee0899640 1 --2- 172.21.15.47:0/3759200426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9edc0d0a90 0x7f9edc0cb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:43.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.324+0000 7f9ee0899640 1 --2- 172.21.15.47:0/3759200426 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9edc0cfdd0 0x7f9edc158420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:43.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.324+0000 7f9ee0899640 1 -- 172.21.15.47:0/3759200426 >> 172.21.15.47:0/3759200426 conn(0x7f9edc0d2ec0 msgr2=0x7f9edc071990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:43.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.324+0000 7f9ee0899640 1 -- 172.21.15.47:0/3759200426 shutdown_connections 2024-04-26T23:20:43.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.324+0000 7f9ee0899640 1 -- 172.21.15.47:0/3759200426 wait complete. 2024-04-26T23:20:43.340 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T23:20:43.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.450+0000 7f12599b7640 1 Processor -- start 2024-04-26T23:20:43.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.450+0000 7f12599b7640 1 -- start start 2024-04-26T23:20:43.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.451+0000 7f12599b7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f125415f340 0x7f12540d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:43.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.451+0000 7f12599b7640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f125415fe30 0x7f12540cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:43.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.451+0000 7f12599b7640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1254163160 con 0x7f12541606c0 2024-04-26T23:20:43.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.451+0000 7f12599b7640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f1254162f10 con 0x7f125415fe30 2024-04-26T23:20:43.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.451+0000 7f12599b7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f1254163d70 con 0x7f125415f340 2024-04-26T23:20:43.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.451+0000 7f12537fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f125415fe30 0x7f12540cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:43.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.451+0000 7f12537fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f125415fe30 0x7f12540cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39616/0 (socket says 172.21.15.47:39616) 2024-04-26T23:20:43.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.451+0000 7f12537fe640 1 -- 172.21.15.47:0/3473433097 learned_addr learned my addr 172.21.15.47:0/3473433097 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:43.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.451+0000 7f12537fe640 1 -- 172.21.15.47:0/3473433097 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f125415f340 msgr2=0x7f12540d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:43.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.451+0000 7f12537fe640 1 --2- 172.21.15.47:0/3473433097 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f125415f340 0x7f12540d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:43.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.452+0000 7f12537fe640 1 -- 172.21.15.47:0/3473433097 >> v1:172.21.15.203:6789/0 conn(0x7f12541606c0 legacy=0x7f125415f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:43.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.452+0000 7f12537fe640 1 -- 172.21.15.47:0/3473433097 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1254163440 con 0x7f125415fe30 2024-04-26T23:20:43.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.452+0000 7f12537fe640 1 --2- 172.21.15.47:0/3473433097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f125415fe30 0x7f12540cf640 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f1244009e50 tx=0x7f124402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25cb9adc1d1a01a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:43.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.452+0000 7f1251ffb640 1 -- 172.21.15.47:0/3473433097 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1244036070 con 0x7f125415fe30 2024-04-26T23:20:43.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.452+0000 7f1251ffb640 1 -- 172.21.15.47:0/3473433097 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1244031600 con 0x7f125415fe30 2024-04-26T23:20:43.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.452+0000 7f1251ffb640 1 -- 172.21.15.47:0/3473433097 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12440319d0 con 0x7f125415fe30 2024-04-26T23:20:43.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.453+0000 7f12599b7640 1 -- 172.21.15.47:0/3473433097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f125415fe30 msgr2=0x7f12540cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:43.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.453+0000 7f12599b7640 1 --2- 172.21.15.47:0/3473433097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f125415fe30 0x7f12540cf640 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f1244009e50 tx=0x7f124402a6c0 comp rx=0 tx=0).stop 2024-04-26T23:20:43.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.453+0000 7f12599b7640 1 -- 172.21.15.47:0/3473433097 shutdown_connections 2024-04-26T23:20:43.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.453+0000 7f12599b7640 1 --2- 172.21.15.47:0/3473433097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f125415fe30 0x7f12540cf640 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:43.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.453+0000 7f12599b7640 1 --2- 172.21.15.47:0/3473433097 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f125415f340 0x7f12540d2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:43.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.453+0000 7f12599b7640 1 -- 172.21.15.47:0/3473433097 >> 172.21.15.47:0/3473433097 conn(0x7f125409ba50 msgr2=0x7f125409be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:43.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.453+0000 7f12599b7640 1 -- 172.21.15.47:0/3473433097 shutdown_connections 2024-04-26T23:20:43.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.453+0000 7f12599b7640 1 -- 172.21.15.47:0/3473433097 wait complete. 2024-04-26T23:20:43.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.454+0000 7f12599b7640 1 Processor -- start 2024-04-26T23:20:43.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.454+0000 7f12599b7640 1 -- start start 2024-04-26T23:20:43.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.455+0000 7f12599b7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f125415f340 0x7f125415df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:43.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.455+0000 7f12599b7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f125415fe30 0x7f1254157a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:43.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.455+0000 7f12599b7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f12541606c0 0x7f1254159c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:43.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.455+0000 7f12599b7640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f12541602a0 con 0x7f125415f340 2024-04-26T23:20:43.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.455+0000 7f12599b7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1254163a00 con 0x7f125415fe30 2024-04-26T23:20:43.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.455+0000 7f12599b7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f12540d2a90 con 0x7f12541606c0 2024-04-26T23:20:43.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.455+0000 7f1252ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f125415f340 0x7f125415df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:43.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.455+0000 7f1252ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f125415f340 0x7f125415df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39624/0 (socket says 172.21.15.47:39624) 2024-04-26T23:20:43.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.455+0000 7f1252ffd640 1 -- 172.21.15.47:0/208333474 learned_addr learned my addr 172.21.15.47:0/208333474 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:43.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.455+0000 7f12537fe640 1 --2- 172.21.15.47:0/208333474 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f12541606c0 0x7f1254159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:43.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.455+0000 7f12527fc640 1 --2- 172.21.15.47:0/208333474 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f125415fe30 0x7f1254157a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:43.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.456+0000 7f1252ffd640 1 -- 172.21.15.47:0/208333474 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f12541606c0 msgr2=0x7f1254159c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:43.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.456+0000 7f1252ffd640 1 --2- 172.21.15.47:0/208333474 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f12541606c0 0x7f1254159c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:43.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.456+0000 7f1252ffd640 1 -- 172.21.15.47:0/208333474 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f125415fe30 msgr2=0x7f1254157a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:43.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.456+0000 7f1252ffd640 1 --2- 172.21.15.47:0/208333474 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f125415fe30 0x7f1254157a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:43.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.456+0000 7f1252ffd640 1 -- 172.21.15.47:0/208333474 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f125415e4f0 con 0x7f125415f340 2024-04-26T23:20:43.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.456+0000 7f12537fe640 1 --2- 172.21.15.47:0/208333474 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f12541606c0 0x7f1254159c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:20:43.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.456+0000 7f1252ffd640 1 --2- 172.21.15.47:0/208333474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f125415f340 0x7f125415df10 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f124c00b4a0 tx=0x7f124c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:43.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.456+0000 7f124bfff640 1 -- 172.21.15.47:0/208333474 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124c014070 con 0x7f125415f340 2024-04-26T23:20:43.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.456+0000 7f124bfff640 1 -- 172.21.15.47:0/208333474 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f124c010610 con 0x7f125415f340 2024-04-26T23:20:43.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.456+0000 7f124bfff640 1 -- 172.21.15.47:0/208333474 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124c010a00 con 0x7f125415f340 2024-04-26T23:20:43.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.456+0000 7f12599b7640 1 -- 172.21.15.47:0/208333474 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f125415e7d0 con 0x7f125415f340 2024-04-26T23:20:43.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.456+0000 7f12599b7640 1 -- 172.21.15.47:0/208333474 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1254159730 con 0x7f125415f340 2024-04-26T23:20:43.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.457+0000 7f12599b7640 1 -- 172.21.15.47:0/208333474 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1254159930 con 0x7f125415f340 2024-04-26T23:20:43.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.457+0000 7f124bfff640 1 -- 172.21.15.47:0/208333474 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f124c017070 con 0x7f125415f340 2024-04-26T23:20:43.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.458+0000 7f124bfff640 1 --2- 172.21.15.47:0/208333474 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f123406c920 0x7f123406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:43.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.458+0000 7f124bfff640 1 -- 172.21.15.47:0/208333474 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6187+0+0 (secure 0 0 0) 0x7f124c08b6e0 con 0x7f125415f340 2024-04-26T23:20:43.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.458+0000 7f12527fc640 1 --2- 172.21.15.47:0/208333474 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f123406c920 0x7f123406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:43.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.459+0000 7f12527fc640 1 --2- 172.21.15.47:0/208333474 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f123406c920 0x7f123406ede0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f123c006fd0 tx=0x7f123c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:43.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.462+0000 7f124bfff640 1 -- 172.21.15.47:0/208333474 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1254159930 con 0x7f125415f340 2024-04-26T23:20:43.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:43.649+0000 7f12599b7640 1 -- 172.21.15.47:0/208333474 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1254069600 con 0x7f125415f340 2024-04-26T23:20:44.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.424+0000 7f124bfff640 1 -- 172.21.15.47:0/208333474 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v56) ==== 181+0+0 (secure 0 0 0) 0x7f124c059930 con 0x7f125415f340 2024-04-26T23:20:44.426 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' removed 2024-04-26T23:20:44.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.429+0000 7f12599b7640 1 -- 172.21.15.47:0/208333474 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f123406c920 msgr2=0x7f123406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:44.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.429+0000 7f12599b7640 1 --2- 172.21.15.47:0/208333474 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f123406c920 0x7f123406ede0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f123c006fd0 tx=0x7f123c008040 comp rx=0 tx=0).stop 2024-04-26T23:20:44.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.429+0000 7f12599b7640 1 -- 172.21.15.47:0/208333474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f125415f340 msgr2=0x7f125415df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:44.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.429+0000 7f12599b7640 1 --2- 172.21.15.47:0/208333474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f125415f340 0x7f125415df10 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f124c00b4a0 tx=0x7f124c00b960 comp rx=0 tx=0).stop 2024-04-26T23:20:44.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.429+0000 7f12599b7640 1 -- 172.21.15.47:0/208333474 shutdown_connections 2024-04-26T23:20:44.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.429+0000 7f12599b7640 1 --2- 172.21.15.47:0/208333474 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f123406c920 0x7f123406ede0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:44.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.429+0000 7f12599b7640 1 --2- 172.21.15.47:0/208333474 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f12541606c0 0x7f1254159c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:44.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.429+0000 7f12599b7640 1 --2- 172.21.15.47:0/208333474 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f125415fe30 0x7f1254157a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:44.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.429+0000 7f12599b7640 1 --2- 172.21.15.47:0/208333474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f125415f340 0x7f125415df10 secure :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f124c00b4a0 tx=0x7f124c00b960 comp rx=0 tx=0).stop 2024-04-26T23:20:44.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.429+0000 7f12599b7640 1 -- 172.21.15.47:0/208333474 >> 172.21.15.47:0/208333474 conn(0x7f125409ba50 msgr2=0x7f125415bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:44.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.431+0000 7f12599b7640 1 -- 172.21.15.47:0/208333474 shutdown_connections 2024-04-26T23:20:44.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.431+0000 7f12599b7640 1 -- 172.21.15.47:0/208333474 wait complete. 2024-04-26T23:20:44.449 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T23:20:44.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.560+0000 7f6cd3104640 1 Processor -- start 2024-04-26T23:20:44.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.561+0000 7f6cd3104640 1 -- start start 2024-04-26T23:20:44.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.561+0000 7f6cd3104640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6ccc15f3e0 0x7f6ccc0d2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:44.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.561+0000 7f6cd3104640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f6ccc0d3da0 0x7f6ccc1620a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:44.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.561+0000 7f6cd3104640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ccc160930 con 0x7f6ccc0d31e0 2024-04-26T23:20:44.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.561+0000 7f6cd3104640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f6ccc15fd20 con 0x7f6ccc15f3e0 2024-04-26T23:20:44.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.561+0000 7f6cd3104640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f6ccc160000 con 0x7f6ccc0d3da0 2024-04-26T23:20:44.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.561+0000 7f6cd0ea0640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6ccc15f3e0 0x7f6ccc0d2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:44.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.561+0000 7f6ccbfff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f6ccc0d31e0 0x7f6ccc0d3690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47168/0 (socket says 172.21.15.47:47168) 2024-04-26T23:20:44.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.562+0000 7f6ccbfff640 1 -- 172.21.15.47:0/374676240 learned_addr learned my addr 172.21.15.47:0/374676240 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:44.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.562+0000 7f6cd16a1640 1 --2- 172.21.15.47:0/374676240 >> v2:172.21.15.203:3300/0 conn(0x7f6ccc0d3da0 0x7f6ccc1620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:44.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.562+0000 7f6cd0ea0640 1 -- 172.21.15.47:0/374676240 >> v2:172.21.15.203:3300/0 conn(0x7f6ccc0d3da0 msgr2=0x7f6ccc1620a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:44.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.562+0000 7f6cd0ea0640 1 --2- 172.21.15.47:0/374676240 >> v2:172.21.15.203:3300/0 conn(0x7f6ccc0d3da0 0x7f6ccc1620a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:44.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.562+0000 7f6cd0ea0640 1 -- 172.21.15.47:0/374676240 >> v1:172.21.15.47:6789/0 conn(0x7f6ccc0d31e0 legacy=0x7f6ccc0d3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:44.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.562+0000 7f6cd0ea0640 1 -- 172.21.15.47:0/374676240 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ccc1602e0 con 0x7f6ccc15f3e0 2024-04-26T23:20:44.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.562+0000 7f6ccb7fe640 1 -- 172.21.15.47:0/374676240 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1919558756 0 0) 0x7f6ccc160930 con 0x7f6ccc0d31e0 2024-04-26T23:20:44.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.562+0000 7f6cd16a1640 1 --2- 172.21.15.47:0/374676240 >> v2:172.21.15.203:3300/0 conn(0x7f6ccc0d3da0 0x7f6ccc1620a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:44.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.563+0000 7f6cd0ea0640 1 --2- 172.21.15.47:0/374676240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ccc15f3e0 0x7f6ccc0d2ca0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4009d90 tx=0x7f6cc402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47859cc20bd7e60a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:44.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.563+0000 7f6ccb7fe640 1 -- 172.21.15.47:0/374676240 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc4031600 con 0x7f6ccc15f3e0 2024-04-26T23:20:44.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.563+0000 7f6ccb7fe640 1 -- 172.21.15.47:0/374676240 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cc40317a0 con 0x7f6ccc15f3e0 2024-04-26T23:20:44.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.563+0000 7f6ccb7fe640 1 -- 172.21.15.47:0/374676240 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc4031b70 con 0x7f6ccc15f3e0 2024-04-26T23:20:44.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.563+0000 7f6cd3104640 1 -- 172.21.15.47:0/374676240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ccc15f3e0 msgr2=0x7f6ccc0d2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:44.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.563+0000 7f6cd3104640 1 --2- 172.21.15.47:0/374676240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ccc15f3e0 0x7f6ccc0d2ca0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4009d90 tx=0x7f6cc402a610 comp rx=0 tx=0).stop 2024-04-26T23:20:44.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.563+0000 7f6cd3104640 1 -- 172.21.15.47:0/374676240 shutdown_connections 2024-04-26T23:20:44.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.563+0000 7f6cd3104640 1 --2- 172.21.15.47:0/374676240 >> v2:172.21.15.203:3300/0 conn(0x7f6ccc0d3da0 0x7f6ccc1620a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:44.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.563+0000 7f6cd3104640 1 --2- 172.21.15.47:0/374676240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ccc15f3e0 0x7f6ccc0d2ca0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:44.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.563+0000 7f6cd3104640 1 -- 172.21.15.47:0/374676240 >> 172.21.15.47:0/374676240 conn(0x7f6ccc09e3c0 msgr2=0x7f6ccc0d50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:44.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.563+0000 7f6cd3104640 1 -- 172.21.15.47:0/374676240 shutdown_connections 2024-04-26T23:20:44.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.563+0000 7f6cd3104640 1 -- 172.21.15.47:0/374676240 wait complete. 2024-04-26T23:20:44.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.564+0000 7f6cd3104640 1 Processor -- start 2024-04-26T23:20:44.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.565+0000 7f6cd3104640 1 -- start start 2024-04-26T23:20:44.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.565+0000 7f6cd3104640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ccc0d31e0 0x7f6ccc15c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:44.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.565+0000 7f6cd3104640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ccc0d3da0 0x7f6ccc1f4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:44.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.565+0000 7f6cd3104640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ccc15f3e0 0x7f6ccc0c93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:44.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.565+0000 7f6cd3104640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6ccc15f7c0 con 0x7f6ccc0d3da0 2024-04-26T23:20:44.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.565+0000 7f6cd3104640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6ccc160d40 con 0x7f6ccc0d31e0 2024-04-26T23:20:44.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.565+0000 7f6cd3104640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f6ccc15f940 con 0x7f6ccc15f3e0 2024-04-26T23:20:44.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.566+0000 7f6ccbfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ccc0d3da0 0x7f6ccc1f4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:44.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.566+0000 7f6ccbfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ccc0d3da0 0x7f6ccc1f4f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39636/0 (socket says 172.21.15.47:39636) 2024-04-26T23:20:44.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.566+0000 7f6ccbfff640 1 -- 172.21.15.47:0/3813955997 learned_addr learned my addr 172.21.15.47:0/3813955997 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:44.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.566+0000 7f6cd0ea0640 1 --2- 172.21.15.47:0/3813955997 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ccc0d31e0 0x7f6ccc15c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:44.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.566+0000 7f6cd16a1640 1 --2- 172.21.15.47:0/3813955997 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ccc15f3e0 0x7f6ccc0c93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:44.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.566+0000 7f6ccbfff640 1 -- 172.21.15.47:0/3813955997 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ccc15f3e0 msgr2=0x7f6ccc0c93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:44.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.566+0000 7f6ccbfff640 1 --2- 172.21.15.47:0/3813955997 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ccc15f3e0 0x7f6ccc0c93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:44.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.566+0000 7f6ccbfff640 1 -- 172.21.15.47:0/3813955997 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ccc0d31e0 msgr2=0x7f6ccc15c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:44.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.566+0000 7f6ccbfff640 1 --2- 172.21.15.47:0/3813955997 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ccc0d31e0 0x7f6ccc15c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:44.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.566+0000 7f6ccbfff640 1 -- 172.21.15.47:0/3813955997 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ccc0c8be0 con 0x7f6ccc0d3da0 2024-04-26T23:20:44.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.566+0000 7f6ccbfff640 1 --2- 172.21.15.47:0/3813955997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ccc0d3da0 0x7f6ccc1f4f00 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f6cc000c410 tx=0x7f6cc000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:44.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.566+0000 7f6cc97fa640 1 -- 172.21.15.47:0/3813955997 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc0017070 con 0x7f6ccc0d3da0 2024-04-26T23:20:44.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.566+0000 7f6cc97fa640 1 -- 172.21.15.47:0/3813955997 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cc0005ce0 con 0x7f6ccc0d3da0 2024-04-26T23:20:44.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.566+0000 7f6cd3104640 1 -- 172.21.15.47:0/3813955997 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ccc1f5820 con 0x7f6ccc0d3da0 2024-04-26T23:20:44.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.566+0000 7f6cc97fa640 1 -- 172.21.15.47:0/3813955997 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc00072f0 con 0x7f6ccc0d3da0 2024-04-26T23:20:44.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.566+0000 7f6cd3104640 1 -- 172.21.15.47:0/3813955997 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ccc1f5de0 con 0x7f6ccc0d3da0 2024-04-26T23:20:44.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.567+0000 7f6cd3104640 1 -- 172.21.15.47:0/3813955997 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ccc069600 con 0x7f6ccc0d3da0 2024-04-26T23:20:44.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.568+0000 7f6cc97fa640 1 -- 172.21.15.47:0/3813955997 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6cc001a070 con 0x7f6ccc0d3da0 2024-04-26T23:20:44.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.568+0000 7f6cc97fa640 1 --2- 172.21.15.47:0/3813955997 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f6cb806c920 0x7f6cb806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:44.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.568+0000 7f6cc97fa640 1 -- 172.21.15.47:0/3813955997 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5749+0+0 (secure 0 0 0) 0x7f6cc008af00 con 0x7f6ccc0d3da0 2024-04-26T23:20:44.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.568+0000 7f6cd0ea0640 1 --2- 172.21.15.47:0/3813955997 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f6cb806c920 0x7f6cb806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:44.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.569+0000 7f6cd0ea0640 1 --2- 172.21.15.47:0/3813955997 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f6cb806c920 0x7f6cb806ede0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4034000 tx=0x7f6cc402f000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:44.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.573+0000 7f6cc97fa640 1 -- 172.21.15.47:0/3813955997 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6cc0059300 con 0x7f6ccc0d3da0 2024-04-26T23:20:44.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:44.758+0000 7f6cd3104640 1 -- 172.21.15.47:0/3813955997 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6ccc0d2680 con 0x7f6ccc0d3da0 2024-04-26T23:20:45.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.428+0000 7f6cc97fa640 1 -- 172.21.15.47:0/3813955997 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v57) ==== 169+0+0 (secure 0 0 0) 0x7f6cc0058c00 con 0x7f6ccc0d3da0 2024-04-26T23:20:45.431 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' removed 2024-04-26T23:20:45.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.435+0000 7f6cd3104640 1 -- 172.21.15.47:0/3813955997 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f6cb806c920 msgr2=0x7f6cb806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:45.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.436+0000 7f6cd3104640 1 --2- 172.21.15.47:0/3813955997 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f6cb806c920 0x7f6cb806ede0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4034000 tx=0x7f6cc402f000 comp rx=0 tx=0).stop 2024-04-26T23:20:45.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.436+0000 7f6cd3104640 1 -- 172.21.15.47:0/3813955997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ccc0d3da0 msgr2=0x7f6ccc1f4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:45.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.436+0000 7f6cd3104640 1 --2- 172.21.15.47:0/3813955997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ccc0d3da0 0x7f6ccc1f4f00 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f6cc000c410 tx=0x7f6cc000c8d0 comp rx=0 tx=0).stop 2024-04-26T23:20:45.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.436+0000 7f6cd3104640 1 -- 172.21.15.47:0/3813955997 shutdown_connections 2024-04-26T23:20:45.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.436+0000 7f6cd3104640 1 --2- 172.21.15.47:0/3813955997 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f6cb806c920 0x7f6cb806ede0 secure :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4034000 tx=0x7f6cc402f000 comp rx=0 tx=0).stop 2024-04-26T23:20:45.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.436+0000 7f6cd3104640 1 --2- 172.21.15.47:0/3813955997 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ccc15f3e0 0x7f6ccc0c93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:45.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.436+0000 7f6cd3104640 1 --2- 172.21.15.47:0/3813955997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ccc0d3da0 0x7f6ccc1f4f00 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:45.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.436+0000 7f6cd3104640 1 --2- 172.21.15.47:0/3813955997 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ccc0d31e0 0x7f6ccc15c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:45.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.436+0000 7f6cd3104640 1 -- 172.21.15.47:0/3813955997 >> 172.21.15.47:0/3813955997 conn(0x7f6ccc09e3c0 msgr2=0x7f6ccc06e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:45.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.437+0000 7f6cd3104640 1 -- 172.21.15.47:0/3813955997 shutdown_connections 2024-04-26T23:20:45.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.437+0000 7f6cd3104640 1 -- 172.21.15.47:0/3813955997 wait complete. 2024-04-26T23:20:45.457 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-26T23:20:45.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.563+0000 7f8395512640 1 Processor -- start 2024-04-26T23:20:45.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.563+0000 7f8395512640 1 -- start start 2024-04-26T23:20:45.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.564+0000 7f8395512640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8390162880 0x7f83900d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:45.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.564+0000 7f8395512640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83900b7e50 con 0x7f83900d3bc0 2024-04-26T23:20:45.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.564+0000 7f8395512640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83900d1800 con 0x7f83900b5f20 2024-04-26T23:20:45.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.564+0000 7f8395512640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f83900d0580 con 0x7f8390162880 2024-04-26T23:20:45.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.564+0000 7f838effd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8390162880 0x7f83900d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:45.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.564+0000 7f838effd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8390162880 0x7f83900d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39642/0 (socket says 172.21.15.47:39642) 2024-04-26T23:20:45.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.564+0000 7f838f7fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f83900d3bc0 0x7f83900b8300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47178/0 (socket says 172.21.15.47:47178) 2024-04-26T23:20:45.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.564+0000 7f838effd640 1 -- 172.21.15.47:0/2026378110 learned_addr learned my addr 172.21.15.47:0/2026378110 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:45.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.564+0000 7f838effd640 1 -- 172.21.15.47:0/2026378110 >> v1:172.21.15.203:6789/0 conn(0x7f83900b5f20 legacy=0x7f83900d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:45.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.564+0000 7f838effd640 1 -- 172.21.15.47:0/2026378110 >> v1:172.21.15.47:6789/0 conn(0x7f83900d3bc0 legacy=0x7f83900b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:45.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.564+0000 7f838effd640 1 -- 172.21.15.47:0/2026378110 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83900d07d0 con 0x7f8390162880 2024-04-26T23:20:45.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.564+0000 7f838dffb640 1 -- 172.21.15.47:0/2026378110 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3908762500 0 0) 0x7f83900b7e50 con 0x7f83900d3bc0 2024-04-26T23:20:45.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.565+0000 7f838effd640 1 --2- 172.21.15.47:0/2026378110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8390162880 0x7f83900d2cb0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f8380009870 tx=0x7f838002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57ad806420dba970 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:45.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.565+0000 7f838dffb640 1 -- 172.21.15.47:0/2026378110 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8380037070 con 0x7f8390162880 2024-04-26T23:20:45.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.565+0000 7f838dffb640 1 -- 172.21.15.47:0/2026378110 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8380032440 con 0x7f8390162880 2024-04-26T23:20:45.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.565+0000 7f838dffb640 1 -- 172.21.15.47:0/2026378110 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8380032810 con 0x7f8390162880 2024-04-26T23:20:45.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.566+0000 7f8395512640 1 -- 172.21.15.47:0/2026378110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8390162880 msgr2=0x7f83900d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:45.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.566+0000 7f8395512640 1 --2- 172.21.15.47:0/2026378110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8390162880 0x7f83900d2cb0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f8380009870 tx=0x7f838002a2a0 comp rx=0 tx=0).stop 2024-04-26T23:20:45.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.566+0000 7f8395512640 1 -- 172.21.15.47:0/2026378110 shutdown_connections 2024-04-26T23:20:45.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.566+0000 7f8395512640 1 --2- 172.21.15.47:0/2026378110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8390162880 0x7f83900d2cb0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:45.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.566+0000 7f8395512640 1 -- 172.21.15.47:0/2026378110 >> 172.21.15.47:0/2026378110 conn(0x7f83900d4f60 msgr2=0x7f83900d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:45.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.566+0000 7f8395512640 1 -- 172.21.15.47:0/2026378110 shutdown_connections 2024-04-26T23:20:45.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.566+0000 7f8395512640 1 -- 172.21.15.47:0/2026378110 wait complete. 2024-04-26T23:20:45.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.567+0000 7f8395512640 1 Processor -- start 2024-04-26T23:20:45.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.567+0000 7f8395512640 1 -- start start 2024-04-26T23:20:45.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.568+0000 7f8395512640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83900b5f20 0x7f83900ba690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:45.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.568+0000 7f8395512640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83900d3bc0 0x7f839006e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:45.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.568+0000 7f8395512640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8390162880 0x7f83900b3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:45.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.568+0000 7f8395512640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8390162c60 con 0x7f83900b5f20 2024-04-26T23:20:45.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.568+0000 7f8395512640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f83900d0d90 con 0x7f8390162880 2024-04-26T23:20:45.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.568+0000 7f8395512640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f8390162de0 con 0x7f83900d3bc0 2024-04-26T23:20:45.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.568+0000 7f838effd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83900b5f20 0x7f83900ba690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:45.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.568+0000 7f838effd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83900b5f20 0x7f83900ba690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39648/0 (socket says 172.21.15.47:39648) 2024-04-26T23:20:45.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.568+0000 7f838effd640 1 -- 172.21.15.47:0/3019953057 learned_addr learned my addr 172.21.15.47:0/3019953057 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:45.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.568+0000 7f838e7fc640 1 --2- 172.21.15.47:0/3019953057 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83900d3bc0 0x7f839006e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:45.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.568+0000 7f838f7fe640 1 --2- 172.21.15.47:0/3019953057 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8390162880 0x7f83900b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:45.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.568+0000 7f838effd640 1 -- 172.21.15.47:0/3019953057 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83900d3bc0 msgr2=0x7f839006e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:45.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.568+0000 7f838effd640 1 --2- 172.21.15.47:0/3019953057 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83900d3bc0 0x7f839006e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:45.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.568+0000 7f838effd640 1 -- 172.21.15.47:0/3019953057 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8390162880 msgr2=0x7f83900b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:45.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.568+0000 7f838effd640 1 --2- 172.21.15.47:0/3019953057 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8390162880 0x7f83900b3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:45.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.568+0000 7f838effd640 1 -- 172.21.15.47:0/3019953057 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83900b1fe0 con 0x7f83900b5f20 2024-04-26T23:20:45.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.568+0000 7f838e7fc640 1 --2- 172.21.15.47:0/3019953057 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83900d3bc0 0x7f839006e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:45.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.568+0000 7f838f7fe640 1 --2- 172.21.15.47:0/3019953057 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8390162880 0x7f83900b3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:45.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.569+0000 7f838effd640 1 --2- 172.21.15.47:0/3019953057 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83900b5f20 0x7f83900ba690 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f8380009870 tx=0x7f83800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:45.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.569+0000 7f83777fe640 1 -- 172.21.15.47:0/3019953057 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8380037040 con 0x7f83900b5f20 2024-04-26T23:20:45.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.569+0000 7f83777fe640 1 -- 172.21.15.47:0/3019953057 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8380004250 con 0x7f83900b5f20 2024-04-26T23:20:45.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.569+0000 7f8395512640 1 -- 172.21.15.47:0/3019953057 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83900b22c0 con 0x7f83900b5f20 2024-04-26T23:20:45.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.569+0000 7f83777fe640 1 -- 172.21.15.47:0/3019953057 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838003c730 con 0x7f83900b5f20 2024-04-26T23:20:45.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.569+0000 7f8395512640 1 -- 172.21.15.47:0/3019953057 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83900b2b60 con 0x7f83900b5f20 2024-04-26T23:20:45.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.570+0000 7f8395512640 1 -- 172.21.15.47:0/3019953057 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83900b2490 con 0x7f83900b5f20 2024-04-26T23:20:45.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.570+0000 7f83777fe640 1 -- 172.21.15.47:0/3019953057 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f838003a070 con 0x7f83900b5f20 2024-04-26T23:20:45.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.571+0000 7f83777fe640 1 --2- 172.21.15.47:0/3019953057 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f837c06c8f0 0x7f837c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:45.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.571+0000 7f83777fe640 1 -- 172.21.15.47:0/3019953057 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5335+0+0 (secure 0 0 0) 0x7f83800ac400 con 0x7f83900b5f20 2024-04-26T23:20:45.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.571+0000 7f838e7fc640 1 --2- 172.21.15.47:0/3019953057 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f837c06c8f0 0x7f837c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:45.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.572+0000 7f838e7fc640 1 --2- 172.21.15.47:0/3019953057 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f837c06c8f0 0x7f837c06edb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f83901f7f90 tx=0x7f8384008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:45.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.575+0000 7f83777fe640 1 -- 172.21.15.47:0/3019953057 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8380041020 con 0x7f83900b5f20 2024-04-26T23:20:45.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.765+0000 7f8395512640 1 -- 172.21.15.47:0/3019953057 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f83900694f0 con 0x7f83900b5f20 2024-04-26T23:20:45.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.768+0000 7f83777fe640 1 -- 172.21.15.47:0/3019953057 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) ==== 104+0+0 (secure 0 0 0) 0x7f83900d1800 con 0x7f83900b5f20 2024-04-26T23:20:45.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.770+0000 7f8395512640 1 -- 172.21.15.47:0/3019953057 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f837c06c8f0 msgr2=0x7f837c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:45.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.770+0000 7f8395512640 1 --2- 172.21.15.47:0/3019953057 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f837c06c8f0 0x7f837c06edb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f83901f7f90 tx=0x7f8384008040 comp rx=0 tx=0).stop 2024-04-26T23:20:45.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.770+0000 7f8395512640 1 -- 172.21.15.47:0/3019953057 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83900b5f20 msgr2=0x7f83900ba690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:45.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.770+0000 7f8395512640 1 --2- 172.21.15.47:0/3019953057 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83900b5f20 0x7f83900ba690 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f8380009870 tx=0x7f83800046c0 comp rx=0 tx=0).stop 2024-04-26T23:20:45.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.770+0000 7f8395512640 1 -- 172.21.15.47:0/3019953057 shutdown_connections 2024-04-26T23:20:45.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.770+0000 7f8395512640 1 --2- 172.21.15.47:0/3019953057 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f837c06c8f0 0x7f837c06edb0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:45.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.770+0000 7f8395512640 1 --2- 172.21.15.47:0/3019953057 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8390162880 0x7f83900b3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:45.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.770+0000 7f8395512640 1 --2- 172.21.15.47:0/3019953057 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83900d3bc0 0x7f839006e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:45.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.770+0000 7f8395512640 1 --2- 172.21.15.47:0/3019953057 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83900b5f20 0x7f83900ba690 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:45.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.770+0000 7f8395512640 1 -- 172.21.15.47:0/3019953057 >> 172.21.15.47:0/3019953057 conn(0x7f83900d4f60 msgr2=0x7f83900d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:45.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.770+0000 7f8395512640 1 -- 172.21.15.47:0/3019953057 shutdown_connections 2024-04-26T23:20:45.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.770+0000 7f8395512640 1 -- 172.21.15.47:0/3019953057 wait complete. 2024-04-26T23:20:45.783 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm' 2024-04-26T23:20:45.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.883+0000 7fdd9042a640 1 Processor -- start 2024-04-26T23:20:45.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.883+0000 7fdd9042a640 1 -- start start 2024-04-26T23:20:45.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.884+0000 7fdd9042a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd88162880 0x7fdd880d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:45.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.884+0000 7fdd9042a640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdd880b5f20 0x7fdd880d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:45.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.884+0000 7fdd9042a640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd880b7e50 con 0x7fdd880d39f0 2024-04-26T23:20:45.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.884+0000 7fdd9042a640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fdd880d0580 con 0x7fdd880b5f20 2024-04-26T23:20:45.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.884+0000 7fdd9042a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fdd880d1800 con 0x7fdd88162880 2024-04-26T23:20:45.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.884+0000 7fdd8d9c5640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdd880b5f20 0x7fdd880d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:45.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.884+0000 7fdd8d9c5640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdd880b5f20 0x7fdd880d34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39660/0 (socket says 172.21.15.47:39660) 2024-04-26T23:20:45.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.884+0000 7fdd8e9c7640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fdd880d39f0 0x7fdd880b69e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47186/0 (socket says 172.21.15.47:47186) 2024-04-26T23:20:45.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.884+0000 7fdd8d9c5640 1 -- 172.21.15.47:0/2174023314 learned_addr learned my addr 172.21.15.47:0/2174023314 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:45.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.884+0000 7fdd8e1c6640 1 --2- 172.21.15.47:0/2174023314 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd88162880 0x7fdd880d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:45.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.884+0000 7fdd8d9c5640 1 -- 172.21.15.47:0/2174023314 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd88162880 msgr2=0x7fdd880d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:45.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.884+0000 7fdd8d9c5640 1 --2- 172.21.15.47:0/2174023314 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd88162880 0x7fdd880d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:45.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.884+0000 7fdd8d9c5640 1 -- 172.21.15.47:0/2174023314 >> v1:172.21.15.47:6789/0 conn(0x7fdd880d39f0 legacy=0x7fdd880b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:45.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.885+0000 7fdd8d9c5640 1 -- 172.21.15.47:0/2174023314 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd880d07d0 con 0x7fdd880b5f20 2024-04-26T23:20:45.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.885+0000 7fdd8d1c4640 1 -- 172.21.15.47:0/2174023314 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3161301710 0 0) 0x7fdd880b7e50 con 0x7fdd880d39f0 2024-04-26T23:20:45.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.885+0000 7fdd8e1c6640 1 --2- 172.21.15.47:0/2174023314 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd88162880 0x7fdd880d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:20:45.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.885+0000 7fdd8d9c5640 1 --2- 172.21.15.47:0/2174023314 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd880b5f20 0x7fdd880d34b0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fdd78009870 tx=0x7fdd7802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=779d82f82d9158d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:45.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.885+0000 7fdd8d1c4640 1 -- 172.21.15.47:0/2174023314 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd78037070 con 0x7fdd880b5f20 2024-04-26T23:20:45.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.885+0000 7fdd8d1c4640 1 -- 172.21.15.47:0/2174023314 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd78032440 con 0x7fdd880b5f20 2024-04-26T23:20:45.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.885+0000 7fdd8d1c4640 1 -- 172.21.15.47:0/2174023314 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd78032810 con 0x7fdd880b5f20 2024-04-26T23:20:45.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.886+0000 7fdd9042a640 1 -- 172.21.15.47:0/2174023314 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd880b5f20 msgr2=0x7fdd880d34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:45.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.886+0000 7fdd9042a640 1 --2- 172.21.15.47:0/2174023314 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd880b5f20 0x7fdd880d34b0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fdd78009870 tx=0x7fdd7802a270 comp rx=0 tx=0).stop 2024-04-26T23:20:45.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.886+0000 7fdd9042a640 1 -- 172.21.15.47:0/2174023314 shutdown_connections 2024-04-26T23:20:45.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.886+0000 7fdd9042a640 1 --2- 172.21.15.47:0/2174023314 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd880b5f20 0x7fdd880d34b0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:45.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.886+0000 7fdd9042a640 1 --2- 172.21.15.47:0/2174023314 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd88162880 0x7fdd880d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:45.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.886+0000 7fdd9042a640 1 -- 172.21.15.47:0/2174023314 >> 172.21.15.47:0/2174023314 conn(0x7fdd880d4f60 msgr2=0x7fdd880d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:45.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.886+0000 7fdd9042a640 1 -- 172.21.15.47:0/2174023314 shutdown_connections 2024-04-26T23:20:45.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.886+0000 7fdd9042a640 1 -- 172.21.15.47:0/2174023314 wait complete. 2024-04-26T23:20:45.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.887+0000 7fdd9042a640 1 Processor -- start 2024-04-26T23:20:45.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.887+0000 7fdd9042a640 1 -- start start 2024-04-26T23:20:45.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.888+0000 7fdd9042a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd880b5f20 0x7fdd8815c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:45.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.888+0000 7fdd9042a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd880d39f0 0x7fdd880cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:45.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.888+0000 7fdd9042a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd88162880 0x7fdd880b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:45.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.888+0000 7fdd9042a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fdd88162c60 con 0x7fdd880b5f20 2024-04-26T23:20:45.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.888+0000 7fdd9042a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdd880d0d90 con 0x7fdd88162880 2024-04-26T23:20:45.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.888+0000 7fdd9042a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fdd88162de0 con 0x7fdd880d39f0 2024-04-26T23:20:45.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.888+0000 7fdd8e1c6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd880b5f20 0x7fdd8815c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:45.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.888+0000 7fdd8e1c6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd880b5f20 0x7fdd8815c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39676/0 (socket says 172.21.15.47:39676) 2024-04-26T23:20:45.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.888+0000 7fdd8e1c6640 1 -- 172.21.15.47:0/2596284765 learned_addr learned my addr 172.21.15.47:0/2596284765 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:45.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.888+0000 7fdd8d9c5640 1 --2- 172.21.15.47:0/2596284765 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd880d39f0 0x7fdd880cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:45.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.888+0000 7fdd8e9c7640 1 --2- 172.21.15.47:0/2596284765 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd88162880 0x7fdd880b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:45.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.888+0000 7fdd8e1c6640 1 -- 172.21.15.47:0/2596284765 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd880d39f0 msgr2=0x7fdd880cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:45.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.888+0000 7fdd8e1c6640 1 --2- 172.21.15.47:0/2596284765 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd880d39f0 0x7fdd880cdf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:45.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.888+0000 7fdd8e1c6640 1 -- 172.21.15.47:0/2596284765 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd88162880 msgr2=0x7fdd880b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:45.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.888+0000 7fdd8e1c6640 1 --2- 172.21.15.47:0/2596284765 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd88162880 0x7fdd880b4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:45.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.888+0000 7fdd8e1c6640 1 -- 172.21.15.47:0/2596284765 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd880cca50 con 0x7fdd880b5f20 2024-04-26T23:20:45.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.888+0000 7fdd8d9c5640 1 --2- 172.21.15.47:0/2596284765 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd880d39f0 0x7fdd880cdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:20:45.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.889+0000 7fdd8e1c6640 1 --2- 172.21.15.47:0/2596284765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd880b5f20 0x7fdd8815c2b0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fdd8400e7e0 tx=0x7fdd8400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:45.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.889+0000 7fdd7effd640 1 -- 172.21.15.47:0/2596284765 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8400f040 con 0x7fdd880b5f20 2024-04-26T23:20:45.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.889+0000 7fdd7effd640 1 -- 172.21.15.47:0/2596284765 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd84004030 con 0x7fdd880b5f20 2024-04-26T23:20:45.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.889+0000 7fdd9042a640 1 -- 172.21.15.47:0/2596284765 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd880ccd30 con 0x7fdd880b5f20 2024-04-26T23:20:45.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.889+0000 7fdd7effd640 1 -- 172.21.15.47:0/2596284765 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd84004400 con 0x7fdd880b5f20 2024-04-26T23:20:45.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.889+0000 7fdd9042a640 1 -- 172.21.15.47:0/2596284765 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd880cd2f0 con 0x7fdd880b5f20 2024-04-26T23:20:45.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.890+0000 7fdd9042a640 1 -- 172.21.15.47:0/2596284765 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd880694f0 con 0x7fdd880b5f20 2024-04-26T23:20:45.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.890+0000 7fdd7effd640 1 -- 172.21.15.47:0/2596284765 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdd8400c6d0 con 0x7fdd880b5f20 2024-04-26T23:20:45.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.891+0000 7fdd7effd640 1 --2- 172.21.15.47:0/2596284765 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdd6c06c8f0 0x7fdd6c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:45.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.891+0000 7fdd7effd640 1 -- 172.21.15.47:0/2596284765 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5335+0+0 (secure 0 0 0) 0x7fdd8408b7c0 con 0x7fdd880b5f20 2024-04-26T23:20:45.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.891+0000 7fdd8d9c5640 1 --2- 172.21.15.47:0/2596284765 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdd6c06c8f0 0x7fdd6c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:45.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.892+0000 7fdd8d9c5640 1 --2- 172.21.15.47:0/2596284765 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdd6c06c8f0 0x7fdd6c06edb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fdd78034000 tx=0x7fdd7802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:45.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:45.895+0000 7fdd7effd640 1 -- 172.21.15.47:0/2596284765 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdd84017020 con 0x7fdd880b5f20 2024-04-26T23:20:46.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:46.151+0000 7fdd9042a640 1 -- 172.21.15.47:0/2596284765 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]} v 0) -- 0x7fdd880d1260 con 0x7fdd880b5f20 2024-04-26T23:20:46.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:46.430+0000 7fdd7effd640 1 -- 172.21.15.47:0/2596284765 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]}]=0 v154) ==== 134+0+0 (secure 0 0 0) 0x7fdd84059d60 con 0x7fdd880b5f20 2024-04-26T23:20:46.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:46.432+0000 7fdd9042a640 1 -- 172.21.15.47:0/2596284765 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdd6c06c8f0 msgr2=0x7fdd6c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:46.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:46.432+0000 7fdd9042a640 1 --2- 172.21.15.47:0/2596284765 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdd6c06c8f0 0x7fdd6c06edb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fdd78034000 tx=0x7fdd7802e000 comp rx=0 tx=0).stop 2024-04-26T23:20:46.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:46.432+0000 7fdd9042a640 1 -- 172.21.15.47:0/2596284765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd880b5f20 msgr2=0x7fdd8815c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:46.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:46.432+0000 7fdd9042a640 1 --2- 172.21.15.47:0/2596284765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd880b5f20 0x7fdd8815c2b0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fdd8400e7e0 tx=0x7fdd8400eca0 comp rx=0 tx=0).stop 2024-04-26T23:20:46.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:46.432+0000 7fdd9042a640 1 -- 172.21.15.47:0/2596284765 shutdown_connections 2024-04-26T23:20:46.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:46.432+0000 7fdd9042a640 1 --2- 172.21.15.47:0/2596284765 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdd6c06c8f0 0x7fdd6c06edb0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:46.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:46.432+0000 7fdd9042a640 1 --2- 172.21.15.47:0/2596284765 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd88162880 0x7fdd880b4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:46.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:46.432+0000 7fdd9042a640 1 --2- 172.21.15.47:0/2596284765 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd880d39f0 0x7fdd880cdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:46.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:46.432+0000 7fdd9042a640 1 --2- 172.21.15.47:0/2596284765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd880b5f20 0x7fdd8815c2b0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:46.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:46.432+0000 7fdd9042a640 1 -- 172.21.15.47:0/2596284765 >> 172.21.15.47:0/2596284765 conn(0x7fdd880d4f60 msgr2=0x7fdd880ba160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:46.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:46.432+0000 7fdd9042a640 1 -- 172.21.15.47:0/2596284765 shutdown_connections 2024-04-26T23:20:46.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:46.432+0000 7fdd9042a640 1 -- 172.21.15.47:0/2596284765 wait complete. 2024-04-26T23:20:46.446 INFO:tasks.cephfs_test_runner:test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) ... ok 2024-04-26T23:20:46.446 INFO:tasks.cephfs_test_runner: 2024-04-26T23:20:46.446 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-26T23:20:46.446 INFO:tasks.cephfs_test_runner:Ran 2 tests in 88.721s 2024-04-26T23:20:46.446 INFO:tasks.cephfs_test_runner: 2024-04-26T23:20:46.447 INFO:tasks.cephfs_test_runner:OK 2024-04-26T23:20:46.448 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-26T23:20:46.457 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-26T23:20:46.498 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-26T23:20:46.498 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:20:46.498 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:20:46.515 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=361 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T23:20:46.515 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-26T23:20:46.515 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:20:46.571 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=361 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T23:20:46.571 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-26T23:20:46.571 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:20:46.590 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056088,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:20:46.590 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-26T23:20:46.590 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:20:46.648 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056088,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T23:20:46.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:20:46.648 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-04-26T23:20:46.665 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T23:20:46.665 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T23:20:46.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:20:46.665 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T23:20:46.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:20:46.730 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T23:20:46.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:20:46.796 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link delete ceph-brx 2024-04-26T23:20:46.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:20:46.887 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-04-26T23:20:46.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:20:46.903 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link delete ceph-brx 2024-04-26T23:20:46.979 INFO:teuthology.orchestra.run.smithi047.stderr:Cannot find device "ceph-brx" 2024-04-26T23:20:46.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T23:20:46.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:20:46.981 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list 2024-04-26T23:20:47.000 INFO:teuthology.orchestra.run.smithi203.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-26T23:20:47.000 INFO:teuthology.orchestra.run.smithi203.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-26T23:20:47.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:20:47.001 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T23:20:47.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:20:47.071 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T23:20:47.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:20:47.145 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link delete ceph-brx 2024-04-26T23:20:47.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:20:47.244 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list 2024-04-26T23:20:47.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:20:47.263 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link delete ceph-brx 2024-04-26T23:20:47.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T23:20:47.335 INFO:teuthology.orchestra.run.smithi203.stderr:Cannot find device "ceph-brx" 2024-04-26T23:20:47.336 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-26T23:20:47.487 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-26T23:20:47.487 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T23:20:47.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.592+0000 7fb84d486640 1 Processor -- start 2024-04-26T23:20:47.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.592+0000 7fb84d486640 1 -- start start 2024-04-26T23:20:47.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.593+0000 7fb84d486640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb84806fc40 0x7fb848073360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:47.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.593+0000 7fb84d486640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb848156840 con 0x7fb848156460 2024-04-26T23:20:47.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.593+0000 7fb84d486640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8480cbf20 con 0x7fb84806f790 2024-04-26T23:20:47.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.593+0000 7fb84d486640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fb8480cc200 con 0x7fb84806fc40 2024-04-26T23:20:47.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.593+0000 7fb846ffd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb848156460 0x7fb8480cb4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47200/0 (socket says 172.21.15.47:47200) 2024-04-26T23:20:47.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.593+0000 7fb846ffd640 1 -- 172.21.15.47:0/2289705387 learned_addr learned my addr 172.21.15.47:0/2289705387 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:47.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.594+0000 7fb8477fe640 1 --2- 172.21.15.47:0/2289705387 >> v2:172.21.15.203:3300/0 conn(0x7fb84806fc40 0x7fb848073360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:47.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.594+0000 7fb845ffb640 1 -- 172.21.15.47:0/2289705387 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3607862922 0 0) 0x7fb848156840 con 0x7fb848156460 2024-04-26T23:20:47.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.594+0000 7fb845ffb640 1 -- 172.21.15.47:0/2289705387 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb830005730 con 0x7fb848156460 2024-04-26T23:20:47.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.594+0000 7fb845ffb640 1 -- 172.21.15.47:0/2289705387 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1586378827 0 0) 0x7fb8480cbf20 con 0x7fb84806f790 2024-04-26T23:20:47.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.594+0000 7fb845ffb640 1 -- 172.21.15.47:0/2289705387 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb848156840 con 0x7fb84806f790 2024-04-26T23:20:47.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.594+0000 7fb8477fe640 1 -- 172.21.15.47:0/2289705387 >> v1:172.21.15.203:6789/0 conn(0x7fb84806f790 legacy=0x7fb848072c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:47.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.594+0000 7fb8477fe640 1 -- 172.21.15.47:0/2289705387 >> v1:172.21.15.47:6789/0 conn(0x7fb848156460 legacy=0x7fb8480cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:47.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.594+0000 7fb8477fe640 1 -- 172.21.15.47:0/2289705387 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8480cc4e0 con 0x7fb84806fc40 2024-04-26T23:20:47.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.595+0000 7fb8477fe640 1 --2- 172.21.15.47:0/2289705387 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb84806fc40 0x7fb848073360 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb840009d90 tx=0x7fb84002a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3572d5d1369f8f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:47.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.595+0000 7fb845ffb640 1 -- 172.21.15.47:0/2289705387 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb840007740 con 0x7fb84806fc40 2024-04-26T23:20:47.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.595+0000 7fb845ffb640 1 -- 172.21.15.47:0/2289705387 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8400078e0 con 0x7fb84806fc40 2024-04-26T23:20:47.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.595+0000 7fb845ffb640 1 -- 172.21.15.47:0/2289705387 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb840007c90 con 0x7fb84806fc40 2024-04-26T23:20:47.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.595+0000 7fb84d486640 1 -- 172.21.15.47:0/2289705387 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb84806fc40 msgr2=0x7fb848073360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:47.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.595+0000 7fb84d486640 1 --2- 172.21.15.47:0/2289705387 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb84806fc40 0x7fb848073360 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb840009d90 tx=0x7fb84002a6f0 comp rx=0 tx=0).stop 2024-04-26T23:20:47.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.595+0000 7fb84d486640 1 -- 172.21.15.47:0/2289705387 shutdown_connections 2024-04-26T23:20:47.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.595+0000 7fb84d486640 1 --2- 172.21.15.47:0/2289705387 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb84806fc40 0x7fb848073360 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:47.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.595+0000 7fb84d486640 1 -- 172.21.15.47:0/2289705387 >> 172.21.15.47:0/2289705387 conn(0x7fb8480d0be0 msgr2=0x7fb8480d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:47.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.595+0000 7fb84d486640 1 -- 172.21.15.47:0/2289705387 shutdown_connections 2024-04-26T23:20:47.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.595+0000 7fb84d486640 1 -- 172.21.15.47:0/2289705387 wait complete. 2024-04-26T23:20:47.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.597+0000 7fb84d486640 1 Processor -- start 2024-04-26T23:20:47.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.597+0000 7fb84d486640 1 -- start start 2024-04-26T23:20:47.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.597+0000 7fb84d486640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb84806f790 0x7fb8480c7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:47.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.597+0000 7fb84d486640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb84806fc40 0x7fb84814f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:47.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.597+0000 7fb84d486640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb848156460 0x7fb8481f3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:47.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.597+0000 7fb84d486640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb848070050 con 0x7fb84806f790 2024-04-26T23:20:47.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.597+0000 7fb84d486640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb848072700 con 0x7fb84806fc40 2024-04-26T23:20:47.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.597+0000 7fb84d486640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb8480cbbe0 con 0x7fb848156460 2024-04-26T23:20:47.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.598+0000 7fb846ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb84806f790 0x7fb8480c7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:47.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.598+0000 7fb846ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb84806f790 0x7fb8480c7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39690/0 (socket says 172.21.15.47:39690) 2024-04-26T23:20:47.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.598+0000 7fb846ffd640 1 -- 172.21.15.47:0/257236391 learned_addr learned my addr 172.21.15.47:0/257236391 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:47.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.598+0000 7fb8477fe640 1 --2- 172.21.15.47:0/257236391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb848156460 0x7fb8481f3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:47.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.598+0000 7fb8467fc640 1 --2- 172.21.15.47:0/257236391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb84806fc40 0x7fb84814f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:47.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.598+0000 7fb846ffd640 1 -- 172.21.15.47:0/257236391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb848156460 msgr2=0x7fb8481f3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:47.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.598+0000 7fb846ffd640 1 --2- 172.21.15.47:0/257236391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb848156460 0x7fb8481f3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:47.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.598+0000 7fb846ffd640 1 -- 172.21.15.47:0/257236391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb84806fc40 msgr2=0x7fb84814f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:47.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.598+0000 7fb846ffd640 1 --2- 172.21.15.47:0/257236391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb84806fc40 0x7fb84814f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:47.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.598+0000 7fb846ffd640 1 -- 172.21.15.47:0/257236391 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8480c7cb0 con 0x7fb84806f790 2024-04-26T23:20:47.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.598+0000 7fb846ffd640 1 --2- 172.21.15.47:0/257236391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb84806f790 0x7fb8480c7140 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fb83800c410 tx=0x7fb83800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:47.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.598+0000 7fb82f7fe640 1 -- 172.21.15.47:0/257236391 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb838017070 con 0x7fb84806f790 2024-04-26T23:20:47.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.598+0000 7fb82f7fe640 1 -- 172.21.15.47:0/257236391 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb838005ce0 con 0x7fb84806f790 2024-04-26T23:20:47.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.598+0000 7fb82f7fe640 1 -- 172.21.15.47:0/257236391 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8380072d0 con 0x7fb84806f790 2024-04-26T23:20:47.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.598+0000 7fb84d486640 1 -- 172.21.15.47:0/257236391 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8481f4540 con 0x7fb84806f790 2024-04-26T23:20:47.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.598+0000 7fb84d486640 1 -- 172.21.15.47:0/257236391 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8481f4b00 con 0x7fb84806f790 2024-04-26T23:20:47.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.599+0000 7fb84d486640 1 -- 172.21.15.47:0/257236391 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8480694f0 con 0x7fb84806f790 2024-04-26T23:20:47.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.600+0000 7fb82f7fe640 1 -- 172.21.15.47:0/257236391 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb83801a070 con 0x7fb84806f790 2024-04-26T23:20:47.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.600+0000 7fb82f7fe640 1 --2- 172.21.15.47:0/257236391 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb83406c8f0 0x7fb83406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:47.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.600+0000 7fb82f7fe640 1 -- 172.21.15.47:0/257236391 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5335+0+0 (secure 0 0 0) 0x7fb83808ac70 con 0x7fb84806f790 2024-04-26T23:20:47.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.601+0000 7fb8467fc640 1 --2- 172.21.15.47:0/257236391 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb83406c8f0 0x7fb83406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:47.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.601+0000 7fb8467fc640 1 --2- 172.21.15.47:0/257236391 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb83406c8f0 0x7fb83406edb0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb83c007fd0 tx=0x7fb83c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:47.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.605+0000 7fb82f7fe640 1 -- 172.21.15.47:0/257236391 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb838058c50 con 0x7fb84806f790 2024-04-26T23:20:47.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.786+0000 7fb84d486640 1 -- 172.21.15.47:0/257236391 --> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb84814fa80 con 0x7fb83406c8f0 2024-04-26T23:20:47.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.788+0000 7fb82f7fe640 1 -- 172.21.15.47:0/257236391 <== mgr.4099 v2:172.21.15.203:6832/992162732 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13403 (secure 0 0 0) 0x7fb83c00f850 con 0x7fb83406c8f0 2024-04-26T23:20:47.790 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:20:47.790 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-26T23:20:47.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.789+0000 7fb84d486640 1 -- 172.21.15.47:0/257236391 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb83406c8f0 msgr2=0x7fb83406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:47.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.789+0000 7fb84d486640 1 --2- 172.21.15.47:0/257236391 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb83406c8f0 0x7fb83406edb0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb83c007fd0 tx=0x7fb83c009040 comp rx=0 tx=0).stop 2024-04-26T23:20:47.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.789+0000 7fb84d486640 1 -- 172.21.15.47:0/257236391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb84806f790 msgr2=0x7fb8480c7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:47.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.789+0000 7fb84d486640 1 --2- 172.21.15.47:0/257236391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb84806f790 0x7fb8480c7140 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fb83800c410 tx=0x7fb83800c8d0 comp rx=0 tx=0).stop 2024-04-26T23:20:47.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.789+0000 7fb84d486640 1 -- 172.21.15.47:0/257236391 shutdown_connections 2024-04-26T23:20:47.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.789+0000 7fb84d486640 1 --2- 172.21.15.47:0/257236391 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fb83406c8f0 0x7fb83406edb0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:47.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.789+0000 7fb84d486640 1 --2- 172.21.15.47:0/257236391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb848156460 0x7fb8481f3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:47.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.789+0000 7fb84d486640 1 --2- 172.21.15.47:0/257236391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb84806fc40 0x7fb84814f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:47.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.789+0000 7fb84d486640 1 --2- 172.21.15.47:0/257236391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb84806f790 0x7fb8480c7140 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:47.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.789+0000 7fb84d486640 1 -- 172.21.15.47:0/257236391 >> 172.21.15.47:0/257236391 conn(0x7fb8480d0be0 msgr2=0x7fb8480cf5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:47.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.789+0000 7fb84d486640 1 -- 172.21.15.47:0/257236391 shutdown_connections 2024-04-26T23:20:47.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.789+0000 7fb84d486640 1 -- 172.21.15.47:0/257236391 wait complete. 2024-04-26T23:20:47.802 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":125,"stamp":"2024-04-26T23:20:46.961092+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239140,"kb_used_data":6340,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735580,"statfs":{"total":773094113280,"available":772849233920,"internally_reserved":0,"allocated":6492160,"data_stored":3577713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17361,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-67,"num_write_kb":-40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001480"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":234,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-26T23:20:42.324231+0000","last_change":"2024-04-26T23:18:18.862631+0000","last_active":"2024-04-26T23:20:42.324231+0000","last_peered":"2024-04-26T23:20:42.324231+0000","last_clean":"2024-04-26T23:20:42.324231+0000","last_became_active":"2024-04-26T23:18:18.861536+0000","last_became_peered":"2024-04-26T23:18:18.861536+0000","last_unstale":"2024-04-26T23:20:42.324231+0000","last_undegraded":"2024-04-26T23:20:42.324231+0000","last_fullsized":"2024-04-26T23:20:42.324231+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:17.808736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:17.808736+0000","last_clean_scrub_stamp":"2024-04-26T23:18:17.808736+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:59:26.359282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542176,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28248,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343592,"statfs":{"total":96636764160,"available":96607838208,"internally_reserved":0,"allocated":1257472,"data_stored":887666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542178,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27660,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344180,"statfs":{"total":96636764160,"available":96608440320,"internally_reserved":0,"allocated":655360,"data_stored":297138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542177,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":663552,"data_stored":297790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542177,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27688,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344152,"statfs":{"total":96636764160,"available":96608411648,"internally_reserved":0,"allocated":684032,"data_stored":315986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574880,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31820,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340020,"statfs":{"total":96636764160,"available":96604180480,"internally_reserved":0,"allocated":651264,"data_stored":290721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31816,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340024,"statfs":{"total":96636764160,"available":96604184576,"internally_reserved":0,"allocated":651264,"data_stored":290721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574879,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31824,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340016,"statfs":{"total":96636764160,"available":96604176384,"internally_reserved":0,"allocated":659456,"data_stored":297228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574880,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32416,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339424,"statfs":{"total":96636764160,"available":96603570176,"internally_reserved":0,"allocated":1269760,"data_stored":900463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:20:47.803 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T23:20:47.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.902+0000 7f4e1a847640 1 Processor -- start 2024-04-26T23:20:47.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.902+0000 7f4e1a847640 1 -- start start 2024-04-26T23:20:47.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.902+0000 7f4e1a847640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e1415f3a0 0x7f4e140d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:47.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.902+0000 7f4e1a847640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e140d0e20 con 0x7f4e140d1770 2024-04-26T23:20:47.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.902+0000 7f4e1a847640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e140b1d70 con 0x7f4e140d2300 2024-04-26T23:20:47.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.902+0000 7f4e1a847640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f4e140b2050 con 0x7f4e1415f3a0 2024-04-26T23:20:47.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.903+0000 7f4e137fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4e140d1770 0x7f4e140d1bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47212/0 (socket says 172.21.15.47:47212) 2024-04-26T23:20:47.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.903+0000 7f4e137fe640 1 -- 172.21.15.47:0/3698942866 learned_addr learned my addr 172.21.15.47:0/3698942866 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:47.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.903+0000 7f4e13fff640 1 --2- 172.21.15.47:0/3698942866 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e1415f3a0 0x7f4e140d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:47.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.903+0000 7f4e12ffd640 1 -- 172.21.15.47:0/3698942866 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3828831022 0 0) 0x7f4e140d0e20 con 0x7f4e140d1770 2024-04-26T23:20:47.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.903+0000 7f4e12ffd640 1 -- 172.21.15.47:0/3698942866 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e00003520 con 0x7f4e140d1770 2024-04-26T23:20:47.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.903+0000 7f4e12ffd640 1 -- 172.21.15.47:0/3698942866 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2399989096 0 0) 0x7f4e140b1d70 con 0x7f4e140d2300 2024-04-26T23:20:47.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.903+0000 7f4e12ffd640 1 -- 172.21.15.47:0/3698942866 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e140d0e20 con 0x7f4e140d2300 2024-04-26T23:20:47.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.903+0000 7f4e12ffd640 1 -- 172.21.15.47:0/3698942866 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3043260586 0 0) 0x7f4e00003520 con 0x7f4e140d1770 2024-04-26T23:20:47.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.903+0000 7f4e12ffd640 1 -- 172.21.15.47:0/3698942866 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e1415f3a0 msgr2=0x7f4e140d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:47.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.903+0000 7f4e12ffd640 1 --2- 172.21.15.47:0/3698942866 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e1415f3a0 0x7f4e140d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:47.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.903+0000 7f4e12ffd640 1 -- 172.21.15.47:0/3698942866 >> v1:172.21.15.203:6789/0 conn(0x7f4e140d2300 legacy=0x7f4e140b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:47.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.903+0000 7f4e12ffd640 1 -- 172.21.15.47:0/3698942866 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e140b2330 con 0x7f4e140d1770 2024-04-26T23:20:47.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.903+0000 7f4e12ffd640 1 -- 172.21.15.47:0/3698942866 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7f4e04001670 con 0x7f4e140d1770 2024-04-26T23:20:47.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.903+0000 7f4e12ffd640 1 -- 172.21.15.47:0/3698942866 >> v1:172.21.15.47:6789/0 conn(0x7f4e140d1770 legacy=0x7f4e140d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:47.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.903+0000 7f4e12ffd640 1 --2- 172.21.15.47:0/3698942866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e00003b70 0x7f4e00003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:47.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.903+0000 7f4e12ffd640 1 --2- 172.21.15.47:0/3698942866 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e1415f3a0 0x7f4e00004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:47.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.903+0000 7f4e12ffd640 1 --2- 172.21.15.47:0/3698942866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e00004d40 0x7f4e000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:47.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.903+0000 7f4e12ffd640 1 -- 172.21.15.47:0/3698942866 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4e140b2050 con 0x7f4e00003b70 2024-04-26T23:20:47.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.903+0000 7f4e12ffd640 1 -- 172.21.15.47:0/3698942866 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4e0000ba70 con 0x7f4e00004d40 2024-04-26T23:20:47.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.903+0000 7f4e12ffd640 1 -- 172.21.15.47:0/3698942866 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f4e0000bc20 con 0x7f4e1415f3a0 2024-04-26T23:20:47.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.903+0000 7f4e13fff640 1 --2- 172.21.15.47:0/3698942866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e00003b70 0x7f4e00003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:47.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.903+0000 7f4e137fe640 1 --2- 172.21.15.47:0/3698942866 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e1415f3a0 0x7f4e00004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:47.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.903+0000 7f4e18de4640 1 --2- 172.21.15.47:0/3698942866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e00004d40 0x7f4e000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:47.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.904+0000 7f4e13fff640 1 -- 172.21.15.47:0/3698942866 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e1415f3a0 msgr2=0x7f4e00004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:47.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.904+0000 7f4e13fff640 1 --2- 172.21.15.47:0/3698942866 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e1415f3a0 0x7f4e00004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:47.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.904+0000 7f4e13fff640 1 -- 172.21.15.47:0/3698942866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e00004d40 msgr2=0x7f4e000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:47.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.904+0000 7f4e13fff640 1 --2- 172.21.15.47:0/3698942866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e00004d40 0x7f4e000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:47.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.904+0000 7f4e13fff640 1 -- 172.21.15.47:0/3698942866 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e00003520 con 0x7f4e00003b70 2024-04-26T23:20:47.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.904+0000 7f4e18de4640 1 --2- 172.21.15.47:0/3698942866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e00004d40 0x7f4e000051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:20:47.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.904+0000 7f4e13fff640 1 --2- 172.21.15.47:0/3698942866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e00003b70 0x7f4e00003f70 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc009ac0 tx=0x7f4e00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=157d4ce1548ff561 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:47.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.904+0000 7f4e12ffd640 1 -- 172.21.15.47:0/3698942866 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc036070 con 0x7f4e00003b70 2024-04-26T23:20:47.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.905+0000 7f4e12ffd640 1 -- 172.21.15.47:0/3698942866 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dfc031400 con 0x7f4e00003b70 2024-04-26T23:20:47.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.905+0000 7f4e12ffd640 1 -- 172.21.15.47:0/3698942866 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc009690 con 0x7f4e00003b70 2024-04-26T23:20:47.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.905+0000 7f4e1a847640 1 -- 172.21.15.47:0/3698942866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e00003b70 msgr2=0x7f4e00003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:47.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.905+0000 7f4e1a847640 1 --2- 172.21.15.47:0/3698942866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e00003b70 0x7f4e00003f70 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc009ac0 tx=0x7f4e00005830 comp rx=0 tx=0).stop 2024-04-26T23:20:47.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.905+0000 7f4e18de4640 1 -- 172.21.15.47:0/3698942866 reap_dead start 2024-04-26T23:20:47.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.905+0000 7f4e1a847640 1 -- 172.21.15.47:0/3698942866 shutdown_connections 2024-04-26T23:20:47.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.905+0000 7f4e1a847640 1 -- 172.21.15.47:0/3698942866 >> 172.21.15.47:0/3698942866 conn(0x7f4e140d4f90 msgr2=0x7f4e140d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:47.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.905+0000 7f4e1a847640 1 -- 172.21.15.47:0/3698942866 shutdown_connections 2024-04-26T23:20:47.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.905+0000 7f4e1a847640 1 -- 172.21.15.47:0/3698942866 wait complete. 2024-04-26T23:20:47.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.906+0000 7f4e1a847640 1 Processor -- start 2024-04-26T23:20:47.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.906+0000 7f4e1a847640 1 -- start start 2024-04-26T23:20:47.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.907+0000 7f4e1a847640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e00003b70 0x7f4e140cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:47.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.907+0000 7f4e1a847640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e140b5860 0x7f4e140b9d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:47.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.907+0000 7f4e1a847640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e140b5ce0 0x7f4e140ba290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:47.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.907+0000 7f4e1a847640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4e140d0870 con 0x7f4e140b5ce0 2024-04-26T23:20:47.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.907+0000 7f4e1a847640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4e140d06f0 con 0x7f4e00003b70 2024-04-26T23:20:47.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.907+0000 7f4e1a847640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f4e140d09f0 con 0x7f4e140b5860 2024-04-26T23:20:47.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.907+0000 7f4e18de4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e140b5ce0 0x7f4e140ba290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:47.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.907+0000 7f4e18de4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e140b5ce0 0x7f4e140ba290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39718/0 (socket says 172.21.15.47:39718) 2024-04-26T23:20:47.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.907+0000 7f4e18de4640 1 -- 172.21.15.47:0/4247047823 learned_addr learned my addr 172.21.15.47:0/4247047823 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:47.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.907+0000 7f4e13fff640 1 --2- 172.21.15.47:0/4247047823 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e00003b70 0x7f4e140cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:47.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.907+0000 7f4e137fe640 1 --2- 172.21.15.47:0/4247047823 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e140b5860 0x7f4e140b9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:47.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.907+0000 7f4e18de4640 1 -- 172.21.15.47:0/4247047823 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e140b5860 msgr2=0x7f4e140b9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:47.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.908+0000 7f4e18de4640 1 --2- 172.21.15.47:0/4247047823 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e140b5860 0x7f4e140b9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:47.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.908+0000 7f4e18de4640 1 -- 172.21.15.47:0/4247047823 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e00003b70 msgr2=0x7f4e140cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:47.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.908+0000 7f4e18de4640 1 --2- 172.21.15.47:0/4247047823 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e00003b70 0x7f4e140cafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:47.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.908+0000 7f4e18de4640 1 -- 172.21.15.47:0/4247047823 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e141624c0 con 0x7f4e140b5ce0 2024-04-26T23:20:47.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.908+0000 7f4e137fe640 1 --2- 172.21.15.47:0/4247047823 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e140b5860 0x7f4e140b9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:47.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.908+0000 7f4e13fff640 1 --2- 172.21.15.47:0/4247047823 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e00003b70 0x7f4e140cafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:47.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.908+0000 7f4e18de4640 1 --2- 172.21.15.47:0/4247047823 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e140b5ce0 0x7f4e140ba290 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c001330 tx=0x7f4e0c00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:47.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.908+0000 7f4e10ff9640 1 -- 172.21.15.47:0/4247047823 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c00ee50 con 0x7f4e140b5ce0 2024-04-26T23:20:47.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.908+0000 7f4e10ff9640 1 -- 172.21.15.47:0/4247047823 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e0c003b80 con 0x7f4e140b5ce0 2024-04-26T23:20:47.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.908+0000 7f4e1a847640 1 -- 172.21.15.47:0/4247047823 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e141627a0 con 0x7f4e140b5ce0 2024-04-26T23:20:47.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.908+0000 7f4e10ff9640 1 -- 172.21.15.47:0/4247047823 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c003d70 con 0x7f4e140b5ce0 2024-04-26T23:20:47.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.908+0000 7f4e1a847640 1 -- 172.21.15.47:0/4247047823 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e140c8c60 con 0x7f4e140b5ce0 2024-04-26T23:20:47.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.909+0000 7f4e1a847640 1 -- 172.21.15.47:0/4247047823 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e140694f0 con 0x7f4e140b5ce0 2024-04-26T23:20:47.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.909+0000 7f4e10ff9640 1 -- 172.21.15.47:0/4247047823 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4e0c01a020 con 0x7f4e140b5ce0 2024-04-26T23:20:47.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.910+0000 7f4e10ff9640 1 --2- 172.21.15.47:0/4247047823 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4df406c920 0x7f4df406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:47.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.910+0000 7f4e10ff9640 1 -- 172.21.15.47:0/4247047823 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5335+0+0 (secure 0 0 0) 0x7f4e0c08af20 con 0x7f4e140b5ce0 2024-04-26T23:20:47.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.910+0000 7f4e13fff640 1 --2- 172.21.15.47:0/4247047823 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4df406c920 0x7f4df406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:47.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.911+0000 7f4e13fff640 1 --2- 172.21.15.47:0/4247047823 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4df406c920 0x7f4df406ede0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc033000 tx=0x7f4dfc008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:47.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:47.914+0000 7f4e10ff9640 1 -- 172.21.15.47:0/4247047823 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4e0c021020 con 0x7f4e140b5ce0 2024-04-26T23:20:48.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.095+0000 7f4e1a847640 1 -- 172.21.15.47:0/4247047823 --> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4e140b6860 con 0x7f4df406c920 2024-04-26T23:20:48.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.096+0000 7f4e10ff9640 1 -- 172.21.15.47:0/4247047823 <== mgr.4099 v2:172.21.15.203:6832/992162732 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13403 (secure 0 0 0) 0x7f4e140b6860 con 0x7f4df406c920 2024-04-26T23:20:48.098 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:20:48.099 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-26T23:20:48.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.098+0000 7f4e1a847640 1 -- 172.21.15.47:0/4247047823 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4df406c920 msgr2=0x7f4df406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:48.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.098+0000 7f4e1a847640 1 --2- 172.21.15.47:0/4247047823 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4df406c920 0x7f4df406ede0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc033000 tx=0x7f4dfc008030 comp rx=0 tx=0).stop 2024-04-26T23:20:48.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.098+0000 7f4e1a847640 1 -- 172.21.15.47:0/4247047823 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e140b5ce0 msgr2=0x7f4e140ba290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:48.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.098+0000 7f4e1a847640 1 --2- 172.21.15.47:0/4247047823 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e140b5ce0 0x7f4e140ba290 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c001330 tx=0x7f4e0c00ec00 comp rx=0 tx=0).stop 2024-04-26T23:20:48.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.098+0000 7f4e1a847640 1 -- 172.21.15.47:0/4247047823 shutdown_connections 2024-04-26T23:20:48.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.098+0000 7f4e1a847640 1 --2- 172.21.15.47:0/4247047823 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4df406c920 0x7f4df406ede0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:48.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.098+0000 7f4e1a847640 1 --2- 172.21.15.47:0/4247047823 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e140b5ce0 0x7f4e140ba290 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:48.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.098+0000 7f4e1a847640 1 --2- 172.21.15.47:0/4247047823 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e140b5860 0x7f4e140b9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:48.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.098+0000 7f4e1a847640 1 --2- 172.21.15.47:0/4247047823 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e00003b70 0x7f4e140cafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:48.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.098+0000 7f4e1a847640 1 -- 172.21.15.47:0/4247047823 >> 172.21.15.47:0/4247047823 conn(0x7f4e140d4f90 msgr2=0x7f4e1406da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:48.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.098+0000 7f4e1a847640 1 -- 172.21.15.47:0/4247047823 shutdown_connections 2024-04-26T23:20:48.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.098+0000 7f4e1a847640 1 -- 172.21.15.47:0/4247047823 wait complete. 2024-04-26T23:20:48.111 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":125,"stamp":"2024-04-26T23:20:46.961092+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239140,"kb_used_data":6340,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735580,"statfs":{"total":773094113280,"available":772849233920,"internally_reserved":0,"allocated":6492160,"data_stored":3577713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17361,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-67,"num_write_kb":-40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001480"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":234,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-26T23:20:42.324231+0000","last_change":"2024-04-26T23:18:18.862631+0000","last_active":"2024-04-26T23:20:42.324231+0000","last_peered":"2024-04-26T23:20:42.324231+0000","last_clean":"2024-04-26T23:20:42.324231+0000","last_became_active":"2024-04-26T23:18:18.861536+0000","last_became_peered":"2024-04-26T23:18:18.861536+0000","last_unstale":"2024-04-26T23:20:42.324231+0000","last_undegraded":"2024-04-26T23:20:42.324231+0000","last_fullsized":"2024-04-26T23:20:42.324231+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:17.808736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:17.808736+0000","last_clean_scrub_stamp":"2024-04-26T23:18:17.808736+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:59:26.359282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542176,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28248,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343592,"statfs":{"total":96636764160,"available":96607838208,"internally_reserved":0,"allocated":1257472,"data_stored":887666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542178,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27660,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344180,"statfs":{"total":96636764160,"available":96608440320,"internally_reserved":0,"allocated":655360,"data_stored":297138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542177,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":663552,"data_stored":297790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542177,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27688,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344152,"statfs":{"total":96636764160,"available":96608411648,"internally_reserved":0,"allocated":684032,"data_stored":315986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574880,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31820,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340020,"statfs":{"total":96636764160,"available":96604180480,"internally_reserved":0,"allocated":651264,"data_stored":290721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31816,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340024,"statfs":{"total":96636764160,"available":96604184576,"internally_reserved":0,"allocated":651264,"data_stored":290721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574879,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31824,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340016,"statfs":{"total":96636764160,"available":96604176384,"internally_reserved":0,"allocated":659456,"data_stored":297228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574880,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32416,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339424,"statfs":{"total":96636764160,"available":96603570176,"internally_reserved":0,"allocated":1269760,"data_stored":900463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:20:48.111 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-26T23:20:48.111 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T23:20:48.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.210+0000 7f4db8304640 1 Processor -- start 2024-04-26T23:20:48.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.210+0000 7f4db8304640 1 -- start start 2024-04-26T23:20:48.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.211+0000 7f4db8304640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4db00d1b30 0x7f4db00d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:48.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.211+0000 7f4db8304640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4db00d10d0 con 0x7f4db0162810 2024-04-26T23:20:48.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.211+0000 7f4db8304640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4db00d1370 con 0x7f4db00b2340 2024-04-26T23:20:48.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.211+0000 7f4db8304640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f4db00d0750 con 0x7f4db00d1b30 2024-04-26T23:20:48.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.211+0000 7f4db68a1640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4db00d1b30 0x7f4db00d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:48.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.211+0000 7f4db60a0640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4db0162810 0x7f4db0070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47224/0 (socket says 172.21.15.47:47224) 2024-04-26T23:20:48.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.211+0000 7f4db60a0640 1 -- 172.21.15.47:0/615416944 learned_addr learned my addr 172.21.15.47:0/615416944 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:48.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.211+0000 7f4db68a1640 1 -- 172.21.15.47:0/615416944 >> v1:172.21.15.203:6789/0 conn(0x7f4db00b2340 legacy=0x7f4db00cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:48.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.211+0000 7f4db68a1640 1 -- 172.21.15.47:0/615416944 >> v1:172.21.15.47:6789/0 conn(0x7f4db0162810 legacy=0x7f4db0070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:48.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.211+0000 7f4db68a1640 1 -- 172.21.15.47:0/615416944 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4db00d01b0 con 0x7f4db00d1b30 2024-04-26T23:20:48.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.211+0000 7f4db509e640 1 -- 172.21.15.47:0/615416944 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 960391019 0 0) 0x7f4db00d10d0 con 0x7f4db0162810 2024-04-26T23:20:48.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.212+0000 7f4db68a1640 1 --2- 172.21.15.47:0/615416944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4db00d1b30 0x7f4db00d1fc0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f4da8009d90 tx=0x7f4da802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cbfbe686ddde971 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:48.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.212+0000 7f4db509e640 1 -- 172.21.15.47:0/615416944 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da8007740 con 0x7f4db00d1b30 2024-04-26T23:20:48.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.212+0000 7f4db509e640 1 -- 172.21.15.47:0/615416944 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4da80078e0 con 0x7f4db00d1b30 2024-04-26T23:20:48.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.212+0000 7f4db509e640 1 -- 172.21.15.47:0/615416944 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da8007cb0 con 0x7f4db00d1b30 2024-04-26T23:20:48.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.213+0000 7f4db8304640 1 -- 172.21.15.47:0/615416944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4db00d1b30 msgr2=0x7f4db00d1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:48.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.213+0000 7f4db8304640 1 --2- 172.21.15.47:0/615416944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4db00d1b30 0x7f4db00d1fc0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f4da8009d90 tx=0x7f4da802a6f0 comp rx=0 tx=0).stop 2024-04-26T23:20:48.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.213+0000 7f4db8304640 1 -- 172.21.15.47:0/615416944 shutdown_connections 2024-04-26T23:20:48.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.213+0000 7f4db8304640 1 --2- 172.21.15.47:0/615416944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4db00d1b30 0x7f4db00d1fc0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:48.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.213+0000 7f4db8304640 1 -- 172.21.15.47:0/615416944 >> 172.21.15.47:0/615416944 conn(0x7f4db00d2da0 msgr2=0x7f4db00d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:48.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.213+0000 7f4db8304640 1 -- 172.21.15.47:0/615416944 shutdown_connections 2024-04-26T23:20:48.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.213+0000 7f4db8304640 1 -- 172.21.15.47:0/615416944 wait complete. 2024-04-26T23:20:48.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.214+0000 7f4db8304640 1 Processor -- start 2024-04-26T23:20:48.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.214+0000 7f4db8304640 1 -- start start 2024-04-26T23:20:48.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.215+0000 7f4db8304640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4db00b2340 0x7f4db01f52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:48.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.215+0000 7f4db8304640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4db00d1b30 0x7f4db00c72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:48.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.215+0000 7f4db8304640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4db0162810 0x7f4db01f7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:48.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.215+0000 7f4db8304640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4db00cdb00 con 0x7f4db0162810 2024-04-26T23:20:48.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.215+0000 7f4db8304640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4db00d16a0 con 0x7f4db00d1b30 2024-04-26T23:20:48.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.215+0000 7f4db8304640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f4db0070b50 con 0x7f4db00b2340 2024-04-26T23:20:48.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.215+0000 7f4db68a1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4db0162810 0x7f4db01f7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:48.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.215+0000 7f4db68a1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4db0162810 0x7f4db01f7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39730/0 (socket says 172.21.15.47:39730) 2024-04-26T23:20:48.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.215+0000 7f4db68a1640 1 -- 172.21.15.47:0/3220790391 learned_addr learned my addr 172.21.15.47:0/3220790391 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:48.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.215+0000 7f4db60a0640 1 --2- 172.21.15.47:0/3220790391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4db00b2340 0x7f4db01f52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:48.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.215+0000 7f4db589f640 1 --2- 172.21.15.47:0/3220790391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4db00d1b30 0x7f4db00c72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:48.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.215+0000 7f4db68a1640 1 -- 172.21.15.47:0/3220790391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4db00b2340 msgr2=0x7f4db01f52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:48.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.215+0000 7f4db68a1640 1 --2- 172.21.15.47:0/3220790391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4db00b2340 0x7f4db01f52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:48.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.215+0000 7f4db68a1640 1 -- 172.21.15.47:0/3220790391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4db00d1b30 msgr2=0x7f4db00c72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:48.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.215+0000 7f4db68a1640 1 --2- 172.21.15.47:0/3220790391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4db00d1b30 0x7f4db00c72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:48.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.215+0000 7f4db68a1640 1 -- 172.21.15.47:0/3220790391 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4db00c7800 con 0x7f4db0162810 2024-04-26T23:20:48.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.215+0000 7f4db589f640 1 --2- 172.21.15.47:0/3220790391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4db00d1b30 0x7f4db00c72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:20:48.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.215+0000 7f4db68a1640 1 --2- 172.21.15.47:0/3220790391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4db0162810 0x7f4db01f7fa0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f4da8009eb0 tx=0x7f4da8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:48.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.216+0000 7f4da6ffd640 1 -- 172.21.15.47:0/3220790391 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da8040070 con 0x7f4db0162810 2024-04-26T23:20:48.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.216+0000 7f4da6ffd640 1 -- 172.21.15.47:0/3220790391 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4da8031c90 con 0x7f4db0162810 2024-04-26T23:20:48.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.216+0000 7f4db8304640 1 -- 172.21.15.47:0/3220790391 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4db00c7e90 con 0x7f4db0162810 2024-04-26T23:20:48.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.216+0000 7f4da6ffd640 1 -- 172.21.15.47:0/3220790391 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da803c640 con 0x7f4db0162810 2024-04-26T23:20:48.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.216+0000 7f4db8304640 1 -- 172.21.15.47:0/3220790391 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4db00c8450 con 0x7f4db0162810 2024-04-26T23:20:48.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.217+0000 7f4db8304640 1 -- 172.21.15.47:0/3220790391 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4db00694f0 con 0x7f4db0162810 2024-04-26T23:20:48.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.217+0000 7f4da6ffd640 1 -- 172.21.15.47:0/3220790391 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4da8039040 con 0x7f4db0162810 2024-04-26T23:20:48.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.218+0000 7f4da6ffd640 1 --2- 172.21.15.47:0/3220790391 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4d9406c8f0 0x7f4d9406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:48.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.218+0000 7f4da6ffd640 1 -- 172.21.15.47:0/3220790391 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5335+0+0 (secure 0 0 0) 0x7f4da80ad020 con 0x7f4db0162810 2024-04-26T23:20:48.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.218+0000 7f4db60a0640 1 --2- 172.21.15.47:0/3220790391 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4d9406c8f0 0x7f4d9406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:48.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.218+0000 7f4db60a0640 1 --2- 172.21.15.47:0/3220790391 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4d9406c8f0 0x7f4d9406edb0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4dac000f60 tx=0x7f4dac007d00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:48.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.222+0000 7f4da6ffd640 1 -- 172.21.15.47:0/3220790391 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4db00c7800 con 0x7f4db0162810 2024-04-26T23:20:48.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.402+0000 7f4db8304640 1 -- 172.21.15.47:0/3220790391 --> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4db00b1ce0 con 0x7f4d9406c8f0 2024-04-26T23:20:48.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.404+0000 7f4da6ffd640 1 -- 172.21.15.47:0/3220790391 <== mgr.4099 v2:172.21.15.203:6832/992162732 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13403 (secure 0 0 0) 0x7f4db00b1ce0 con 0x7f4d9406c8f0 2024-04-26T23:20:48.405 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:20:48.406 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-26T23:20:48.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.405+0000 7f4db8304640 1 -- 172.21.15.47:0/3220790391 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4d9406c8f0 msgr2=0x7f4d9406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:48.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.405+0000 7f4db8304640 1 --2- 172.21.15.47:0/3220790391 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4d9406c8f0 0x7f4d9406edb0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4dac000f60 tx=0x7f4dac007d00 comp rx=0 tx=0).stop 2024-04-26T23:20:48.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.405+0000 7f4db8304640 1 -- 172.21.15.47:0/3220790391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4db0162810 msgr2=0x7f4db01f7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:48.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.405+0000 7f4db8304640 1 --2- 172.21.15.47:0/3220790391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4db0162810 0x7f4db01f7fa0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f4da8009eb0 tx=0x7f4da8004a40 comp rx=0 tx=0).stop 2024-04-26T23:20:48.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.405+0000 7f4db8304640 1 -- 172.21.15.47:0/3220790391 shutdown_connections 2024-04-26T23:20:48.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.405+0000 7f4db8304640 1 --2- 172.21.15.47:0/3220790391 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f4d9406c8f0 0x7f4d9406edb0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:48.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.405+0000 7f4db8304640 1 --2- 172.21.15.47:0/3220790391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4db0162810 0x7f4db01f7fa0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:48.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.405+0000 7f4db8304640 1 --2- 172.21.15.47:0/3220790391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4db00d1b30 0x7f4db00c72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:48.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.405+0000 7f4db8304640 1 --2- 172.21.15.47:0/3220790391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4db00b2340 0x7f4db01f52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:48.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.405+0000 7f4db8304640 1 -- 172.21.15.47:0/3220790391 >> 172.21.15.47:0/3220790391 conn(0x7f4db00d2da0 msgr2=0x7f4db00b2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:48.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.405+0000 7f4db8304640 1 -- 172.21.15.47:0/3220790391 shutdown_connections 2024-04-26T23:20:48.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.405+0000 7f4db8304640 1 -- 172.21.15.47:0/3220790391 wait complete. 2024-04-26T23:20:48.418 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":125,"stamp":"2024-04-26T23:20:46.961092+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239140,"kb_used_data":6340,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735580,"statfs":{"total":773094113280,"available":772849233920,"internally_reserved":0,"allocated":6492160,"data_stored":3577713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17361,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-67,"num_write_kb":-40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001480"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":234,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-26T23:20:42.324231+0000","last_change":"2024-04-26T23:18:18.862631+0000","last_active":"2024-04-26T23:20:42.324231+0000","last_peered":"2024-04-26T23:20:42.324231+0000","last_clean":"2024-04-26T23:20:42.324231+0000","last_became_active":"2024-04-26T23:18:18.861536+0000","last_became_peered":"2024-04-26T23:18:18.861536+0000","last_unstale":"2024-04-26T23:20:42.324231+0000","last_undegraded":"2024-04-26T23:20:42.324231+0000","last_fullsized":"2024-04-26T23:20:42.324231+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:17.808736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:17.808736+0000","last_clean_scrub_stamp":"2024-04-26T23:18:17.808736+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:59:26.359282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542176,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28248,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343592,"statfs":{"total":96636764160,"available":96607838208,"internally_reserved":0,"allocated":1257472,"data_stored":887666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542178,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27660,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344180,"statfs":{"total":96636764160,"available":96608440320,"internally_reserved":0,"allocated":655360,"data_stored":297138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542177,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":663552,"data_stored":297790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542177,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27688,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344152,"statfs":{"total":96636764160,"available":96608411648,"internally_reserved":0,"allocated":684032,"data_stored":315986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574880,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31820,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340020,"statfs":{"total":96636764160,"available":96604180480,"internally_reserved":0,"allocated":651264,"data_stored":290721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31816,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340024,"statfs":{"total":96636764160,"available":96604184576,"internally_reserved":0,"allocated":651264,"data_stored":290721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574879,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31824,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340016,"statfs":{"total":96636764160,"available":96604176384,"internally_reserved":0,"allocated":659456,"data_stored":297228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574880,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32416,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339424,"statfs":{"total":96636764160,"available":96603570176,"internally_reserved":0,"allocated":1269760,"data_stored":900463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:20:48.418 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:20:48.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.517+0000 7f872c5b4640 1 Processor -- start 2024-04-26T23:20:48.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.518+0000 7f872c5b4640 1 -- start start 2024-04-26T23:20:48.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.518+0000 7f872c5b4640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f872415f3a0 0x7f87240d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:48.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.518+0000 7f872c5b4640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f87240d2330 0x7f87240b2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:48.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.518+0000 7f872c5b4640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87240d0e20 con 0x7f87240d1770 2024-04-26T23:20:48.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.518+0000 7f872c5b4640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f87240b1d70 con 0x7f87240d2330 2024-04-26T23:20:48.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.518+0000 7f872c5b4640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f87240b2050 con 0x7f872415f3a0 2024-04-26T23:20:48.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.519+0000 7f872ab51640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f87240d2330 0x7f87240b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:48.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.519+0000 7f872ab51640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f87240d2330 0x7f87240b2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39742/0 (socket says 172.21.15.47:39742) 2024-04-26T23:20:48.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.519+0000 7f8729b4f640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f87240d1770 0x7f87240d1c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47234/0 (socket says 172.21.15.47:47234) 2024-04-26T23:20:48.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.519+0000 7f872ab51640 1 -- 172.21.15.47:0/2638813755 learned_addr learned my addr 172.21.15.47:0/2638813755 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:48.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.519+0000 7f872a350640 1 --2- 172.21.15.47:0/2638813755 >> v2:172.21.15.203:3300/0 conn(0x7f872415f3a0 0x7f87240d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:48.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.519+0000 7f872ab51640 1 -- 172.21.15.47:0/2638813755 >> v2:172.21.15.203:3300/0 conn(0x7f872415f3a0 msgr2=0x7f87240d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:48.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.519+0000 7f872ab51640 1 --2- 172.21.15.47:0/2638813755 >> v2:172.21.15.203:3300/0 conn(0x7f872415f3a0 0x7f87240d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:48.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.519+0000 7f872ab51640 1 -- 172.21.15.47:0/2638813755 >> v1:172.21.15.47:6789/0 conn(0x7f87240d1770 legacy=0x7f87240d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:48.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.519+0000 7f872934e640 1 -- 172.21.15.47:0/2638813755 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 761995469 0 0) 0x7f87240d0e20 con 0x7f87240d1770 2024-04-26T23:20:48.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.519+0000 7f872ab51640 1 -- 172.21.15.47:0/2638813755 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87240b2330 con 0x7f87240d2330 2024-04-26T23:20:48.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.520+0000 7f872ab51640 1 --2- 172.21.15.47:0/2638813755 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87240d2330 0x7f87240b2850 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f8724073b50 tx=0x7f871c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb4ae7a3f833b54e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:48.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.520+0000 7f872934e640 1 -- 172.21.15.47:0/2638813755 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871c036070 con 0x7f87240d2330 2024-04-26T23:20:48.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.520+0000 7f872934e640 1 -- 172.21.15.47:0/2638813755 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f871c031600 con 0x7f87240d2330 2024-04-26T23:20:48.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.520+0000 7f872934e640 1 -- 172.21.15.47:0/2638813755 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871c0319d0 con 0x7f87240d2330 2024-04-26T23:20:48.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.520+0000 7f872c5b4640 1 -- 172.21.15.47:0/2638813755 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87240d2330 msgr2=0x7f87240b2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:48.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.520+0000 7f872c5b4640 1 --2- 172.21.15.47:0/2638813755 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87240d2330 0x7f87240b2850 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f8724073b50 tx=0x7f871c02a730 comp rx=0 tx=0).stop 2024-04-26T23:20:48.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.520+0000 7f872c5b4640 1 -- 172.21.15.47:0/2638813755 shutdown_connections 2024-04-26T23:20:48.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.520+0000 7f872c5b4640 1 --2- 172.21.15.47:0/2638813755 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87240d2330 0x7f87240b2850 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:48.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.520+0000 7f872c5b4640 1 --2- 172.21.15.47:0/2638813755 >> v2:172.21.15.203:3300/0 conn(0x7f872415f3a0 0x7f87240d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:48.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.520+0000 7f872c5b4640 1 -- 172.21.15.47:0/2638813755 >> 172.21.15.47:0/2638813755 conn(0x7f87240d4f90 msgr2=0x7f87240d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:48.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.520+0000 7f872c5b4640 1 -- 172.21.15.47:0/2638813755 shutdown_connections 2024-04-26T23:20:48.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.520+0000 7f872c5b4640 1 -- 172.21.15.47:0/2638813755 wait complete. 2024-04-26T23:20:48.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.522+0000 7f872c5b4640 1 Processor -- start 2024-04-26T23:20:48.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.522+0000 7f872c5b4640 1 -- start start 2024-04-26T23:20:48.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.522+0000 7f872c5b4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f87240d1770 0x7f8724153460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:48.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.522+0000 7f872c5b4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87240d2330 0x7f87240b8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:48.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.522+0000 7f872c5b4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f872415f3a0 0x7f87241604e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:48.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.522+0000 7f872c5b4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f87240b2f10 con 0x7f872415f3a0 2024-04-26T23:20:48.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.522+0000 7f872c5b4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f87240b2d90 con 0x7f87240d2330 2024-04-26T23:20:48.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.522+0000 7f872c5b4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f87240b3090 con 0x7f87240d1770 2024-04-26T23:20:48.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.522+0000 7f872ab51640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f872415f3a0 0x7f87241604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:48.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.522+0000 7f872ab51640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f872415f3a0 0x7f87241604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39758/0 (socket says 172.21.15.47:39758) 2024-04-26T23:20:48.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.522+0000 7f872ab51640 1 -- 172.21.15.47:0/1799250750 learned_addr learned my addr 172.21.15.47:0/1799250750 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:48.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.523+0000 7f872a350640 1 --2- 172.21.15.47:0/1799250750 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f87240d1770 0x7f8724153460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:48.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.523+0000 7f8729b4f640 1 --2- 172.21.15.47:0/1799250750 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87240d2330 0x7f87240b8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:48.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.523+0000 7f872ab51640 1 -- 172.21.15.47:0/1799250750 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f87240d1770 msgr2=0x7f8724153460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:48.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.523+0000 7f872ab51640 1 --2- 172.21.15.47:0/1799250750 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f87240d1770 0x7f8724153460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:48.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.523+0000 7f872ab51640 1 -- 172.21.15.47:0/1799250750 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87240d2330 msgr2=0x7f87240b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:48.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.523+0000 7f872ab51640 1 --2- 172.21.15.47:0/1799250750 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87240d2330 0x7f87240b8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:48.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.523+0000 7f872ab51640 1 -- 172.21.15.47:0/1799250750 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87240b8e10 con 0x7f872415f3a0 2024-04-26T23:20:48.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.523+0000 7f872a350640 1 --2- 172.21.15.47:0/1799250750 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f87240d1770 0x7f8724153460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:48.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.523+0000 7f8729b4f640 1 --2- 172.21.15.47:0/1799250750 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87240d2330 0x7f87240b8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:48.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.523+0000 7f872ab51640 1 --2- 172.21.15.47:0/1799250750 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f872415f3a0 0x7f87241604e0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f871c034040 tx=0x7f871c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:48.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.523+0000 7f871affd640 1 -- 172.21.15.47:0/1799250750 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871c041020 con 0x7f872415f3a0 2024-04-26T23:20:48.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.523+0000 7f871affd640 1 -- 172.21.15.47:0/1799250750 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f871c037040 con 0x7f872415f3a0 2024-04-26T23:20:48.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.523+0000 7f872c5b4640 1 -- 172.21.15.47:0/1799250750 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87240b90f0 con 0x7f872415f3a0 2024-04-26T23:20:48.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.523+0000 7f871affd640 1 -- 172.21.15.47:0/1799250750 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871c036050 con 0x7f872415f3a0 2024-04-26T23:20:48.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.523+0000 7f872c5b4640 1 -- 172.21.15.47:0/1799250750 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87240b96b0 con 0x7f872415f3a0 2024-04-26T23:20:48.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.524+0000 7f872c5b4640 1 -- 172.21.15.47:0/1799250750 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87240694f0 con 0x7f872415f3a0 2024-04-26T23:20:48.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.525+0000 7f871affd640 1 -- 172.21.15.47:0/1799250750 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f871c007bf0 con 0x7f872415f3a0 2024-04-26T23:20:48.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.525+0000 7f871affd640 1 --2- 172.21.15.47:0/1799250750 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f870806c8f0 0x7f870806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:48.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.525+0000 7f871affd640 1 -- 172.21.15.47:0/1799250750 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5335+0+0 (secure 0 0 0) 0x7f871c0ac2d0 con 0x7f872415f3a0 2024-04-26T23:20:48.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.525+0000 7f872a350640 1 --2- 172.21.15.47:0/1799250750 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f870806c8f0 0x7f870806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:48.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.526+0000 7f872a350640 1 --2- 172.21.15.47:0/1799250750 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f870806c8f0 0x7f870806edb0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f872000b820 tx=0x7f8720008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:48.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.529+0000 7f871affd640 1 -- 172.21.15.47:0/1799250750 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f871c04b060 con 0x7f872415f3a0 2024-04-26T23:20:48.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.716+0000 7f872c5b4640 1 -- 172.21.15.47:0/1799250750 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f87240b6860 con 0x7f872415f3a0 2024-04-26T23:20:48.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.717+0000 7f871affd640 1 -- 172.21.15.47:0/1799250750 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) ==== 74+0+13259 (secure 0 0 0) 0x7f871c07a880 con 0x7f872415f3a0 2024-04-26T23:20:48.719 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:20:48.719 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":57,"fsid":"399006d3-4db1-43e6-ba81-a79a276eabfd","created":"2024-04-26T23:18:06.731211+0000","modified":"2024-04-26T23:20:45.426030+0000","last_up_change":"2024-04-26T23:18:17.808736+0000","last_in_change":"2024-04-26T23:18:08.945859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:18:16.937492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ee091567-fea0-4a14-bf4e-0e16c20bb2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6819","nonce":326520779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6821","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6823","nonce":326520779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6829","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6831","nonce":326520779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6825","nonce":326520779},{"type":"v1","addr":"172.21.15.47:6827","nonce":326520779}]},"public_addr":"172.21.15.47:6819/326520779","cluster_addr":"172.21.15.47:6823/326520779","heartbeat_back_addr":"172.21.15.47:6831/326520779","heartbeat_front_addr":"172.21.15.47:6827/326520779","state":["exists","up"]},{"osd":1,"uuid":"c14440a0-bba5-4931-a6db-b736c08e869b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6809","nonce":1912610931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6811","nonce":1912610931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6815","nonce":1912610931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":1912610931},{"type":"v1","addr":"172.21.15.47:6813","nonce":1912610931}]},"public_addr":"172.21.15.47:6809/1912610931","cluster_addr":"172.21.15.47:6811/1912610931","heartbeat_back_addr":"172.21.15.47:6815/1912610931","heartbeat_front_addr":"172.21.15.47:6813/1912610931","state":["exists","up"]},{"osd":2,"uuid":"3940f8de-c9b2-4cd3-a911-9c301729eb30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6801","nonce":2569869305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6803","nonce":2569869305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6807","nonce":2569869305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":2569869305},{"type":"v1","addr":"172.21.15.47:6805","nonce":2569869305}]},"public_addr":"172.21.15.47:6801/2569869305","cluster_addr":"172.21.15.47:6803/2569869305","heartbeat_back_addr":"172.21.15.47:6807/2569869305","heartbeat_front_addr":"172.21.15.47:6805/2569869305","state":["exists","up"]},{"osd":3,"uuid":"1286a67e-8958-426a-8c2e-bc85498c619c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6818","nonce":2791411683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6822","nonce":2791411683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6830","nonce":2791411683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2791411683},{"type":"v1","addr":"172.21.15.47:6826","nonce":2791411683}]},"public_addr":"172.21.15.47:6818/2791411683","cluster_addr":"172.21.15.47:6822/2791411683","heartbeat_back_addr":"172.21.15.47:6830/2791411683","heartbeat_front_addr":"172.21.15.47:6826/2791411683","state":["exists","up"]},{"osd":4,"uuid":"d8ee2c20-e5b7-46c2-9999-b234070126cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6809","nonce":441313104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6811","nonce":441313104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6815","nonce":441313104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":441313104},{"type":"v1","addr":"172.21.15.203:6813","nonce":441313104}]},"public_addr":"172.21.15.203:6809/441313104","cluster_addr":"172.21.15.203:6811/441313104","heartbeat_back_addr":"172.21.15.203:6815/441313104","heartbeat_front_addr":"172.21.15.203:6813/441313104","state":["exists","up"]},{"osd":5,"uuid":"b453a8f0-f324-4840-9ae0-d67bdadb9f8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6801","nonce":56838444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6803","nonce":56838444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6807","nonce":56838444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":56838444},{"type":"v1","addr":"172.21.15.203:6805","nonce":56838444}]},"public_addr":"172.21.15.203:6801/56838444","cluster_addr":"172.21.15.203:6803/56838444","heartbeat_back_addr":"172.21.15.203:6807/56838444","heartbeat_front_addr":"172.21.15.203:6805/56838444","state":["exists","up"]},{"osd":6,"uuid":"a6d31951-2d4b-4e8d-a510-6283b01dc158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6825","nonce":2738517080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6827","nonce":2738517080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6831","nonce":2738517080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2738517080},{"type":"v1","addr":"172.21.15.203:6829","nonce":2738517080}]},"public_addr":"172.21.15.203:6825/2738517080","cluster_addr":"172.21.15.203:6827/2738517080","heartbeat_back_addr":"172.21.15.203:6831/2738517080","heartbeat_front_addr":"172.21.15.203:6829/2738517080","state":["exists","up"]},{"osd":7,"uuid":"7119b8c0-fe96-402d-9e75-7b509d9d44d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6817","nonce":1315565058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6819","nonce":1315565058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6823","nonce":1315565058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1315565058},{"type":"v1","addr":"172.21.15.203:6821","nonce":1315565058}]},"public_addr":"172.21.15.203:6817/1315565058","cluster_addr":"172.21.15.203:6819/1315565058","heartbeat_back_addr":"172.21.15.203:6823/1315565058","heartbeat_front_addr":"172.21.15.203:6821/1315565058","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.121921+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.137494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.084814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.111593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.072430+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.125066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.097867+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:18:14.057061+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:6836/1168022146":"2024-04-27T23:20:42.319267+0000","172.21.15.203:6834/1168022146":"2024-04-27T23:20:42.319267+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:20:48.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.719+0000 7f872c5b4640 1 -- 172.21.15.47:0/1799250750 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f870806c8f0 msgr2=0x7f870806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:48.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.719+0000 7f872c5b4640 1 --2- 172.21.15.47:0/1799250750 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f870806c8f0 0x7f870806edb0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f872000b820 tx=0x7f8720008040 comp rx=0 tx=0).stop 2024-04-26T23:20:48.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.719+0000 7f872c5b4640 1 -- 172.21.15.47:0/1799250750 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f872415f3a0 msgr2=0x7f87241604e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:48.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.719+0000 7f872c5b4640 1 --2- 172.21.15.47:0/1799250750 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f872415f3a0 0x7f87241604e0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f871c034040 tx=0x7f871c004a40 comp rx=0 tx=0).stop 2024-04-26T23:20:48.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.720+0000 7f872c5b4640 1 -- 172.21.15.47:0/1799250750 shutdown_connections 2024-04-26T23:20:48.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.720+0000 7f872c5b4640 1 --2- 172.21.15.47:0/1799250750 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f870806c8f0 0x7f870806edb0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:48.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.720+0000 7f872c5b4640 1 --2- 172.21.15.47:0/1799250750 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f872415f3a0 0x7f87241604e0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:48.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.720+0000 7f872c5b4640 1 --2- 172.21.15.47:0/1799250750 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87240d2330 0x7f87240b8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:48.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.720+0000 7f872c5b4640 1 --2- 172.21.15.47:0/1799250750 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f87240d1770 0x7f8724153460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:48.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.720+0000 7f872c5b4640 1 -- 172.21.15.47:0/1799250750 >> 172.21.15.47:0/1799250750 conn(0x7f87240d4f90 msgr2=0x7f87240b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:48.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.720+0000 7f872c5b4640 1 -- 172.21.15.47:0/1799250750 shutdown_connections 2024-04-26T23:20:48.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:48.720+0000 7f872c5b4640 1 -- 172.21.15.47:0/1799250750 wait complete. 2024-04-26T23:20:49.734 INFO:tasks.ceph:Scrubbing osd.0 2024-04-26T23:20:49.734 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-26T23:20:49.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.850+0000 7f5b0f699640 1 Processor -- start 2024-04-26T23:20:49.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.850+0000 7f5b0f699640 1 -- start start 2024-04-26T23:20:49.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.851+0000 7f5b0f699640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5b0815f7c0 0x7f5b080d3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:49.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.851+0000 7f5b0f699640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b08160930 con 0x7f5b0815f3e0 2024-04-26T23:20:49.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.851+0000 7f5b0f699640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b0815fd20 con 0x7f5b080d3bd0 2024-04-26T23:20:49.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.851+0000 7f5b0f699640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f5b080d4040 con 0x7f5b0815f7c0 2024-04-26T23:20:49.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.851+0000 7f5b0cc34640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5b0815f7c0 0x7f5b080d3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:49.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.851+0000 7f5b0cc34640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5b0815f7c0 0x7f5b080d3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48080/0 (socket says 172.21.15.47:48080) 2024-04-26T23:20:49.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.851+0000 7f5b0d435640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f5b0815f3e0 0x7f5b080d2ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:52906/0 (socket says 172.21.15.47:52906) 2024-04-26T23:20:49.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.851+0000 7f5b0cc34640 1 -- 172.21.15.47:0/375673698 learned_addr learned my addr 172.21.15.47:0/375673698 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:49.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.852+0000 7f5b0cc34640 1 -- 172.21.15.47:0/375673698 >> v1:172.21.15.203:6789/0 conn(0x7f5b080d3bd0 legacy=0x7f5b081620a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:49.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.852+0000 7f5b0cc34640 1 -- 172.21.15.47:0/375673698 >> v1:172.21.15.47:6789/0 conn(0x7f5b0815f3e0 legacy=0x7f5b080d2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:49.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.852+0000 7f5afffff640 1 -- 172.21.15.47:0/375673698 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2131222712 0 0) 0x7f5b08160930 con 0x7f5b0815f3e0 2024-04-26T23:20:49.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.852+0000 7f5b0cc34640 1 -- 172.21.15.47:0/375673698 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b08160000 con 0x7f5b0815f7c0 2024-04-26T23:20:49.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.852+0000 7f5b0cc34640 1 --2- 172.21.15.47:0/375673698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b0815f7c0 0x7f5b080d3690 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f5af8009870 tx=0x7f5af802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdcea1b6ba7e85d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:49.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.852+0000 7f5afffff640 1 -- 172.21.15.47:0/375673698 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af8037070 con 0x7f5b0815f7c0 2024-04-26T23:20:49.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.853+0000 7f5afffff640 1 -- 172.21.15.47:0/375673698 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5af8032440 con 0x7f5b0815f7c0 2024-04-26T23:20:49.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.853+0000 7f5afffff640 1 -- 172.21.15.47:0/375673698 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af8032810 con 0x7f5b0815f7c0 2024-04-26T23:20:49.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.853+0000 7f5b0f699640 1 -- 172.21.15.47:0/375673698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b0815f7c0 msgr2=0x7f5b080d3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:49.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.853+0000 7f5b0f699640 1 --2- 172.21.15.47:0/375673698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b0815f7c0 0x7f5b080d3690 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f5af8009870 tx=0x7f5af802a210 comp rx=0 tx=0).stop 2024-04-26T23:20:49.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.853+0000 7f5b0f699640 1 -- 172.21.15.47:0/375673698 shutdown_connections 2024-04-26T23:20:49.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.853+0000 7f5b0f699640 1 --2- 172.21.15.47:0/375673698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b0815f7c0 0x7f5b080d3690 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:49.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.853+0000 7f5b0f699640 1 -- 172.21.15.47:0/375673698 >> 172.21.15.47:0/375673698 conn(0x7f5b0809e3c0 msgr2=0x7f5b080d50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:49.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.853+0000 7f5b0f699640 1 -- 172.21.15.47:0/375673698 shutdown_connections 2024-04-26T23:20:49.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.853+0000 7f5b0f699640 1 -- 172.21.15.47:0/375673698 wait complete. 2024-04-26T23:20:49.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.854+0000 7f5b0f699640 1 Processor -- start 2024-04-26T23:20:49.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.854+0000 7f5b0f699640 1 -- start start 2024-04-26T23:20:49.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.855+0000 7f5b0f699640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b080d3bd0 0x7f5b080cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:49.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.855+0000 7f5b0f699640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b0815f3e0 0x7f5b080ba290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:49.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.855+0000 7f5b0f699640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b0815f7c0 0x7f5b0815a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:49.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.855+0000 7f5b0f699640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5b08160d40 con 0x7f5b080d3bd0 2024-04-26T23:20:49.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.855+0000 7f5b0f699640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5b081605c0 con 0x7f5b0815f3e0 2024-04-26T23:20:49.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.855+0000 7f5b0f699640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f5b080d1040 con 0x7f5b0815f7c0 2024-04-26T23:20:49.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.855+0000 7f5b0d435640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b080d3bd0 0x7f5b080cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:49.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.855+0000 7f5b0d435640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b080d3bd0 0x7f5b080cb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48094/0 (socket says 172.21.15.47:48094) 2024-04-26T23:20:49.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.855+0000 7f5b0d435640 1 -- 172.21.15.47:0/2852400216 learned_addr learned my addr 172.21.15.47:0/2852400216 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:49.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.856+0000 7f5b0cc34640 1 --2- 172.21.15.47:0/2852400216 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b0815f3e0 0x7f5b080ba290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:49.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.856+0000 7f5b0d435640 1 -- 172.21.15.47:0/2852400216 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b0815f7c0 msgr2=0x7f5b0815a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:49.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.856+0000 7f5b0dc36640 1 --2- 172.21.15.47:0/2852400216 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b0815f7c0 0x7f5b0815a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:49.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.856+0000 7f5b0d435640 1 --2- 172.21.15.47:0/2852400216 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b0815f7c0 0x7f5b0815a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:49.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.856+0000 7f5b0d435640 1 -- 172.21.15.47:0/2852400216 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b0815f3e0 msgr2=0x7f5b080ba290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:49.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.856+0000 7f5b0d435640 1 --2- 172.21.15.47:0/2852400216 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b0815f3e0 0x7f5b080ba290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:49.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.856+0000 7f5b0d435640 1 -- 172.21.15.47:0/2852400216 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b080cce90 con 0x7f5b080d3bd0 2024-04-26T23:20:49.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.856+0000 7f5b0cc34640 1 --2- 172.21.15.47:0/2852400216 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b0815f3e0 0x7f5b080ba290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:49.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.856+0000 7f5b0dc36640 1 --2- 172.21.15.47:0/2852400216 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b0815f7c0 0x7f5b0815a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:49.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.856+0000 7f5b0d435640 1 --2- 172.21.15.47:0/2852400216 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b080d3bd0 0x7f5b080cb9e0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f5b0400c4d0 tx=0x7f5b0400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:49.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.856+0000 7f5afdffb640 1 -- 172.21.15.47:0/2852400216 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b04018070 con 0x7f5b080d3bd0 2024-04-26T23:20:49.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.856+0000 7f5afdffb640 1 -- 172.21.15.47:0/2852400216 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b04007500 con 0x7f5b080d3bd0 2024-04-26T23:20:49.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.856+0000 7f5afdffb640 1 -- 172.21.15.47:0/2852400216 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b04004a20 con 0x7f5b080d3bd0 2024-04-26T23:20:49.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.856+0000 7f5b0f699640 1 -- 172.21.15.47:0/2852400216 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b080cd170 con 0x7f5b080d3bd0 2024-04-26T23:20:49.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.856+0000 7f5b0f699640 1 -- 172.21.15.47:0/2852400216 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b080cda10 con 0x7f5b080d3bd0 2024-04-26T23:20:49.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.857+0000 7f5b0f699640 1 -- 172.21.15.47:0/2852400216 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5b08069600 con 0x7f5b080d3bd0 2024-04-26T23:20:49.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.858+0000 7f5afdffb640 1 -- 172.21.15.47:0/2852400216 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5b0401b070 con 0x7f5b080d3bd0 2024-04-26T23:20:49.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.858+0000 7f5afdffb640 1 --2- 172.21.15.47:0/2852400216 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f5aec06c8f0 0x7f5aec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:49.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.858+0000 7f5afdffb640 1 -- 172.21.15.47:0/2852400216 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5335+0+0 (secure 0 0 0) 0x7f5b0408ace0 con 0x7f5b080d3bd0 2024-04-26T23:20:49.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.859+0000 7f5afdffb640 1 --2- 172.21.15.47:0/2852400216 >> [v2:172.21.15.47:6817/326520779,v1:172.21.15.47:6819/326520779] conn(0x7f5aec072d90 0x7f5aec0751d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:49.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.859+0000 7f5afdffb640 1 -- 172.21.15.47:0/2852400216 --> [v2:172.21.15.47:6817/326520779,v1:172.21.15.47:6819/326520779] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5aec0758a0 con 0x7f5aec072d90 2024-04-26T23:20:49.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.859+0000 7f5afdffb640 1 -- 172.21.15.47:0/2852400216 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f5b0408d380 con 0x7f5b080d3bd0 2024-04-26T23:20:49.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.859+0000 7f5b0cc34640 1 --2- 172.21.15.47:0/2852400216 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f5aec06c8f0 0x7f5aec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:49.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.859+0000 7f5b0dc36640 1 --2- 172.21.15.47:0/2852400216 >> [v2:172.21.15.47:6817/326520779,v1:172.21.15.47:6819/326520779] conn(0x7f5aec072d90 0x7f5aec0751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:49.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.859+0000 7f5b0dc36640 1 --2- 172.21.15.47:0/2852400216 >> [v2:172.21.15.47:6817/326520779,v1:172.21.15.47:6819/326520779] conn(0x7f5aec072d90 0x7f5aec0751d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:49.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.859+0000 7f5b0cc34640 1 --2- 172.21.15.47:0/2852400216 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f5aec06c8f0 0x7f5aec06edb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5af8034000 tx=0x7f5af802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:49.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.860+0000 7f5afdffb640 1 -- 172.21.15.47:0/2852400216 <== osd.0 v2:172.21.15.47:6817/326520779 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5aec0758a0 con 0x7f5aec072d90 2024-04-26T23:20:49.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.873+0000 7f5b0f699640 1 -- 172.21.15.47:0/2852400216 --> [v2:172.21.15.47:6817/326520779,v1:172.21.15.47:6819/326520779] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f5b080d2680 con 0x7f5aec072d90 2024-04-26T23:20:49.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.873+0000 7f5afdffb640 1 -- 172.21.15.47:0/2852400216 <== osd.0 v2:172.21.15.47:6817/326520779 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f5b080d2680 con 0x7f5aec072d90 2024-04-26T23:20:49.875 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-26T23:20:49.875 INFO:teuthology.orchestra.run.smithi047.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T23:20:49.876 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-26T23:20:49.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.873+0000 7f5b0f699640 1 -- 172.21.15.47:0/2852400216 >> [v2:172.21.15.47:6817/326520779,v1:172.21.15.47:6819/326520779] conn(0x7f5aec072d90 msgr2=0x7f5aec0751d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:49.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.873+0000 7f5b0f699640 1 --2- 172.21.15.47:0/2852400216 >> [v2:172.21.15.47:6817/326520779,v1:172.21.15.47:6819/326520779] conn(0x7f5aec072d90 0x7f5aec0751d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:49.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.874+0000 7f5b0f699640 1 -- 172.21.15.47:0/2852400216 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f5aec06c8f0 msgr2=0x7f5aec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:49.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.874+0000 7f5b0f699640 1 --2- 172.21.15.47:0/2852400216 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f5aec06c8f0 0x7f5aec06edb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5af8034000 tx=0x7f5af802e000 comp rx=0 tx=0).stop 2024-04-26T23:20:49.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.874+0000 7f5b0f699640 1 -- 172.21.15.47:0/2852400216 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b080d3bd0 msgr2=0x7f5b080cb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:49.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.874+0000 7f5b0f699640 1 --2- 172.21.15.47:0/2852400216 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b080d3bd0 0x7f5b080cb9e0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f5b0400c4d0 tx=0x7f5b0400c990 comp rx=0 tx=0).stop 2024-04-26T23:20:49.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.874+0000 7f5b0dc36640 1 -- 172.21.15.47:0/2852400216 reap_dead start 2024-04-26T23:20:49.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.874+0000 7f5b0f699640 1 -- 172.21.15.47:0/2852400216 shutdown_connections 2024-04-26T23:20:49.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.874+0000 7f5b0f699640 1 -- 172.21.15.47:0/2852400216 >> 172.21.15.47:0/2852400216 conn(0x7f5b0809e3c0 msgr2=0x7f5b080d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:49.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.874+0000 7f5b0f699640 1 -- 172.21.15.47:0/2852400216 shutdown_connections 2024-04-26T23:20:49.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.874+0000 7f5b0f699640 1 -- 172.21.15.47:0/2852400216 wait complete. 2024-04-26T23:20:49.887 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-26T23:20:49.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.987+0000 7fae07d24640 1 Processor -- start 2024-04-26T23:20:49.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.987+0000 7fae07d24640 1 -- start start 2024-04-26T23:20:49.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.988+0000 7fae07d24640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fae00162880 0x7fae000d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:49.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.988+0000 7fae07d24640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fae000b5f20 0x7fae000d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:49.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.988+0000 7fae07d24640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae000b7e50 con 0x7fae000d39f0 2024-04-26T23:20:49.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.988+0000 7fae07d24640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fae000d3e40 con 0x7fae000b5f20 2024-04-26T23:20:49.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.988+0000 7fae07d24640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fae000d0580 con 0x7fae00162880 2024-04-26T23:20:49.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.988+0000 7fae052bf640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fae000b5f20 0x7fae000d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:49.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.988+0000 7fae062c1640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fae000d39f0 0x7fae000b69e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:52912/0 (socket says 172.21.15.47:52912) 2024-04-26T23:20:49.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.988+0000 7fae052bf640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fae000b5f20 0x7fae000d34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48104/0 (socket says 172.21.15.47:48104) 2024-04-26T23:20:49.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.988+0000 7fae062c1640 1 -- 172.21.15.47:0/2380588773 learned_addr learned my addr 172.21.15.47:0/2380588773 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:49.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.989+0000 7fae05ac0640 1 --2- 172.21.15.47:0/2380588773 >> v2:172.21.15.203:3300/0 conn(0x7fae00162880 0x7fae000d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:49.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.989+0000 7fae052bf640 1 -- 172.21.15.47:0/2380588773 >> v2:172.21.15.203:3300/0 conn(0x7fae00162880 msgr2=0x7fae000d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:49.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.989+0000 7fae052bf640 1 --2- 172.21.15.47:0/2380588773 >> v2:172.21.15.203:3300/0 conn(0x7fae00162880 0x7fae000d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:49.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.989+0000 7fae052bf640 1 -- 172.21.15.47:0/2380588773 >> v1:172.21.15.47:6789/0 conn(0x7fae000d39f0 legacy=0x7fae000b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:49.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.989+0000 7fae04abe640 1 -- 172.21.15.47:0/2380588773 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 188742750 0 0) 0x7fae000b7e50 con 0x7fae000d39f0 2024-04-26T23:20:49.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.989+0000 7fae052bf640 1 -- 172.21.15.47:0/2380588773 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae000d1800 con 0x7fae000b5f20 2024-04-26T23:20:49.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.989+0000 7fae052bf640 1 --2- 172.21.15.47:0/2380588773 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae000b5f20 0x7fae000d34b0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fae0008d490 tx=0x7fadf402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e6ae84a3fc70ef4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:49.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.989+0000 7fae04abe640 1 -- 172.21.15.47:0/2380588773 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf4037070 con 0x7fae000b5f20 2024-04-26T23:20:49.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.989+0000 7fae04abe640 1 -- 172.21.15.47:0/2380588773 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadf4032440 con 0x7fae000b5f20 2024-04-26T23:20:49.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.990+0000 7fae04abe640 1 -- 172.21.15.47:0/2380588773 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf4032810 con 0x7fae000b5f20 2024-04-26T23:20:49.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.990+0000 7fae07d24640 1 -- 172.21.15.47:0/2380588773 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae000b5f20 msgr2=0x7fae000d34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:49.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.990+0000 7fae07d24640 1 --2- 172.21.15.47:0/2380588773 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae000b5f20 0x7fae000d34b0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fae0008d490 tx=0x7fadf402a1a0 comp rx=0 tx=0).stop 2024-04-26T23:20:49.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.990+0000 7fae07d24640 1 -- 172.21.15.47:0/2380588773 shutdown_connections 2024-04-26T23:20:49.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.990+0000 7fae07d24640 1 --2- 172.21.15.47:0/2380588773 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae000b5f20 0x7fae000d34b0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:49.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.990+0000 7fae07d24640 1 --2- 172.21.15.47:0/2380588773 >> v2:172.21.15.203:3300/0 conn(0x7fae00162880 0x7fae000d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:49.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.990+0000 7fae07d24640 1 -- 172.21.15.47:0/2380588773 >> 172.21.15.47:0/2380588773 conn(0x7fae000d4f60 msgr2=0x7fae000d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:49.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.990+0000 7fae07d24640 1 -- 172.21.15.47:0/2380588773 shutdown_connections 2024-04-26T23:20:49.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.990+0000 7fae07d24640 1 -- 172.21.15.47:0/2380588773 wait complete. 2024-04-26T23:20:49.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.991+0000 7fae07d24640 1 Processor -- start 2024-04-26T23:20:49.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.991+0000 7fae07d24640 1 -- start start 2024-04-26T23:20:49.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.992+0000 7fae07d24640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae000b5f20 0x7fae000a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:49.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.992+0000 7fae07d24640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae000d39f0 0x7fae000b4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:49.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.992+0000 7fae07d24640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae00162880 0x7fae000b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:49.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.992+0000 7fae07d24640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fae000d0d90 con 0x7fae000b5f20 2024-04-26T23:20:49.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.992+0000 7fae07d24640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fae000d0ab0 con 0x7fae00162880 2024-04-26T23:20:49.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.992+0000 7fae07d24640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fae00162c60 con 0x7fae000d39f0 2024-04-26T23:20:49.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.992+0000 7fae05ac0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae000b5f20 0x7fae000a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:49.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.992+0000 7fae05ac0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae000b5f20 0x7fae000a1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48118/0 (socket says 172.21.15.47:48118) 2024-04-26T23:20:49.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.992+0000 7fae05ac0640 1 -- 172.21.15.47:0/1127182545 learned_addr learned my addr 172.21.15.47:0/1127182545 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:49.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.992+0000 7fae062c1640 1 --2- 172.21.15.47:0/1127182545 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae00162880 0x7fae000b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:49.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.992+0000 7fae052bf640 1 --2- 172.21.15.47:0/1127182545 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae000d39f0 0x7fae000b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:49.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.992+0000 7fae05ac0640 1 -- 172.21.15.47:0/1127182545 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae000d39f0 msgr2=0x7fae000b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:49.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.992+0000 7fae05ac0640 1 --2- 172.21.15.47:0/1127182545 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae000d39f0 0x7fae000b4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:49.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.992+0000 7fae05ac0640 1 -- 172.21.15.47:0/1127182545 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae00162880 msgr2=0x7fae000b4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:49.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.992+0000 7fae05ac0640 1 --2- 172.21.15.47:0/1127182545 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae00162880 0x7fae000b4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:49.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.992+0000 7fae05ac0640 1 -- 172.21.15.47:0/1127182545 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae0015e160 con 0x7fae000b5f20 2024-04-26T23:20:49.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.992+0000 7fae062c1640 1 --2- 172.21.15.47:0/1127182545 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae00162880 0x7fae000b4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:49.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.993+0000 7fae05ac0640 1 --2- 172.21.15.47:0/1127182545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae000b5f20 0x7fae000a1690 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fadf800d800 tx=0x7fadf800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:49.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.993+0000 7fadfe7fc640 1 -- 172.21.15.47:0/1127182545 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf800b6c0 con 0x7fae000b5f20 2024-04-26T23:20:49.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.993+0000 7fadfe7fc640 1 -- 172.21.15.47:0/1127182545 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadf8019070 con 0x7fae000b5f20 2024-04-26T23:20:49.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.993+0000 7fae07d24640 1 -- 172.21.15.47:0/1127182545 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae0015e420 con 0x7fae000b5f20 2024-04-26T23:20:49.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.993+0000 7fadfe7fc640 1 -- 172.21.15.47:0/1127182545 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf800bac0 con 0x7fae000b5f20 2024-04-26T23:20:49.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.993+0000 7fae07d24640 1 -- 172.21.15.47:0/1127182545 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae0015e9e0 con 0x7fae000b5f20 2024-04-26T23:20:49.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.994+0000 7fae07d24640 1 -- 172.21.15.47:0/1127182545 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae000694f0 con 0x7fae000b5f20 2024-04-26T23:20:49.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.994+0000 7fadfe7fc640 1 -- 172.21.15.47:0/1127182545 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fadf800bc60 con 0x7fae000b5f20 2024-04-26T23:20:49.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.995+0000 7fadfe7fc640 1 --2- 172.21.15.47:0/1127182545 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fadec06c8f0 0x7fadec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:49.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.995+0000 7fadfe7fc640 1 -- 172.21.15.47:0/1127182545 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5335+0+0 (secure 0 0 0) 0x7fadf808be60 con 0x7fae000b5f20 2024-04-26T23:20:49.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.995+0000 7fae052bf640 1 --2- 172.21.15.47:0/1127182545 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fadec06c8f0 0x7fadec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:49.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:49.996+0000 7fae052bf640 1 --2- 172.21.15.47:0/1127182545 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fadec06c8f0 0x7fadec06edb0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fadf4034000 tx=0x7fadf402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:50.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.000+0000 7fadfe7fc640 1 -- 172.21.15.47:0/1127182545 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fadf805a490 con 0x7fae000b5f20 2024-04-26T23:20:50.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.185+0000 7fae07d24640 1 -- 172.21.15.47:0/1127182545 --> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fae000cfa30 con 0x7fadec06c8f0 2024-04-26T23:20:50.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.186+0000 7fadfe7fc640 1 -- 172.21.15.47:0/1127182545 <== mgr.4099 v2:172.21.15.203:6832/992162732 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fae000cfa30 con 0x7fadec06c8f0 2024-04-26T23:20:50.188 INFO:teuthology.orchestra.run.smithi047.stderr:instructed osd(s) 0 to deep-scrub 2024-04-26T23:20:50.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.188+0000 7fae07d24640 1 -- 172.21.15.47:0/1127182545 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fadec06c8f0 msgr2=0x7fadec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:50.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.188+0000 7fae07d24640 1 --2- 172.21.15.47:0/1127182545 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fadec06c8f0 0x7fadec06edb0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fadf4034000 tx=0x7fadf402e000 comp rx=0 tx=0).stop 2024-04-26T23:20:50.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.188+0000 7fae07d24640 1 -- 172.21.15.47:0/1127182545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae000b5f20 msgr2=0x7fae000a1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:50.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.188+0000 7fae07d24640 1 --2- 172.21.15.47:0/1127182545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae000b5f20 0x7fae000a1690 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fadf800d800 tx=0x7fadf800dcc0 comp rx=0 tx=0).stop 2024-04-26T23:20:50.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.188+0000 7fae07d24640 1 -- 172.21.15.47:0/1127182545 shutdown_connections 2024-04-26T23:20:50.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.188+0000 7fae07d24640 1 --2- 172.21.15.47:0/1127182545 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fadec06c8f0 0x7fadec06edb0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:50.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.188+0000 7fae07d24640 1 --2- 172.21.15.47:0/1127182545 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae00162880 0x7fae000b4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:50.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.188+0000 7fae07d24640 1 --2- 172.21.15.47:0/1127182545 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae000d39f0 0x7fae000b4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:50.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.188+0000 7fae07d24640 1 --2- 172.21.15.47:0/1127182545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae000b5f20 0x7fae000a1690 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:50.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.188+0000 7fae07d24640 1 -- 172.21.15.47:0/1127182545 >> 172.21.15.47:0/1127182545 conn(0x7fae000d4f60 msgr2=0x7fae000d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:50.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.188+0000 7fae07d24640 1 -- 172.21.15.47:0/1127182545 shutdown_connections 2024-04-26T23:20:50.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.188+0000 7fae07d24640 1 -- 172.21.15.47:0/1127182545 wait complete. 2024-04-26T23:20:50.201 INFO:tasks.ceph:Scrubbing osd.1 2024-04-26T23:20:50.201 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-26T23:20:50.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.304+0000 7f6e6eccf640 1 Processor -- start 2024-04-26T23:20:50.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.304+0000 7f6e6eccf640 1 -- start start 2024-04-26T23:20:50.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.305+0000 7f6e6eccf640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6e68145d80 0x7f6e680b71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:50.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.305+0000 7f6e6eccf640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e680d00a0 con 0x7f6e680cfb60 2024-04-26T23:20:50.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.305+0000 7f6e6eccf640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e68160a80 con 0x7f6e680b7730 2024-04-26T23:20:50.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.305+0000 7f6e6eccf640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f6e68160d60 con 0x7f6e68145d80 2024-04-26T23:20:50.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.305+0000 7f6e6ca6b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6e68145d80 0x7f6e680b71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:50.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.305+0000 7f6e6ca6b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6e68145d80 0x7f6e680b71f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48126/0 (socket says 172.21.15.47:48126) 2024-04-26T23:20:50.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.305+0000 7f6e6ca6b640 1 -- 172.21.15.47:0/1249689572 learned_addr learned my addr 172.21.15.47:0/1249689572 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:50.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.305+0000 7f6e6d26c640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f6e680cfb60 0x7f6e680708f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:52920/0 (socket says 172.21.15.47:52920) 2024-04-26T23:20:50.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.305+0000 7f6e6ca6b640 1 -- 172.21.15.47:0/1249689572 >> v1:172.21.15.203:6789/0 conn(0x7f6e680b7730 legacy=0x7f6e680cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:50.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.305+0000 7f6e6ca6b640 1 -- 172.21.15.47:0/1249689572 >> v1:172.21.15.47:6789/0 conn(0x7f6e680cfb60 legacy=0x7f6e680708f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:50.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.305+0000 7f6e6ca6b640 1 -- 172.21.15.47:0/1249689572 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e68161040 con 0x7f6e68145d80 2024-04-26T23:20:50.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.306+0000 7f6e6ca6b640 1 --2- 172.21.15.47:0/1249689572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e68145d80 0x7f6e680b71f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c0098a0 tx=0x7f6e5c02a320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=177601012aa068a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:50.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.306+0000 7f6e677fe640 1 -- 172.21.15.47:0/1249689572 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c037070 con 0x7f6e68145d80 2024-04-26T23:20:50.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.306+0000 7f6e677fe640 1 -- 172.21.15.47:0/1249689572 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e5c032640 con 0x7f6e68145d80 2024-04-26T23:20:50.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.306+0000 7f6e677fe640 1 -- 172.21.15.47:0/1249689572 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c032a10 con 0x7f6e68145d80 2024-04-26T23:20:50.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.307+0000 7f6e6eccf640 1 -- 172.21.15.47:0/1249689572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e68145d80 msgr2=0x7f6e680b71f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:50.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.307+0000 7f6e6eccf640 1 --2- 172.21.15.47:0/1249689572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e68145d80 0x7f6e680b71f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c0098a0 tx=0x7f6e5c02a320 comp rx=0 tx=0).stop 2024-04-26T23:20:50.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.307+0000 7f6e6eccf640 1 -- 172.21.15.47:0/1249689572 shutdown_connections 2024-04-26T23:20:50.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.307+0000 7f6e6eccf640 1 --2- 172.21.15.47:0/1249689572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e68145d80 0x7f6e680b71f0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:50.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.307+0000 7f6e6eccf640 1 -- 172.21.15.47:0/1249689572 >> 172.21.15.47:0/1249689572 conn(0x7f6e680b3be0 msgr2=0x7f6e680b3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:50.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.307+0000 7f6e6eccf640 1 -- 172.21.15.47:0/1249689572 shutdown_connections 2024-04-26T23:20:50.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.307+0000 7f6e6eccf640 1 -- 172.21.15.47:0/1249689572 wait complete. 2024-04-26T23:20:50.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.308+0000 7f6e6eccf640 1 Processor -- start 2024-04-26T23:20:50.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.308+0000 7f6e6eccf640 1 -- start start 2024-04-26T23:20:50.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.309+0000 7f6e6eccf640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e680b7730 0x7f6e680b69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:50.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.309+0000 7f6e6eccf640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6e680cfb60 0x7f6e68138b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:50.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.309+0000 7f6e6eccf640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e68145d80 0x7f6e6815bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:50.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.309+0000 7f6e6eccf640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6e68071000 con 0x7f6e680b7730 2024-04-26T23:20:50.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.309+0000 7f6e6eccf640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6e68161840 con 0x7f6e680cfb60 2024-04-26T23:20:50.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.309+0000 7f6e6eccf640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f6e68071180 con 0x7f6e68145d80 2024-04-26T23:20:50.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.309+0000 7f6e6ca6b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e680b7730 0x7f6e680b69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:50.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.309+0000 7f6e6ca6b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e680b7730 0x7f6e680b69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48136/0 (socket says 172.21.15.47:48136) 2024-04-26T23:20:50.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.309+0000 7f6e6ca6b640 1 -- 172.21.15.47:0/346177039 learned_addr learned my addr 172.21.15.47:0/346177039 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:50.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.309+0000 7f6e6ca6b640 1 -- 172.21.15.47:0/346177039 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e68145d80 msgr2=0x7f6e6815bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:50.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.309+0000 7f6e6ca6b640 1 --2- 172.21.15.47:0/346177039 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e68145d80 0x7f6e6815bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:50.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.309+0000 7f6e6ca6b640 1 -- 172.21.15.47:0/346177039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6e680cfb60 msgr2=0x7f6e68138b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:50.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.309+0000 7f6e6ca6b640 1 --2- 172.21.15.47:0/346177039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6e680cfb60 0x7f6e68138b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:50.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.309+0000 7f6e6ca6b640 1 -- 172.21.15.47:0/346177039 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e680b7300 con 0x7f6e680b7730 2024-04-26T23:20:50.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.309+0000 7f6e6ca6b640 1 --2- 172.21.15.47:0/346177039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e680b7730 0x7f6e680b69d0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c000c00 tx=0x7f6e5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:50.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.309+0000 7f6e657fa640 1 -- 172.21.15.47:0/346177039 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c037040 con 0x7f6e680b7730 2024-04-26T23:20:50.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.309+0000 7f6e657fa640 1 -- 172.21.15.47:0/346177039 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e5c042070 con 0x7f6e680b7730 2024-04-26T23:20:50.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.309+0000 7f6e6eccf640 1 -- 172.21.15.47:0/346177039 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e680ae8c0 con 0x7f6e680b7730 2024-04-26T23:20:50.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.309+0000 7f6e657fa640 1 -- 172.21.15.47:0/346177039 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c030bd0 con 0x7f6e680b7730 2024-04-26T23:20:50.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.310+0000 7f6e6eccf640 1 -- 172.21.15.47:0/346177039 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e680af160 con 0x7f6e680b7730 2024-04-26T23:20:50.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.310+0000 7f6e6eccf640 1 -- 172.21.15.47:0/346177039 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6e68070e20 con 0x7f6e680b7730 2024-04-26T23:20:50.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.311+0000 7f6e657fa640 1 -- 172.21.15.47:0/346177039 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6e5c032500 con 0x7f6e680b7730 2024-04-26T23:20:50.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.311+0000 7f6e657fa640 1 --2- 172.21.15.47:0/346177039 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f6e4c06c8f0 0x7f6e4c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:50.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.311+0000 7f6e657fa640 1 -- 172.21.15.47:0/346177039 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5335+0+0 (secure 0 0 0) 0x7f6e5c0ac380 con 0x7f6e680b7730 2024-04-26T23:20:50.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.311+0000 7f6e657fa640 1 --2- 172.21.15.47:0/346177039 >> [v2:172.21.15.47:6808/1912610931,v1:172.21.15.47:6809/1912610931] conn(0x7f6e4c072d90 0x7f6e4c0751d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:50.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.311+0000 7f6e657fa640 1 -- 172.21.15.47:0/346177039 --> [v2:172.21.15.47:6808/1912610931,v1:172.21.15.47:6809/1912610931] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6e5c008f00 con 0x7f6e4c072d90 2024-04-26T23:20:50.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.311+0000 7f6e657fa640 1 -- 172.21.15.47:0/346177039 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f6e5c0b3050 con 0x7f6e680b7730 2024-04-26T23:20:50.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.312+0000 7f6e67fff640 1 --2- 172.21.15.47:0/346177039 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f6e4c06c8f0 0x7f6e4c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:50.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.312+0000 7f6e6d26c640 1 --2- 172.21.15.47:0/346177039 >> [v2:172.21.15.47:6808/1912610931,v1:172.21.15.47:6809/1912610931] conn(0x7f6e4c072d90 0x7f6e4c0751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:50.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.312+0000 7f6e6d26c640 1 --2- 172.21.15.47:0/346177039 >> [v2:172.21.15.47:6808/1912610931,v1:172.21.15.47:6809/1912610931] conn(0x7f6e4c072d90 0x7f6e4c0751d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:50.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.312+0000 7f6e67fff640 1 --2- 172.21.15.47:0/346177039 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f6e4c06c8f0 0x7f6e4c06edb0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f6e54000f60 tx=0x7f6e540079d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:50.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.313+0000 7f6e657fa640 1 -- 172.21.15.47:0/346177039 <== osd.1 v2:172.21.15.47:6808/1912610931 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6e5c008f00 con 0x7f6e4c072d90 2024-04-26T23:20:50.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.326+0000 7f6e6eccf640 1 -- 172.21.15.47:0/346177039 --> [v2:172.21.15.47:6808/1912610931,v1:172.21.15.47:6809/1912610931] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f6e68069600 con 0x7f6e4c072d90 2024-04-26T23:20:50.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.326+0000 7f6e657fa640 1 -- 172.21.15.47:0/346177039 <== osd.1 v2:172.21.15.47:6808/1912610931 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f6e68069600 con 0x7f6e4c072d90 2024-04-26T23:20:50.330 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-26T23:20:50.330 INFO:teuthology.orchestra.run.smithi047.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T23:20:50.330 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-26T23:20:50.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.326+0000 7f6e6eccf640 1 -- 172.21.15.47:0/346177039 >> [v2:172.21.15.47:6808/1912610931,v1:172.21.15.47:6809/1912610931] conn(0x7f6e4c072d90 msgr2=0x7f6e4c0751d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:50.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.326+0000 7f6e6eccf640 1 --2- 172.21.15.47:0/346177039 >> [v2:172.21.15.47:6808/1912610931,v1:172.21.15.47:6809/1912610931] conn(0x7f6e4c072d90 0x7f6e4c0751d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:50.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.327+0000 7f6e6eccf640 1 -- 172.21.15.47:0/346177039 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f6e4c06c8f0 msgr2=0x7f6e4c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:50.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.327+0000 7f6e6eccf640 1 --2- 172.21.15.47:0/346177039 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f6e4c06c8f0 0x7f6e4c06edb0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f6e54000f60 tx=0x7f6e540079d0 comp rx=0 tx=0).stop 2024-04-26T23:20:50.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.327+0000 7f6e6eccf640 1 -- 172.21.15.47:0/346177039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e680b7730 msgr2=0x7f6e680b69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:50.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.327+0000 7f6e6eccf640 1 --2- 172.21.15.47:0/346177039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e680b7730 0x7f6e680b69d0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c000c00 tx=0x7f6e5c0046c0 comp rx=0 tx=0).stop 2024-04-26T23:20:50.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.327+0000 7f6e6d26c640 1 -- 172.21.15.47:0/346177039 reap_dead start 2024-04-26T23:20:50.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.327+0000 7f6e6eccf640 1 -- 172.21.15.47:0/346177039 shutdown_connections 2024-04-26T23:20:50.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.327+0000 7f6e6eccf640 1 -- 172.21.15.47:0/346177039 >> 172.21.15.47:0/346177039 conn(0x7f6e680b3be0 msgr2=0x7f6e68141690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:50.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.327+0000 7f6e6eccf640 1 -- 172.21.15.47:0/346177039 shutdown_connections 2024-04-26T23:20:50.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.327+0000 7f6e6eccf640 1 -- 172.21.15.47:0/346177039 wait complete. 2024-04-26T23:20:50.339 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-26T23:20:50.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.439+0000 7f140cd0a640 1 Processor -- start 2024-04-26T23:20:50.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.439+0000 7f140cd0a640 1 -- start start 2024-04-26T23:20:50.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.440+0000 7f140cd0a640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f14080d3bc0 0x7f14080d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:50.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.440+0000 7f140cd0a640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14080b7e50 con 0x7f1408162880 2024-04-26T23:20:50.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.440+0000 7f140cd0a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14080d1800 con 0x7f14080b5f20 2024-04-26T23:20:50.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.440+0000 7f140cd0a640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f14080d0580 con 0x7f14080d3bc0 2024-04-26T23:20:50.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.440+0000 7f1405d9b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f14080d3bc0 0x7f14080d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:50.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.440+0000 7f140659c640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1408162880 0x7f14080d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:52922/0 (socket says 172.21.15.47:52922) 2024-04-26T23:20:50.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.440+0000 7f1405d9b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f14080d3bc0 0x7f14080d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48138/0 (socket says 172.21.15.47:48138) 2024-04-26T23:20:50.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.440+0000 7f140659c640 1 -- 172.21.15.47:0/649210638 learned_addr learned my addr 172.21.15.47:0/649210638 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:50.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.440+0000 7f1405d9b640 1 -- 172.21.15.47:0/649210638 >> v1:172.21.15.203:6789/0 conn(0x7f14080b5f20 legacy=0x7f14080b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:50.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.440+0000 7f1405d9b640 1 -- 172.21.15.47:0/649210638 >> v1:172.21.15.47:6789/0 conn(0x7f1408162880 legacy=0x7f14080d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:50.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.440+0000 7f1405d9b640 1 -- 172.21.15.47:0/649210638 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14080d07d0 con 0x7f14080d3bc0 2024-04-26T23:20:50.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.441+0000 7f1405d9b640 1 --2- 172.21.15.47:0/649210638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14080d3bc0 0x7f14080d2cb0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f13fc0097a0 tx=0x7f13fc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=221dc21496eb3db8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:50.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.443+0000 7f140559a640 1 -- 172.21.15.47:0/649210638 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13fc037070 con 0x7f14080d3bc0 2024-04-26T23:20:50.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.443+0000 7f140559a640 1 -- 172.21.15.47:0/649210638 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13fc032440 con 0x7f14080d3bc0 2024-04-26T23:20:50.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.443+0000 7f140559a640 1 -- 172.21.15.47:0/649210638 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13fc032810 con 0x7f14080d3bc0 2024-04-26T23:20:50.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.443+0000 7f140cd0a640 1 -- 172.21.15.47:0/649210638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14080d3bc0 msgr2=0x7f14080d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:50.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.443+0000 7f140cd0a640 1 --2- 172.21.15.47:0/649210638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14080d3bc0 0x7f14080d2cb0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f13fc0097a0 tx=0x7f13fc02a1e0 comp rx=0 tx=0).stop 2024-04-26T23:20:50.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.443+0000 7f140cd0a640 1 -- 172.21.15.47:0/649210638 shutdown_connections 2024-04-26T23:20:50.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.443+0000 7f140cd0a640 1 --2- 172.21.15.47:0/649210638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14080d3bc0 0x7f14080d2cb0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:50.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.443+0000 7f140cd0a640 1 -- 172.21.15.47:0/649210638 >> 172.21.15.47:0/649210638 conn(0x7f14080d4f60 msgr2=0x7f14080d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:50.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.443+0000 7f140cd0a640 1 -- 172.21.15.47:0/649210638 shutdown_connections 2024-04-26T23:20:50.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.443+0000 7f140cd0a640 1 -- 172.21.15.47:0/649210638 wait complete. 2024-04-26T23:20:50.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.445+0000 7f140cd0a640 1 Processor -- start 2024-04-26T23:20:50.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.445+0000 7f140cd0a640 1 -- start start 2024-04-26T23:20:50.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.445+0000 7f140cd0a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14080b5f20 0x7f14081f49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:50.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.445+0000 7f140cd0a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14080d3bc0 0x7f14081f76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:50.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.445+0000 7f140cd0a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1408162880 0x7f14080b2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:50.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.445+0000 7f140cd0a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1408162c60 con 0x7f1408162880 2024-04-26T23:20:50.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.445+0000 7f140cd0a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f14080d0d90 con 0x7f14080d3bc0 2024-04-26T23:20:50.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.445+0000 7f140cd0a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f1408162de0 con 0x7f14080b5f20 2024-04-26T23:20:50.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.446+0000 7f1406d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1408162880 0x7f14080b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:50.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.446+0000 7f1406d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1408162880 0x7f14080b2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48154/0 (socket says 172.21.15.47:48154) 2024-04-26T23:20:50.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.446+0000 7f1406d9d640 1 -- 172.21.15.47:0/3313197395 learned_addr learned my addr 172.21.15.47:0/3313197395 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:50.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.446+0000 7f140659c640 1 --2- 172.21.15.47:0/3313197395 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14080b5f20 0x7f14081f49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:50.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.446+0000 7f1405d9b640 1 --2- 172.21.15.47:0/3313197395 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14080d3bc0 0x7f14081f76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:50.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.446+0000 7f1406d9d640 1 -- 172.21.15.47:0/3313197395 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14080b5f20 msgr2=0x7f14081f49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:50.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.446+0000 7f1406d9d640 1 --2- 172.21.15.47:0/3313197395 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14080b5f20 0x7f14081f49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:50.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.446+0000 7f1406d9d640 1 -- 172.21.15.47:0/3313197395 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14080d3bc0 msgr2=0x7f14081f76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:50.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.446+0000 7f1406d9d640 1 --2- 172.21.15.47:0/3313197395 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14080d3bc0 0x7f14081f76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:50.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.446+0000 7f1406d9d640 1 -- 172.21.15.47:0/3313197395 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14080b30d0 con 0x7f1408162880 2024-04-26T23:20:50.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.446+0000 7f1406d9d640 1 --2- 172.21.15.47:0/3313197395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1408162880 0x7f14080b2560 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f140000c780 tx=0x7f140000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:50.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.446+0000 7f13f6ffd640 1 -- 172.21.15.47:0/3313197395 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1400004b60 con 0x7f1408162880 2024-04-26T23:20:50.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.446+0000 7f13f6ffd640 1 -- 172.21.15.47:0/3313197395 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1400004d00 con 0x7f1408162880 2024-04-26T23:20:50.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.446+0000 7f140cd0a640 1 -- 172.21.15.47:0/3313197395 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14080b33b0 con 0x7f1408162880 2024-04-26T23:20:50.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.446+0000 7f13f6ffd640 1 -- 172.21.15.47:0/3313197395 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140000aa90 con 0x7f1408162880 2024-04-26T23:20:50.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.446+0000 7f140cd0a640 1 -- 172.21.15.47:0/3313197395 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14080b3970 con 0x7f1408162880 2024-04-26T23:20:50.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.447+0000 7f140cd0a640 1 -- 172.21.15.47:0/3313197395 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14080b3e40 con 0x7f1408162880 2024-04-26T23:20:50.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.448+0000 7f13f6ffd640 1 -- 172.21.15.47:0/3313197395 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f140000ac30 con 0x7f1408162880 2024-04-26T23:20:50.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.448+0000 7f13f6ffd640 1 --2- 172.21.15.47:0/3313197395 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f13e806c8f0 0x7f13e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:50.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.448+0000 7f13f6ffd640 1 -- 172.21.15.47:0/3313197395 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5335+0+0 (secure 0 0 0) 0x7f140008bd90 con 0x7f1408162880 2024-04-26T23:20:50.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.449+0000 7f140659c640 1 --2- 172.21.15.47:0/3313197395 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f13e806c8f0 0x7f13e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:50.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.449+0000 7f140659c640 1 --2- 172.21.15.47:0/3313197395 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f13e806c8f0 0x7f13e806edb0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f13f8007fd0 tx=0x7f13f8007e80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:50.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.453+0000 7f13f6ffd640 1 -- 172.21.15.47:0/3313197395 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1400059d70 con 0x7f1408162880 2024-04-26T23:20:50.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.644+0000 7f140cd0a640 1 -- 172.21.15.47:0/3313197395 --> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f14080694f0 con 0x7f13e806c8f0 2024-04-26T23:20:50.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.644+0000 7f13f6ffd640 1 -- 172.21.15.47:0/3313197395 <== mgr.4099 v2:172.21.15.203:6832/992162732 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f14080694f0 con 0x7f13e806c8f0 2024-04-26T23:20:50.646 INFO:teuthology.orchestra.run.smithi047.stderr:instructed osd(s) 1 to deep-scrub 2024-04-26T23:20:50.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.647+0000 7f140cd0a640 1 -- 172.21.15.47:0/3313197395 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f13e806c8f0 msgr2=0x7f13e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:50.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.647+0000 7f140cd0a640 1 --2- 172.21.15.47:0/3313197395 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f13e806c8f0 0x7f13e806edb0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f13f8007fd0 tx=0x7f13f8007e80 comp rx=0 tx=0).stop 2024-04-26T23:20:50.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.647+0000 7f140cd0a640 1 -- 172.21.15.47:0/3313197395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1408162880 msgr2=0x7f14080b2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:50.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.647+0000 7f140cd0a640 1 --2- 172.21.15.47:0/3313197395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1408162880 0x7f14080b2560 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f140000c780 tx=0x7f140000cc40 comp rx=0 tx=0).stop 2024-04-26T23:20:50.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.647+0000 7f140cd0a640 1 -- 172.21.15.47:0/3313197395 shutdown_connections 2024-04-26T23:20:50.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.647+0000 7f140cd0a640 1 --2- 172.21.15.47:0/3313197395 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f13e806c8f0 0x7f13e806edb0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:50.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.647+0000 7f140cd0a640 1 --2- 172.21.15.47:0/3313197395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1408162880 0x7f14080b2560 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:50.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.647+0000 7f140cd0a640 1 --2- 172.21.15.47:0/3313197395 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14080d3bc0 0x7f14081f76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:50.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.647+0000 7f140cd0a640 1 --2- 172.21.15.47:0/3313197395 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14080b5f20 0x7f14081f49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:50.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.647+0000 7f140cd0a640 1 -- 172.21.15.47:0/3313197395 >> 172.21.15.47:0/3313197395 conn(0x7f14080d4f60 msgr2=0x7f140806cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:50.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.647+0000 7f140cd0a640 1 -- 172.21.15.47:0/3313197395 shutdown_connections 2024-04-26T23:20:50.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.647+0000 7f140cd0a640 1 -- 172.21.15.47:0/3313197395 wait complete. 2024-04-26T23:20:50.659 INFO:tasks.ceph:Scrubbing osd.2 2024-04-26T23:20:50.660 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-26T23:20:50.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.763+0000 7f41623f7640 1 Processor -- start 2024-04-26T23:20:50.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.763+0000 7f41623f7640 1 -- start start 2024-04-26T23:20:50.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.764+0000 7f41623f7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f415c15f340 0x7f415c0d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:50.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.764+0000 7f41623f7640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f415c1606c0 0x7f415c15f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:50.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.764+0000 7f41623f7640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f415c163160 con 0x7f415c15fc60 2024-04-26T23:20:50.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.764+0000 7f41623f7640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f415c162f10 con 0x7f415c1606c0 2024-04-26T23:20:50.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.764+0000 7f41623f7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f415c163d70 con 0x7f415c15f340 2024-04-26T23:20:50.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.764+0000 7f415b7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f415c1606c0 0x7f415c15f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:50.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.764+0000 7f4160994640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f415c15fc60 0x7f415c0d2e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:52934/0 (socket says 172.21.15.47:52934) 2024-04-26T23:20:50.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.764+0000 7f4160994640 1 -- 172.21.15.47:0/2216379377 learned_addr learned my addr 172.21.15.47:0/2216379377 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:50.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.765+0000 7f415bfff640 1 --2- 172.21.15.47:0/2216379377 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f415c15f340 0x7f415c0d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:50.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.765+0000 7f415b7fe640 1 -- 172.21.15.47:0/2216379377 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f415c15f340 msgr2=0x7f415c0d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:50.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.765+0000 7f415b7fe640 1 --2- 172.21.15.47:0/2216379377 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f415c15f340 0x7f415c0d2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:50.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.765+0000 7f415b7fe640 1 -- 172.21.15.47:0/2216379377 >> v1:172.21.15.47:6789/0 conn(0x7f415c15fc60 legacy=0x7f415c0d2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:50.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.765+0000 7f415affd640 1 -- 172.21.15.47:0/2216379377 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3266750866 0 0) 0x7f415c163160 con 0x7f415c15fc60 2024-04-26T23:20:50.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.765+0000 7f415b7fe640 1 -- 172.21.15.47:0/2216379377 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f415c163440 con 0x7f415c1606c0 2024-04-26T23:20:50.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.765+0000 7f415bfff640 1 --2- 172.21.15.47:0/2216379377 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f415c15f340 0x7f415c0d2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:50.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.765+0000 7f415b7fe640 1 --2- 172.21.15.47:0/2216379377 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f415c1606c0 0x7f415c15f720 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f414c0097a0 tx=0x7f414c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a21dfede3619f9a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:50.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.766+0000 7f415affd640 1 -- 172.21.15.47:0/2216379377 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414c037070 con 0x7f415c1606c0 2024-04-26T23:20:50.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.766+0000 7f415affd640 1 -- 172.21.15.47:0/2216379377 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f414c032440 con 0x7f415c1606c0 2024-04-26T23:20:50.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.766+0000 7f415affd640 1 -- 172.21.15.47:0/2216379377 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414c032810 con 0x7f415c1606c0 2024-04-26T23:20:50.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.766+0000 7f41623f7640 1 -- 172.21.15.47:0/2216379377 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f415c1606c0 msgr2=0x7f415c15f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:50.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.766+0000 7f41623f7640 1 --2- 172.21.15.47:0/2216379377 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f415c1606c0 0x7f415c15f720 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f414c0097a0 tx=0x7f414c02a180 comp rx=0 tx=0).stop 2024-04-26T23:20:50.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.766+0000 7f41623f7640 1 -- 172.21.15.47:0/2216379377 shutdown_connections 2024-04-26T23:20:50.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.766+0000 7f41623f7640 1 --2- 172.21.15.47:0/2216379377 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f415c1606c0 0x7f415c15f720 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:50.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.766+0000 7f41623f7640 1 --2- 172.21.15.47:0/2216379377 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f415c15f340 0x7f415c0d2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:50.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.766+0000 7f41623f7640 1 -- 172.21.15.47:0/2216379377 >> 172.21.15.47:0/2216379377 conn(0x7f415c09ba50 msgr2=0x7f415c09be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:50.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.766+0000 7f41623f7640 1 -- 172.21.15.47:0/2216379377 shutdown_connections 2024-04-26T23:20:50.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.766+0000 7f41623f7640 1 -- 172.21.15.47:0/2216379377 wait complete. 2024-04-26T23:20:50.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.767+0000 7f41623f7640 1 Processor -- start 2024-04-26T23:20:50.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.768+0000 7f41623f7640 1 -- start start 2024-04-26T23:20:50.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.768+0000 7f41623f7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f415c15f340 0x7f415c14c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:50.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.768+0000 7f41623f7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f415c15fc60 0x7f415c06e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:50.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.768+0000 7f41623f7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f415c1606c0 0x7f415c1f52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:50.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.768+0000 7f41623f7640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f415c0d2a90 con 0x7f415c1606c0 2024-04-26T23:20:50.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.768+0000 7f41623f7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f415c163a00 con 0x7f415c15f340 2024-04-26T23:20:50.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.768+0000 7f41623f7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f415c1600d0 con 0x7f415c15fc60 2024-04-26T23:20:50.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.768+0000 7f4160994640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f415c1606c0 0x7f415c1f52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:50.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.768+0000 7f4160994640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f415c1606c0 0x7f415c1f52e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48180/0 (socket says 172.21.15.47:48180) 2024-04-26T23:20:50.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.768+0000 7f4160994640 1 -- 172.21.15.47:0/1998087910 learned_addr learned my addr 172.21.15.47:0/1998087910 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:50.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.769+0000 7f415b7fe640 1 --2- 172.21.15.47:0/1998087910 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f415c15fc60 0x7f415c06e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:50.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.769+0000 7f415bfff640 1 --2- 172.21.15.47:0/1998087910 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f415c15f340 0x7f415c14c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:50.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.769+0000 7f4160994640 1 -- 172.21.15.47:0/1998087910 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f415c15fc60 msgr2=0x7f415c06e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:50.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.769+0000 7f4160994640 1 --2- 172.21.15.47:0/1998087910 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f415c15fc60 0x7f415c06e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:50.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.769+0000 7f4160994640 1 -- 172.21.15.47:0/1998087910 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f415c15f340 msgr2=0x7f415c14c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:50.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.769+0000 7f4160994640 1 --2- 172.21.15.47:0/1998087910 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f415c15f340 0x7f415c14c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:50.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.769+0000 7f4160994640 1 -- 172.21.15.47:0/1998087910 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f415c06d7c0 con 0x7f415c1606c0 2024-04-26T23:20:50.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.769+0000 7f4160994640 1 --2- 172.21.15.47:0/1998087910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f415c1606c0 0x7f415c1f52e0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f415400c7a0 tx=0x7f415400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:50.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.769+0000 7f4158ff9640 1 -- 172.21.15.47:0/1998087910 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4154018070 con 0x7f415c1606c0 2024-04-26T23:20:50.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.769+0000 7f4158ff9640 1 -- 172.21.15.47:0/1998087910 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f415400a640 con 0x7f415c1606c0 2024-04-26T23:20:50.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.769+0000 7f41623f7640 1 -- 172.21.15.47:0/1998087910 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f415c06daa0 con 0x7f415c1606c0 2024-04-26T23:20:50.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.769+0000 7f4158ff9640 1 -- 172.21.15.47:0/1998087910 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f415400aa70 con 0x7f415c1606c0 2024-04-26T23:20:50.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.769+0000 7f41623f7640 1 -- 172.21.15.47:0/1998087910 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f415c06e060 con 0x7f415c1606c0 2024-04-26T23:20:50.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.770+0000 7f41623f7640 1 -- 172.21.15.47:0/1998087910 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f415c069600 con 0x7f415c1606c0 2024-04-26T23:20:50.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.771+0000 7f4158ff9640 1 -- 172.21.15.47:0/1998087910 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f415401b070 con 0x7f415c1606c0 2024-04-26T23:20:50.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.771+0000 7f4158ff9640 1 --2- 172.21.15.47:0/1998087910 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f413c06c8f0 0x7f413c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:50.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.771+0000 7f4158ff9640 1 -- 172.21.15.47:0/1998087910 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5335+0+0 (secure 0 0 0) 0x7f415408b010 con 0x7f415c1606c0 2024-04-26T23:20:50.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.771+0000 7f4158ff9640 1 --2- 172.21.15.47:0/1998087910 >> [v2:172.21.15.47:6800/2569869305,v1:172.21.15.47:6801/2569869305] conn(0x7f413c072d90 0x7f413c0751d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:50.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.771+0000 7f4158ff9640 1 -- 172.21.15.47:0/1998087910 --> [v2:172.21.15.47:6800/2569869305,v1:172.21.15.47:6801/2569869305] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f413c0758a0 con 0x7f413c072d90 2024-04-26T23:20:50.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.771+0000 7f4158ff9640 1 -- 172.21.15.47:0/1998087910 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f4154092050 con 0x7f415c1606c0 2024-04-26T23:20:50.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.771+0000 7f415bfff640 1 --2- 172.21.15.47:0/1998087910 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f413c06c8f0 0x7f413c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:50.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.771+0000 7f415b7fe640 1 --2- 172.21.15.47:0/1998087910 >> [v2:172.21.15.47:6800/2569869305,v1:172.21.15.47:6801/2569869305] conn(0x7f413c072d90 0x7f413c0751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:50.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.772+0000 7f415b7fe640 1 --2- 172.21.15.47:0/1998087910 >> [v2:172.21.15.47:6800/2569869305,v1:172.21.15.47:6801/2569869305] conn(0x7f413c072d90 0x7f413c0751d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:50.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.772+0000 7f415bfff640 1 --2- 172.21.15.47:0/1998087910 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f413c06c8f0 0x7f413c06edb0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f415c0cf580 tx=0x7f4144008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:50.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.772+0000 7f4158ff9640 1 -- 172.21.15.47:0/1998087910 <== osd.2 v2:172.21.15.47:6800/2569869305 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f413c0758a0 con 0x7f413c072d90 2024-04-26T23:20:50.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.786+0000 7f41623f7640 1 -- 172.21.15.47:0/1998087910 --> [v2:172.21.15.47:6800/2569869305,v1:172.21.15.47:6801/2569869305] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f415c068e00 con 0x7f413c072d90 2024-04-26T23:20:50.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.786+0000 7f4158ff9640 1 -- 172.21.15.47:0/1998087910 <== osd.2 v2:172.21.15.47:6800/2569869305 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f415c068e00 con 0x7f413c072d90 2024-04-26T23:20:50.788 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-26T23:20:50.788 INFO:teuthology.orchestra.run.smithi047.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T23:20:50.788 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-26T23:20:50.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.786+0000 7f41623f7640 1 -- 172.21.15.47:0/1998087910 >> [v2:172.21.15.47:6800/2569869305,v1:172.21.15.47:6801/2569869305] conn(0x7f413c072d90 msgr2=0x7f413c0751d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:50.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.786+0000 7f41623f7640 1 --2- 172.21.15.47:0/1998087910 >> [v2:172.21.15.47:6800/2569869305,v1:172.21.15.47:6801/2569869305] conn(0x7f413c072d90 0x7f413c0751d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:50.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.786+0000 7f41623f7640 1 -- 172.21.15.47:0/1998087910 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f413c06c8f0 msgr2=0x7f413c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:50.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.786+0000 7f41623f7640 1 --2- 172.21.15.47:0/1998087910 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f413c06c8f0 0x7f413c06edb0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f415c0cf580 tx=0x7f4144008040 comp rx=0 tx=0).stop 2024-04-26T23:20:50.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.786+0000 7f41623f7640 1 -- 172.21.15.47:0/1998087910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f415c1606c0 msgr2=0x7f415c1f52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:50.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.786+0000 7f41623f7640 1 --2- 172.21.15.47:0/1998087910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f415c1606c0 0x7f415c1f52e0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f415400c7a0 tx=0x7f415400cc60 comp rx=0 tx=0).stop 2024-04-26T23:20:50.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.786+0000 7f4160994640 1 -- 172.21.15.47:0/1998087910 reap_dead start 2024-04-26T23:20:50.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.786+0000 7f41623f7640 1 -- 172.21.15.47:0/1998087910 shutdown_connections 2024-04-26T23:20:50.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.786+0000 7f41623f7640 1 -- 172.21.15.47:0/1998087910 >> 172.21.15.47:0/1998087910 conn(0x7f415c09ba50 msgr2=0x7f415c0709c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:50.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.786+0000 7f41623f7640 1 -- 172.21.15.47:0/1998087910 shutdown_connections 2024-04-26T23:20:50.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.786+0000 7f41623f7640 1 -- 172.21.15.47:0/1998087910 wait complete. 2024-04-26T23:20:50.799 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-26T23:20:50.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.898+0000 7f9b8d609640 1 Processor -- start 2024-04-26T23:20:50.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.898+0000 7f9b8d609640 1 -- start start 2024-04-26T23:20:50.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.899+0000 7f9b8d609640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b881608f0 0x7f9b880d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:50.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.899+0000 7f9b8d609640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9b880cffa0 0x7f9b880b2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:50.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.899+0000 7f9b8d609640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b880cde20 con 0x7f9b880d0a90 2024-04-26T23:20:50.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.899+0000 7f9b8d609640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f9b880cea30 con 0x7f9b880cffa0 2024-04-26T23:20:50.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.899+0000 7f9b8d609640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f9b880ce100 con 0x7f9b881608f0 2024-04-26T23:20:50.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.899+0000 7f9b86ffd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b881608f0 0x7f9b880d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:50.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.899+0000 7f9b86ffd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b881608f0 0x7f9b880d1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.47:49356/0 (socket says 172.21.15.47:49356) 2024-04-26T23:20:50.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.899+0000 7f9b86ffd640 1 -- 172.21.15.47:0/357009406 learned_addr learned my addr 172.21.15.47:0/357009406 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:50.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.899+0000 7f9b877fe640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9b880cffa0 0x7f9b880b2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:50.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.899+0000 7f9b867fc640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f9b880d0a90 0x7f9b880cf890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.47:54214/0 (socket says 172.21.15.47:54214) 2024-04-26T23:20:50.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.900+0000 7f9b86ffd640 1 -- 172.21.15.47:0/357009406 >> v2:172.21.15.203:3300/0 conn(0x7f9b880cffa0 msgr2=0x7f9b880b2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:50.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.900+0000 7f9b86ffd640 1 --2- 172.21.15.47:0/357009406 >> v2:172.21.15.203:3300/0 conn(0x7f9b880cffa0 0x7f9b880b2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:50.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.900+0000 7f9b86ffd640 1 -- 172.21.15.47:0/357009406 >> v1:172.21.15.203:6789/0 conn(0x7f9b880d0a90 legacy=0x7f9b880cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:50.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.900+0000 7f9b86ffd640 1 -- 172.21.15.47:0/357009406 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b880ce3e0 con 0x7f9b881608f0 2024-04-26T23:20:50.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.900+0000 7f9b86ffd640 1 --2- 172.21.15.47:0/357009406 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b881608f0 0x7f9b880d1bf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9b80009e50 tx=0x7f9b8002a730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adf8737f6cee60ff server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:50.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.901+0000 7f9b85ffb640 1 -- 172.21.15.47:0/357009406 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b80036070 con 0x7f9b881608f0 2024-04-26T23:20:50.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.901+0000 7f9b85ffb640 1 -- 172.21.15.47:0/357009406 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b80031600 con 0x7f9b881608f0 2024-04-26T23:20:50.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.901+0000 7f9b85ffb640 1 -- 172.21.15.47:0/357009406 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b800319d0 con 0x7f9b881608f0 2024-04-26T23:20:50.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.901+0000 7f9b8d609640 1 -- 172.21.15.47:0/357009406 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b881608f0 msgr2=0x7f9b880d1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:50.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.901+0000 7f9b8d609640 1 --2- 172.21.15.47:0/357009406 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b881608f0 0x7f9b880d1bf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9b80009e50 tx=0x7f9b8002a730 comp rx=0 tx=0).stop 2024-04-26T23:20:50.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.901+0000 7f9b8d609640 1 -- 172.21.15.47:0/357009406 shutdown_connections 2024-04-26T23:20:50.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.901+0000 7f9b8d609640 1 --2- 172.21.15.47:0/357009406 >> v2:172.21.15.203:3300/0 conn(0x7f9b880cffa0 0x7f9b880b2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:50.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.901+0000 7f9b8d609640 1 --2- 172.21.15.47:0/357009406 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b881608f0 0x7f9b880d1bf0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:50.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.901+0000 7f9b8d609640 1 -- 172.21.15.47:0/357009406 >> 172.21.15.47:0/357009406 conn(0x7f9b880d2ec0 msgr2=0x7f9b880d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:50.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.901+0000 7f9b8d609640 1 -- 172.21.15.47:0/357009406 shutdown_connections 2024-04-26T23:20:50.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.901+0000 7f9b8d609640 1 -- 172.21.15.47:0/357009406 wait complete. 2024-04-26T23:20:50.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.903+0000 7f9b8d609640 1 Processor -- start 2024-04-26T23:20:50.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.903+0000 7f9b8d609640 1 -- start start 2024-04-26T23:20:50.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.903+0000 7f9b8d609640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b880cffa0 0x7f9b880ca000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:50.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.903+0000 7f9b8d609640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b880d0a90 0x7f9b881e6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:50.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.903+0000 7f9b8d609640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b881608f0 0x7f9b881e78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:50.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.903+0000 7f9b8d609640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9b880ceec0 con 0x7f9b880cffa0 2024-04-26T23:20:50.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.903+0000 7f9b8d609640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9b880d0ef0 con 0x7f9b880d0a90 2024-04-26T23:20:50.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.903+0000 7f9b8d609640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f9b880cda40 con 0x7f9b881608f0 2024-04-26T23:20:50.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.903+0000 7f9b86ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b880cffa0 0x7f9b880ca000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:50.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.904+0000 7f9b86ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b880cffa0 0x7f9b880ca000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48194/0 (socket says 172.21.15.47:48194) 2024-04-26T23:20:50.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.904+0000 7f9b86ffd640 1 -- 172.21.15.47:0/3797129238 learned_addr learned my addr 172.21.15.47:0/3797129238 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:50.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.904+0000 7f9b877fe640 1 --2- 172.21.15.47:0/3797129238 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b881608f0 0x7f9b881e78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:50.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.904+0000 7f9b867fc640 1 --2- 172.21.15.47:0/3797129238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b880d0a90 0x7f9b881e6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:50.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.904+0000 7f9b86ffd640 1 -- 172.21.15.47:0/3797129238 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b881608f0 msgr2=0x7f9b881e78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:50.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.904+0000 7f9b86ffd640 1 --2- 172.21.15.47:0/3797129238 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b881608f0 0x7f9b881e78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:50.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.904+0000 7f9b86ffd640 1 -- 172.21.15.47:0/3797129238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b880d0a90 msgr2=0x7f9b881e6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:50.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.904+0000 7f9b86ffd640 1 --2- 172.21.15.47:0/3797129238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b880d0a90 0x7f9b881e6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:50.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.904+0000 7f9b86ffd640 1 -- 172.21.15.47:0/3797129238 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b880cd100 con 0x7f9b880cffa0 2024-04-26T23:20:50.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.904+0000 7f9b877fe640 1 --2- 172.21.15.47:0/3797129238 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b881608f0 0x7f9b881e78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:50.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.904+0000 7f9b867fc640 1 --2- 172.21.15.47:0/3797129238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b880d0a90 0x7f9b881e6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:50.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.904+0000 7f9b86ffd640 1 --2- 172.21.15.47:0/3797129238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b880cffa0 0x7f9b880ca000 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f9b80034040 tx=0x7f9b80004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:50.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.904+0000 7f9b7f7fe640 1 -- 172.21.15.47:0/3797129238 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b80036050 con 0x7f9b880cffa0 2024-04-26T23:20:50.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.904+0000 7f9b7f7fe640 1 -- 172.21.15.47:0/3797129238 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b80042070 con 0x7f9b880cffa0 2024-04-26T23:20:50.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.904+0000 7f9b8d609640 1 -- 172.21.15.47:0/3797129238 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b881e7240 con 0x7f9b880cffa0 2024-04-26T23:20:50.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.904+0000 7f9b7f7fe640 1 -- 172.21.15.47:0/3797129238 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b80009510 con 0x7f9b880cffa0 2024-04-26T23:20:50.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.904+0000 7f9b8d609640 1 -- 172.21.15.47:0/3797129238 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b880cce20 con 0x7f9b880cffa0 2024-04-26T23:20:50.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.905+0000 7f9b8d609640 1 -- 172.21.15.47:0/3797129238 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b880c78e0 con 0x7f9b880cffa0 2024-04-26T23:20:50.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.906+0000 7f9b7f7fe640 1 -- 172.21.15.47:0/3797129238 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9b80007c50 con 0x7f9b880cffa0 2024-04-26T23:20:50.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.906+0000 7f9b7f7fe640 1 --2- 172.21.15.47:0/3797129238 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9b6806c8f0 0x7f9b6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:50.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.906+0000 7f9b7f7fe640 1 -- 172.21.15.47:0/3797129238 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5335+0+0 (secure 0 0 0) 0x7f9b800ac6e0 con 0x7f9b880cffa0 2024-04-26T23:20:50.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.907+0000 7f9b867fc640 1 --2- 172.21.15.47:0/3797129238 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9b6806c8f0 0x7f9b6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:50.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.907+0000 7f9b867fc640 1 --2- 172.21.15.47:0/3797129238 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9b6806c8f0 0x7f9b6806edb0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9b881f6c10 tx=0x7f9b70009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:50.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:50.911+0000 7f9b7f7fe640 1 -- 172.21.15.47:0/3797129238 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9b80045360 con 0x7f9b880cffa0 2024-04-26T23:20:51.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.095+0000 7f9b8d609640 1 -- 172.21.15.47:0/3797129238 --> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f9b880694f0 con 0x7f9b6806c8f0 2024-04-26T23:20:51.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.095+0000 7f9b7f7fe640 1 -- 172.21.15.47:0/3797129238 <== mgr.4099 v2:172.21.15.203:6832/992162732 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f9b880694f0 con 0x7f9b6806c8f0 2024-04-26T23:20:51.097 INFO:teuthology.orchestra.run.smithi047.stderr:instructed osd(s) 2 to deep-scrub 2024-04-26T23:20:51.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.097+0000 7f9b8d609640 1 -- 172.21.15.47:0/3797129238 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9b6806c8f0 msgr2=0x7f9b6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:51.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.097+0000 7f9b8d609640 1 --2- 172.21.15.47:0/3797129238 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9b6806c8f0 0x7f9b6806edb0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9b881f6c10 tx=0x7f9b70009040 comp rx=0 tx=0).stop 2024-04-26T23:20:51.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.097+0000 7f9b8d609640 1 -- 172.21.15.47:0/3797129238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b880cffa0 msgr2=0x7f9b880ca000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:51.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.097+0000 7f9b8d609640 1 --2- 172.21.15.47:0/3797129238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b880cffa0 0x7f9b880ca000 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f9b80034040 tx=0x7f9b80004a40 comp rx=0 tx=0).stop 2024-04-26T23:20:51.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.097+0000 7f9b8d609640 1 -- 172.21.15.47:0/3797129238 shutdown_connections 2024-04-26T23:20:51.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.097+0000 7f9b8d609640 1 --2- 172.21.15.47:0/3797129238 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f9b6806c8f0 0x7f9b6806edb0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:51.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.097+0000 7f9b8d609640 1 --2- 172.21.15.47:0/3797129238 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b881608f0 0x7f9b881e78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:51.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.097+0000 7f9b8d609640 1 --2- 172.21.15.47:0/3797129238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b880d0a90 0x7f9b881e6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:51.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.097+0000 7f9b8d609640 1 --2- 172.21.15.47:0/3797129238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b880cffa0 0x7f9b880ca000 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:51.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.097+0000 7f9b8d609640 1 -- 172.21.15.47:0/3797129238 >> 172.21.15.47:0/3797129238 conn(0x7f9b880d2ec0 msgr2=0x7f9b880b4dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:51.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.097+0000 7f9b8d609640 1 -- 172.21.15.47:0/3797129238 shutdown_connections 2024-04-26T23:20:51.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.097+0000 7f9b8d609640 1 -- 172.21.15.47:0/3797129238 wait complete. 2024-04-26T23:20:51.110 INFO:tasks.ceph:Scrubbing osd.3 2024-04-26T23:20:51.110 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-26T23:20:51.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.214+0000 7f106719e640 1 Processor -- start 2024-04-26T23:20:51.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.214+0000 7f106719e640 1 -- start start 2024-04-26T23:20:51.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.215+0000 7f106719e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f106015f420 0x7f10600d0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:51.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.215+0000 7f106719e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f10600d3b60 0x7f10600b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:51.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.215+0000 7f106719e640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10600d0d50 con 0x7f10600b3d90 2024-04-26T23:20:51.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.215+0000 7f106719e640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f10600d0a70 con 0x7f10600d3b60 2024-04-26T23:20:51.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.215+0000 7f106719e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f10600b1bf0 con 0x7f106015f420 2024-04-26T23:20:51.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.215+0000 7f106699d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f10600d3b60 0x7f10600b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:51.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.215+0000 7f106699d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f10600d3b60 0x7f10600b26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48196/0 (socket says 172.21.15.47:48196) 2024-04-26T23:20:51.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.215+0000 7f106599b640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f10600b3d90 0x7f10600d3450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:52948/0 (socket says 172.21.15.47:52948) 2024-04-26T23:20:51.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.215+0000 7f106699d640 1 -- 172.21.15.47:0/2846016242 learned_addr learned my addr 172.21.15.47:0/2846016242 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:51.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.215+0000 7f106619c640 1 --2- 172.21.15.47:0/2846016242 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f106015f420 0x7f10600d0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:51.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.215+0000 7f106699d640 1 -- 172.21.15.47:0/2846016242 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f106015f420 msgr2=0x7f10600d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:51.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.215+0000 7f106699d640 1 --2- 172.21.15.47:0/2846016242 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f106015f420 0x7f10600d0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:51.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.215+0000 7f106699d640 1 -- 172.21.15.47:0/2846016242 >> v1:172.21.15.47:6789/0 conn(0x7f10600b3d90 legacy=0x7f10600d3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:51.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.215+0000 7f106699d640 1 -- 172.21.15.47:0/2846016242 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10600b1ed0 con 0x7f10600d3b60 2024-04-26T23:20:51.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.215+0000 7f106519a640 1 -- 172.21.15.47:0/2846016242 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3692756501 0 0) 0x7f10600d0d50 con 0x7f10600b3d90 2024-04-26T23:20:51.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.216+0000 7f106699d640 1 --2- 172.21.15.47:0/2846016242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10600d3b60 0x7f10600b26d0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f1058009e50 tx=0x7f105802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=396f31e480ae601 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:51.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.216+0000 7f106519a640 1 -- 172.21.15.47:0/2846016242 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1058036070 con 0x7f10600d3b60 2024-04-26T23:20:51.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.216+0000 7f106519a640 1 -- 172.21.15.47:0/2846016242 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1058031600 con 0x7f10600d3b60 2024-04-26T23:20:51.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.216+0000 7f106519a640 1 -- 172.21.15.47:0/2846016242 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10580319d0 con 0x7f10600d3b60 2024-04-26T23:20:51.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.217+0000 7f106719e640 1 -- 172.21.15.47:0/2846016242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10600d3b60 msgr2=0x7f10600b26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:51.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.217+0000 7f106719e640 1 --2- 172.21.15.47:0/2846016242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10600d3b60 0x7f10600b26d0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f1058009e50 tx=0x7f105802a680 comp rx=0 tx=0).stop 2024-04-26T23:20:51.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.217+0000 7f106719e640 1 -- 172.21.15.47:0/2846016242 shutdown_connections 2024-04-26T23:20:51.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.217+0000 7f106719e640 1 --2- 172.21.15.47:0/2846016242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10600d3b60 0x7f10600b26d0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:51.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.217+0000 7f106719e640 1 --2- 172.21.15.47:0/2846016242 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f106015f420 0x7f10600d0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:51.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.217+0000 7f106719e640 1 -- 172.21.15.47:0/2846016242 >> 172.21.15.47:0/2846016242 conn(0x7f106009e1d0 msgr2=0x7f10600d4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:51.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.217+0000 7f106719e640 1 -- 172.21.15.47:0/2846016242 shutdown_connections 2024-04-26T23:20:51.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.217+0000 7f106719e640 1 -- 172.21.15.47:0/2846016242 wait complete. 2024-04-26T23:20:51.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.218+0000 7f106719e640 1 Processor -- start 2024-04-26T23:20:51.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.218+0000 7f106719e640 1 -- start start 2024-04-26T23:20:51.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.219+0000 7f106719e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10600b3d90 0x7f1060151560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:51.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.219+0000 7f106719e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10600d3b60 0x7f10600c88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:51.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.219+0000 7f106719e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f106015f420 0x7f106015dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:51.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.219+0000 7f106719e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f10600b2c10 con 0x7f10600d3b60 2024-04-26T23:20:51.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.219+0000 7f106719e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f10600d2e50 con 0x7f106015f420 2024-04-26T23:20:51.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.219+0000 7f106719e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f10600b2d90 con 0x7f10600b3d90 2024-04-26T23:20:51.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.219+0000 7f106599b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10600d3b60 0x7f10600c88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:51.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.219+0000 7f106599b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10600d3b60 0x7f10600c88d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48204/0 (socket says 172.21.15.47:48204) 2024-04-26T23:20:51.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.219+0000 7f106599b640 1 -- 172.21.15.47:0/375010397 learned_addr learned my addr 172.21.15.47:0/375010397 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:51.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.219+0000 7f106599b640 1 -- 172.21.15.47:0/375010397 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10600b3d90 msgr2=0x7f1060151560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:51.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.219+0000 7f106699d640 1 --2- 172.21.15.47:0/375010397 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f106015f420 0x7f106015dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:51.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.219+0000 7f106619c640 1 --2- 172.21.15.47:0/375010397 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10600b3d90 0x7f1060151560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:51.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.219+0000 7f106599b640 1 --2- 172.21.15.47:0/375010397 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10600b3d90 0x7f1060151560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:51.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.219+0000 7f106599b640 1 -- 172.21.15.47:0/375010397 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f106015f420 msgr2=0x7f106015dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:51.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.219+0000 7f106599b640 1 --2- 172.21.15.47:0/375010397 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f106015f420 0x7f106015dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:51.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.219+0000 7f106599b640 1 -- 172.21.15.47:0/375010397 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10601613e0 con 0x7f10600d3b60 2024-04-26T23:20:51.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.219+0000 7f106699d640 1 --2- 172.21.15.47:0/375010397 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f106015f420 0x7f106015dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:51.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.219+0000 7f106619c640 1 --2- 172.21.15.47:0/375010397 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10600b3d90 0x7f1060151560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:51.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.219+0000 7f106599b640 1 --2- 172.21.15.47:0/375010397 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10600d3b60 0x7f10600c88d0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f105000c4d0 tx=0x7f105000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:51.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.220+0000 7f1056ffd640 1 -- 172.21.15.47:0/375010397 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1050018070 con 0x7f10600d3b60 2024-04-26T23:20:51.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.220+0000 7f1056ffd640 1 -- 172.21.15.47:0/375010397 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1050007500 con 0x7f10600d3b60 2024-04-26T23:20:51.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.220+0000 7f1056ffd640 1 -- 172.21.15.47:0/375010397 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1050004a20 con 0x7f10600d3b60 2024-04-26T23:20:51.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.220+0000 7f106719e640 1 -- 172.21.15.47:0/375010397 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10601616c0 con 0x7f10600d3b60 2024-04-26T23:20:51.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.220+0000 7f106719e640 1 -- 172.21.15.47:0/375010397 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1060161c80 con 0x7f10600d3b60 2024-04-26T23:20:51.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.220+0000 7f106719e640 1 -- 172.21.15.47:0/375010397 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1060069370 con 0x7f10600d3b60 2024-04-26T23:20:51.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.221+0000 7f1056ffd640 1 -- 172.21.15.47:0/375010397 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f105001b070 con 0x7f10600d3b60 2024-04-26T23:20:51.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.222+0000 7f1056ffd640 1 --2- 172.21.15.47:0/375010397 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f104406c920 0x7f104406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:51.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.222+0000 7f1056ffd640 1 -- 172.21.15.47:0/375010397 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5335+0+0 (secure 0 0 0) 0x7f105008b2b0 con 0x7f10600d3b60 2024-04-26T23:20:51.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.222+0000 7f1056ffd640 1 --2- 172.21.15.47:0/375010397 >> [v2:172.21.15.47:6816/2791411683,v1:172.21.15.47:6818/2791411683] conn(0x7f1044072dc0 0x7f1044075200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:51.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.222+0000 7f1056ffd640 1 -- 172.21.15.47:0/375010397 --> [v2:172.21.15.47:6816/2791411683,v1:172.21.15.47:6818/2791411683] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f10440758d0 con 0x7f1044072dc0 2024-04-26T23:20:51.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.222+0000 7f1056ffd640 1 -- 172.21.15.47:0/375010397 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f105008b660 con 0x7f10600d3b60 2024-04-26T23:20:51.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.222+0000 7f106619c640 1 --2- 172.21.15.47:0/375010397 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f104406c920 0x7f104406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:51.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.222+0000 7f106699d640 1 --2- 172.21.15.47:0/375010397 >> [v2:172.21.15.47:6816/2791411683,v1:172.21.15.47:6818/2791411683] conn(0x7f1044072dc0 0x7f1044075200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:51.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.222+0000 7f106699d640 1 --2- 172.21.15.47:0/375010397 >> [v2:172.21.15.47:6816/2791411683,v1:172.21.15.47:6818/2791411683] conn(0x7f1044072dc0 0x7f1044075200 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:51.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.222+0000 7f106619c640 1 --2- 172.21.15.47:0/375010397 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f104406c920 0x7f104406ede0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f105c006fd0 tx=0x7f105c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:51.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.223+0000 7f1056ffd640 1 -- 172.21.15.47:0/375010397 <== osd.3 v2:172.21.15.47:6816/2791411683 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f10440758d0 con 0x7f1044072dc0 2024-04-26T23:20:51.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.236+0000 7f106719e640 1 -- 172.21.15.47:0/375010397 --> [v2:172.21.15.47:6816/2791411683,v1:172.21.15.47:6818/2791411683] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f10600d3160 con 0x7f1044072dc0 2024-04-26T23:20:51.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.236+0000 7f1056ffd640 1 -- 172.21.15.47:0/375010397 <== osd.3 v2:172.21.15.47:6816/2791411683 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f10600d3160 con 0x7f1044072dc0 2024-04-26T23:20:51.239 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-26T23:20:51.239 INFO:teuthology.orchestra.run.smithi047.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T23:20:51.239 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-26T23:20:51.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.237+0000 7f106719e640 1 -- 172.21.15.47:0/375010397 >> [v2:172.21.15.47:6816/2791411683,v1:172.21.15.47:6818/2791411683] conn(0x7f1044072dc0 msgr2=0x7f1044075200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:51.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.237+0000 7f106719e640 1 --2- 172.21.15.47:0/375010397 >> [v2:172.21.15.47:6816/2791411683,v1:172.21.15.47:6818/2791411683] conn(0x7f1044072dc0 0x7f1044075200 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:51.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.237+0000 7f106719e640 1 -- 172.21.15.47:0/375010397 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f104406c920 msgr2=0x7f104406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:51.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.237+0000 7f106719e640 1 --2- 172.21.15.47:0/375010397 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f104406c920 0x7f104406ede0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f105c006fd0 tx=0x7f105c008040 comp rx=0 tx=0).stop 2024-04-26T23:20:51.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.237+0000 7f106719e640 1 -- 172.21.15.47:0/375010397 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10600d3b60 msgr2=0x7f10600c88d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:51.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.237+0000 7f106719e640 1 --2- 172.21.15.47:0/375010397 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10600d3b60 0x7f10600c88d0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f105000c4d0 tx=0x7f105000c990 comp rx=0 tx=0).stop 2024-04-26T23:20:51.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.237+0000 7f106699d640 1 -- 172.21.15.47:0/375010397 reap_dead start 2024-04-26T23:20:51.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.237+0000 7f106719e640 1 -- 172.21.15.47:0/375010397 shutdown_connections 2024-04-26T23:20:51.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.237+0000 7f106719e640 1 -- 172.21.15.47:0/375010397 >> 172.21.15.47:0/375010397 conn(0x7f106009e1d0 msgr2=0x7f10600d3450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:51.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.237+0000 7f106719e640 1 -- 172.21.15.47:0/375010397 shutdown_connections 2024-04-26T23:20:51.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.237+0000 7f106719e640 1 -- 172.21.15.47:0/375010397 wait complete. 2024-04-26T23:20:51.249 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-26T23:20:51.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.348+0000 7f3cf0b2b640 1 Processor -- start 2024-04-26T23:20:51.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.348+0000 7f3cf0b2b640 1 -- start start 2024-04-26T23:20:51.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.349+0000 7f3cf0b2b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cec0d3bc0 0x7f3cec0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:51.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.349+0000 7f3cf0b2b640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cec0b7e50 con 0x7f3cec162880 2024-04-26T23:20:51.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.349+0000 7f3cf0b2b640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cec0d1800 con 0x7f3cec0b5f20 2024-04-26T23:20:51.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.349+0000 7f3cf0b2b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3cec0d0580 con 0x7f3cec0d3bc0 2024-04-26T23:20:51.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.349+0000 7f3cea59c640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f3cec162880 0x7f3cec0d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:52954/0 (socket says 172.21.15.47:52954) 2024-04-26T23:20:51.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.349+0000 7f3cea59c640 1 -- 172.21.15.47:0/1737368359 learned_addr learned my addr 172.21.15.47:0/1737368359 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:51.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.349+0000 7f3ce9d9b640 1 --2- 172.21.15.47:0/1737368359 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cec0d3bc0 0x7f3cec0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:51.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.349+0000 7f3ce959a640 1 -- 172.21.15.47:0/1737368359 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1796461329 0 0) 0x7f3cec0b7e50 con 0x7f3cec162880 2024-04-26T23:20:51.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.349+0000 7f3ce959a640 1 -- 172.21.15.47:0/1737368359 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3cd8003520 con 0x7f3cec162880 2024-04-26T23:20:51.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.349+0000 7f3ce959a640 1 -- 172.21.15.47:0/1737368359 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1986115988 0 0) 0x7f3cdc001490 con 0x7f3cec0b5f20 2024-04-26T23:20:51.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.349+0000 7f3ce959a640 1 -- 172.21.15.47:0/1737368359 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3cec0b7e50 con 0x7f3cec0b5f20 2024-04-26T23:20:51.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.350+0000 7f3ce9d9b640 1 -- 172.21.15.47:0/1737368359 >> v1:172.21.15.203:6789/0 conn(0x7f3cec0b5f20 legacy=0x7f3cec0b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:51.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.350+0000 7f3ce9d9b640 1 -- 172.21.15.47:0/1737368359 >> v1:172.21.15.47:6789/0 conn(0x7f3cec162880 legacy=0x7f3cec0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:51.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.350+0000 7f3ce9d9b640 1 -- 172.21.15.47:0/1737368359 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cec0d07d0 con 0x7f3cec0d3bc0 2024-04-26T23:20:51.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.350+0000 7f3ce9d9b640 1 --2- 172.21.15.47:0/1737368359 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cec0d3bc0 0x7f3cec0d2cb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3cd40097d0 tx=0x7f3cd402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6577ab73918cbe5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:51.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.351+0000 7f3ce959a640 1 -- 172.21.15.47:0/1737368359 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd4037070 con 0x7f3cec0d3bc0 2024-04-26T23:20:51.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.351+0000 7f3ce959a640 1 -- 172.21.15.47:0/1737368359 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cd4032440 con 0x7f3cec0d3bc0 2024-04-26T23:20:51.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.351+0000 7f3ce959a640 1 -- 172.21.15.47:0/1737368359 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd40327f0 con 0x7f3cec0d3bc0 2024-04-26T23:20:51.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.351+0000 7f3cf0b2b640 1 -- 172.21.15.47:0/1737368359 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cec0d3bc0 msgr2=0x7f3cec0d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:51.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.351+0000 7f3cf0b2b640 1 --2- 172.21.15.47:0/1737368359 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cec0d3bc0 0x7f3cec0d2cb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3cd40097d0 tx=0x7f3cd402a210 comp rx=0 tx=0).stop 2024-04-26T23:20:51.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.351+0000 7f3cf0b2b640 1 -- 172.21.15.47:0/1737368359 shutdown_connections 2024-04-26T23:20:51.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.351+0000 7f3cf0b2b640 1 --2- 172.21.15.47:0/1737368359 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cec0d3bc0 0x7f3cec0d2cb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:51.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.351+0000 7f3cf0b2b640 1 -- 172.21.15.47:0/1737368359 >> 172.21.15.47:0/1737368359 conn(0x7f3cec0d4f60 msgr2=0x7f3cec0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:51.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.351+0000 7f3cf0b2b640 1 -- 172.21.15.47:0/1737368359 shutdown_connections 2024-04-26T23:20:51.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.351+0000 7f3cf0b2b640 1 -- 172.21.15.47:0/1737368359 wait complete. 2024-04-26T23:20:51.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.353+0000 7f3cf0b2b640 1 Processor -- start 2024-04-26T23:20:51.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.353+0000 7f3cf0b2b640 1 -- start start 2024-04-26T23:20:51.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.353+0000 7f3cf0b2b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cec0b5f20 0x7f3cec0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:51.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.353+0000 7f3cf0b2b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cec0d3bc0 0x7f3cec0b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:51.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.353+0000 7f3cf0b2b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3cec162880 0x7f3cec1f4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:51.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.353+0000 7f3cf0b2b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3cec162c60 con 0x7f3cec0d3bc0 2024-04-26T23:20:51.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.353+0000 7f3cf0b2b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3cec0d0d90 con 0x7f3cec162880 2024-04-26T23:20:51.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.353+0000 7f3cf0b2b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3cec162de0 con 0x7f3cec0b5f20 2024-04-26T23:20:51.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.353+0000 7f3ce9d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cec0d3bc0 0x7f3cec0b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:51.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.353+0000 7f3ce9d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cec0d3bc0 0x7f3cec0b48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48210/0 (socket says 172.21.15.47:48210) 2024-04-26T23:20:51.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.353+0000 7f3ce9d9b640 1 -- 172.21.15.47:0/2876707520 learned_addr learned my addr 172.21.15.47:0/2876707520 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:51.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.354+0000 7f3cea59c640 1 --2- 172.21.15.47:0/2876707520 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cec0b5f20 0x7f3cec0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:51.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.354+0000 7f3cead9d640 1 --2- 172.21.15.47:0/2876707520 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3cec162880 0x7f3cec1f4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:51.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.354+0000 7f3ce9d9b640 1 -- 172.21.15.47:0/2876707520 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cec0b5f20 msgr2=0x7f3cec0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:51.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.354+0000 7f3ce9d9b640 1 --2- 172.21.15.47:0/2876707520 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cec0b5f20 0x7f3cec0ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:51.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.354+0000 7f3ce9d9b640 1 -- 172.21.15.47:0/2876707520 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3cec162880 msgr2=0x7f3cec1f4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:51.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.354+0000 7f3ce9d9b640 1 --2- 172.21.15.47:0/2876707520 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3cec162880 0x7f3cec1f4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:51.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.354+0000 7f3ce9d9b640 1 -- 172.21.15.47:0/2876707520 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cec0cd5d0 con 0x7f3cec0d3bc0 2024-04-26T23:20:51.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.354+0000 7f3ce9d9b640 1 --2- 172.21.15.47:0/2876707520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cec0d3bc0 0x7f3cec0b48b0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4035040 tx=0x7f3cd40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:51.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.354+0000 7f3ce2ffd640 1 -- 172.21.15.47:0/2876707520 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd4037040 con 0x7f3cec0d3bc0 2024-04-26T23:20:51.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.354+0000 7f3ce2ffd640 1 -- 172.21.15.47:0/2876707520 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cd40044c0 con 0x7f3cec0d3bc0 2024-04-26T23:20:51.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.354+0000 7f3ce2ffd640 1 -- 172.21.15.47:0/2876707520 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd40325f0 con 0x7f3cec0d3bc0 2024-04-26T23:20:51.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.354+0000 7f3cf0b2b640 1 -- 172.21.15.47:0/2876707520 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cec0cd8b0 con 0x7f3cec0d3bc0 2024-04-26T23:20:51.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.354+0000 7f3cf0b2b640 1 -- 172.21.15.47:0/2876707520 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cec0cde70 con 0x7f3cec0d3bc0 2024-04-26T23:20:51.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.355+0000 7f3cf0b2b640 1 -- 172.21.15.47:0/2876707520 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cec0ce0b0 con 0x7f3cec0d3bc0 2024-04-26T23:20:51.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.356+0000 7f3ce2ffd640 1 -- 172.21.15.47:0/2876707520 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3cd4044020 con 0x7f3cec0d3bc0 2024-04-26T23:20:51.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.356+0000 7f3ce2ffd640 1 --2- 172.21.15.47:0/2876707520 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3ccc06c8f0 0x7f3ccc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:51.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.356+0000 7f3ce2ffd640 1 -- 172.21.15.47:0/2876707520 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5335+0+0 (secure 0 0 0) 0x7f3cd40ab930 con 0x7f3cec0d3bc0 2024-04-26T23:20:51.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.357+0000 7f3cea59c640 1 --2- 172.21.15.47:0/2876707520 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3ccc06c8f0 0x7f3ccc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:51.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.357+0000 7f3cea59c640 1 --2- 172.21.15.47:0/2876707520 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3ccc06c8f0 0x7f3ccc06edb0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4000f60 tx=0x7f3ce4007d70 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:51.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.361+0000 7f3ce2ffd640 1 -- 172.21.15.47:0/2876707520 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3cd404b020 con 0x7f3cec0d3bc0 2024-04-26T23:20:51.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.551+0000 7f3cf0b2b640 1 -- 172.21.15.47:0/2876707520 --> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f3cec0694f0 con 0x7f3ccc06c8f0 2024-04-26T23:20:51.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.552+0000 7f3ce2ffd640 1 -- 172.21.15.47:0/2876707520 <== mgr.4099 v2:172.21.15.203:6832/992162732 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f3cec0694f0 con 0x7f3ccc06c8f0 2024-04-26T23:20:51.554 INFO:teuthology.orchestra.run.smithi047.stderr:instructed osd(s) 3 to deep-scrub 2024-04-26T23:20:51.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.554+0000 7f3cf0b2b640 1 -- 172.21.15.47:0/2876707520 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3ccc06c8f0 msgr2=0x7f3ccc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:51.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.554+0000 7f3cf0b2b640 1 --2- 172.21.15.47:0/2876707520 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3ccc06c8f0 0x7f3ccc06edb0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4000f60 tx=0x7f3ce4007d70 comp rx=0 tx=0).stop 2024-04-26T23:20:51.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.554+0000 7f3cf0b2b640 1 -- 172.21.15.47:0/2876707520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cec0d3bc0 msgr2=0x7f3cec0b48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:51.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.554+0000 7f3cf0b2b640 1 --2- 172.21.15.47:0/2876707520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cec0d3bc0 0x7f3cec0b48b0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4035040 tx=0x7f3cd40046c0 comp rx=0 tx=0).stop 2024-04-26T23:20:51.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.554+0000 7f3cf0b2b640 1 -- 172.21.15.47:0/2876707520 shutdown_connections 2024-04-26T23:20:51.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.554+0000 7f3cf0b2b640 1 --2- 172.21.15.47:0/2876707520 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f3ccc06c8f0 0x7f3ccc06edb0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:51.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.554+0000 7f3cf0b2b640 1 --2- 172.21.15.47:0/2876707520 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3cec162880 0x7f3cec1f4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:51.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.554+0000 7f3cf0b2b640 1 --2- 172.21.15.47:0/2876707520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cec0d3bc0 0x7f3cec0b48b0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:51.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.554+0000 7f3cf0b2b640 1 --2- 172.21.15.47:0/2876707520 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cec0b5f20 0x7f3cec0ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:51.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.554+0000 7f3cf0b2b640 1 -- 172.21.15.47:0/2876707520 >> 172.21.15.47:0/2876707520 conn(0x7f3cec0d4f60 msgr2=0x7f3cec0d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:51.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.554+0000 7f3cf0b2b640 1 -- 172.21.15.47:0/2876707520 shutdown_connections 2024-04-26T23:20:51.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.554+0000 7f3cf0b2b640 1 -- 172.21.15.47:0/2876707520 wait complete. 2024-04-26T23:20:51.567 INFO:tasks.ceph:Scrubbing osd.4 2024-04-26T23:20:51.567 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-26T23:20:51.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.669+0000 7fde915f2640 1 Processor -- start 2024-04-26T23:20:51.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.669+0000 7fde915f2640 1 -- start start 2024-04-26T23:20:51.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.670+0000 7fde915f2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde8c0d3480 0x7fde8c169e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:51.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.670+0000 7fde915f2640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde8c06dcc0 con 0x7fde8c06ee20 2024-04-26T23:20:51.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.670+0000 7fde915f2640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde8c06dfa0 con 0x7fde8c15f160 2024-04-26T23:20:51.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.670+0000 7fde915f2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fde8c06da60 con 0x7fde8c0d3480 2024-04-26T23:20:51.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.670+0000 7fde8a7fc640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fde8c06ee20 0x7fde8c0d2d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:52968/0 (socket says 172.21.15.47:52968) 2024-04-26T23:20:51.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.670+0000 7fde8a7fc640 1 -- 172.21.15.47:0/2906764593 learned_addr learned my addr 172.21.15.47:0/2906764593 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:51.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.670+0000 7fde89ffb640 1 -- 172.21.15.47:0/2906764593 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 289590417 0 0) 0x7fde8c06dcc0 con 0x7fde8c06ee20 2024-04-26T23:20:51.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.670+0000 7fde89ffb640 1 -- 172.21.15.47:0/2906764593 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde74003520 con 0x7fde8c06ee20 2024-04-26T23:20:51.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.670+0000 7fde8b7fe640 1 --2- 172.21.15.47:0/2906764593 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde8c0d3480 0x7fde8c169e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:51.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.671+0000 7fde89ffb640 1 -- 172.21.15.47:0/2906764593 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3961780114 0 0) 0x7fde74003520 con 0x7fde8c06ee20 2024-04-26T23:20:51.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.671+0000 7fde89ffb640 1 -- 172.21.15.47:0/2906764593 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde8c0d3480 msgr2=0x7fde8c169e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:51.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.671+0000 7fde89ffb640 1 --2- 172.21.15.47:0/2906764593 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde8c0d3480 0x7fde8c169e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:51.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.671+0000 7fde89ffb640 1 -- 172.21.15.47:0/2906764593 >> v1:172.21.15.203:6789/0 conn(0x7fde8c15f160 legacy=0x7fde8c06d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:51.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.671+0000 7fde89ffb640 1 -- 172.21.15.47:0/2906764593 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde8c06e280 con 0x7fde8c06ee20 2024-04-26T23:20:51.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.671+0000 7fde89ffb640 1 -- 172.21.15.47:0/2906764593 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7fde80001670 con 0x7fde8c06ee20 2024-04-26T23:20:51.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.671+0000 7fde89ffb640 1 -- 172.21.15.47:0/2906764593 >> v1:172.21.15.47:6789/0 conn(0x7fde8c06ee20 legacy=0x7fde8c0d2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:51.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.671+0000 7fde89ffb640 1 --2- 172.21.15.47:0/2906764593 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde8c0d3480 0x7fde740039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:51.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.671+0000 7fde89ffb640 1 --2- 172.21.15.47:0/2906764593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde74003f60 0x7fde74004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:51.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.671+0000 7fde89ffb640 1 --2- 172.21.15.47:0/2906764593 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde74004d40 0x7fde740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:51.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.671+0000 7fde89ffb640 1 -- 172.21.15.47:0/2906764593 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fde8c06da60 con 0x7fde74003f60 2024-04-26T23:20:51.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.671+0000 7fde89ffb640 1 -- 172.21.15.47:0/2906764593 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fde7400bb30 con 0x7fde74004d40 2024-04-26T23:20:51.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.671+0000 7fde89ffb640 1 -- 172.21.15.47:0/2906764593 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fde7400bd30 con 0x7fde8c0d3480 2024-04-26T23:20:51.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.671+0000 7fde8a7fc640 1 --2- 172.21.15.47:0/2906764593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde74003f60 0x7fde74004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:51.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.671+0000 7fde8affd640 1 --2- 172.21.15.47:0/2906764593 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde8c0d3480 0x7fde740039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:51.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.671+0000 7fde8b7fe640 1 --2- 172.21.15.47:0/2906764593 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde74004d40 0x7fde740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:51.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.671+0000 7fde8a7fc640 1 -- 172.21.15.47:0/2906764593 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde8c0d3480 msgr2=0x7fde740039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:51.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.671+0000 7fde8a7fc640 1 --2- 172.21.15.47:0/2906764593 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde8c0d3480 0x7fde740039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:51.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.671+0000 7fde8a7fc640 1 -- 172.21.15.47:0/2906764593 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde74004d40 msgr2=0x7fde740051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:51.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.671+0000 7fde8a7fc640 1 --2- 172.21.15.47:0/2906764593 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde74004d40 0x7fde740051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:51.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.671+0000 7fde8a7fc640 1 -- 172.21.15.47:0/2906764593 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde74003520 con 0x7fde74003f60 2024-04-26T23:20:51.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.671+0000 7fde8b7fe640 1 --2- 172.21.15.47:0/2906764593 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde74004d40 0x7fde740051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:20:51.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.672+0000 7fde8a7fc640 1 --2- 172.21.15.47:0/2906764593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde74003f60 0x7fde74004800 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fde8000c240 tx=0x7fde74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef96d71bf1b149fd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:51.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.672+0000 7fde89ffb640 1 -- 172.21.15.47:0/2906764593 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde80007d10 con 0x7fde74003f60 2024-04-26T23:20:51.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.672+0000 7fde89ffb640 1 -- 172.21.15.47:0/2906764593 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde80005ad0 con 0x7fde74003f60 2024-04-26T23:20:51.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.672+0000 7fde89ffb640 1 -- 172.21.15.47:0/2906764593 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde80007070 con 0x7fde74003f60 2024-04-26T23:20:51.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.673+0000 7fde915f2640 1 -- 172.21.15.47:0/2906764593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde74003f60 msgr2=0x7fde74004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:51.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.673+0000 7fde915f2640 1 --2- 172.21.15.47:0/2906764593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde74003f60 0x7fde74004800 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fde8000c240 tx=0x7fde74005830 comp rx=0 tx=0).stop 2024-04-26T23:20:51.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.673+0000 7fde8b7fe640 1 -- 172.21.15.47:0/2906764593 reap_dead start 2024-04-26T23:20:51.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.673+0000 7fde915f2640 1 -- 172.21.15.47:0/2906764593 shutdown_connections 2024-04-26T23:20:51.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.673+0000 7fde915f2640 1 -- 172.21.15.47:0/2906764593 >> 172.21.15.47:0/2906764593 conn(0x7fde8c09ba50 msgr2=0x7fde8c09be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:51.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.673+0000 7fde915f2640 1 -- 172.21.15.47:0/2906764593 shutdown_connections 2024-04-26T23:20:51.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.673+0000 7fde915f2640 1 -- 172.21.15.47:0/2906764593 wait complete. 2024-04-26T23:20:51.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.674+0000 7fde915f2640 1 Processor -- start 2024-04-26T23:20:51.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.674+0000 7fde915f2640 1 -- start start 2024-04-26T23:20:51.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.675+0000 7fde915f2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde8c0c8a90 0x7fde8c0d0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:51.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.675+0000 7fde915f2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde8c0c8e70 0x7fde8c1472c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:51.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.675+0000 7fde915f2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde8c0cc6d0 0x7fde8c0ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:51.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.675+0000 7fde915f2640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fde8c15f540 con 0x7fde8c0c8a90 2024-04-26T23:20:51.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.675+0000 7fde915f2640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fde8c06e840 con 0x7fde8c0c8e70 2024-04-26T23:20:51.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.675+0000 7fde915f2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fde8c15f6c0 con 0x7fde8c0cc6d0 2024-04-26T23:20:51.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.675+0000 7fde8affd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde8c0c8a90 0x7fde8c0d0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:51.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.675+0000 7fde8affd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde8c0c8a90 0x7fde8c0d0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48242/0 (socket says 172.21.15.47:48242) 2024-04-26T23:20:51.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.675+0000 7fde8affd640 1 -- 172.21.15.47:0/2802998084 learned_addr learned my addr 172.21.15.47:0/2802998084 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:51.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.675+0000 7fde8b7fe640 1 --2- 172.21.15.47:0/2802998084 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde8c0cc6d0 0x7fde8c0ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:51.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.675+0000 7fde8a7fc640 1 --2- 172.21.15.47:0/2802998084 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde8c0c8e70 0x7fde8c1472c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:51.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.675+0000 7fde8affd640 1 -- 172.21.15.47:0/2802998084 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde8c0cc6d0 msgr2=0x7fde8c0ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:51.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.675+0000 7fde8affd640 1 --2- 172.21.15.47:0/2802998084 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde8c0cc6d0 0x7fde8c0ccb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:51.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.675+0000 7fde8affd640 1 -- 172.21.15.47:0/2802998084 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde8c0c8e70 msgr2=0x7fde8c1472c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:51.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.675+0000 7fde8affd640 1 --2- 172.21.15.47:0/2802998084 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde8c0c8e70 0x7fde8c1472c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:51.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.675+0000 7fde8affd640 1 -- 172.21.15.47:0/2802998084 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde8c0c92d0 con 0x7fde8c0c8a90 2024-04-26T23:20:51.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.675+0000 7fde8b7fe640 1 --2- 172.21.15.47:0/2802998084 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde8c0cc6d0 0x7fde8c0ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:20:51.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.675+0000 7fde8affd640 1 --2- 172.21.15.47:0/2802998084 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde8c0c8a90 0x7fde8c0d0030 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fde8400f820 tx=0x7fde8400fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:51.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.675+0000 7fde737fe640 1 -- 172.21.15.47:0/2802998084 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde840050d0 con 0x7fde8c0c8a90 2024-04-26T23:20:51.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.675+0000 7fde737fe640 1 -- 172.21.15.47:0/2802998084 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde84018070 con 0x7fde8c0c8a90 2024-04-26T23:20:51.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.676+0000 7fde915f2640 1 -- 172.21.15.47:0/2802998084 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde8c0cb910 con 0x7fde8c0c8a90 2024-04-26T23:20:51.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.676+0000 7fde737fe640 1 -- 172.21.15.47:0/2802998084 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde84005490 con 0x7fde8c0c8a90 2024-04-26T23:20:51.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.676+0000 7fde915f2640 1 -- 172.21.15.47:0/2802998084 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde8c0cbed0 con 0x7fde8c0c8a90 2024-04-26T23:20:51.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.676+0000 7fde915f2640 1 -- 172.21.15.47:0/2802998084 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fde8c0d06d0 con 0x7fde8c0c8a90 2024-04-26T23:20:51.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.677+0000 7fde737fe640 1 -- 172.21.15.47:0/2802998084 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fde8400d730 con 0x7fde8c0c8a90 2024-04-26T23:20:51.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.677+0000 7fde737fe640 1 --2- 172.21.15.47:0/2802998084 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fde7806c8f0 0x7fde7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:51.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.678+0000 7fde737fe640 1 -- 172.21.15.47:0/2802998084 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5335+0+0 (secure 0 0 0) 0x7fde8408b900 con 0x7fde8c0c8a90 2024-04-26T23:20:51.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.678+0000 7fde737fe640 1 --2- 172.21.15.47:0/2802998084 >> [v2:172.21.15.203:6808/441313104,v1:172.21.15.203:6809/441313104] conn(0x7fde78072d90 0x7fde780751d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:51.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.678+0000 7fde737fe640 1 -- 172.21.15.47:0/2802998084 --> [v2:172.21.15.203:6808/441313104,v1:172.21.15.203:6809/441313104] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fde84009e50 con 0x7fde78072d90 2024-04-26T23:20:51.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.678+0000 7fde737fe640 1 -- 172.21.15.47:0/2802998084 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fde840902c0 con 0x7fde8c0c8a90 2024-04-26T23:20:51.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.678+0000 7fde8a7fc640 1 --2- 172.21.15.47:0/2802998084 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fde7806c8f0 0x7fde7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:51.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.678+0000 7fde8b7fe640 1 --2- 172.21.15.47:0/2802998084 >> [v2:172.21.15.203:6808/441313104,v1:172.21.15.203:6809/441313104] conn(0x7fde78072d90 0x7fde780751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:51.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.678+0000 7fde8a7fc640 1 --2- 172.21.15.47:0/2802998084 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fde7806c8f0 0x7fde7806edb0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fde80033000 tx=0x7fde8002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:51.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.678+0000 7fde8b7fe640 1 --2- 172.21.15.47:0/2802998084 >> [v2:172.21.15.203:6808/441313104,v1:172.21.15.203:6809/441313104] conn(0x7fde78072d90 0x7fde780751d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:51.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.679+0000 7fde737fe640 1 -- 172.21.15.47:0/2802998084 <== osd.4 v2:172.21.15.203:6808/441313104 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fde84009e50 con 0x7fde78072d90 2024-04-26T23:20:51.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.692+0000 7fde915f2640 1 -- 172.21.15.47:0/2802998084 --> [v2:172.21.15.203:6808/441313104,v1:172.21.15.203:6809/441313104] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fde8c069600 con 0x7fde78072d90 2024-04-26T23:20:51.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.693+0000 7fde737fe640 1 -- 172.21.15.47:0/2802998084 <== osd.4 v2:172.21.15.203:6808/441313104 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fde8c069600 con 0x7fde78072d90 2024-04-26T23:20:51.695 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-26T23:20:51.695 INFO:teuthology.orchestra.run.smithi047.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T23:20:51.695 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-26T23:20:51.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.693+0000 7fde915f2640 1 -- 172.21.15.47:0/2802998084 >> [v2:172.21.15.203:6808/441313104,v1:172.21.15.203:6809/441313104] conn(0x7fde78072d90 msgr2=0x7fde780751d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:51.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.693+0000 7fde915f2640 1 --2- 172.21.15.47:0/2802998084 >> [v2:172.21.15.203:6808/441313104,v1:172.21.15.203:6809/441313104] conn(0x7fde78072d90 0x7fde780751d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:51.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.693+0000 7fde915f2640 1 -- 172.21.15.47:0/2802998084 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fde7806c8f0 msgr2=0x7fde7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:51.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.693+0000 7fde915f2640 1 --2- 172.21.15.47:0/2802998084 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fde7806c8f0 0x7fde7806edb0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fde80033000 tx=0x7fde8002e000 comp rx=0 tx=0).stop 2024-04-26T23:20:51.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.693+0000 7fde915f2640 1 -- 172.21.15.47:0/2802998084 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde8c0c8a90 msgr2=0x7fde8c0d0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:51.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.693+0000 7fde915f2640 1 --2- 172.21.15.47:0/2802998084 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde8c0c8a90 0x7fde8c0d0030 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fde8400f820 tx=0x7fde8400fce0 comp rx=0 tx=0).stop 2024-04-26T23:20:51.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.693+0000 7fde8b7fe640 1 -- 172.21.15.47:0/2802998084 reap_dead start 2024-04-26T23:20:51.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.693+0000 7fde915f2640 1 -- 172.21.15.47:0/2802998084 shutdown_connections 2024-04-26T23:20:51.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.693+0000 7fde915f2640 1 -- 172.21.15.47:0/2802998084 >> 172.21.15.47:0/2802998084 conn(0x7fde8c09ba50 msgr2=0x7fde8c15bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:51.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.693+0000 7fde915f2640 1 -- 172.21.15.47:0/2802998084 shutdown_connections 2024-04-26T23:20:51.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.693+0000 7fde915f2640 1 -- 172.21.15.47:0/2802998084 wait complete. 2024-04-26T23:20:51.706 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-26T23:20:51.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.806+0000 7ff06f607640 1 Processor -- start 2024-04-26T23:20:51.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.806+0000 7ff06f607640 1 -- start start 2024-04-26T23:20:51.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.807+0000 7ff06f607640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff06815b600 0x7ff068071f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:51.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.807+0000 7ff06f607640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff0680cf5e0 0x7ff0680ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:51.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.807+0000 7ff06f607640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff0680cbbd0 0x7ff0680cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:51.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.807+0000 7ff06f607640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7ff0680cb9b0 con 0x7ff0680cf5e0 2024-04-26T23:20:51.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.807+0000 7ff06f607640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7ff068072450 con 0x7ff06815b600 2024-04-26T23:20:51.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.807+0000 7ff06f607640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7ff06815b9e0 con 0x7ff0680cbbd0 2024-04-26T23:20:51.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.807+0000 7ff06cba2640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff0680cf5e0 0x7ff0680ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:51.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.807+0000 7ff06cba2640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff0680cf5e0 0x7ff0680ce9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48244/0 (socket says 172.21.15.47:48244) 2024-04-26T23:20:51.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.807+0000 7ff06cba2640 1 -- 172.21.15.47:0/3418213557 learned_addr learned my addr 172.21.15.47:0/3418213557 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:51.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.807+0000 7ff06d3a3640 1 --2- 172.21.15.47:0/3418213557 >> v2:172.21.15.203:3300/0 conn(0x7ff06815b600 0x7ff068071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:51.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.807+0000 7ff06dba4640 1 --2- 172.21.15.47:0/3418213557 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff0680cbbd0 0x7ff0680cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:51.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.807+0000 7ff06cba2640 1 -- 172.21.15.47:0/3418213557 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff0680cbbd0 msgr2=0x7ff0680cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:51.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.807+0000 7ff06cba2640 1 --2- 172.21.15.47:0/3418213557 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff0680cbbd0 0x7ff0680cef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:51.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.807+0000 7ff06cba2640 1 -- 172.21.15.47:0/3418213557 >> v2:172.21.15.203:3300/0 conn(0x7ff06815b600 msgr2=0x7ff068071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:51.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.807+0000 7ff06cba2640 1 --2- 172.21.15.47:0/3418213557 >> v2:172.21.15.203:3300/0 conn(0x7ff06815b600 0x7ff068071f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:51.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.807+0000 7ff06cba2640 1 -- 172.21.15.47:0/3418213557 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0680d0740 con 0x7ff0680cf5e0 2024-04-26T23:20:51.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.808+0000 7ff06cba2640 1 --2- 172.21.15.47:0/3418213557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0680cf5e0 0x7ff0680ce9d0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ff068071ad0 tx=0x7ff05802b2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2af430495f81566 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:51.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.808+0000 7ff05ffff640 1 -- 172.21.15.47:0/3418213557 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff058038070 con 0x7ff0680cf5e0 2024-04-26T23:20:51.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.808+0000 7ff05ffff640 1 -- 172.21.15.47:0/3418213557 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff058033640 con 0x7ff0680cf5e0 2024-04-26T23:20:51.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.808+0000 7ff05ffff640 1 -- 172.21.15.47:0/3418213557 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff058033a10 con 0x7ff0680cf5e0 2024-04-26T23:20:51.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.809+0000 7ff06f607640 1 -- 172.21.15.47:0/3418213557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0680cf5e0 msgr2=0x7ff0680ce9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:51.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.809+0000 7ff06f607640 1 --2- 172.21.15.47:0/3418213557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0680cf5e0 0x7ff0680ce9d0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ff068071ad0 tx=0x7ff05802b2f0 comp rx=0 tx=0).stop 2024-04-26T23:20:51.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.809+0000 7ff06f607640 1 -- 172.21.15.47:0/3418213557 shutdown_connections 2024-04-26T23:20:51.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.809+0000 7ff06f607640 1 --2- 172.21.15.47:0/3418213557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0680cf5e0 0x7ff0680ce9d0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:51.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.809+0000 7ff06f607640 1 --2- 172.21.15.47:0/3418213557 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff0680cbbd0 0x7ff0680cef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:51.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.809+0000 7ff06f607640 1 --2- 172.21.15.47:0/3418213557 >> v2:172.21.15.203:3300/0 conn(0x7ff06815b600 0x7ff068071f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:51.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.809+0000 7ff06f607640 1 -- 172.21.15.47:0/3418213557 >> 172.21.15.47:0/3418213557 conn(0x7ff0680d0be0 msgr2=0x7ff0680d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:51.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.809+0000 7ff06f607640 1 -- 172.21.15.47:0/3418213557 shutdown_connections 2024-04-26T23:20:51.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.809+0000 7ff06f607640 1 -- 172.21.15.47:0/3418213557 wait complete. 2024-04-26T23:20:51.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.810+0000 7ff06f607640 1 Processor -- start 2024-04-26T23:20:51.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.810+0000 7ff06f607640 1 -- start start 2024-04-26T23:20:51.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.811+0000 7ff06f607640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0680cbbd0 0x7ff0681f3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:51.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.811+0000 7ff06f607640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0680cf5e0 0x7ff06815a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:51.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.811+0000 7ff06f607640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff06815b600 0x7ff06815bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:51.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.811+0000 7ff06f607640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff0680d02f0 con 0x7ff0680cf5e0 2024-04-26T23:20:51.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.811+0000 7ff06f607640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff0680cb640 con 0x7ff0680cbbd0 2024-04-26T23:20:51.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.811+0000 7ff06f607640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7ff0680d0470 con 0x7ff06815b600 2024-04-26T23:20:51.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.811+0000 7ff06cba2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0680cf5e0 0x7ff06815a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:51.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.811+0000 7ff06cba2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0680cf5e0 0x7ff06815a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48260/0 (socket says 172.21.15.47:48260) 2024-04-26T23:20:51.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.811+0000 7ff06cba2640 1 -- 172.21.15.47:0/2834768732 learned_addr learned my addr 172.21.15.47:0/2834768732 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:51.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.811+0000 7ff06dba4640 1 --2- 172.21.15.47:0/2834768732 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff06815b600 0x7ff06815bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:51.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.811+0000 7ff06cba2640 1 -- 172.21.15.47:0/2834768732 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff06815b600 msgr2=0x7ff06815bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:51.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.811+0000 7ff06cba2640 1 --2- 172.21.15.47:0/2834768732 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff06815b600 0x7ff06815bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:51.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.811+0000 7ff06d3a3640 1 --2- 172.21.15.47:0/2834768732 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0680cbbd0 0x7ff0681f3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:51.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.811+0000 7ff06cba2640 1 -- 172.21.15.47:0/2834768732 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0680cbbd0 msgr2=0x7ff0681f3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:51.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.811+0000 7ff06cba2640 1 --2- 172.21.15.47:0/2834768732 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0680cbbd0 0x7ff0681f3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:51.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.811+0000 7ff06cba2640 1 -- 172.21.15.47:0/2834768732 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff06815d7e0 con 0x7ff0680cf5e0 2024-04-26T23:20:51.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.811+0000 7ff06cba2640 1 --2- 172.21.15.47:0/2834768732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0680cf5e0 0x7ff06815a290 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ff058000c00 tx=0x7ff0580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:51.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.811+0000 7ff05dffb640 1 -- 172.21.15.47:0/2834768732 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff058043070 con 0x7ff0680cf5e0 2024-04-26T23:20:51.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.811+0000 7ff05dffb640 1 -- 172.21.15.47:0/2834768732 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff058031770 con 0x7ff0680cf5e0 2024-04-26T23:20:51.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.811+0000 7ff05dffb640 1 -- 172.21.15.47:0/2834768732 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff058038040 con 0x7ff0680cf5e0 2024-04-26T23:20:51.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.811+0000 7ff06f607640 1 -- 172.21.15.47:0/2834768732 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff06815dac0 con 0x7ff0680cf5e0 2024-04-26T23:20:51.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.812+0000 7ff06f607640 1 -- 172.21.15.47:0/2834768732 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff06815e080 con 0x7ff0680cf5e0 2024-04-26T23:20:51.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.813+0000 7ff06f607640 1 -- 172.21.15.47:0/2834768732 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0680694f0 con 0x7ff0680cf5e0 2024-04-26T23:20:51.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.813+0000 7ff05dffb640 1 -- 172.21.15.47:0/2834768732 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff058046070 con 0x7ff0680cf5e0 2024-04-26T23:20:51.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.814+0000 7ff05dffb640 1 --2- 172.21.15.47:0/2834768732 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7ff04c06c8f0 0x7ff04c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:51.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.814+0000 7ff05dffb640 1 -- 172.21.15.47:0/2834768732 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5335+0+0 (secure 0 0 0) 0x7ff0580ab970 con 0x7ff0680cf5e0 2024-04-26T23:20:51.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.814+0000 7ff06d3a3640 1 --2- 172.21.15.47:0/2834768732 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7ff04c06c8f0 0x7ff04c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:51.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.814+0000 7ff06d3a3640 1 --2- 172.21.15.47:0/2834768732 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7ff04c06c8f0 0x7ff04c06edb0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff064006fd0 tx=0x7ff064008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:51.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:51.818+0000 7ff05dffb640 1 -- 172.21.15.47:0/2834768732 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff058079e70 con 0x7ff0680cf5e0 2024-04-26T23:20:52.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.003+0000 7ff06f607640 1 -- 172.21.15.47:0/2834768732 --> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7ff0680ce7b0 con 0x7ff04c06c8f0 2024-04-26T23:20:52.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.004+0000 7ff05dffb640 1 -- 172.21.15.47:0/2834768732 <== mgr.4099 v2:172.21.15.203:6832/992162732 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7ff0680ce7b0 con 0x7ff04c06c8f0 2024-04-26T23:20:52.006 INFO:teuthology.orchestra.run.smithi047.stderr:instructed osd(s) 4 to deep-scrub 2024-04-26T23:20:52.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.005+0000 7ff06f607640 1 -- 172.21.15.47:0/2834768732 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7ff04c06c8f0 msgr2=0x7ff04c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:52.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.006+0000 7ff06f607640 1 --2- 172.21.15.47:0/2834768732 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7ff04c06c8f0 0x7ff04c06edb0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff064006fd0 tx=0x7ff064008040 comp rx=0 tx=0).stop 2024-04-26T23:20:52.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.006+0000 7ff06f607640 1 -- 172.21.15.47:0/2834768732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0680cf5e0 msgr2=0x7ff06815a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:52.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.006+0000 7ff06f607640 1 --2- 172.21.15.47:0/2834768732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0680cf5e0 0x7ff06815a290 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ff058000c00 tx=0x7ff0580046c0 comp rx=0 tx=0).stop 2024-04-26T23:20:52.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.006+0000 7ff06f607640 1 -- 172.21.15.47:0/2834768732 shutdown_connections 2024-04-26T23:20:52.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.006+0000 7ff06f607640 1 --2- 172.21.15.47:0/2834768732 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7ff04c06c8f0 0x7ff04c06edb0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:52.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.006+0000 7ff06f607640 1 --2- 172.21.15.47:0/2834768732 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff06815b600 0x7ff06815bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:52.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.006+0000 7ff06f607640 1 --2- 172.21.15.47:0/2834768732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0680cf5e0 0x7ff06815a290 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:52.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.006+0000 7ff06f607640 1 --2- 172.21.15.47:0/2834768732 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0680cbbd0 0x7ff0681f3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:52.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.006+0000 7ff06f607640 1 -- 172.21.15.47:0/2834768732 >> 172.21.15.47:0/2834768732 conn(0x7ff0680d0be0 msgr2=0x7ff068156f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:52.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.006+0000 7ff06f607640 1 -- 172.21.15.47:0/2834768732 shutdown_connections 2024-04-26T23:20:52.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.006+0000 7ff06f607640 1 -- 172.21.15.47:0/2834768732 wait complete. 2024-04-26T23:20:52.018 INFO:tasks.ceph:Scrubbing osd.5 2024-04-26T23:20:52.019 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-26T23:20:52.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.121+0000 7f5a612a9640 1 Processor -- start 2024-04-26T23:20:52.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.121+0000 7f5a612a9640 1 -- start start 2024-04-26T23:20:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.122+0000 7f5a612a9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a5c15cea0 0x7f5a5c160380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.122+0000 7f5a612a9640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5a5c06e8d0 0x7f5a5c0b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.122+0000 7f5a612a9640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a5c13f100 con 0x7f5a5c06e490 2024-04-26T23:20:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.122+0000 7f5a612a9640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f5a5c15d280 con 0x7f5a5c06e8d0 2024-04-26T23:20:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.122+0000 7f5a612a9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f5a5c141840 con 0x7f5a5c15cea0 2024-04-26T23:20:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.123+0000 7f5a5affd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a5c15cea0 0x7f5a5c160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.123+0000 7f5a5a7fc640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f5a5c06e490 0x7f5a5c0b2a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.47:54230/0 (socket says 172.21.15.47:54230) 2024-04-26T23:20:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.123+0000 7f5a5a7fc640 1 -- 172.21.15.47:0/3007646044 learned_addr learned my addr 172.21.15.47:0/3007646044 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.123+0000 7f5a5b7fe640 1 --2- 172.21.15.47:0/3007646044 >> v2:172.21.15.203:3300/0 conn(0x7f5a5c06e8d0 0x7f5a5c0b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.123+0000 7f5a5affd640 1 -- 172.21.15.47:0/3007646044 >> v2:172.21.15.203:3300/0 conn(0x7f5a5c06e8d0 msgr2=0x7f5a5c0b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.123+0000 7f5a5affd640 1 --2- 172.21.15.47:0/3007646044 >> v2:172.21.15.203:3300/0 conn(0x7f5a5c06e8d0 0x7f5a5c0b3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.123+0000 7f5a5affd640 1 -- 172.21.15.47:0/3007646044 >> v1:172.21.15.203:6789/0 conn(0x7f5a5c06e490 legacy=0x7f5a5c0b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.123+0000 7f5a5affd640 1 -- 172.21.15.47:0/3007646044 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a5c1423c0 con 0x7f5a5c15cea0 2024-04-26T23:20:52.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.124+0000 7f5a5affd640 1 --2- 172.21.15.47:0/3007646044 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a5c15cea0 0x7f5a5c160380 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c0097a0 tx=0x7f5a4c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4477d5b31cffab92 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:52.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.124+0000 7f5a59ffb640 1 -- 172.21.15.47:0/3007646044 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c037070 con 0x7f5a5c15cea0 2024-04-26T23:20:52.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.124+0000 7f5a59ffb640 1 -- 172.21.15.47:0/3007646044 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a4c032440 con 0x7f5a5c15cea0 2024-04-26T23:20:52.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.124+0000 7f5a59ffb640 1 -- 172.21.15.47:0/3007646044 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c032810 con 0x7f5a5c15cea0 2024-04-26T23:20:52.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.124+0000 7f5a612a9640 1 -- 172.21.15.47:0/3007646044 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a5c15cea0 msgr2=0x7f5a5c160380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:52.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.124+0000 7f5a612a9640 1 --2- 172.21.15.47:0/3007646044 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a5c15cea0 0x7f5a5c160380 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c0097a0 tx=0x7f5a4c02a180 comp rx=0 tx=0).stop 2024-04-26T23:20:52.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.125+0000 7f5a612a9640 1 -- 172.21.15.47:0/3007646044 shutdown_connections 2024-04-26T23:20:52.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.125+0000 7f5a612a9640 1 --2- 172.21.15.47:0/3007646044 >> v2:172.21.15.203:3300/0 conn(0x7f5a5c06e8d0 0x7f5a5c0b3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:52.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.125+0000 7f5a612a9640 1 --2- 172.21.15.47:0/3007646044 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a5c15cea0 0x7f5a5c160380 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:52.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.125+0000 7f5a612a9640 1 -- 172.21.15.47:0/3007646044 >> 172.21.15.47:0/3007646044 conn(0x7f5a5c06f310 msgr2=0x7f5a5c06f720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:52.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.125+0000 7f5a612a9640 1 -- 172.21.15.47:0/3007646044 shutdown_connections 2024-04-26T23:20:52.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.125+0000 7f5a612a9640 1 -- 172.21.15.47:0/3007646044 wait complete. 2024-04-26T23:20:52.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.126+0000 7f5a612a9640 1 Processor -- start 2024-04-26T23:20:52.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.126+0000 7f5a612a9640 1 -- start start 2024-04-26T23:20:52.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.127+0000 7f5a612a9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a5c06e490 0x7f5a5c151b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:52.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.127+0000 7f5a612a9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a5c06e8d0 0x7f5a5c1f5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:52.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.127+0000 7f5a612a9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a5c15cea0 0x7f5a5c1f5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:52.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.127+0000 7f5a612a9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5c1420e0 con 0x7f5a5c15cea0 2024-04-26T23:20:52.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.127+0000 7f5a612a9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5c141e00 con 0x7f5a5c06e490 2024-04-26T23:20:52.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.127+0000 7f5a612a9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f5a5c06ed60 con 0x7f5a5c06e8d0 2024-04-26T23:20:52.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.127+0000 7f5a5b7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a5c15cea0 0x7f5a5c1f5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:52.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.127+0000 7f5a5b7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a5c15cea0 0x7f5a5c1f5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48266/0 (socket says 172.21.15.47:48266) 2024-04-26T23:20:52.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.127+0000 7f5a5b7fe640 1 -- 172.21.15.47:0/3894630265 learned_addr learned my addr 172.21.15.47:0/3894630265 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:52.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.127+0000 7f5a5a7fc640 1 --2- 172.21.15.47:0/3894630265 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a5c06e8d0 0x7f5a5c1f5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:52.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.127+0000 7f5a5affd640 1 --2- 172.21.15.47:0/3894630265 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a5c06e490 0x7f5a5c151b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:52.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.127+0000 7f5a5b7fe640 1 -- 172.21.15.47:0/3894630265 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a5c06e8d0 msgr2=0x7f5a5c1f5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:52.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.127+0000 7f5a5b7fe640 1 --2- 172.21.15.47:0/3894630265 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a5c06e8d0 0x7f5a5c1f5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:52.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.127+0000 7f5a5b7fe640 1 -- 172.21.15.47:0/3894630265 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a5c06e490 msgr2=0x7f5a5c151b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:52.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.127+0000 7f5a5b7fe640 1 --2- 172.21.15.47:0/3894630265 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a5c06e490 0x7f5a5c151b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:52.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.127+0000 7f5a5b7fe640 1 -- 172.21.15.47:0/3894630265 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a5c152060 con 0x7f5a5c15cea0 2024-04-26T23:20:52.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.127+0000 7f5a5affd640 1 --2- 172.21.15.47:0/3894630265 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a5c06e490 0x7f5a5c151b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:52.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.127+0000 7f5a5a7fc640 1 --2- 172.21.15.47:0/3894630265 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a5c06e8d0 0x7f5a5c1f5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:20:52.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.127+0000 7f5a5b7fe640 1 --2- 172.21.15.47:0/3894630265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a5c15cea0 0x7f5a5c1f5950 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f5a5400efc0 tx=0x7f5a5400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:52.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.127+0000 7f5a437fe640 1 -- 172.21.15.47:0/3894630265 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a54018020 con 0x7f5a5c15cea0 2024-04-26T23:20:52.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.128+0000 7f5a437fe640 1 -- 172.21.15.47:0/3894630265 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a54014480 con 0x7f5a5c15cea0 2024-04-26T23:20:52.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.128+0000 7f5a612a9640 1 -- 172.21.15.47:0/3894630265 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a5c1f5e90 con 0x7f5a5c15cea0 2024-04-26T23:20:52.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.128+0000 7f5a437fe640 1 -- 172.21.15.47:0/3894630265 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a54014870 con 0x7f5a5c15cea0 2024-04-26T23:20:52.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.128+0000 7f5a612a9640 1 -- 172.21.15.47:0/3894630265 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a5c0cb380 con 0x7f5a5c15cea0 2024-04-26T23:20:52.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.128+0000 7f5a612a9640 1 -- 172.21.15.47:0/3894630265 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5a5c069600 con 0x7f5a5c15cea0 2024-04-26T23:20:52.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.129+0000 7f5a437fe640 1 -- 172.21.15.47:0/3894630265 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5a54005d10 con 0x7f5a5c15cea0 2024-04-26T23:20:52.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.129+0000 7f5a437fe640 1 --2- 172.21.15.47:0/3894630265 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f5a4806c920 0x7f5a4806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:52.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.130+0000 7f5a437fe640 1 -- 172.21.15.47:0/3894630265 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5335+0+0 (secure 0 0 0) 0x7f5a5408c600 con 0x7f5a5c15cea0 2024-04-26T23:20:52.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.130+0000 7f5a437fe640 1 --2- 172.21.15.47:0/3894630265 >> [v2:172.21.15.203:6800/56838444,v1:172.21.15.203:6801/56838444] conn(0x7f5a48072dc0 0x7f5a48075200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:52.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.130+0000 7f5a437fe640 1 -- 172.21.15.47:0/3894630265 --> [v2:172.21.15.203:6800/56838444,v1:172.21.15.203:6801/56838444] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5a5400be50 con 0x7f5a48072dc0 2024-04-26T23:20:52.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.130+0000 7f5a437fe640 1 -- 172.21.15.47:0/3894630265 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f5a54093050 con 0x7f5a5c15cea0 2024-04-26T23:20:52.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.130+0000 7f5a5affd640 1 --2- 172.21.15.47:0/3894630265 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f5a4806c920 0x7f5a4806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:52.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.130+0000 7f5a5a7fc640 1 --2- 172.21.15.47:0/3894630265 >> [v2:172.21.15.203:6800/56838444,v1:172.21.15.203:6801/56838444] conn(0x7f5a48072dc0 0x7f5a48075200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:52.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.130+0000 7f5a5affd640 1 --2- 172.21.15.47:0/3894630265 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f5a4806c920 0x7f5a4806ede0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c034000 tx=0x7f5a4c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:52.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.130+0000 7f5a5a7fc640 1 --2- 172.21.15.47:0/3894630265 >> [v2:172.21.15.203:6800/56838444,v1:172.21.15.203:6801/56838444] conn(0x7f5a48072dc0 0x7f5a48075200 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:52.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.131+0000 7f5a437fe640 1 -- 172.21.15.47:0/3894630265 <== osd.5 v2:172.21.15.203:6800/56838444 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5a5400be50 con 0x7f5a48072dc0 2024-04-26T23:20:52.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.145+0000 7f5a612a9640 1 -- 172.21.15.47:0/3894630265 --> [v2:172.21.15.203:6800/56838444,v1:172.21.15.203:6801/56838444] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f5a5c0b5410 con 0x7f5a48072dc0 2024-04-26T23:20:52.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.145+0000 7f5a437fe640 1 -- 172.21.15.47:0/3894630265 <== osd.5 v2:172.21.15.203:6800/56838444 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f5a5c0b5410 con 0x7f5a48072dc0 2024-04-26T23:20:52.147 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-26T23:20:52.147 INFO:teuthology.orchestra.run.smithi047.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T23:20:52.147 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-26T23:20:52.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.145+0000 7f5a612a9640 1 -- 172.21.15.47:0/3894630265 >> [v2:172.21.15.203:6800/56838444,v1:172.21.15.203:6801/56838444] conn(0x7f5a48072dc0 msgr2=0x7f5a48075200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:52.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.145+0000 7f5a612a9640 1 --2- 172.21.15.47:0/3894630265 >> [v2:172.21.15.203:6800/56838444,v1:172.21.15.203:6801/56838444] conn(0x7f5a48072dc0 0x7f5a48075200 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:52.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.145+0000 7f5a612a9640 1 -- 172.21.15.47:0/3894630265 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f5a4806c920 msgr2=0x7f5a4806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:52.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.145+0000 7f5a612a9640 1 --2- 172.21.15.47:0/3894630265 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f5a4806c920 0x7f5a4806ede0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c034000 tx=0x7f5a4c02e000 comp rx=0 tx=0).stop 2024-04-26T23:20:52.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.145+0000 7f5a612a9640 1 -- 172.21.15.47:0/3894630265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a5c15cea0 msgr2=0x7f5a5c1f5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:52.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.145+0000 7f5a612a9640 1 --2- 172.21.15.47:0/3894630265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a5c15cea0 0x7f5a5c1f5950 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f5a5400efc0 tx=0x7f5a5400c490 comp rx=0 tx=0).stop 2024-04-26T23:20:52.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.145+0000 7f5a5b7fe640 1 -- 172.21.15.47:0/3894630265 reap_dead start 2024-04-26T23:20:52.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.145+0000 7f5a612a9640 1 -- 172.21.15.47:0/3894630265 shutdown_connections 2024-04-26T23:20:52.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.145+0000 7f5a612a9640 1 -- 172.21.15.47:0/3894630265 >> 172.21.15.47:0/3894630265 conn(0x7f5a5c06f310 msgr2=0x7f5a5c0b4ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:52.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.146+0000 7f5a612a9640 1 -- 172.21.15.47:0/3894630265 shutdown_connections 2024-04-26T23:20:52.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.146+0000 7f5a612a9640 1 -- 172.21.15.47:0/3894630265 wait complete. 2024-04-26T23:20:52.158 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-26T23:20:52.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.258+0000 7f45e8809640 1 Processor -- start 2024-04-26T23:20:52.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.258+0000 7f45e8809640 1 -- start start 2024-04-26T23:20:52.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.259+0000 7f45e8809640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45e40d06f0 0x7f45e40d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:52.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.259+0000 7f45e8809640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45e40d0b70 con 0x7f45e40b2850 2024-04-26T23:20:52.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.259+0000 7f45e8809640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45e40d0e20 con 0x7f45e415f3a0 2024-04-26T23:20:52.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.259+0000 7f45e8809640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f45e40b1d70 con 0x7f45e40d06f0 2024-04-26T23:20:52.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.259+0000 7f45e2d9d640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f45e40b2850 0x7f45e40b2cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:52976/0 (socket says 172.21.15.47:52976) 2024-04-26T23:20:52.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.259+0000 7f45e2d9d640 1 -- 172.21.15.47:0/2793159070 learned_addr learned my addr 172.21.15.47:0/2793159070 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:52.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.259+0000 7f45d9d9b640 1 --2- 172.21.15.47:0/2793159070 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45e40d06f0 0x7f45e40d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:52.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.259+0000 7f45e1d9b640 1 -- 172.21.15.47:0/2793159070 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2857979247 0 0) 0x7f45e40d0b70 con 0x7f45e40b2850 2024-04-26T23:20:52.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.259+0000 7f45e1d9b640 1 -- 172.21.15.47:0/2793159070 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f45d0003520 con 0x7f45e40b2850 2024-04-26T23:20:52.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.259+0000 7f45e1d9b640 1 -- 172.21.15.47:0/2793159070 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 942908166 0 0) 0x7f45d0003520 con 0x7f45e40b2850 2024-04-26T23:20:52.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.259+0000 7f45e1d9b640 1 -- 172.21.15.47:0/2793159070 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45e40d06f0 msgr2=0x7f45e40d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:52.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.259+0000 7f45e1d9b640 1 --2- 172.21.15.47:0/2793159070 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45e40d06f0 0x7f45e40d1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:52.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.259+0000 7f45e1d9b640 1 -- 172.21.15.47:0/2793159070 >> v1:172.21.15.203:6789/0 conn(0x7f45e415f3a0 legacy=0x7f45e40d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:52.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.259+0000 7f45e1d9b640 1 -- 172.21.15.47:0/2793159070 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45e40b2050 con 0x7f45e40b2850 2024-04-26T23:20:52.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.259+0000 7f45e1d9b640 1 -- 172.21.15.47:0/2793159070 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 117215439 0 0) 0x7f45dc0019f0 con 0x7f45e40b2850 2024-04-26T23:20:52.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.259+0000 7f45e1d9b640 1 -- 172.21.15.47:0/2793159070 >> v1:172.21.15.47:6789/0 conn(0x7f45e40b2850 legacy=0x7f45e40b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:52.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.259+0000 7f45e1d9b640 1 --2- 172.21.15.47:0/2793159070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45d00039e0 0x7f45d0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:52.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.259+0000 7f45e1d9b640 1 --2- 172.21.15.47:0/2793159070 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45e40d06f0 0x7f45d0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:52.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.259+0000 7f45e1d9b640 1 --2- 172.21.15.47:0/2793159070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45d0004d40 0x7f45d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:52.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.259+0000 7f45e1d9b640 1 -- 172.21.15.47:0/2793159070 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f45e40b1d70 con 0x7f45d00039e0 2024-04-26T23:20:52.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.259+0000 7f45e1d9b640 1 -- 172.21.15.47:0/2793159070 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f45d000ba70 con 0x7f45d0004d40 2024-04-26T23:20:52.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.259+0000 7f45e1d9b640 1 -- 172.21.15.47:0/2793159070 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f45d000bc70 con 0x7f45e40d06f0 2024-04-26T23:20:52.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.260+0000 7f45e259c640 1 --2- 172.21.15.47:0/2793159070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45d00039e0 0x7f45d0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:52.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.260+0000 7f45d9d9b640 1 --2- 172.21.15.47:0/2793159070 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45e40d06f0 0x7f45d0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:52.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.260+0000 7f45e2d9d640 1 --2- 172.21.15.47:0/2793159070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45d0004d40 0x7f45d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:52.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.260+0000 7f45e259c640 1 -- 172.21.15.47:0/2793159070 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45e40d06f0 msgr2=0x7f45d0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:52.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.260+0000 7f45e259c640 1 --2- 172.21.15.47:0/2793159070 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45e40d06f0 0x7f45d0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:52.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.260+0000 7f45e259c640 1 -- 172.21.15.47:0/2793159070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45d0004d40 msgr2=0x7f45d00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:52.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.260+0000 7f45e259c640 1 --2- 172.21.15.47:0/2793159070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45d0004d40 0x7f45d00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:52.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.260+0000 7f45e259c640 1 -- 172.21.15.47:0/2793159070 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45d0003520 con 0x7f45d00039e0 2024-04-26T23:20:52.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.261+0000 7f45e259c640 1 --2- 172.21.15.47:0/2793159070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45d00039e0 0x7f45d0003e00 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f45d4001720 tx=0x7f45d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aaf7c8b2375d043 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:52.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.261+0000 7f45e1d9b640 1 -- 172.21.15.47:0/2793159070 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d4001370 con 0x7f45d00039e0 2024-04-26T23:20:52.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.261+0000 7f45e1d9b640 1 -- 172.21.15.47:0/2793159070 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45d40052d0 con 0x7f45d00039e0 2024-04-26T23:20:52.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.261+0000 7f45e1d9b640 1 -- 172.21.15.47:0/2793159070 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d402abc0 con 0x7f45d00039e0 2024-04-26T23:20:52.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.261+0000 7f45e8809640 1 -- 172.21.15.47:0/2793159070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45d00039e0 msgr2=0x7f45d0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:52.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.261+0000 7f45e8809640 1 --2- 172.21.15.47:0/2793159070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45d00039e0 0x7f45d0003e00 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f45d4001720 tx=0x7f45d0005830 comp rx=0 tx=0).stop 2024-04-26T23:20:52.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.261+0000 7f45e2d9d640 1 -- 172.21.15.47:0/2793159070 reap_dead start 2024-04-26T23:20:52.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.261+0000 7f45e8809640 1 -- 172.21.15.47:0/2793159070 shutdown_connections 2024-04-26T23:20:52.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.261+0000 7f45e8809640 1 -- 172.21.15.47:0/2793159070 >> 172.21.15.47:0/2793159070 conn(0x7f45e40d4f90 msgr2=0x7f45e40d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:52.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.261+0000 7f45e8809640 1 -- 172.21.15.47:0/2793159070 shutdown_connections 2024-04-26T23:20:52.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.261+0000 7f45e8809640 1 -- 172.21.15.47:0/2793159070 wait complete. 2024-04-26T23:20:52.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.263+0000 7f45e8809640 1 Processor -- start 2024-04-26T23:20:52.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.263+0000 7f45e8809640 1 -- start start 2024-04-26T23:20:52.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.264+0000 7f45e8809640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45d00039e0 0x7f45e41f5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:52.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.264+0000 7f45e8809640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45e415fa80 0x7f45e4160c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:52.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.264+0000 7f45e8809640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45e41f69b0 0x7f45e41f6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:52.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.264+0000 7f45e8809640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f45e415f900 con 0x7f45e415fa80 2024-04-26T23:20:52.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.264+0000 7f45e8809640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f45e415f780 con 0x7f45e41f69b0 2024-04-26T23:20:52.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.264+0000 7f45e8809640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f45e40d1e80 con 0x7f45d00039e0 2024-04-26T23:20:52.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.264+0000 7f45d9d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45e415fa80 0x7f45e4160c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:52.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.264+0000 7f45d9d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45e415fa80 0x7f45e4160c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48294/0 (socket says 172.21.15.47:48294) 2024-04-26T23:20:52.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.264+0000 7f45d9d9b640 1 -- 172.21.15.47:0/14834999 learned_addr learned my addr 172.21.15.47:0/14834999 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:52.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.264+0000 7f45e259c640 1 --2- 172.21.15.47:0/14834999 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45d00039e0 0x7f45e41f5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:52.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.264+0000 7f45d9d9b640 1 -- 172.21.15.47:0/14834999 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45d00039e0 msgr2=0x7f45e41f5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:52.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.264+0000 7f45d9d9b640 1 --2- 172.21.15.47:0/14834999 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45d00039e0 0x7f45e41f5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:52.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.264+0000 7f45d9d9b640 1 -- 172.21.15.47:0/14834999 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45e41f69b0 msgr2=0x7f45e41f6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:52.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.264+0000 7f45d9d9b640 1 --2- 172.21.15.47:0/14834999 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45e41f69b0 0x7f45e41f6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:52.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.264+0000 7f45d9d9b640 1 -- 172.21.15.47:0/14834999 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45e4160260 con 0x7f45e415fa80 2024-04-26T23:20:52.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.264+0000 7f45d9d9b640 1 --2- 172.21.15.47:0/14834999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45e415fa80 0x7f45e4160c10 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f45cc0119d0 tx=0x7f45cc011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:52.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.264+0000 7f45db7fe640 1 -- 172.21.15.47:0/14834999 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45cc00fad0 con 0x7f45e415fa80 2024-04-26T23:20:52.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.264+0000 7f45db7fe640 1 -- 172.21.15.47:0/14834999 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45cc00fc70 con 0x7f45e415fa80 2024-04-26T23:20:52.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.264+0000 7f45e8809640 1 -- 172.21.15.47:0/14834999 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45e4160540 con 0x7f45e415fa80 2024-04-26T23:20:52.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.264+0000 7f45db7fe640 1 -- 172.21.15.47:0/14834999 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45cc014440 con 0x7f45e415fa80 2024-04-26T23:20:52.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.264+0000 7f45e8809640 1 -- 172.21.15.47:0/14834999 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45e41f6490 con 0x7f45e415fa80 2024-04-26T23:20:52.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.265+0000 7f45e8809640 1 -- 172.21.15.47:0/14834999 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45e40694f0 con 0x7f45e415fa80 2024-04-26T23:20:52.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.266+0000 7f45db7fe640 1 -- 172.21.15.47:0/14834999 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f45cc012040 con 0x7f45e415fa80 2024-04-26T23:20:52.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.266+0000 7f45db7fe640 1 --2- 172.21.15.47:0/14834999 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f45c406c8f0 0x7f45c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:52.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.266+0000 7f45db7fe640 1 -- 172.21.15.47:0/14834999 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5335+0+0 (secure 0 0 0) 0x7f45cc08bff0 con 0x7f45e415fa80 2024-04-26T23:20:52.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.267+0000 7f45e259c640 1 --2- 172.21.15.47:0/14834999 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f45c406c8f0 0x7f45c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:52.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.267+0000 7f45e259c640 1 --2- 172.21.15.47:0/14834999 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f45c406c8f0 0x7f45c406edb0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f45d4032000 tx=0x7f45d402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:52.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.271+0000 7f45db7fe640 1 -- 172.21.15.47:0/14834999 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f45cc019030 con 0x7f45e415fa80 2024-04-26T23:20:52.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.456+0000 7f45e8809640 1 -- 172.21.15.47:0/14834999 --> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f45e40b6860 con 0x7f45c406c8f0 2024-04-26T23:20:52.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.456+0000 7f45db7fe640 1 -- 172.21.15.47:0/14834999 <== mgr.4099 v2:172.21.15.203:6832/992162732 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f45e40b6860 con 0x7f45c406c8f0 2024-04-26T23:20:52.459 INFO:teuthology.orchestra.run.smithi047.stderr:instructed osd(s) 5 to deep-scrub 2024-04-26T23:20:52.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.459+0000 7f45e8809640 1 -- 172.21.15.47:0/14834999 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f45c406c8f0 msgr2=0x7f45c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:52.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.459+0000 7f45e8809640 1 --2- 172.21.15.47:0/14834999 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f45c406c8f0 0x7f45c406edb0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f45d4032000 tx=0x7f45d402e000 comp rx=0 tx=0).stop 2024-04-26T23:20:52.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.459+0000 7f45e8809640 1 -- 172.21.15.47:0/14834999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45e415fa80 msgr2=0x7f45e4160c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:52.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.459+0000 7f45e8809640 1 --2- 172.21.15.47:0/14834999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45e415fa80 0x7f45e4160c10 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f45cc0119d0 tx=0x7f45cc011e90 comp rx=0 tx=0).stop 2024-04-26T23:20:52.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.459+0000 7f45e8809640 1 -- 172.21.15.47:0/14834999 shutdown_connections 2024-04-26T23:20:52.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.459+0000 7f45e8809640 1 --2- 172.21.15.47:0/14834999 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f45c406c8f0 0x7f45c406edb0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:52.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.459+0000 7f45e8809640 1 --2- 172.21.15.47:0/14834999 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45e41f69b0 0x7f45e41f6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:52.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.459+0000 7f45e8809640 1 --2- 172.21.15.47:0/14834999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45e415fa80 0x7f45e4160c10 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:52.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.459+0000 7f45e8809640 1 --2- 172.21.15.47:0/14834999 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45d00039e0 0x7f45e41f5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:52.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.459+0000 7f45e8809640 1 -- 172.21.15.47:0/14834999 >> 172.21.15.47:0/14834999 conn(0x7f45e40d4f90 msgr2=0x7f45e406df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:52.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.459+0000 7f45e8809640 1 -- 172.21.15.47:0/14834999 shutdown_connections 2024-04-26T23:20:52.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.459+0000 7f45e8809640 1 -- 172.21.15.47:0/14834999 wait complete. 2024-04-26T23:20:52.471 INFO:tasks.ceph:Scrubbing osd.6 2024-04-26T23:20:52.472 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-26T23:20:52.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.574+0000 7fdc3d009640 1 Processor -- start 2024-04-26T23:20:52.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.574+0000 7fdc3d009640 1 -- start start 2024-04-26T23:20:52.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.575+0000 7fdc3d009640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdc3815f2b0 0x7fdc380d0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:52.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.575+0000 7fdc3d009640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdc380d2080 0x7fdc380d0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:52.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.575+0000 7fdc3d009640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc380d2e50 con 0x7fdc380d3ef0 2024-04-26T23:20:52.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.575+0000 7fdc3d009640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fdc380d2c00 con 0x7fdc3815f2b0 2024-04-26T23:20:52.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.575+0000 7fdc3d009640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fdc380d3a60 con 0x7fdc380d2080 2024-04-26T23:20:52.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.575+0000 7fdc36d9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdc3815f2b0 0x7fdc380d0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:52.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.575+0000 7fdc3659c640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fdc380d3ef0 0x7fdc380d1970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:52984/0 (socket says 172.21.15.47:52984) 2024-04-26T23:20:52.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.575+0000 7fdc3659c640 1 -- 172.21.15.47:0/2723184751 learned_addr learned my addr 172.21.15.47:0/2723184751 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:20:52.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.575+0000 7fdc3759e640 1 --2- 172.21.15.47:0/2723184751 >> v2:172.21.15.203:3300/0 conn(0x7fdc380d2080 0x7fdc380d0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:52.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.575+0000 7fdc36d9d640 1 -- 172.21.15.47:0/2723184751 >> v2:172.21.15.203:3300/0 conn(0x7fdc380d2080 msgr2=0x7fdc380d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:52.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.575+0000 7fdc36d9d640 1 --2- 172.21.15.47:0/2723184751 >> v2:172.21.15.203:3300/0 conn(0x7fdc380d2080 0x7fdc380d0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:52.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.575+0000 7fdc36d9d640 1 -- 172.21.15.47:0/2723184751 >> v1:172.21.15.47:6789/0 conn(0x7fdc380d3ef0 legacy=0x7fdc380d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:52.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.575+0000 7fdc36d9d640 1 -- 172.21.15.47:0/2723184751 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc380d3130 con 0x7fdc3815f2b0 2024-04-26T23:20:52.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.576+0000 7fdc36d9d640 1 --2- 172.21.15.47:0/2723184751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc3815f2b0 0x7fdc380d0750 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fdc380d2d80 tx=0x7fdc3002a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61fd9291154e3dac server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:52.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.576+0000 7fdc35d9b640 1 -- 172.21.15.47:0/2723184751 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc30037070 con 0x7fdc3815f2b0 2024-04-26T23:20:52.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.576+0000 7fdc35d9b640 1 -- 172.21.15.47:0/2723184751 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc3002ad30 con 0x7fdc3815f2b0 2024-04-26T23:20:52.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.576+0000 7fdc35d9b640 1 -- 172.21.15.47:0/2723184751 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc30032660 con 0x7fdc3815f2b0 2024-04-26T23:20:52.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.577+0000 7fdc3d009640 1 -- 172.21.15.47:0/2723184751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc3815f2b0 msgr2=0x7fdc380d0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:52.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.577+0000 7fdc3d009640 1 --2- 172.21.15.47:0/2723184751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc3815f2b0 0x7fdc380d0750 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fdc380d2d80 tx=0x7fdc3002a0a0 comp rx=0 tx=0).stop 2024-04-26T23:20:52.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.577+0000 7fdc3d009640 1 -- 172.21.15.47:0/2723184751 shutdown_connections 2024-04-26T23:20:52.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.577+0000 7fdc3d009640 1 --2- 172.21.15.47:0/2723184751 >> v2:172.21.15.203:3300/0 conn(0x7fdc380d2080 0x7fdc380d0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:52.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.577+0000 7fdc3d009640 1 --2- 172.21.15.47:0/2723184751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc3815f2b0 0x7fdc380d0750 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:52.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.577+0000 7fdc3d009640 1 -- 172.21.15.47:0/2723184751 >> 172.21.15.47:0/2723184751 conn(0x7fdc380d5030 msgr2=0x7fdc380d5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:52.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.577+0000 7fdc3d009640 1 -- 172.21.15.47:0/2723184751 shutdown_connections 2024-04-26T23:20:52.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.577+0000 7fdc3d009640 1 -- 172.21.15.47:0/2723184751 wait complete. 2024-04-26T23:20:52.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.578+0000 7fdc3d009640 1 Processor -- start 2024-04-26T23:20:52.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.578+0000 7fdc3d009640 1 -- start start 2024-04-26T23:20:52.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.579+0000 7fdc3d009640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc380d2080 0x7fdc3809ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:52.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.579+0000 7fdc3d009640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc380d3ef0 0x7fdc380cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:52.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.579+0000 7fdc3d009640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc3815f2b0 0x7fdc38160860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:52.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.579+0000 7fdc3d009640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fdc3815f690 con 0x7fdc3815f2b0 2024-04-26T23:20:52.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.579+0000 7fdc3d009640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdc380d36f0 con 0x7fdc380d2080 2024-04-26T23:20:52.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.579+0000 7fdc3d009640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fdc380b7e80 con 0x7fdc380d3ef0 2024-04-26T23:20:52.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.579+0000 7fdc3759e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc3815f2b0 0x7fdc38160860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:52.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.579+0000 7fdc3759e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc3815f2b0 0x7fdc38160860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48322/0 (socket says 172.21.15.47:48322) 2024-04-26T23:20:52.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.579+0000 7fdc3759e640 1 -- 172.21.15.47:0/4099847719 learned_addr learned my addr 172.21.15.47:0/4099847719 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:52.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.579+0000 7fdc36d9d640 1 --2- 172.21.15.47:0/4099847719 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc380d2080 0x7fdc3809ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:52.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.579+0000 7fdc3659c640 1 --2- 172.21.15.47:0/4099847719 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc380d3ef0 0x7fdc380cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:52.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.580+0000 7fdc3759e640 1 -- 172.21.15.47:0/4099847719 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc380d3ef0 msgr2=0x7fdc380cb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:52.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.580+0000 7fdc3759e640 1 --2- 172.21.15.47:0/4099847719 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc380d3ef0 0x7fdc380cb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:52.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.580+0000 7fdc3759e640 1 -- 172.21.15.47:0/4099847719 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc380d2080 msgr2=0x7fdc3809ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:52.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.580+0000 7fdc3759e640 1 --2- 172.21.15.47:0/4099847719 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc380d2080 0x7fdc3809ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:52.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.580+0000 7fdc3759e640 1 -- 172.21.15.47:0/4099847719 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc380cb9f0 con 0x7fdc3815f2b0 2024-04-26T23:20:52.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.580+0000 7fdc3659c640 1 --2- 172.21.15.47:0/4099847719 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc380d3ef0 0x7fdc380cb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:20:52.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.580+0000 7fdc3759e640 1 --2- 172.21.15.47:0/4099847719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc3815f2b0 0x7fdc38160860 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fdc28007e10 tx=0x7fdc2800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:52.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.580+0000 7fdc1f7fe640 1 -- 172.21.15.47:0/4099847719 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc28014620 con 0x7fdc3815f2b0 2024-04-26T23:20:52.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.580+0000 7fdc1f7fe640 1 -- 172.21.15.47:0/4099847719 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc28019070 con 0x7fdc3815f2b0 2024-04-26T23:20:52.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.580+0000 7fdc3d009640 1 -- 172.21.15.47:0/4099847719 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc3814e4b0 con 0x7fdc3815f2b0 2024-04-26T23:20:52.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.580+0000 7fdc1f7fe640 1 -- 172.21.15.47:0/4099847719 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc28014a20 con 0x7fdc3815f2b0 2024-04-26T23:20:52.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.580+0000 7fdc3d009640 1 -- 172.21.15.47:0/4099847719 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc3814ea70 con 0x7fdc3815f2b0 2024-04-26T23:20:52.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.581+0000 7fdc3d009640 1 -- 172.21.15.47:0/4099847719 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdc38069600 con 0x7fdc3815f2b0 2024-04-26T23:20:52.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.581+0000 7fdc1f7fe640 1 -- 172.21.15.47:0/4099847719 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdc28014bc0 con 0x7fdc3815f2b0 2024-04-26T23:20:52.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.582+0000 7fdc1f7fe640 1 --2- 172.21.15.47:0/4099847719 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdc2406c8f0 0x7fdc2406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:52.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.582+0000 7fdc1f7fe640 1 -- 172.21.15.47:0/4099847719 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5335+0+0 (secure 0 0 0) 0x7fdc2808c200 con 0x7fdc3815f2b0 2024-04-26T23:20:52.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.582+0000 7fdc1f7fe640 1 --2- 172.21.15.47:0/4099847719 >> [v2:172.21.15.203:6824/2738517080,v1:172.21.15.203:6825/2738517080] conn(0x7fdc24072d90 0x7fdc240751d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:52.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.582+0000 7fdc1f7fe640 1 -- 172.21.15.47:0/4099847719 --> [v2:172.21.15.203:6824/2738517080,v1:172.21.15.203:6825/2738517080] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdc2800ce50 con 0x7fdc24072d90 2024-04-26T23:20:52.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.582+0000 7fdc1f7fe640 1 -- 172.21.15.47:0/4099847719 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fdc28093050 con 0x7fdc3815f2b0 2024-04-26T23:20:52.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.582+0000 7fdc36d9d640 1 --2- 172.21.15.47:0/4099847719 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdc2406c8f0 0x7fdc2406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:52.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.582+0000 7fdc3659c640 1 --2- 172.21.15.47:0/4099847719 >> [v2:172.21.15.203:6824/2738517080,v1:172.21.15.203:6825/2738517080] conn(0x7fdc24072d90 0x7fdc240751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:52.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.582+0000 7fdc36d9d640 1 --2- 172.21.15.47:0/4099847719 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdc2406c8f0 0x7fdc2406edb0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fdc30034000 tx=0x7fdc3002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:52.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.583+0000 7fdc3659c640 1 --2- 172.21.15.47:0/4099847719 >> [v2:172.21.15.203:6824/2738517080,v1:172.21.15.203:6825/2738517080] conn(0x7fdc24072d90 0x7fdc240751d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:52.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.583+0000 7fdc1f7fe640 1 -- 172.21.15.47:0/4099847719 <== osd.6 v2:172.21.15.203:6824/2738517080 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdc2800ce50 con 0x7fdc24072d90 2024-04-26T23:20:52.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.597+0000 7fdc3d009640 1 -- 172.21.15.47:0/4099847719 --> [v2:172.21.15.203:6824/2738517080,v1:172.21.15.203:6825/2738517080] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fdc3808eb50 con 0x7fdc24072d90 2024-04-26T23:20:52.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.597+0000 7fdc1f7fe640 1 -- 172.21.15.47:0/4099847719 <== osd.6 v2:172.21.15.203:6824/2738517080 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fdc3808eb50 con 0x7fdc24072d90 2024-04-26T23:20:52.599 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-26T23:20:52.600 INFO:teuthology.orchestra.run.smithi047.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T23:20:52.600 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-26T23:20:52.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.597+0000 7fdc3d009640 1 -- 172.21.15.47:0/4099847719 >> [v2:172.21.15.203:6824/2738517080,v1:172.21.15.203:6825/2738517080] conn(0x7fdc24072d90 msgr2=0x7fdc240751d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:52.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.597+0000 7fdc3d009640 1 --2- 172.21.15.47:0/4099847719 >> [v2:172.21.15.203:6824/2738517080,v1:172.21.15.203:6825/2738517080] conn(0x7fdc24072d90 0x7fdc240751d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:52.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.597+0000 7fdc3d009640 1 -- 172.21.15.47:0/4099847719 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdc2406c8f0 msgr2=0x7fdc2406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:52.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.597+0000 7fdc3d009640 1 --2- 172.21.15.47:0/4099847719 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdc2406c8f0 0x7fdc2406edb0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fdc30034000 tx=0x7fdc3002e000 comp rx=0 tx=0).stop 2024-04-26T23:20:52.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.597+0000 7fdc3d009640 1 -- 172.21.15.47:0/4099847719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc3815f2b0 msgr2=0x7fdc38160860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:52.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.597+0000 7fdc3d009640 1 --2- 172.21.15.47:0/4099847719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc3815f2b0 0x7fdc38160860 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fdc28007e10 tx=0x7fdc2800e4b0 comp rx=0 tx=0).stop 2024-04-26T23:20:52.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.598+0000 7fdc3759e640 1 -- 172.21.15.47:0/4099847719 reap_dead start 2024-04-26T23:20:52.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.598+0000 7fdc3d009640 1 -- 172.21.15.47:0/4099847719 shutdown_connections 2024-04-26T23:20:52.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.598+0000 7fdc3d009640 1 -- 172.21.15.47:0/4099847719 >> 172.21.15.47:0/4099847719 conn(0x7fdc380d5030 msgr2=0x7fdc380b90a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:52.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.598+0000 7fdc3d009640 1 -- 172.21.15.47:0/4099847719 shutdown_connections 2024-04-26T23:20:52.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.598+0000 7fdc3d009640 1 -- 172.21.15.47:0/4099847719 wait complete. 2024-04-26T23:20:52.610 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-26T23:20:52.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.709+0000 7f04cecd1640 1 Processor -- start 2024-04-26T23:20:52.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.710+0000 7f04cecd1640 1 -- start start 2024-04-26T23:20:52.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.710+0000 7f04cecd1640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f04c80d3bc0 0x7f04c80d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:52.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.710+0000 7f04cecd1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f04c80b5f20 0x7f04c80b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:52.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.710+0000 7f04cecd1640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04c80b7e50 con 0x7f04c8162880 2024-04-26T23:20:52.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.710+0000 7f04cecd1640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f04c80d0580 con 0x7f04c80d3bc0 2024-04-26T23:20:52.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.710+0000 7f04cecd1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f04c80d1800 con 0x7f04c80b5f20 2024-04-26T23:20:52.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.711+0000 7f04cd26e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f04c80b5f20 0x7f04c80b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:52.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.711+0000 7f04cd26e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f04c80b5f20 0x7f04c80b8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.47:49492/0 (socket says 172.21.15.47:49492) 2024-04-26T23:20:52.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.711+0000 7f04c7fff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f04c80d3bc0 0x7f04c80d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:52.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.711+0000 7f04cd26e640 1 -- 172.21.15.47:0/1500285790 learned_addr learned my addr 172.21.15.47:0/1500285790 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:52.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.711+0000 7f04cd26e640 1 -- 172.21.15.47:0/1500285790 >> v2:172.21.15.203:3300/0 conn(0x7f04c80d3bc0 msgr2=0x7f04c80d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:52.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.711+0000 7f04cd26e640 1 --2- 172.21.15.47:0/1500285790 >> v2:172.21.15.203:3300/0 conn(0x7f04c80d3bc0 0x7f04c80d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:52.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.711+0000 7f04cd26e640 1 -- 172.21.15.47:0/1500285790 >> v1:172.21.15.203:6789/0 conn(0x7f04c8162880 legacy=0x7f04c80d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:52.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.711+0000 7f04cd26e640 1 -- 172.21.15.47:0/1500285790 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04c80d07d0 con 0x7f04c80b5f20 2024-04-26T23:20:52.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.712+0000 7f04cd26e640 1 --2- 172.21.15.47:0/1500285790 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f04c80b5f20 0x7f04c80b8300 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f04b0009870 tx=0x7f04b002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b402101396d565f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:52.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.712+0000 7f04c77fe640 1 -- 172.21.15.47:0/1500285790 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b0037070 con 0x7f04c80b5f20 2024-04-26T23:20:52.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.712+0000 7f04c77fe640 1 -- 172.21.15.47:0/1500285790 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04b0032440 con 0x7f04c80b5f20 2024-04-26T23:20:52.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.712+0000 7f04c77fe640 1 -- 172.21.15.47:0/1500285790 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b0032810 con 0x7f04c80b5f20 2024-04-26T23:20:52.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.713+0000 7f04cecd1640 1 -- 172.21.15.47:0/1500285790 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f04c80b5f20 msgr2=0x7f04c80b8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:52.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.713+0000 7f04cecd1640 1 --2- 172.21.15.47:0/1500285790 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f04c80b5f20 0x7f04c80b8300 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f04b0009870 tx=0x7f04b002a270 comp rx=0 tx=0).stop 2024-04-26T23:20:52.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.713+0000 7f04cecd1640 1 -- 172.21.15.47:0/1500285790 shutdown_connections 2024-04-26T23:20:52.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.713+0000 7f04cecd1640 1 --2- 172.21.15.47:0/1500285790 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f04c80b5f20 0x7f04c80b8300 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:52.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.713+0000 7f04cecd1640 1 --2- 172.21.15.47:0/1500285790 >> v2:172.21.15.203:3300/0 conn(0x7f04c80d3bc0 0x7f04c80d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:52.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.713+0000 7f04cecd1640 1 -- 172.21.15.47:0/1500285790 >> 172.21.15.47:0/1500285790 conn(0x7f04c80d4f60 msgr2=0x7f04c80d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:52.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.713+0000 7f04cecd1640 1 -- 172.21.15.47:0/1500285790 shutdown_connections 2024-04-26T23:20:52.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.713+0000 7f04cecd1640 1 -- 172.21.15.47:0/1500285790 wait complete. 2024-04-26T23:20:52.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.714+0000 7f04cecd1640 1 Processor -- start 2024-04-26T23:20:52.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.714+0000 7f04cecd1640 1 -- start start 2024-04-26T23:20:52.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.715+0000 7f04cecd1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04c80b5f20 0x7f04c80cc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:52.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.715+0000 7f04cecd1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f04c80d3bc0 0x7f04c815b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:52.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.715+0000 7f04cecd1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04c8162880 0x7f04c80b9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:52.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.715+0000 7f04cecd1640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f04c8162c60 con 0x7f04c80b5f20 2024-04-26T23:20:52.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.715+0000 7f04cecd1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f04c80d0d90 con 0x7f04c8162880 2024-04-26T23:20:52.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.715+0000 7f04cecd1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f04c8162de0 con 0x7f04c80d3bc0 2024-04-26T23:20:52.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.715+0000 7f04cca6d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04c80b5f20 0x7f04c80cc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:52.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.715+0000 7f04cca6d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04c80b5f20 0x7f04c80cc9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48326/0 (socket says 172.21.15.47:48326) 2024-04-26T23:20:52.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.715+0000 7f04cca6d640 1 -- 172.21.15.47:0/4159959431 learned_addr learned my addr 172.21.15.47:0/4159959431 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:52.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.715+0000 7f04c7fff640 1 --2- 172.21.15.47:0/4159959431 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f04c80d3bc0 0x7f04c815b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:52.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.715+0000 7f04cd26e640 1 --2- 172.21.15.47:0/4159959431 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04c8162880 0x7f04c80b9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:52.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.715+0000 7f04cca6d640 1 -- 172.21.15.47:0/4159959431 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f04c80d3bc0 msgr2=0x7f04c815b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:52.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.715+0000 7f04cca6d640 1 --2- 172.21.15.47:0/4159959431 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f04c80d3bc0 0x7f04c815b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:52.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.715+0000 7f04cca6d640 1 -- 172.21.15.47:0/4159959431 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04c8162880 msgr2=0x7f04c80b9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:52.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.715+0000 7f04cca6d640 1 --2- 172.21.15.47:0/4159959431 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04c8162880 0x7f04c80b9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:52.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.715+0000 7f04cca6d640 1 -- 172.21.15.47:0/4159959431 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04c80b5a80 con 0x7f04c80b5f20 2024-04-26T23:20:52.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.715+0000 7f04cd26e640 1 --2- 172.21.15.47:0/4159959431 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04c8162880 0x7f04c80b9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:20:52.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.715+0000 7f04c7fff640 1 --2- 172.21.15.47:0/4159959431 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f04c80d3bc0 0x7f04c815b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:20:52.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.715+0000 7f04cca6d640 1 --2- 172.21.15.47:0/4159959431 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04c80b5f20 0x7f04c80cc9b0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f04c000c790 tx=0x7f04c000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:52.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.716+0000 7f04c57fa640 1 -- 172.21.15.47:0/4159959431 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c0016020 con 0x7f04c80b5f20 2024-04-26T23:20:52.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.716+0000 7f04c57fa640 1 -- 172.21.15.47:0/4159959431 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04c0006d50 con 0x7f04c80b5f20 2024-04-26T23:20:52.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.716+0000 7f04cecd1640 1 -- 172.21.15.47:0/4159959431 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04c80cdf00 con 0x7f04c80b5f20 2024-04-26T23:20:52.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.716+0000 7f04c57fa640 1 -- 172.21.15.47:0/4159959431 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c0003bb0 con 0x7f04c80b5f20 2024-04-26T23:20:52.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.716+0000 7f04cecd1640 1 -- 172.21.15.47:0/4159959431 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04c80ce4c0 con 0x7f04c80b5f20 2024-04-26T23:20:52.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.717+0000 7f04cecd1640 1 -- 172.21.15.47:0/4159959431 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04c80694f0 con 0x7f04c80b5f20 2024-04-26T23:20:52.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.717+0000 7f04c57fa640 1 -- 172.21.15.47:0/4159959431 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f04c000a850 con 0x7f04c80b5f20 2024-04-26T23:20:52.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.717+0000 7f04c57fa640 1 --2- 172.21.15.47:0/4159959431 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f04a806c920 0x7f04a806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:52.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.718+0000 7f04c57fa640 1 -- 172.21.15.47:0/4159959431 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5335+0+0 (secure 0 0 0) 0x7f04c008b670 con 0x7f04c80b5f20 2024-04-26T23:20:52.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.718+0000 7f04c7fff640 1 --2- 172.21.15.47:0/4159959431 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f04a806c920 0x7f04a806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:52.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.718+0000 7f04c7fff640 1 --2- 172.21.15.47:0/4159959431 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f04a806c920 0x7f04a806ede0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f04c80cd820 tx=0x7f04b8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:52.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.722+0000 7f04c57fa640 1 -- 172.21.15.47:0/4159959431 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f04c0020020 con 0x7f04c80b5f20 2024-04-26T23:20:52.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.906+0000 7f04cecd1640 1 -- 172.21.15.47:0/4159959431 --> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f04c8069c00 con 0x7f04a806c920 2024-04-26T23:20:52.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.906+0000 7f04c57fa640 1 -- 172.21.15.47:0/4159959431 <== mgr.4099 v2:172.21.15.203:6832/992162732 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f04c8069c00 con 0x7f04a806c920 2024-04-26T23:20:52.908 INFO:teuthology.orchestra.run.smithi047.stderr:instructed osd(s) 6 to deep-scrub 2024-04-26T23:20:52.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.908+0000 7f04cecd1640 1 -- 172.21.15.47:0/4159959431 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f04a806c920 msgr2=0x7f04a806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:52.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.908+0000 7f04cecd1640 1 --2- 172.21.15.47:0/4159959431 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f04a806c920 0x7f04a806ede0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f04c80cd820 tx=0x7f04b8008040 comp rx=0 tx=0).stop 2024-04-26T23:20:52.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.908+0000 7f04cecd1640 1 -- 172.21.15.47:0/4159959431 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04c80b5f20 msgr2=0x7f04c80cc9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:52.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.908+0000 7f04cecd1640 1 --2- 172.21.15.47:0/4159959431 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04c80b5f20 0x7f04c80cc9b0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f04c000c790 tx=0x7f04c000cc50 comp rx=0 tx=0).stop 2024-04-26T23:20:52.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.908+0000 7f04cecd1640 1 -- 172.21.15.47:0/4159959431 shutdown_connections 2024-04-26T23:20:52.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.908+0000 7f04cecd1640 1 --2- 172.21.15.47:0/4159959431 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f04a806c920 0x7f04a806ede0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:52.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.908+0000 7f04cecd1640 1 --2- 172.21.15.47:0/4159959431 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04c8162880 0x7f04c80b9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:52.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.908+0000 7f04cecd1640 1 --2- 172.21.15.47:0/4159959431 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f04c80d3bc0 0x7f04c815b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:52.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.908+0000 7f04cecd1640 1 --2- 172.21.15.47:0/4159959431 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04c80b5f20 0x7f04c80cc9b0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:52.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.908+0000 7f04cecd1640 1 -- 172.21.15.47:0/4159959431 >> 172.21.15.47:0/4159959431 conn(0x7f04c80d4f60 msgr2=0x7f04c81614f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:52.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.909+0000 7f04cecd1640 1 -- 172.21.15.47:0/4159959431 shutdown_connections 2024-04-26T23:20:52.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:52.909+0000 7f04cecd1640 1 -- 172.21.15.47:0/4159959431 wait complete. 2024-04-26T23:20:52.921 INFO:tasks.ceph:Scrubbing osd.7 2024-04-26T23:20:52.922 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-26T23:20:53.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.025+0000 7fdaddfb4640 1 Processor -- start 2024-04-26T23:20:53.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.025+0000 7fdaddfb4640 1 -- start start 2024-04-26T23:20:53.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.026+0000 7fdaddfb4640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdad815f460 0x7fdad80d3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:53.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.026+0000 7fdaddfb4640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdad80d0760 0x7fdad80d0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:53.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.026+0000 7fdaddfb4640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdad815f840 con 0x7fdad80d1a40 2024-04-26T23:20:53.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.026+0000 7fdaddfb4640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fdad80d41c0 con 0x7fdad80d0760 2024-04-26T23:20:53.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.026+0000 7fdaddfb4640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fdad80d37f0 con 0x7fdad815f460 2024-04-26T23:20:53.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.026+0000 7fdad6ffd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdad80d0760 0x7fdad80d0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:53.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.026+0000 7fdad6ffd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdad80d0760 0x7fdad80d0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48340/0 (socket says 172.21.15.47:48340) 2024-04-26T23:20:53.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.026+0000 7fdad7fff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fdad80d1a40 0x7fdad806ce20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:52994/0 (socket says 172.21.15.47:52994) 2024-04-26T23:20:53.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.026+0000 7fdad6ffd640 1 -- 172.21.15.47:0/556572041 learned_addr learned my addr 172.21.15.47:0/556572041 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:53.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.026+0000 7fdad77fe640 1 --2- 172.21.15.47:0/556572041 >> v2:172.21.15.203:3300/0 conn(0x7fdad815f460 0x7fdad80d3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:53.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.026+0000 7fdad6ffd640 1 -- 172.21.15.47:0/556572041 >> v2:172.21.15.203:3300/0 conn(0x7fdad815f460 msgr2=0x7fdad80d3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:53.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.026+0000 7fdad6ffd640 1 --2- 172.21.15.47:0/556572041 >> v2:172.21.15.203:3300/0 conn(0x7fdad815f460 0x7fdad80d3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:53.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.026+0000 7fdad6ffd640 1 -- 172.21.15.47:0/556572041 >> v1:172.21.15.47:6789/0 conn(0x7fdad80d1a40 legacy=0x7fdad806ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:53.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.026+0000 7fdad6ffd640 1 -- 172.21.15.47:0/556572041 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdad80d2ec0 con 0x7fdad80d0760 2024-04-26T23:20:53.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.026+0000 7fdad77fe640 1 --2- 172.21.15.47:0/556572041 >> v2:172.21.15.203:3300/0 conn(0x7fdad815f460 0x7fdad80d3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:53.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.026+0000 7fdad67fc640 1 -- 172.21.15.47:0/556572041 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4012712514 0 0) 0x7fdad815f840 con 0x7fdad80d1a40 2024-04-26T23:20:53.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.027+0000 7fdad6ffd640 1 --2- 172.21.15.47:0/556572041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdad80d0760 0x7fdad80d0bc0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fdad808f280 tx=0x7fdac802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9fce7ea6dcb7960 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:53.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.027+0000 7fdad67fc640 1 -- 172.21.15.47:0/556572041 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac8037070 con 0x7fdad80d0760 2024-04-26T23:20:53.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.027+0000 7fdad67fc640 1 -- 172.21.15.47:0/556572041 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdac8032440 con 0x7fdad80d0760 2024-04-26T23:20:53.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.027+0000 7fdad67fc640 1 -- 172.21.15.47:0/556572041 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac8032810 con 0x7fdad80d0760 2024-04-26T23:20:53.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.028+0000 7fdaddfb4640 1 -- 172.21.15.47:0/556572041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdad80d0760 msgr2=0x7fdad80d0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:53.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.028+0000 7fdaddfb4640 1 --2- 172.21.15.47:0/556572041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdad80d0760 0x7fdad80d0bc0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fdad808f280 tx=0x7fdac802a1e0 comp rx=0 tx=0).stop 2024-04-26T23:20:53.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.028+0000 7fdaddfb4640 1 -- 172.21.15.47:0/556572041 shutdown_connections 2024-04-26T23:20:53.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.028+0000 7fdaddfb4640 1 --2- 172.21.15.47:0/556572041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdad80d0760 0x7fdad80d0bc0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:53.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.028+0000 7fdaddfb4640 1 --2- 172.21.15.47:0/556572041 >> v2:172.21.15.203:3300/0 conn(0x7fdad815f460 0x7fdad80d3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:53.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.028+0000 7fdaddfb4640 1 -- 172.21.15.47:0/556572041 >> 172.21.15.47:0/556572041 conn(0x7fdad809e3f0 msgr2=0x7fdad80d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:53.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.028+0000 7fdaddfb4640 1 -- 172.21.15.47:0/556572041 shutdown_connections 2024-04-26T23:20:53.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.028+0000 7fdaddfb4640 1 -- 172.21.15.47:0/556572041 wait complete. 2024-04-26T23:20:53.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.029+0000 7fdaddfb4640 1 Processor -- start 2024-04-26T23:20:53.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.029+0000 7fdaddfb4640 1 -- start start 2024-04-26T23:20:53.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.030+0000 7fdaddfb4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdad80d0760 0x7fdad80cb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:53.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.030+0000 7fdaddfb4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdad80d1a40 0x7fdad815fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:53.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.030+0000 7fdaddfb4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdad815f460 0x7fdad81505a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:53.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.030+0000 7fdaddfb4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fdad80d5f30 con 0x7fdad815f460 2024-04-26T23:20:53.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.030+0000 7fdaddfb4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdad80d3480 con 0x7fdad80d1a40 2024-04-26T23:20:53.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.030+0000 7fdaddfb4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fdad80d1ed0 con 0x7fdad80d0760 2024-04-26T23:20:53.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.030+0000 7fdad7fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdad815f460 0x7fdad81505a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:53.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.030+0000 7fdad7fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdad815f460 0x7fdad81505a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48354/0 (socket says 172.21.15.47:48354) 2024-04-26T23:20:53.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.030+0000 7fdad7fff640 1 -- 172.21.15.47:0/3455187205 learned_addr learned my addr 172.21.15.47:0/3455187205 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:53.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.030+0000 7fdad77fe640 1 --2- 172.21.15.47:0/3455187205 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdad80d0760 0x7fdad80cb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:53.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.030+0000 7fdad6ffd640 1 --2- 172.21.15.47:0/3455187205 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdad80d1a40 0x7fdad815fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:53.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.030+0000 7fdad7fff640 1 -- 172.21.15.47:0/3455187205 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdad80d0760 msgr2=0x7fdad80cb700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:53.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.030+0000 7fdad7fff640 1 --2- 172.21.15.47:0/3455187205 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdad80d0760 0x7fdad80cb700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:53.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.030+0000 7fdad7fff640 1 -- 172.21.15.47:0/3455187205 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdad80d1a40 msgr2=0x7fdad815fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:53.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.030+0000 7fdad7fff640 1 --2- 172.21.15.47:0/3455187205 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdad80d1a40 0x7fdad815fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:53.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.030+0000 7fdad7fff640 1 -- 172.21.15.47:0/3455187205 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdad815ff90 con 0x7fdad815f460 2024-04-26T23:20:53.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.030+0000 7fdad7fff640 1 --2- 172.21.15.47:0/3455187205 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdad815f460 0x7fdad81505a0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fdad000c7a0 tx=0x7fdad000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:53.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.031+0000 7fdabffff640 1 -- 172.21.15.47:0/3455187205 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad0018070 con 0x7fdad815f460 2024-04-26T23:20:53.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.031+0000 7fdabffff640 1 -- 172.21.15.47:0/3455187205 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdad000a640 con 0x7fdad815f460 2024-04-26T23:20:53.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.031+0000 7fdabffff640 1 -- 172.21.15.47:0/3455187205 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad000aab0 con 0x7fdad815f460 2024-04-26T23:20:53.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.031+0000 7fdaddfb4640 1 -- 172.21.15.47:0/3455187205 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdad80ca660 con 0x7fdad815f460 2024-04-26T23:20:53.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.031+0000 7fdaddfb4640 1 -- 172.21.15.47:0/3455187205 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdad80cac20 con 0x7fdad815f460 2024-04-26T23:20:53.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.031+0000 7fdaddfb4640 1 -- 172.21.15.47:0/3455187205 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdad8069600 con 0x7fdad815f460 2024-04-26T23:20:53.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.032+0000 7fdabffff640 1 -- 172.21.15.47:0/3455187205 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdad001b070 con 0x7fdad815f460 2024-04-26T23:20:53.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.033+0000 7fdabffff640 1 --2- 172.21.15.47:0/3455187205 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdac406c8f0 0x7fdac406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:53.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.033+0000 7fdabffff640 1 -- 172.21.15.47:0/3455187205 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5335+0+0 (secure 0 0 0) 0x7fdad008b530 con 0x7fdad815f460 2024-04-26T23:20:53.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.033+0000 7fdabffff640 1 --2- 172.21.15.47:0/3455187205 >> [v2:172.21.15.203:6816/1315565058,v1:172.21.15.203:6817/1315565058] conn(0x7fdac4072d90 0x7fdac40751d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:53.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.033+0000 7fdabffff640 1 -- 172.21.15.47:0/3455187205 --> [v2:172.21.15.203:6816/1315565058,v1:172.21.15.203:6817/1315565058] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdad001fe50 con 0x7fdac4072d90 2024-04-26T23:20:53.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.033+0000 7fdabffff640 1 -- 172.21.15.47:0/3455187205 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fdad008b9c0 con 0x7fdad815f460 2024-04-26T23:20:53.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.033+0000 7fdad77fe640 1 --2- 172.21.15.47:0/3455187205 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdac406c8f0 0x7fdac406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:53.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.033+0000 7fdad6ffd640 1 --2- 172.21.15.47:0/3455187205 >> [v2:172.21.15.203:6816/1315565058,v1:172.21.15.203:6817/1315565058] conn(0x7fdac4072d90 0x7fdac40751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:53.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.033+0000 7fdad77fe640 1 --2- 172.21.15.47:0/3455187205 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdac406c8f0 0x7fdac406edb0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fdacc00aac0 tx=0x7fdacc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:53.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.034+0000 7fdad6ffd640 1 --2- 172.21.15.47:0/3455187205 >> [v2:172.21.15.203:6816/1315565058,v1:172.21.15.203:6817/1315565058] conn(0x7fdac4072d90 0x7fdac40751d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:53.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.034+0000 7fdabffff640 1 -- 172.21.15.47:0/3455187205 <== osd.7 v2:172.21.15.203:6816/1315565058 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdad001fe50 con 0x7fdac4072d90 2024-04-26T23:20:53.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.047+0000 7fdaddfb4640 1 -- 172.21.15.47:0/3455187205 --> [v2:172.21.15.203:6816/1315565058,v1:172.21.15.203:6817/1315565058] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fdad80d29a0 con 0x7fdac4072d90 2024-04-26T23:20:53.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.048+0000 7fdabffff640 1 -- 172.21.15.47:0/3455187205 <== osd.7 v2:172.21.15.203:6816/1315565058 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fdad80d29a0 con 0x7fdac4072d90 2024-04-26T23:20:53.050 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-26T23:20:53.050 INFO:teuthology.orchestra.run.smithi047.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T23:20:53.051 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-26T23:20:53.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.048+0000 7fdaddfb4640 1 -- 172.21.15.47:0/3455187205 >> [v2:172.21.15.203:6816/1315565058,v1:172.21.15.203:6817/1315565058] conn(0x7fdac4072d90 msgr2=0x7fdac40751d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:53.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.049+0000 7fdaddfb4640 1 --2- 172.21.15.47:0/3455187205 >> [v2:172.21.15.203:6816/1315565058,v1:172.21.15.203:6817/1315565058] conn(0x7fdac4072d90 0x7fdac40751d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:53.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.049+0000 7fdaddfb4640 1 -- 172.21.15.47:0/3455187205 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdac406c8f0 msgr2=0x7fdac406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:53.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.049+0000 7fdaddfb4640 1 --2- 172.21.15.47:0/3455187205 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fdac406c8f0 0x7fdac406edb0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fdacc00aac0 tx=0x7fdacc008040 comp rx=0 tx=0).stop 2024-04-26T23:20:53.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.049+0000 7fdaddfb4640 1 -- 172.21.15.47:0/3455187205 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdad815f460 msgr2=0x7fdad81505a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:53.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.049+0000 7fdaddfb4640 1 --2- 172.21.15.47:0/3455187205 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdad815f460 0x7fdad81505a0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fdad000c7a0 tx=0x7fdad000cc60 comp rx=0 tx=0).stop 2024-04-26T23:20:53.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.049+0000 7fdad7fff640 1 -- 172.21.15.47:0/3455187205 reap_dead start 2024-04-26T23:20:53.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.049+0000 7fdaddfb4640 1 -- 172.21.15.47:0/3455187205 shutdown_connections 2024-04-26T23:20:53.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.049+0000 7fdaddfb4640 1 -- 172.21.15.47:0/3455187205 >> 172.21.15.47:0/3455187205 conn(0x7fdad809e3f0 msgr2=0x7fdad8074ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:53.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.049+0000 7fdaddfb4640 1 -- 172.21.15.47:0/3455187205 shutdown_connections 2024-04-26T23:20:53.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.049+0000 7fdaddfb4640 1 -- 172.21.15.47:0/3455187205 wait complete. 2024-04-26T23:20:53.062 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-26T23:20:53.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.161+0000 7f3698aea640 1 Processor -- start 2024-04-26T23:20:53.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.161+0000 7f3698aea640 1 -- start start 2024-04-26T23:20:53.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.162+0000 7f3698aea640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f36940b2340 0x7f36940cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:53.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.162+0000 7f3698aea640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f36940cd930 0x7f36940d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:53.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.162+0000 7f3698aea640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36940d10d0 con 0x7f3694162810 2024-04-26T23:20:53.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.162+0000 7f3698aea640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f36940d0750 con 0x7f36940b2340 2024-04-26T23:20:53.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.162+0000 7f3698aea640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f36940d16a0 con 0x7f36940cd930 2024-04-26T23:20:53.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.162+0000 7f3691d9b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f36940b2340 0x7f36940cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:53.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.162+0000 7f3691d9b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f36940b2340 0x7f36940cd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48360/0 (socket says 172.21.15.47:48360) 2024-04-26T23:20:53.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.162+0000 7f3691d9b640 1 -- 172.21.15.47:0/987443314 learned_addr learned my addr 172.21.15.47:0/987443314 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:53.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.162+0000 7f369259c640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f3694162810 0x7f3694070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:53002/0 (socket says 172.21.15.47:53002) 2024-04-26T23:20:53.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.163+0000 7f3691d9b640 1 -- 172.21.15.47:0/987443314 >> v2:172.21.15.203:3300/0 conn(0x7f36940cd930 msgr2=0x7f36940d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:53.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.162+0000 7f3692d9d640 1 --2- 172.21.15.47:0/987443314 >> v2:172.21.15.203:3300/0 conn(0x7f36940cd930 0x7f36940d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:53.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.163+0000 7f3691d9b640 1 --2- 172.21.15.47:0/987443314 >> v2:172.21.15.203:3300/0 conn(0x7f36940cd930 0x7f36940d1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:53.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.163+0000 7f3691d9b640 1 -- 172.21.15.47:0/987443314 >> v1:172.21.15.47:6789/0 conn(0x7f3694162810 legacy=0x7f3694070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:53.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.163+0000 7f369159a640 1 -- 172.21.15.47:0/987443314 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 149416050 0 0) 0x7f36940d10d0 con 0x7f3694162810 2024-04-26T23:20:53.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.163+0000 7f3691d9b640 1 -- 172.21.15.47:0/987443314 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36940d1370 con 0x7f36940b2340 2024-04-26T23:20:53.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.163+0000 7f3692d9d640 1 --2- 172.21.15.47:0/987443314 >> v2:172.21.15.203:3300/0 conn(0x7f36940cd930 0x7f36940d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:20:53.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.163+0000 7f3691d9b640 1 --2- 172.21.15.47:0/987443314 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36940b2340 0x7f36940cd3f0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f36940b2280 tx=0x7f367c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c99223e00cab38 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:53.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.163+0000 7f369159a640 1 -- 172.21.15.47:0/987443314 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f367c037070 con 0x7f36940b2340 2024-04-26T23:20:53.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.163+0000 7f369159a640 1 -- 172.21.15.47:0/987443314 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f367c032440 con 0x7f36940b2340 2024-04-26T23:20:53.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.163+0000 7f369159a640 1 -- 172.21.15.47:0/987443314 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f367c032810 con 0x7f36940b2340 2024-04-26T23:20:53.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.164+0000 7f3698aea640 1 -- 172.21.15.47:0/987443314 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36940b2340 msgr2=0x7f36940cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:53.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.164+0000 7f3698aea640 1 --2- 172.21.15.47:0/987443314 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36940b2340 0x7f36940cd3f0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f36940b2280 tx=0x7f367c02a270 comp rx=0 tx=0).stop 2024-04-26T23:20:53.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.164+0000 7f3698aea640 1 -- 172.21.15.47:0/987443314 shutdown_connections 2024-04-26T23:20:53.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.164+0000 7f3698aea640 1 --2- 172.21.15.47:0/987443314 >> v2:172.21.15.203:3300/0 conn(0x7f36940cd930 0x7f36940d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:53.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.164+0000 7f3698aea640 1 --2- 172.21.15.47:0/987443314 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36940b2340 0x7f36940cd3f0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:53.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.164+0000 7f3698aea640 1 -- 172.21.15.47:0/987443314 >> 172.21.15.47:0/987443314 conn(0x7f36940d2da0 msgr2=0x7f36940d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:53.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.164+0000 7f3698aea640 1 -- 172.21.15.47:0/987443314 shutdown_connections 2024-04-26T23:20:53.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.164+0000 7f3698aea640 1 -- 172.21.15.47:0/987443314 wait complete. 2024-04-26T23:20:53.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.165+0000 7f3698aea640 1 Processor -- start 2024-04-26T23:20:53.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.165+0000 7f3698aea640 1 -- start start 2024-04-26T23:20:53.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.166+0000 7f3698aea640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36940b2340 0x7f369414c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:53.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.166+0000 7f3698aea640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36940cd930 0x7f3694149450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:53.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.166+0000 7f3698aea640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3694162810 0x7f36940c94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:53.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.166+0000 7f3698aea640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f36940d2070 con 0x7f36940b2340 2024-04-26T23:20:53.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.166+0000 7f3698aea640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f36940d0470 con 0x7f36940cd930 2024-04-26T23:20:53.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.166+0000 7f3698aea640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f36940d21f0 con 0x7f3694162810 2024-04-26T23:20:53.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.166+0000 7f369259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36940b2340 0x7f369414c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:53.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.166+0000 7f369259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36940b2340 0x7f369414c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48362/0 (socket says 172.21.15.47:48362) 2024-04-26T23:20:53.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.166+0000 7f369259c640 1 -- 172.21.15.47:0/4012032349 learned_addr learned my addr 172.21.15.47:0/4012032349 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:53.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.166+0000 7f3691d9b640 1 --2- 172.21.15.47:0/4012032349 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36940cd930 0x7f3694149450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:53.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.166+0000 7f3692d9d640 1 --2- 172.21.15.47:0/4012032349 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3694162810 0x7f36940c94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:53.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.167+0000 7f369259c640 1 -- 172.21.15.47:0/4012032349 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3694162810 msgr2=0x7f36940c94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:53.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.167+0000 7f369259c640 1 --2- 172.21.15.47:0/4012032349 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3694162810 0x7f36940c94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:53.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.167+0000 7f369259c640 1 -- 172.21.15.47:0/4012032349 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36940cd930 msgr2=0x7f3694149450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:53.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.167+0000 7f369259c640 1 --2- 172.21.15.47:0/4012032349 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36940cd930 0x7f3694149450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:53.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.167+0000 7f369259c640 1 -- 172.21.15.47:0/4012032349 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f369414ccf0 con 0x7f36940b2340 2024-04-26T23:20:53.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.167+0000 7f3691d9b640 1 --2- 172.21.15.47:0/4012032349 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36940cd930 0x7f3694149450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:20:53.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.167+0000 7f3692d9d640 1 --2- 172.21.15.47:0/4012032349 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3694162810 0x7f36940c94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:20:53.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.167+0000 7f369259c640 1 --2- 172.21.15.47:0/4012032349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36940b2340 0x7f369414c7b0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f368c00c460 tx=0x7f368c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:53.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.167+0000 7f368affd640 1 -- 172.21.15.47:0/4012032349 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368c018070 con 0x7f36940b2340 2024-04-26T23:20:53.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.167+0000 7f368affd640 1 -- 172.21.15.47:0/4012032349 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f368c007500 con 0x7f36940b2340 2024-04-26T23:20:53.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.167+0000 7f3698aea640 1 -- 172.21.15.47:0/4012032349 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36940c8440 con 0x7f36940b2340 2024-04-26T23:20:53.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.167+0000 7f368affd640 1 -- 172.21.15.47:0/4012032349 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368c004a20 con 0x7f36940b2340 2024-04-26T23:20:53.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.167+0000 7f3698aea640 1 -- 172.21.15.47:0/4012032349 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36940c8a00 con 0x7f36940b2340 2024-04-26T23:20:53.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.168+0000 7f3698aea640 1 -- 172.21.15.47:0/4012032349 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36940694f0 con 0x7f36940b2340 2024-04-26T23:20:53.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.169+0000 7f368affd640 1 -- 172.21.15.47:0/4012032349 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f368c01b070 con 0x7f36940b2340 2024-04-26T23:20:53.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.169+0000 7f368affd640 1 --2- 172.21.15.47:0/4012032349 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f367406c8f0 0x7f367406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:53.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.169+0000 7f368affd640 1 -- 172.21.15.47:0/4012032349 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5335+0+0 (secure 0 0 0) 0x7f368c08ac00 con 0x7f36940b2340 2024-04-26T23:20:53.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.169+0000 7f3691d9b640 1 --2- 172.21.15.47:0/4012032349 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f367406c8f0 0x7f367406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:53.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.170+0000 7f3691d9b640 1 --2- 172.21.15.47:0/4012032349 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f367406c8f0 0x7f367406edb0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f367c034000 tx=0x7f367c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:53.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.174+0000 7f368affd640 1 -- 172.21.15.47:0/4012032349 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f368c091080 con 0x7f36940b2340 2024-04-26T23:20:53.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.361+0000 7f3698aea640 1 -- 172.21.15.47:0/4012032349 --> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f36940b1e80 con 0x7f367406c8f0 2024-04-26T23:20:53.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.363+0000 7f368affd640 1 -- 172.21.15.47:0/4012032349 <== mgr.4099 v2:172.21.15.203:6832/992162732 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f36940b1e80 con 0x7f367406c8f0 2024-04-26T23:20:53.364 INFO:teuthology.orchestra.run.smithi047.stderr:instructed osd(s) 7 to deep-scrub 2024-04-26T23:20:53.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.364+0000 7f3698aea640 1 -- 172.21.15.47:0/4012032349 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f367406c8f0 msgr2=0x7f367406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:53.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.364+0000 7f3698aea640 1 --2- 172.21.15.47:0/4012032349 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f367406c8f0 0x7f367406edb0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f367c034000 tx=0x7f367c02e000 comp rx=0 tx=0).stop 2024-04-26T23:20:53.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.364+0000 7f3698aea640 1 -- 172.21.15.47:0/4012032349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36940b2340 msgr2=0x7f369414c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:53.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.364+0000 7f3698aea640 1 --2- 172.21.15.47:0/4012032349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36940b2340 0x7f369414c7b0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f368c00c460 tx=0x7f368c00c920 comp rx=0 tx=0).stop 2024-04-26T23:20:53.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.364+0000 7f3698aea640 1 -- 172.21.15.47:0/4012032349 shutdown_connections 2024-04-26T23:20:53.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.364+0000 7f3698aea640 1 --2- 172.21.15.47:0/4012032349 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f367406c8f0 0x7f367406edb0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:53.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.364+0000 7f3698aea640 1 --2- 172.21.15.47:0/4012032349 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3694162810 0x7f36940c94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:53.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.364+0000 7f3698aea640 1 --2- 172.21.15.47:0/4012032349 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36940cd930 0x7f3694149450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:53.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.364+0000 7f3698aea640 1 --2- 172.21.15.47:0/4012032349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36940b2340 0x7f369414c7b0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:53.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.364+0000 7f3698aea640 1 -- 172.21.15.47:0/4012032349 >> 172.21.15.47:0/4012032349 conn(0x7f36940d2da0 msgr2=0x7f36940b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:53.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.365+0000 7f3698aea640 1 -- 172.21.15.47:0/4012032349 shutdown_connections 2024-04-26T23:20:53.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.365+0000 7f3698aea640 1 -- 172.21.15.47:0/4012032349 wait complete. 2024-04-26T23:20:53.378 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T23:20:53.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.477+0000 7f50a42da640 1 Processor -- start 2024-04-26T23:20:53.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.477+0000 7f50a42da640 1 -- start start 2024-04-26T23:20:53.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.478+0000 7f50a42da640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f509c162880 0x7f509c0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:53.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.478+0000 7f50a42da640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f509c0b5f20 0x7f509c0d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:53.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.478+0000 7f50a42da640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f509c0d39f0 0x7f509c0b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:53.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.478+0000 7f50a42da640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f509c0d0580 con 0x7f509c0b5f20 2024-04-26T23:20:53.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.478+0000 7f50a42da640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f509c0b7e50 con 0x7f509c162880 2024-04-26T23:20:53.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.478+0000 7f50a42da640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f509c0d1800 con 0x7f509c0d39f0 2024-04-26T23:20:53.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.478+0000 7f50a1875640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f509c0b5f20 0x7f509c0d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:53.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.478+0000 7f50a1875640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f509c0b5f20 0x7f509c0d34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48364/0 (socket says 172.21.15.47:48364) 2024-04-26T23:20:53.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.478+0000 7f50a1875640 1 -- 172.21.15.47:0/2639766150 learned_addr learned my addr 172.21.15.47:0/2639766150 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:53.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.478+0000 7f50a2076640 1 --2- 172.21.15.47:0/2639766150 >> v2:172.21.15.203:3300/0 conn(0x7f509c162880 0x7f509c0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:53.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.478+0000 7f50a2877640 1 --2- 172.21.15.47:0/2639766150 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f509c0d39f0 0x7f509c0b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:53.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.478+0000 7f50a1875640 1 -- 172.21.15.47:0/2639766150 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f509c0d39f0 msgr2=0x7f509c0b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:53.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.478+0000 7f50a1875640 1 --2- 172.21.15.47:0/2639766150 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f509c0d39f0 0x7f509c0b69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:53.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.478+0000 7f50a1875640 1 -- 172.21.15.47:0/2639766150 >> v2:172.21.15.203:3300/0 conn(0x7f509c162880 msgr2=0x7f509c0d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:20:53.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.478+0000 7f50a1875640 1 --2- 172.21.15.47:0/2639766150 >> v2:172.21.15.203:3300/0 conn(0x7f509c162880 0x7f509c0d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:53.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.479+0000 7f50a1875640 1 -- 172.21.15.47:0/2639766150 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f509c0d07d0 con 0x7f509c0b5f20 2024-04-26T23:20:53.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.479+0000 7f50a1875640 1 --2- 172.21.15.47:0/2639766150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f509c0b5f20 0x7f509c0d34b0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f509c08d490 tx=0x7f508c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57c14c54c7f0f02f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:53.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.479+0000 7f50a1074640 1 -- 172.21.15.47:0/2639766150 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508c037070 con 0x7f509c0b5f20 2024-04-26T23:20:53.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.479+0000 7f50a1074640 1 -- 172.21.15.47:0/2639766150 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f508c032440 con 0x7f509c0b5f20 2024-04-26T23:20:53.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.479+0000 7f50a1074640 1 -- 172.21.15.47:0/2639766150 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508c032810 con 0x7f509c0b5f20 2024-04-26T23:20:53.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.480+0000 7f50a42da640 1 -- 172.21.15.47:0/2639766150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f509c0b5f20 msgr2=0x7f509c0d34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:53.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.480+0000 7f50a42da640 1 --2- 172.21.15.47:0/2639766150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f509c0b5f20 0x7f509c0d34b0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f509c08d490 tx=0x7f508c02a1e0 comp rx=0 tx=0).stop 2024-04-26T23:20:53.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.480+0000 7f50a42da640 1 -- 172.21.15.47:0/2639766150 shutdown_connections 2024-04-26T23:20:53.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.480+0000 7f50a42da640 1 --2- 172.21.15.47:0/2639766150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f509c0b5f20 0x7f509c0d34b0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:53.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.480+0000 7f50a42da640 1 --2- 172.21.15.47:0/2639766150 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f509c0d39f0 0x7f509c0b69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:53.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.480+0000 7f50a42da640 1 --2- 172.21.15.47:0/2639766150 >> v2:172.21.15.203:3300/0 conn(0x7f509c162880 0x7f509c0d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:53.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.480+0000 7f50a42da640 1 -- 172.21.15.47:0/2639766150 >> 172.21.15.47:0/2639766150 conn(0x7f509c0d4f60 msgr2=0x7f509c0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:53.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.480+0000 7f50a42da640 1 -- 172.21.15.47:0/2639766150 shutdown_connections 2024-04-26T23:20:53.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.480+0000 7f50a42da640 1 -- 172.21.15.47:0/2639766150 wait complete. 2024-04-26T23:20:53.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.481+0000 7f50a42da640 1 Processor -- start 2024-04-26T23:20:53.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.481+0000 7f50a42da640 1 -- start start 2024-04-26T23:20:53.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.482+0000 7f50a42da640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f509c0b5f20 0x7f509c14c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:53.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.482+0000 7f50a42da640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f509c0d39f0 0x7f509c1f4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:53.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.482+0000 7f50a42da640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f509c162880 0x7f509c0b4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:53.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.482+0000 7f50a42da640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f509c162c60 con 0x7f509c0d39f0 2024-04-26T23:20:53.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.482+0000 7f50a42da640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f509c0d0d90 con 0x7f509c0b5f20 2024-04-26T23:20:53.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.482+0000 7f50a42da640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f509c162de0 con 0x7f509c162880 2024-04-26T23:20:53.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.482+0000 7f50a1875640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f509c0d39f0 0x7f509c1f4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:53.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.482+0000 7f50a1875640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f509c0d39f0 0x7f509c1f4a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48378/0 (socket says 172.21.15.47:48378) 2024-04-26T23:20:53.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.482+0000 7f50a1875640 1 -- 172.21.15.47:0/2265935 learned_addr learned my addr 172.21.15.47:0/2265935 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:20:53.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.482+0000 7f50a2076640 1 --2- 172.21.15.47:0/2265935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f509c0b5f20 0x7f509c14c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:53.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.482+0000 7f50a2877640 1 --2- 172.21.15.47:0/2265935 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f509c162880 0x7f509c0b4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:53.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.482+0000 7f50a1875640 1 -- 172.21.15.47:0/2265935 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f509c162880 msgr2=0x7f509c0b4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:53.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.482+0000 7f50a1875640 1 --2- 172.21.15.47:0/2265935 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f509c162880 0x7f509c0b4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:53.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.482+0000 7f50a1875640 1 -- 172.21.15.47:0/2265935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f509c0b5f20 msgr2=0x7f509c14c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:53.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.482+0000 7f50a1875640 1 --2- 172.21.15.47:0/2265935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f509c0b5f20 0x7f509c14c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:53.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.482+0000 7f50a1875640 1 -- 172.21.15.47:0/2265935 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f509c14cb60 con 0x7f509c0d39f0 2024-04-26T23:20:53.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.482+0000 7f50a2877640 1 --2- 172.21.15.47:0/2265935 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f509c162880 0x7f509c0b4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:20:53.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.483+0000 7f50a1875640 1 --2- 172.21.15.47:0/2265935 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f509c0d39f0 0x7f509c1f4a50 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f508c000c00 tx=0x7f508c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:53.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.483+0000 7f5092ffd640 1 -- 172.21.15.47:0/2265935 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508c041020 con 0x7f509c0d39f0 2024-04-26T23:20:53.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.483+0000 7f5092ffd640 1 -- 172.21.15.47:0/2265935 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f508c0306c0 con 0x7f509c0d39f0 2024-04-26T23:20:53.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.483+0000 7f50a42da640 1 -- 172.21.15.47:0/2265935 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f509c1f5430 con 0x7f509c0d39f0 2024-04-26T23:20:53.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.483+0000 7f5092ffd640 1 -- 172.21.15.47:0/2265935 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508c037040 con 0x7f509c0d39f0 2024-04-26T23:20:53.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.483+0000 7f50a42da640 1 -- 172.21.15.47:0/2265935 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f509c0b1c80 con 0x7f509c0d39f0 2024-04-26T23:20:53.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.484+0000 7f50a42da640 1 -- 172.21.15.47:0/2265935 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f509c0694f0 con 0x7f509c0d39f0 2024-04-26T23:20:53.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.484+0000 7f5092ffd640 1 -- 172.21.15.47:0/2265935 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f508c044020 con 0x7f509c0d39f0 2024-04-26T23:20:53.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.485+0000 7f5092ffd640 1 --2- 172.21.15.47:0/2265935 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f508006c8f0 0x7f508006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:20:53.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.485+0000 7f5092ffd640 1 -- 172.21.15.47:0/2265935 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5335+0+0 (secure 0 0 0) 0x7f508c0ac0f0 con 0x7f509c0d39f0 2024-04-26T23:20:53.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.485+0000 7f50a2076640 1 --2- 172.21.15.47:0/2265935 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f508006c8f0 0x7f508006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:20:53.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.485+0000 7f50a2076640 1 --2- 172.21.15.47:0/2265935 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f508006c8f0 0x7f508006edb0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5098006fd0 tx=0x7f5098008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:20:53.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.489+0000 7f5092ffd640 1 -- 172.21.15.47:0/2265935 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f508c04b030 con 0x7f509c0d39f0 2024-04-26T23:20:53.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.670+0000 7f50a42da640 1 -- 172.21.15.47:0/2265935 --> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f509c074590 con 0x7f508006c8f0 2024-04-26T23:20:53.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.672+0000 7f5092ffd640 1 -- 172.21.15.47:0/2265935 <== mgr.4099 v2:172.21.15.203:6832/992162732 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13418 (secure 0 0 0) 0x7f509c074590 con 0x7f508006c8f0 2024-04-26T23:20:53.674 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:20:53.674 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-26T23:20:53.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.673+0000 7f50a42da640 1 -- 172.21.15.47:0/2265935 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f508006c8f0 msgr2=0x7f508006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:53.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.673+0000 7f50a42da640 1 --2- 172.21.15.47:0/2265935 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f508006c8f0 0x7f508006edb0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5098006fd0 tx=0x7f5098008040 comp rx=0 tx=0).stop 2024-04-26T23:20:53.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.673+0000 7f50a42da640 1 -- 172.21.15.47:0/2265935 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f509c0d39f0 msgr2=0x7f509c1f4a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:20:53.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.673+0000 7f50a42da640 1 --2- 172.21.15.47:0/2265935 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f509c0d39f0 0x7f509c1f4a50 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f508c000c00 tx=0x7f508c0046c0 comp rx=0 tx=0).stop 2024-04-26T23:20:53.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.673+0000 7f50a42da640 1 -- 172.21.15.47:0/2265935 shutdown_connections 2024-04-26T23:20:53.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.673+0000 7f50a42da640 1 --2- 172.21.15.47:0/2265935 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f508006c8f0 0x7f508006edb0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:53.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.673+0000 7f50a42da640 1 --2- 172.21.15.47:0/2265935 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f509c162880 0x7f509c0b4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:53.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.673+0000 7f50a42da640 1 --2- 172.21.15.47:0/2265935 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f509c0d39f0 0x7f509c1f4a50 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:53.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.673+0000 7f50a42da640 1 --2- 172.21.15.47:0/2265935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f509c0b5f20 0x7f509c14c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:20:53.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.673+0000 7f50a42da640 1 -- 172.21.15.47:0/2265935 >> 172.21.15.47:0/2265935 conn(0x7f509c0d4f60 msgr2=0x7f509c0ba210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:20:53.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.674+0000 7f50a42da640 1 -- 172.21.15.47:0/2265935 shutdown_connections 2024-04-26T23:20:53.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:20:53.674+0000 7f50a42da640 1 -- 172.21.15.47:0/2265935 wait complete. 2024-04-26T23:20:53.686 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":128,"stamp":"2024-04-26T23:20:52.962183+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":41,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":239360,"kb_used_data":6536,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735360,"statfs":{"total":773094113280,"available":772849008640,"internally_reserved":0,"allocated":6692864,"data_stored":3656656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":-1,"num_object_clones":0,"num_object_copies":-2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.535913"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":238,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-26T23:20:45.660644+0000","last_change":"2024-04-26T23:18:18.862631+0000","last_active":"2024-04-26T23:20:45.660644+0000","last_peered":"2024-04-26T23:20:45.660644+0000","last_clean":"2024-04-26T23:20:45.660644+0000","last_became_active":"2024-04-26T23:18:18.861536+0000","last_became_peered":"2024-04-26T23:18:18.861536+0000","last_unstale":"2024-04-26T23:20:45.660644+0000","last_undegraded":"2024-04-26T23:20:45.660644+0000","last_fullsized":"2024-04-26T23:20:45.660644+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:18:17.808736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:18:17.808736+0000","last_clean_scrub_stamp":"2024-04-26T23:18:17.808736+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:59:26.359282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542177,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28272,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343568,"statfs":{"total":96636764160,"available":96607813632,"internally_reserved":0,"allocated":1282048,"data_stored":899858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542179,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27696,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":688128,"data_stored":309490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542178,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27696,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":688128,"data_stored":309490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542178,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27696,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":688128,"data_stored":309490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31860,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339980,"statfs":{"total":96636764160,"available":96604139520,"internally_reserved":0,"allocated":688128,"data_stored":309490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574882,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31852,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339988,"statfs":{"total":96636764160,"available":96604147712,"internally_reserved":0,"allocated":688128,"data_stored":309490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31856,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339984,"statfs":{"total":96636764160,"available":96604143616,"internally_reserved":0,"allocated":688128,"data_stored":309490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32432,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339408,"statfs":{"total":96636764160,"available":96603553792,"internally_reserved":0,"allocated":1282048,"data_stored":899858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:20:53.689 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-26T23:18:17.808736+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=23, tm_min=18, tm_sec=17, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=23, tm_min=20, tm_sec=48, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T23:20:53.690 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-26T23:20:58.367 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:20:58.370 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:21:13.690 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T23:21:13.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.805+0000 7fa6faa8c640 1 Processor -- start 2024-04-26T23:21:13.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.805+0000 7fa6faa8c640 1 -- start start 2024-04-26T23:21:13.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.805+0000 7fa6faa8c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa6f415f3a0 0x7fa6f40d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:21:13.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.805+0000 7fa6faa8c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa6f40d1770 0x7fa6f40d1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:21:13.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.805+0000 7fa6faa8c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6f40d2590 con 0x7fa6f40d2160 2024-04-26T23:21:13.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.805+0000 7fa6faa8c640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fa6f40d0e20 con 0x7fa6f40d1770 2024-04-26T23:21:13.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.805+0000 7fa6faa8c640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fa6f40b1d70 con 0x7fa6f415f3a0 2024-04-26T23:21:13.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.806+0000 7fa6f3fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa6f40d1770 0x7fa6f40d1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:21:13.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.806+0000 7fa6f3fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa6f40d1770 0x7fa6f40d1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37406/0 (socket says 172.21.15.47:37406) 2024-04-26T23:21:13.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.806+0000 7fa6f3fff640 1 -- 172.21.15.47:0/1201367039 learned_addr learned my addr 172.21.15.47:0/1201367039 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:21:13.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.806+0000 7fa6f8828640 1 --2- 172.21.15.47:0/1201367039 >> v2:172.21.15.203:3300/0 conn(0x7fa6f415f3a0 0x7fa6f40d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:21:13.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.806+0000 7fa6f3fff640 1 -- 172.21.15.47:0/1201367039 >> v2:172.21.15.203:3300/0 conn(0x7fa6f415f3a0 msgr2=0x7fa6f40d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:21:13.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.806+0000 7fa6f3fff640 1 --2- 172.21.15.47:0/1201367039 >> v2:172.21.15.203:3300/0 conn(0x7fa6f415f3a0 0x7fa6f40d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:21:13.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.806+0000 7fa6f3fff640 1 -- 172.21.15.47:0/1201367039 >> v1:172.21.15.203:6789/0 conn(0x7fa6f40d2160 legacy=0x7fa6f40d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:21:13.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.806+0000 7fa6f3fff640 1 -- 172.21.15.47:0/1201367039 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6f40b2050 con 0x7fa6f40d1770 2024-04-26T23:21:13.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.807+0000 7fa6f3fff640 1 --2- 172.21.15.47:0/1201367039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6f40d1770 0x7fa6f40d1c20 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4073b50 tx=0x7fa6dc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9341d760424105fd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:21:13.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.807+0000 7fa6f37fe640 1 -- 172.21.15.47:0/1201367039 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6dc037070 con 0x7fa6f40d1770 2024-04-26T23:21:13.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.807+0000 7fa6f37fe640 1 -- 172.21.15.47:0/1201367039 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6dc032440 con 0x7fa6f40d1770 2024-04-26T23:21:13.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.807+0000 7fa6f37fe640 1 -- 172.21.15.47:0/1201367039 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6dc032810 con 0x7fa6f40d1770 2024-04-26T23:21:13.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.807+0000 7fa6faa8c640 1 -- 172.21.15.47:0/1201367039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6f40d1770 msgr2=0x7fa6f40d1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:21:13.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.807+0000 7fa6faa8c640 1 --2- 172.21.15.47:0/1201367039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6f40d1770 0x7fa6f40d1c20 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4073b50 tx=0x7fa6dc02a270 comp rx=0 tx=0).stop 2024-04-26T23:21:13.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.807+0000 7fa6faa8c640 1 -- 172.21.15.47:0/1201367039 shutdown_connections 2024-04-26T23:21:13.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.807+0000 7fa6faa8c640 1 --2- 172.21.15.47:0/1201367039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6f40d1770 0x7fa6f40d1c20 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:21:13.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.807+0000 7fa6faa8c640 1 --2- 172.21.15.47:0/1201367039 >> v2:172.21.15.203:3300/0 conn(0x7fa6f415f3a0 0x7fa6f40d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:21:13.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.807+0000 7fa6faa8c640 1 -- 172.21.15.47:0/1201367039 >> 172.21.15.47:0/1201367039 conn(0x7fa6f40d4f90 msgr2=0x7fa6f40d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:21:13.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.808+0000 7fa6faa8c640 1 -- 172.21.15.47:0/1201367039 shutdown_connections 2024-04-26T23:21:13.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.808+0000 7fa6faa8c640 1 -- 172.21.15.47:0/1201367039 wait complete. 2024-04-26T23:21:13.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.809+0000 7fa6faa8c640 1 Processor -- start 2024-04-26T23:21:13.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.809+0000 7fa6faa8c640 1 -- start start 2024-04-26T23:21:13.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.810+0000 7fa6faa8c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6f40d1770 0x7fa6f41533f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:21:13.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.810+0000 7fa6faa8c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6f40d2160 0x7fa6f40b7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:21:13.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.810+0000 7fa6faa8c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6f415f3a0 0x7fa6f415d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:21:13.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.810+0000 7fa6faa8c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f415f900 con 0x7fa6f40d1770 2024-04-26T23:21:13.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.810+0000 7fa6faa8c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f415f780 con 0x7fa6f415f3a0 2024-04-26T23:21:13.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.810+0000 7fa6faa8c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa6f40b3860 con 0x7fa6f40d2160 2024-04-26T23:21:13.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.810+0000 7fa6f8828640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6f40d1770 0x7fa6f41533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:21:13.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.810+0000 7fa6f8828640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6f40d1770 0x7fa6f41533f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37422/0 (socket says 172.21.15.47:37422) 2024-04-26T23:21:13.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.810+0000 7fa6f8828640 1 -- 172.21.15.47:0/58683078 learned_addr learned my addr 172.21.15.47:0/58683078 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:21:13.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.810+0000 7fa6f9029640 1 --2- 172.21.15.47:0/58683078 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6f415f3a0 0x7fa6f415d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:21:13.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.810+0000 7fa6f3fff640 1 --2- 172.21.15.47:0/58683078 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6f40d2160 0x7fa6f40b7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:21:13.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.810+0000 7fa6f8828640 1 -- 172.21.15.47:0/58683078 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6f40d2160 msgr2=0x7fa6f40b7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:21:13.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.810+0000 7fa6f8828640 1 --2- 172.21.15.47:0/58683078 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6f40d2160 0x7fa6f40b7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:21:13.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.810+0000 7fa6f8828640 1 -- 172.21.15.47:0/58683078 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6f415f3a0 msgr2=0x7fa6f415d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:21:13.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.810+0000 7fa6f8828640 1 --2- 172.21.15.47:0/58683078 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6f415f3a0 0x7fa6f415d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:21:13.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.810+0000 7fa6f8828640 1 -- 172.21.15.47:0/58683078 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6f4153930 con 0x7fa6f40d1770 2024-04-26T23:21:13.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.810+0000 7fa6f8828640 1 --2- 172.21.15.47:0/58683078 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6f40d1770 0x7fa6f41533f0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec007e90 tx=0x7fa6ec00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:21:13.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.811+0000 7fa6f17fa640 1 -- 172.21.15.47:0/58683078 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec014620 con 0x7fa6f40d1770 2024-04-26T23:21:13.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.811+0000 7fa6f17fa640 1 -- 172.21.15.47:0/58683078 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6ec019070 con 0x7fa6f40d1770 2024-04-26T23:21:13.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.811+0000 7fa6faa8c640 1 -- 172.21.15.47:0/58683078 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6f40b84f0 con 0x7fa6f40d1770 2024-04-26T23:21:13.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.811+0000 7fa6f17fa640 1 -- 172.21.15.47:0/58683078 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec014a20 con 0x7fa6f40d1770 2024-04-26T23:21:13.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.811+0000 7fa6faa8c640 1 -- 172.21.15.47:0/58683078 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6f40b8ab0 con 0x7fa6f40d1770 2024-04-26T23:21:13.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.812+0000 7fa6faa8c640 1 -- 172.21.15.47:0/58683078 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6f40694f0 con 0x7fa6f40d1770 2024-04-26T23:21:13.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.812+0000 7fa6f17fa640 1 -- 172.21.15.47:0/58683078 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa6ec014bc0 con 0x7fa6f40d1770 2024-04-26T23:21:13.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.812+0000 7fa6f17fa640 1 --2- 172.21.15.47:0/58683078 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa6d406c8f0 0x7fa6d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:21:13.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.813+0000 7fa6f17fa640 1 -- 172.21.15.47:0/58683078 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5335+0+0 (secure 0 0 0) 0x7fa6ec08c110 con 0x7fa6f40d1770 2024-04-26T23:21:13.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.813+0000 7fa6f3fff640 1 --2- 172.21.15.47:0/58683078 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa6d406c8f0 0x7fa6d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:21:13.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.813+0000 7fa6f3fff640 1 --2- 172.21.15.47:0/58683078 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa6d406c8f0 0x7fa6d406edb0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc034000 tx=0x7fa6dc02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:21:13.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:13.817+0000 7fa6f17fa640 1 -- 172.21.15.47:0/58683078 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa6ec05a0f0 con 0x7fa6f40d1770 2024-04-26T23:21:14.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.000+0000 7fa6faa8c640 1 -- 172.21.15.47:0/58683078 --> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa6f40b6860 con 0x7fa6d406c8f0 2024-04-26T23:21:14.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.001+0000 7fa6f17fa640 1 -- 172.21.15.47:0/58683078 <== mgr.4099 v2:172.21.15.203:6832/992162732 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13359 (secure 0 0 0) 0x7fa6f40b6860 con 0x7fa6d406c8f0 2024-04-26T23:21:14.003 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:21:14.003 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-26T23:21:14.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.003+0000 7fa6faa8c640 1 -- 172.21.15.47:0/58683078 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa6d406c8f0 msgr2=0x7fa6d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:21:14.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.003+0000 7fa6faa8c640 1 --2- 172.21.15.47:0/58683078 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa6d406c8f0 0x7fa6d406edb0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc034000 tx=0x7fa6dc02e000 comp rx=0 tx=0).stop 2024-04-26T23:21:14.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.003+0000 7fa6faa8c640 1 -- 172.21.15.47:0/58683078 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6f40d1770 msgr2=0x7fa6f41533f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:21:14.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.003+0000 7fa6faa8c640 1 --2- 172.21.15.47:0/58683078 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6f40d1770 0x7fa6f41533f0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec007e90 tx=0x7fa6ec00e4b0 comp rx=0 tx=0).stop 2024-04-26T23:21:14.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.003+0000 7fa6faa8c640 1 -- 172.21.15.47:0/58683078 shutdown_connections 2024-04-26T23:21:14.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.003+0000 7fa6faa8c640 1 --2- 172.21.15.47:0/58683078 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7fa6d406c8f0 0x7fa6d406edb0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:21:14.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.003+0000 7fa6faa8c640 1 --2- 172.21.15.47:0/58683078 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6f415f3a0 0x7fa6f415d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:21:14.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.003+0000 7fa6faa8c640 1 --2- 172.21.15.47:0/58683078 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6f40d2160 0x7fa6f40b7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:21:14.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.003+0000 7fa6faa8c640 1 --2- 172.21.15.47:0/58683078 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6f40d1770 0x7fa6f41533f0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:21:14.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.003+0000 7fa6faa8c640 1 -- 172.21.15.47:0/58683078 >> 172.21.15.47:0/58683078 conn(0x7fa6f40d4f90 msgr2=0x7fa6f40b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:21:14.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.003+0000 7fa6faa8c640 1 -- 172.21.15.47:0/58683078 shutdown_connections 2024-04-26T23:21:14.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.003+0000 7fa6faa8c640 1 -- 172.21.15.47:0/58683078 wait complete. 2024-04-26T23:21:14.015 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":138,"stamp":"2024-04-26T23:21:12.965304+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":239352,"kb_used_data":6536,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735368,"statfs":{"total":773094113280,"available":772849016832,"internally_reserved":0,"allocated":6692864,"data_stored":3656656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001920"},"pg_stats":[{"pgid":"1.0","version":"57'119","reported_seq":249,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-26T23:20:53.592439+0000","last_change":"2024-04-26T23:20:53.592480+0000","last_active":"2024-04-26T23:20:53.592439+0000","last_peered":"2024-04-26T23:20:53.592439+0000","last_clean":"2024-04-26T23:20:53.592439+0000","last_became_active":"2024-04-26T23:18:18.861536+0000","last_became_peered":"2024-04-26T23:18:18.861536+0000","last_unstale":"2024-04-26T23:20:53.592439+0000","last_undegraded":"2024-04-26T23:20:53.592439+0000","last_fullsized":"2024-04-26T23:20:53.592439+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'119","last_scrub_stamp":"2024-04-26T23:20:53.592348+0000","last_deep_scrub":"57'119","last_deep_scrub_stamp":"2024-04-26T23:20:53.592348+0000","last_clean_scrub_stamp":"2024-04-26T23:20:53.592348+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:58:20.480516+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542181,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28268,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343572,"statfs":{"total":96636764160,"available":96607817728,"internally_reserved":0,"allocated":1282048,"data_stored":899858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542183,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27696,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":688128,"data_stored":309490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542182,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27696,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":688128,"data_stored":309490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542182,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27696,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":688128,"data_stored":309490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31860,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339980,"statfs":{"total":96636764160,"available":96604139520,"internally_reserved":0,"allocated":688128,"data_stored":309490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31852,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339988,"statfs":{"total":96636764160,"available":96604147712,"internally_reserved":0,"allocated":688128,"data_stored":309490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31856,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339984,"statfs":{"total":96636764160,"available":96604143616,"internally_reserved":0,"allocated":688128,"data_stored":309490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574886,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32428,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1282048,"data_stored":899858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:21:14.016 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-26T23:21:14.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.116+0000 7f01584bb640 1 Processor -- start 2024-04-26T23:21:14.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.116+0000 7f01584bb640 1 -- start start 2024-04-26T23:21:14.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.117+0000 7f01584bb640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f015015f550 0x7f01500d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:21:14.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.117+0000 7f01584bb640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f01500d3bc0 0x7f01500d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:21:14.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.117+0000 7f01584bb640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01500d1040 con 0x7f015015f170 2024-04-26T23:21:14.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.117+0000 7f01584bb640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f01500c9430 con 0x7f015015f550 2024-04-26T23:21:14.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.117+0000 7f01584bb640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f01500c9680 con 0x7f01500d3bc0 2024-04-26T23:21:14.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.117+0000 7f0155a56640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f015015f550 0x7f01500d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:21:14.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.117+0000 7f0156257640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f015015f170 0x7f01500d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:53666/0 (socket says 172.21.15.47:53666) 2024-04-26T23:21:14.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.117+0000 7f0156257640 1 -- 172.21.15.47:0/743276304 learned_addr learned my addr 172.21.15.47:0/743276304 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-26T23:21:14.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.117+0000 7f0156a58640 1 --2- 172.21.15.47:0/743276304 >> v2:172.21.15.203:3300/0 conn(0x7f01500d3bc0 0x7f01500d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:21:14.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.117+0000 7f0155a56640 1 -- 172.21.15.47:0/743276304 >> v2:172.21.15.203:3300/0 conn(0x7f01500d3bc0 msgr2=0x7f01500d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:21:14.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.117+0000 7f0155a56640 1 --2- 172.21.15.47:0/743276304 >> v2:172.21.15.203:3300/0 conn(0x7f01500d3bc0 0x7f01500d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:21:14.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.117+0000 7f0155a56640 1 -- 172.21.15.47:0/743276304 >> v1:172.21.15.47:6789/0 conn(0x7f015015f170 legacy=0x7f01500d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:21:14.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.117+0000 7f0155a56640 1 -- 172.21.15.47:0/743276304 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01500c9960 con 0x7f015015f550 2024-04-26T23:21:14.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.117+0000 7f0155255640 1 -- 172.21.15.47:0/743276304 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4171422343 0 0) 0x7f014c001110 con 0x7f015015f170 2024-04-26T23:21:14.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.118+0000 7f0155a56640 1 --2- 172.21.15.47:0/743276304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f015015f550 0x7f01500d5a80 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f01400097a0 tx=0x7f014002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e06279b340c17f0a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:21:14.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.118+0000 7f0155255640 1 -- 172.21.15.47:0/743276304 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0140037070 con 0x7f015015f550 2024-04-26T23:21:14.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.118+0000 7f0155255640 1 -- 172.21.15.47:0/743276304 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0140032440 con 0x7f015015f550 2024-04-26T23:21:14.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.118+0000 7f0155255640 1 -- 172.21.15.47:0/743276304 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0140032810 con 0x7f015015f550 2024-04-26T23:21:14.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.119+0000 7f01584bb640 1 -- 172.21.15.47:0/743276304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f015015f550 msgr2=0x7f01500d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:21:14.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.119+0000 7f01584bb640 1 --2- 172.21.15.47:0/743276304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f015015f550 0x7f01500d5a80 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f01400097a0 tx=0x7f014002a1e0 comp rx=0 tx=0).stop 2024-04-26T23:21:14.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.119+0000 7f01584bb640 1 -- 172.21.15.47:0/743276304 shutdown_connections 2024-04-26T23:21:14.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.119+0000 7f01584bb640 1 --2- 172.21.15.47:0/743276304 >> v2:172.21.15.203:3300/0 conn(0x7f01500d3bc0 0x7f01500d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:21:14.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.119+0000 7f01584bb640 1 --2- 172.21.15.47:0/743276304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f015015f550 0x7f01500d5a80 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:21:14.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.119+0000 7f01584bb640 1 -- 172.21.15.47:0/743276304 >> 172.21.15.47:0/743276304 conn(0x7f01500d4f60 msgr2=0x7f01500d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:21:14.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.119+0000 7f01584bb640 1 -- 172.21.15.47:0/743276304 shutdown_connections 2024-04-26T23:21:14.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.119+0000 7f01584bb640 1 -- 172.21.15.47:0/743276304 wait complete. 2024-04-26T23:21:14.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.120+0000 7f01584bb640 1 Processor -- start 2024-04-26T23:21:14.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.120+0000 7f01584bb640 1 -- start start 2024-04-26T23:21:14.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.121+0000 7f01584bb640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01500d3bc0 0x7f01500cee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:21:14.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.121+0000 7f01584bb640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f015015f170 0x7f01500cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:21:14.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.121+0000 7f01584bb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0150152210 0x7f015015a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:21:14.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.121+0000 7f01584bb640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f01500ca5c0 con 0x7f0150152210 2024-04-26T23:21:14.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.121+0000 7f01584bb640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f01500ca440 con 0x7f01500d3bc0 2024-04-26T23:21:14.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.121+0000 7f01584bb640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f01500ca740 con 0x7f015015f170 2024-04-26T23:21:14.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.121+0000 7f0156a58640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0150152210 0x7f015015a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:21:14.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.121+0000 7f0156a58640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0150152210 0x7f015015a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37440/0 (socket says 172.21.15.47:37440) 2024-04-26T23:21:14.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.121+0000 7f0156a58640 1 -- 172.21.15.47:0/1401539788 learned_addr learned my addr 172.21.15.47:0/1401539788 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:21:14.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.121+0000 7f0156257640 1 --2- 172.21.15.47:0/1401539788 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01500d3bc0 0x7f01500cee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:21:14.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.121+0000 7f0155a56640 1 --2- 172.21.15.47:0/1401539788 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f015015f170 0x7f01500cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:21:14.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.121+0000 7f0156a58640 1 -- 172.21.15.47:0/1401539788 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f015015f170 msgr2=0x7f01500cbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:21:14.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.121+0000 7f0156a58640 1 --2- 172.21.15.47:0/1401539788 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f015015f170 0x7f01500cbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:21:14.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.121+0000 7f0156a58640 1 -- 172.21.15.47:0/1401539788 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01500d3bc0 msgr2=0x7f01500cee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:21:14.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.121+0000 7f0156a58640 1 --2- 172.21.15.47:0/1401539788 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01500d3bc0 0x7f01500cee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:21:14.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.121+0000 7f0156a58640 1 -- 172.21.15.47:0/1401539788 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01500cc710 con 0x7f0150152210 2024-04-26T23:21:14.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.121+0000 7f0156257640 1 --2- 172.21.15.47:0/1401539788 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01500d3bc0 0x7f01500cee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:21:14.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.121+0000 7f0155a56640 1 --2- 172.21.15.47:0/1401539788 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f015015f170 0x7f01500cbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:21:14.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.121+0000 7f0156a58640 1 --2- 172.21.15.47:0/1401539788 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0150152210 0x7f015015a970 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f0148007e70 tx=0x7f014800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:21:14.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.121+0000 7f0146ffd640 1 -- 172.21.15.47:0/1401539788 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0148014620 con 0x7f0150152210 2024-04-26T23:21:14.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.121+0000 7f0146ffd640 1 -- 172.21.15.47:0/1401539788 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0148019070 con 0x7f0150152210 2024-04-26T23:21:14.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.121+0000 7f01584bb640 1 -- 172.21.15.47:0/1401539788 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01500cc9f0 con 0x7f0150152210 2024-04-26T23:21:14.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.122+0000 7f0146ffd640 1 -- 172.21.15.47:0/1401539788 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0148014a20 con 0x7f0150152210 2024-04-26T23:21:14.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.122+0000 7f01584bb640 1 -- 172.21.15.47:0/1401539788 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01500ccfb0 con 0x7f0150152210 2024-04-26T23:21:14.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.122+0000 7f01584bb640 1 -- 172.21.15.47:0/1401539788 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01500694f0 con 0x7f0150152210 2024-04-26T23:21:14.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.123+0000 7f0146ffd640 1 -- 172.21.15.47:0/1401539788 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0148014bc0 con 0x7f0150152210 2024-04-26T23:21:14.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.123+0000 7f0146ffd640 1 --2- 172.21.15.47:0/1401539788 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f013406c8f0 0x7f013406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:21:14.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.123+0000 7f0146ffd640 1 -- 172.21.15.47:0/1401539788 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5335+0+0 (secure 0 0 0) 0x7f014808c310 con 0x7f0150152210 2024-04-26T23:21:14.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.124+0000 7f0156257640 1 --2- 172.21.15.47:0/1401539788 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f013406c8f0 0x7f013406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:21:14.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.124+0000 7f0156257640 1 --2- 172.21.15.47:0/1401539788 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f013406c8f0 0x7f013406edb0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f014c000f60 tx=0x7f014c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:21:14.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.128+0000 7f0146ffd640 1 -- 172.21.15.47:0/1401539788 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0148011260 con 0x7f0150152210 2024-04-26T23:21:14.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.314+0000 7f01584bb640 1 -- 172.21.15.47:0/1401539788 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f015015ea90 con 0x7f0150152210 2024-04-26T23:21:14.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.317+0000 7f0146ffd640 1 -- 172.21.15.47:0/1401539788 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v14)=0 v14) ==== 123+0+0 (secure 0 0 0) 0x7f014805a940 con 0x7f0150152210 2024-04-26T23:21:14.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.318+0000 7f0146ffd640 1 -- 172.21.15.47:0/1401539788 <== mon.0 v2:172.21.15.47:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f014805a240 con 0x7f0150152210 2024-04-26T23:21:14.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.320+0000 7f01584bb640 1 -- 172.21.15.47:0/1401539788 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f013406c8f0 msgr2=0x7f013406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:21:14.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.320+0000 7f01584bb640 1 --2- 172.21.15.47:0/1401539788 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f013406c8f0 0x7f013406edb0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f014c000f60 tx=0x7f014c009040 comp rx=0 tx=0).stop 2024-04-26T23:21:14.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.320+0000 7f01584bb640 1 -- 172.21.15.47:0/1401539788 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0150152210 msgr2=0x7f015015a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:21:14.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.320+0000 7f01584bb640 1 --2- 172.21.15.47:0/1401539788 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0150152210 0x7f015015a970 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f0148007e70 tx=0x7f014800e4b0 comp rx=0 tx=0).stop 2024-04-26T23:21:14.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.320+0000 7f01584bb640 1 -- 172.21.15.47:0/1401539788 shutdown_connections 2024-04-26T23:21:14.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.320+0000 7f01584bb640 1 --2- 172.21.15.47:0/1401539788 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f013406c8f0 0x7f013406edb0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:21:14.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.320+0000 7f01584bb640 1 --2- 172.21.15.47:0/1401539788 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0150152210 0x7f015015a970 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:21:14.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.320+0000 7f01584bb640 1 --2- 172.21.15.47:0/1401539788 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f015015f170 0x7f01500cbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:21:14.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.320+0000 7f01584bb640 1 --2- 172.21.15.47:0/1401539788 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01500d3bc0 0x7f01500cee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:21:14.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.320+0000 7f01584bb640 1 -- 172.21.15.47:0/1401539788 >> 172.21.15.47:0/1401539788 conn(0x7f01500d4f60 msgr2=0x7f01500d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:21:14.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.320+0000 7f01584bb640 1 -- 172.21.15.47:0/1401539788 shutdown_connections 2024-04-26T23:21:14.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.320+0000 7f01584bb640 1 -- 172.21.15.47:0/1401539788 wait complete. 2024-04-26T23:21:14.332 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T23:21:14.332 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:21:14.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.432+0000 7f7e270da640 1 Processor -- start 2024-04-26T23:21:14.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.432+0000 7f7e270da640 1 -- start start 2024-04-26T23:21:14.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.433+0000 7f7e270da640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7e200d1380 0x7f7e200d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:21:14.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.433+0000 7f7e270da640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e200d1ce0 con 0x7f7e20070ae0 2024-04-26T23:21:14.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.433+0000 7f7e270da640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e200ce840 con 0x7f7e2015d1e0 2024-04-26T23:21:14.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.433+0000 7f7e270da640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f7e200cede0 con 0x7f7e200d1380 2024-04-26T23:21:14.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.433+0000 7f7e1ffff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7e200d1380 0x7f7e200d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:21:14.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.433+0000 7f7e1ffff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7e200d1380 0x7f7e200d17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37452/0 (socket says 172.21.15.47:37452) 2024-04-26T23:21:14.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.433+0000 7f7e25677640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7e20070ae0 0x7f7e20070f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:53670/0 (socket says 172.21.15.47:53670) 2024-04-26T23:21:14.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.433+0000 7f7e1ffff640 1 -- 172.21.15.47:0/3087337201 learned_addr learned my addr 172.21.15.47:0/3087337201 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:21:14.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.434+0000 7f7e1ffff640 1 -- 172.21.15.47:0/3087337201 >> v1:172.21.15.203:6789/0 conn(0x7f7e2015d1e0 legacy=0x7f7e200d09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:21:14.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.434+0000 7f7e1ffff640 1 -- 172.21.15.47:0/3087337201 >> v1:172.21.15.47:6789/0 conn(0x7f7e20070ae0 legacy=0x7f7e20070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:21:14.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.434+0000 7f7e1f7fe640 1 -- 172.21.15.47:0/3087337201 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 398396459 0 0) 0x7f7e200d1ce0 con 0x7f7e20070ae0 2024-04-26T23:21:14.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.434+0000 7f7e1ffff640 1 -- 172.21.15.47:0/3087337201 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e200ceb00 con 0x7f7e200d1380 2024-04-26T23:21:14.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.434+0000 7f7e1ffff640 1 --2- 172.21.15.47:0/3087337201 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e200d1380 0x7f7e200d17a0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f7e10009e50 tx=0x7f7e1002a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=573a4a930b042790 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:21:14.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.435+0000 7f7e1f7fe640 1 -- 172.21.15.47:0/3087337201 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e10036070 con 0x7f7e200d1380 2024-04-26T23:21:14.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.435+0000 7f7e1f7fe640 1 -- 172.21.15.47:0/3087337201 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f7e10031600 con 0x7f7e200d1380 2024-04-26T23:21:14.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.435+0000 7f7e1f7fe640 1 -- 172.21.15.47:0/3087337201 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e10031a10 con 0x7f7e200d1380 2024-04-26T23:21:14.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.435+0000 7f7e270da640 1 -- 172.21.15.47:0/3087337201 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e200d1380 msgr2=0x7f7e200d17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:21:14.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.435+0000 7f7e270da640 1 --2- 172.21.15.47:0/3087337201 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e200d1380 0x7f7e200d17a0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f7e10009e50 tx=0x7f7e1002a7c0 comp rx=0 tx=0).stop 2024-04-26T23:21:14.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.435+0000 7f7e270da640 1 -- 172.21.15.47:0/3087337201 shutdown_connections 2024-04-26T23:21:14.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.435+0000 7f7e270da640 1 --2- 172.21.15.47:0/3087337201 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e200d1380 0x7f7e200d17a0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:21:14.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.435+0000 7f7e270da640 1 -- 172.21.15.47:0/3087337201 >> 172.21.15.47:0/3087337201 conn(0x7f7e200d2da0 msgr2=0x7f7e200d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:21:14.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.435+0000 7f7e270da640 1 -- 172.21.15.47:0/3087337201 shutdown_connections 2024-04-26T23:21:14.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.435+0000 7f7e270da640 1 -- 172.21.15.47:0/3087337201 wait complete. 2024-04-26T23:21:14.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.436+0000 7f7e270da640 1 Processor -- start 2024-04-26T23:21:14.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.436+0000 7f7e270da640 1 -- start start 2024-04-26T23:21:14.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.437+0000 7f7e270da640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e20070ae0 0x7f7e2014d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:21:14.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.437+0000 7f7e270da640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7e200d1380 0x7f7e200cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:21:14.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.437+0000 7f7e270da640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7e2015d1e0 0x7f7e2015ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:21:14.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.437+0000 7f7e270da640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7e200d10f0 con 0x7f7e20070ae0 2024-04-26T23:21:14.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.437+0000 7f7e270da640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7e20070650 con 0x7f7e2015d1e0 2024-04-26T23:21:14.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.437+0000 7f7e270da640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f7e200ce4c0 con 0x7f7e200d1380 2024-04-26T23:21:14.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.437+0000 7f7e24e76640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e20070ae0 0x7f7e2014d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:21:14.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.437+0000 7f7e24e76640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e20070ae0 0x7f7e2014d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37462/0 (socket says 172.21.15.47:37462) 2024-04-26T23:21:14.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.437+0000 7f7e24e76640 1 -- 172.21.15.47:0/2314660271 learned_addr learned my addr 172.21.15.47:0/2314660271 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-26T23:21:14.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.438+0000 7f7e1ffff640 1 --2- 172.21.15.47:0/2314660271 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7e200d1380 0x7f7e200cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:21:14.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.438+0000 7f7e25677640 1 --2- 172.21.15.47:0/2314660271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7e2015d1e0 0x7f7e2015ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:21:14.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.438+0000 7f7e24e76640 1 -- 172.21.15.47:0/2314660271 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7e200d1380 msgr2=0x7f7e200cafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:21:14.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.438+0000 7f7e24e76640 1 --2- 172.21.15.47:0/2314660271 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7e200d1380 0x7f7e200cafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:21:14.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.438+0000 7f7e24e76640 1 -- 172.21.15.47:0/2314660271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7e2015d1e0 msgr2=0x7f7e2015ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:21:14.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.438+0000 7f7e24e76640 1 --2- 172.21.15.47:0/2314660271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7e2015d1e0 0x7f7e2015ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:21:14.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.438+0000 7f7e24e76640 1 -- 172.21.15.47:0/2314660271 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e2014d720 con 0x7f7e20070ae0 2024-04-26T23:21:14.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.438+0000 7f7e24e76640 1 --2- 172.21.15.47:0/2314660271 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e20070ae0 0x7f7e2014d1e0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f7e1800c400 tx=0x7f7e1800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:21:14.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.438+0000 7f7e1d7fa640 1 -- 172.21.15.47:0/2314660271 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e18017070 con 0x7f7e20070ae0 2024-04-26T23:21:14.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.438+0000 7f7e1d7fa640 1 -- 172.21.15.47:0/2314660271 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f7e180052d0 con 0x7f7e20070ae0 2024-04-26T23:21:14.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.438+0000 7f7e270da640 1 -- 172.21.15.47:0/2314660271 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e200cb9d0 con 0x7f7e20070ae0 2024-04-26T23:21:14.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.438+0000 7f7e1d7fa640 1 -- 172.21.15.47:0/2314660271 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e18003850 con 0x7f7e20070ae0 2024-04-26T23:21:14.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.438+0000 7f7e270da640 1 -- 172.21.15.47:0/2314660271 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e200cbf90 con 0x7f7e20070ae0 2024-04-26T23:21:14.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.439+0000 7f7e270da640 1 -- 172.21.15.47:0/2314660271 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e200694f0 con 0x7f7e20070ae0 2024-04-26T23:21:14.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.440+0000 7f7e1d7fa640 1 -- 172.21.15.47:0/2314660271 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7e1801a070 con 0x7f7e20070ae0 2024-04-26T23:21:14.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.440+0000 7f7e1d7fa640 1 --2- 172.21.15.47:0/2314660271 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f7e0006c8f0 0x7f7e0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:21:14.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.440+0000 7f7e1d7fa640 1 -- 172.21.15.47:0/2314660271 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5335+0+0 (secure 0 0 0) 0x7f7e1808ad80 con 0x7f7e20070ae0 2024-04-26T23:21:14.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.440+0000 7f7e1ffff640 1 --2- 172.21.15.47:0/2314660271 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f7e0006c8f0 0x7f7e0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:21:14.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.441+0000 7f7e1ffff640 1 --2- 172.21.15.47:0/2314660271 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f7e0006c8f0 0x7f7e0006edb0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7e10033000 tx=0x7f7e10009000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:21:14.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.444+0000 7f7e1d7fa640 1 -- 172.21.15.47:0/2314660271 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7e2014d720 con 0x7f7e20070ae0 2024-04-26T23:21:14.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.668+0000 7f7e270da640 1 -- 172.21.15.47:0/2314660271 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e2015c840 con 0x7f7e20070ae0 2024-04-26T23:21:14.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.669+0000 7f7e1d7fa640 1 -- 172.21.15.47:0/2314660271 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+3874 (secure 0 0 0) 0x7f7e18059290 con 0x7f7e20070ae0 2024-04-26T23:21:14.670 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-26T23:21:14.670 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":29,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5865,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/805147373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":805147373},{"type":"v1","addr":"172.21.15.47:6833","nonce":805147373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5868,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3515322868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3515322868},{"type":"v1","addr":"172.21.15.47:6835","nonce":3515322868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":15008,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/4223205269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":4223205269},{"type":"v1","addr":"172.21.15.203:6837","nonce":4223205269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":15236,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/4230269935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":4230269935},{"type":"v1","addr":"172.21.15.203:6836","nonce":4230269935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[]} 2024-04-26T23:21:14.670 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 29 2024-04-26T23:21:14.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.671+0000 7f7e270da640 1 -- 172.21.15.47:0/2314660271 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f7e0006c8f0 msgr2=0x7f7e0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:21:14.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.671+0000 7f7e270da640 1 --2- 172.21.15.47:0/2314660271 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f7e0006c8f0 0x7f7e0006edb0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7e10033000 tx=0x7f7e10009000 comp rx=0 tx=0).stop 2024-04-26T23:21:14.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.671+0000 7f7e270da640 1 -- 172.21.15.47:0/2314660271 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e20070ae0 msgr2=0x7f7e2014d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:21:14.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.671+0000 7f7e270da640 1 --2- 172.21.15.47:0/2314660271 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e20070ae0 0x7f7e2014d1e0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f7e1800c400 tx=0x7f7e1800c8c0 comp rx=0 tx=0).stop 2024-04-26T23:21:14.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.671+0000 7f7e270da640 1 -- 172.21.15.47:0/2314660271 shutdown_connections 2024-04-26T23:21:14.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.671+0000 7f7e270da640 1 --2- 172.21.15.47:0/2314660271 >> [v2:172.21.15.203:6832/992162732,v1:172.21.15.203:6833/992162732] conn(0x7f7e0006c8f0 0x7f7e0006edb0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:21:14.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.671+0000 7f7e270da640 1 --2- 172.21.15.47:0/2314660271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7e2015d1e0 0x7f7e2015ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:21:14.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.671+0000 7f7e270da640 1 --2- 172.21.15.47:0/2314660271 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7e200d1380 0x7f7e200cafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:21:14.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.671+0000 7f7e270da640 1 --2- 172.21.15.47:0/2314660271 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e20070ae0 0x7f7e2014d1e0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:21:14.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.671+0000 7f7e270da640 1 -- 172.21.15.47:0/2314660271 >> 172.21.15.47:0/2314660271 conn(0x7f7e200d2da0 msgr2=0x7f7e200d17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:21:14.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.671+0000 7f7e270da640 1 -- 172.21.15.47:0/2314660271 shutdown_connections 2024-04-26T23:21:14.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-26T23:21:14.671+0000 7f7e270da640 1 -- 172.21.15.47:0/2314660271 wait complete. 2024-04-26T23:21:14.683 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-26T23:21:14.683 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-26T23:21:14.684 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T23:21:14.684 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:21:14.693 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T23:21:14.693 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T23:21:14.693 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:21:14.742 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T23:21:14.742 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T23:21:14.743 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:21:14.755 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T23:21:14.755 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T23:21:14.755 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:21:14.808 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T23:21:14.808 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-26T23:21:14.808 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-26T23:21:14.808 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:21:14.838 INFO:tasks.ceph.osd.0:Stopped 2024-04-26T23:21:14.838 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-26T23:21:14.838 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:21:14.913 INFO:tasks.ceph.osd.1:Stopped 2024-04-26T23:21:14.913 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-26T23:21:14.913 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:21:14.979 INFO:tasks.ceph.osd.2:Stopped 2024-04-26T23:21:14.980 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-26T23:21:14.980 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:21:15.048 INFO:tasks.ceph.osd.3:Stopped 2024-04-26T23:21:15.049 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-26T23:21:15.049 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:21:15.082 INFO:tasks.ceph.osd.4:Stopped 2024-04-26T23:21:15.083 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-26T23:21:15.083 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:21:15.159 INFO:tasks.ceph.osd.5:Stopped 2024-04-26T23:21:15.159 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-26T23:21:15.159 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:21:15.232 INFO:tasks.ceph.osd.6:Stopped 2024-04-26T23:21:15.232 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-26T23:21:15.232 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:21:15.299 INFO:tasks.ceph.osd.7:Stopped 2024-04-26T23:21:15.300 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-26T23:21:15.300 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-26T23:21:15.300 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:21:15.324 INFO:tasks.ceph.mgr.y:Stopped 2024-04-26T23:21:15.325 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-26T23:21:15.325 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:21:15.369 INFO:tasks.ceph.mgr.x:Stopped 2024-04-26T23:21:15.369 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-26T23:21:15.370 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-26T23:21:15.370 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:21:15.381 INFO:tasks.ceph.mon.a:Stopped 2024-04-26T23:21:15.381 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-26T23:21:15.381 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:21:15.422 INFO:tasks.ceph.mon.b:Stopped 2024-04-26T23:21:15.422 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-26T23:21:15.422 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:21:15.475 INFO:tasks.ceph.mon.c:Stopped 2024-04-26T23:21:15.475 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-26T23:21:15.476 DEBUG:teuthology.orchestra.run.smithi047:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-26T23:21:15.511 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi047.front.sepia.ceph.com 2024-04-26T23:21:15.511 DEBUG:teuthology.orchestra.run.smithi047:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-26T23:21:15.736 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi047.front.sepia.ceph.com 2024-04-26T23:21:15.736 DEBUG:teuthology.orchestra.run.smithi047:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-26T23:21:15.836 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi047.front.sepia.ceph.com 2024-04-26T23:21:15.836 DEBUG:teuthology.orchestra.run.smithi047:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-26T23:21:15.882 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi047.front.sepia.ceph.com 2024-04-26T23:21:15.883 DEBUG:teuthology.orchestra.run.smithi047:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-26T23:21:15.933 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi203.front.sepia.ceph.com 2024-04-26T23:21:15.934 DEBUG:teuthology.orchestra.run.smithi203:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-26T23:21:16.366 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi203.front.sepia.ceph.com 2024-04-26T23:21:16.367 DEBUG:teuthology.orchestra.run.smithi203:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-26T23:21:16.431 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi203.front.sepia.ceph.com 2024-04-26T23:21:16.432 DEBUG:teuthology.orchestra.run.smithi203:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-26T23:21:16.515 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi203.front.sepia.ceph.com 2024-04-26T23:21:16.515 DEBUG:teuthology.orchestra.run.smithi203:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-26T23:21:16.574 INFO:tasks.ceph:Archiving mon data... 2024-04-26T23:21:16.575 DEBUG:teuthology.misc:Transferring archived files from smithi047:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675294/data/mon.a.tgz 2024-04-26T23:21:16.575 DEBUG:teuthology.orchestra.run.smithi047:> mktemp 2024-04-26T23:21:16.592 INFO:teuthology.orchestra.run.smithi047.stdout:/tmp/tmp.cmTXoCNW94 2024-04-26T23:21:16.592 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.cmTXoCNW94 2024-04-26T23:21:16.738 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0666 /tmp/tmp.cmTXoCNW94 2024-04-26T23:21:16.821 DEBUG:teuthology.orchestra.remote:smithi047:/tmp/tmp.cmTXoCNW94 is 355KB 2024-04-26T23:21:16.918 DEBUG:teuthology.orchestra.run.smithi047:> rm -fr /tmp/tmp.cmTXoCNW94 2024-04-26T23:21:16.936 DEBUG:teuthology.misc:Transferring archived files from smithi203:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675294/data/mon.b.tgz 2024-04-26T23:21:16.936 DEBUG:teuthology.orchestra.run.smithi203:> mktemp 2024-04-26T23:21:16.954 INFO:teuthology.orchestra.run.smithi203.stdout:/tmp/tmp.QCBjFv1G8u 2024-04-26T23:21:16.954 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.QCBjFv1G8u 2024-04-26T23:21:17.127 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 0666 /tmp/tmp.QCBjFv1G8u 2024-04-26T23:21:17.209 DEBUG:teuthology.orchestra.remote:smithi203:/tmp/tmp.QCBjFv1G8u is 465KB 2024-04-26T23:21:17.307 DEBUG:teuthology.orchestra.run.smithi203:> rm -fr /tmp/tmp.QCBjFv1G8u 2024-04-26T23:21:17.322 DEBUG:teuthology.misc:Transferring archived files from smithi203:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675294/data/mon.c.tgz 2024-04-26T23:21:17.322 DEBUG:teuthology.orchestra.run.smithi203:> mktemp 2024-04-26T23:21:17.375 INFO:teuthology.orchestra.run.smithi203.stdout:/tmp/tmp.HN1WBZ9j8G 2024-04-26T23:21:17.376 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.HN1WBZ9j8G 2024-04-26T23:21:17.526 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 0666 /tmp/tmp.HN1WBZ9j8G 2024-04-26T23:21:17.608 DEBUG:teuthology.orchestra.remote:smithi203:/tmp/tmp.HN1WBZ9j8G is 463KB 2024-04-26T23:21:17.706 DEBUG:teuthology.orchestra.run.smithi203:> rm -fr /tmp/tmp.HN1WBZ9j8G 2024-04-26T23:21:17.720 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-26T23:21:17.721 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-26T23:21:17.723 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-26T23:21:17.807 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-26T23:21:17.839 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-26T23:21:17.839 INFO:tasks.ceph:Archiving crash dumps... 2024-04-26T23:21:17.841 DEBUG:teuthology.misc:Transferring archived files from smithi047:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675294/remote/smithi047/crash 2024-04-26T23:21:17.842 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-26T23:21:17.874 DEBUG:teuthology.misc:Transferring archived files from smithi203:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675294/remote/smithi203/crash 2024-04-26T23:21:17.875 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-26T23:21:17.901 INFO:tasks.ceph:Shutting down logrotate 2024-04-26T23:21:17.902 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:21:17.919 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:21:17.987 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-26T23:21:18.012 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-26T23:21:18.051 INFO:tasks.ceph:Compressing logs... 2024-04-26T23:21:18.051 DEBUG:teuthology.orchestra.run.smithi047:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T23:21:18.055 DEBUG:teuthology.orchestra.run.smithi203:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T23:21:18.078 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40046.log 2024-04-26T23:21:18.078 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42954.log 2024-04-26T23:21:18.078 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.40046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46095.log 2024-04-26T23:21:18.149 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41199.log 2024-04-26T23:21:18.149 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34542.log 2024-04-26T23:21:18.149 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.1.41199.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-26T23:21:18.149 INFO:teuthology.orchestra.run.smithi203.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.41199.log.gz 2024-04-26T23:21:18.149 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.admin.34542.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-26T23:21:18.149 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34502.log 2024-04-26T23:21:18.149 INFO:teuthology.orchestra.run.smithi203.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.34542.log.gz 2024-04-26T23:21:18.149 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-26T23:21:18.150 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.admin.34502.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-26T23:21:18.150 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-osd.7.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34502.log.gz 2024-04-26T23:21:18.150 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34633.log 2024-04-26T23:21:18.150 INFO:teuthology.orchestra.run.smithi203.stderr: 91.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-26T23:21:18.150 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-26T23:21:18.150 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-mds.b.log: /var/log/ceph/ceph-client.admin.34633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.41599.log 2024-04-26T23:21:18.150 INFO:teuthology.orchestra.run.smithi203.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34633.log.gz 2024-04-26T23:21:18.150 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-26T23:21:18.150 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.41603.log 2024-04-26T23:21:18.150 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.3.41599.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.41599.log.gz 2024-04-26T23:21:18.150 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34753.log 2024-04-26T23:21:18.150 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.3.41603.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.3.41603.log.gz 2024-04-26T23:21:18.150 INFO:teuthology.orchestra.run.smithi203.stderr: --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-26T23:21:18.151 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.admin.34753.log: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-26T23:21:18.151 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-26T23:21:18.151 INFO:teuthology.orchestra.run.smithi203.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.34753.log.gz 2024-04-26T23:21:18.151 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41204.log 2024-04-26T23:21:18.151 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-26T23:21:18.151 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.1.41204.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.41204.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34793.log 2024-04-26T23:21:18.151 INFO:teuthology.orchestra.run.smithi203.stderr: 2024-04-26T23:21:18.151 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34713.log 2024-04-26T23:21:18.151 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-osd.6.log: /var/log/ceph/ceph-client.admin.34793.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34793.log.gz 2024-04-26T23:21:18.151 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34673.log 2024-04-26T23:21:18.151 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.admin.34713.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.34713.log.gz 2024-04-26T23:21:18.151 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34582.log 2024-04-26T23:21:18.151 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.admin.34673.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34673.log.gz 2024-04-26T23:21:18.151 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-26T23:21:18.151 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.admin.34582.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34582.log.gz 2024-04-26T23:21:18.152 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-26T23:21:18.152 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-mds.d.log: 91.6% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-26T23:21:18.188 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-mon.b.log: 91.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-26T23:21:18.228 INFO:teuthology.orchestra.run.smithi203.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-26T23:21:18.235 INFO:teuthology.orchestra.run.smithi203.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-26T23:21:18.245 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42954.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40046.log.gzgzip 2024-04-26T23:21:18.245 INFO:teuthology.orchestra.run.smithi047.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.45507.log 2024-04-26T23:21:18.245 INFO:teuthology.orchestra.run.smithi047.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42954.log.gz 2024-04-26T23:21:18.245 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.46095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45045.log 2024-04-26T23:21:18.245 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45843.log 2024-04-26T23:21:18.245 INFO:teuthology.orchestra.run.smithi047.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46095.log.gz 2024-04-26T23:21:18.245 INFO:teuthology.orchestra.run.smithi047.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45507.log.gz 2024-04-26T23:21:18.245 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53772.log 2024-04-26T23:21:18.245 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48832.log 2024-04-26T23:21:18.245 INFO:teuthology.orchestra.run.smithi047.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45045.log.gz 2024-04-26T23:21:18.245 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.53772.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45843.log.gz 2024-04-26T23:21:18.246 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40319.log 2024-04-26T23:21:18.246 INFO:teuthology.orchestra.run.smithi047.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53772.log.gz 2024-04-26T23:21:18.246 INFO:teuthology.orchestra.run.smithi047.stderr:gzip/var/log/ceph/ceph-client.admin.48832.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.45885.log 2024-04-26T23:21:18.246 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.40319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49000.log 2024-04-26T23:21:18.246 INFO:teuthology.orchestra.run.smithi047.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48832.log.gz 2024-04-26T23:21:18.246 INFO:teuthology.orchestra.run.smithi047.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40319.log.gz/var/log/ceph/ceph-client.admin.45885.log: 2024-04-26T23:21:18.246 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39827.log 2024-04-26T23:21:18.246 INFO:teuthology.orchestra.run.smithi047.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45885.log.gz 2024-04-26T23:21:18.246 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.49000.log: /var/log/ceph/ceph-client.admin.35045.log 2024-04-26T23:21:18.246 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.39827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46416.log 2024-04-26T23:21:18.246 INFO:teuthology.orchestra.run.smithi047.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49000.log.gz 2024-04-26T23:21:18.246 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.35045.log: gzip -5 --verbose 87.7% -- -- replaced with /var/log/ceph/ceph-client.admin.39827.log.gz 2024-04-26T23:21:18.246 INFO:teuthology.orchestra.run.smithi047.stderr: /var/log/ceph/ceph-client.admin.40617.log 2024-04-26T23:21:18.246 INFO:teuthology.orchestra.run.smithi047.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.35045.log.gz 2024-04-26T23:21:18.247 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52310.log 2024-04-26T23:21:18.247 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.46416.log: /var/log/ceph/ceph-client.admin.40617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55768.log 2024-04-26T23:21:18.247 INFO:teuthology.orchestra.run.smithi047.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46416.log.gz 2024-04-26T23:21:18.247 INFO:teuthology.orchestra.run.smithi047.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40617.log.gz 2024-04-26T23:21:18.247 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.52310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43395.log 2024-04-26T23:21:18.247 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.55768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42617.log 2024-04-26T23:21:18.247 INFO:teuthology.orchestra.run.smithi047.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52310.log.gz 2024-04-26T23:21:18.247 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.43395.log: 88.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.55768.log.gz 2024-04-26T23:21:18.247 INFO:teuthology.orchestra.run.smithi047.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.41507.log 2024-04-26T23:21:18.247 INFO:teuthology.orchestra.run.smithi047.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43395.log.gz 2024-04-26T23:21:18.247 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49504.log 2024-04-26T23:21:18.247 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42617.log: /var/log/ceph/ceph-client.admin.41507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47046.log 2024-04-26T23:21:18.247 INFO:teuthology.orchestra.run.smithi047.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42617.log.gz 2024-04-26T23:21:18.247 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.49504.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41507.log.gz 2024-04-26T23:21:18.247 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52609.log 2024-04-26T23:21:18.248 INFO:teuthology.orchestra.run.smithi047.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49504.log.gz 2024-04-26T23:21:18.248 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45339.log 2024-04-26T23:21:18.248 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.47046.log: /var/log/ceph/ceph-client.admin.52609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55852.log 2024-04-26T23:21:18.248 INFO:teuthology.orchestra.run.smithi047.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47046.log.gz 2024-04-26T23:21:18.248 INFO:teuthology.orchestra.run.smithi047.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52609.log.gz/var/log/ceph/ceph-client.admin.45339.log: 2024-04-26T23:21:18.248 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45465.log 2024-04-26T23:21:18.248 INFO:teuthology.orchestra.run.smithi047.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45339.log.gz 2024-04-26T23:21:18.248 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.55852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55432.log 2024-04-26T23:21:18.248 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46836.log 2024-04-26T23:21:18.248 INFO:teuthology.orchestra.run.smithi047.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55852.log.gz 2024-04-26T23:21:18.248 INFO:teuthology.orchestra.run.smithi047.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45465.log.gz 2024-04-26T23:21:18.248 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.55432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49590.log 2024-04-26T23:21:18.248 INFO:teuthology.orchestra.run.smithi047.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55432.log.gz 2024-04-26T23:21:18.248 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52949.log 2024-04-26T23:21:18.248 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.46836.log: /var/log/ceph/ceph-client.admin.49590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47994.log 2024-04-26T23:21:18.249 INFO:teuthology.orchestra.run.smithi047.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46836.log.gz 2024-04-26T23:21:18.249 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.52949.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49590.log.gz 2024-04-26T23:21:18.249 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52268.log 2024-04-26T23:21:18.249 INFO:teuthology.orchestra.run.smithi047.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52949.log.gz 2024-04-26T23:21:18.249 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49168.log 2024-04-26T23:21:18.249 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.47994.log: /var/log/ceph/ceph-client.admin.52268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53856.log 2024-04-26T23:21:18.249 INFO:teuthology.orchestra.run.smithi047.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47994.log.gz 2024-04-26T23:21:18.249 INFO:teuthology.orchestra.run.smithi047.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52268.log.gz 2024-04-26T23:21:18.249 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.49168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51680.log 2024-04-26T23:21:18.249 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.53856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44387.log 2024-04-26T23:21:18.249 INFO:teuthology.orchestra.run.smithi047.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.49168.log.gz 2024-04-26T23:21:18.249 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.51680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46011.log 2024-04-26T23:21:18.249 INFO:teuthology.orchestra.run.smithi047.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53856.log.gz 2024-04-26T23:21:18.249 INFO:teuthology.orchestra.run.smithi047.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51680.log.gz 2024-04-26T23:21:18.250 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.0.44387.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-26T23:21:18.250 INFO:teuthology.orchestra.run.smithi047.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44387.log.gz 2024-04-26T23:21:18.250 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.46011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55642.log 2024-04-26T23:21:18.250 INFO:teuthology.orchestra.run.smithi047.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46011.log.gz/var/log/ceph/ceph.log: 2024-04-26T23:21:18.250 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43479.log 2024-04-26T23:21:18.250 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.55642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48492.log 2024-04-26T23:21:18.250 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.43479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48450.log 2024-04-26T23:21:18.250 INFO:teuthology.orchestra.run.smithi047.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55642.log.gz 2024-04-26T23:21:18.250 INFO:teuthology.orchestra.run.smithi047.stderr: 85.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-26T23:21:18.250 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.48492.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43479.log.gz 2024-04-26T23:21:18.250 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45297.log 2024-04-26T23:21:18.250 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.48450.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48492.log.gz 2024-04-26T23:21:18.250 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45717.log 2024-04-26T23:21:18.250 INFO:teuthology.orchestra.run.smithi047.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48450.log.gz 2024-04-26T23:21:18.251 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54419.log 2024-04-26T23:21:18.251 INFO:teuthology.orchestra.run.smithi047.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45297.log.gz 2024-04-26T23:21:18.251 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51214.log 2024-04-26T23:21:18.251 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45717.log: /var/log/ceph/ceph-client.admin.54419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56420.log 2024-04-26T23:21:18.251 INFO:teuthology.orchestra.run.smithi047.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45717.log.gz 2024-04-26T23:21:18.251 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.51214.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54419.log.gz 2024-04-26T23:21:18.251 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53075.log 2024-04-26T23:21:18.251 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-26T23:21:18.251 INFO:teuthology.orchestra.run.smithi047.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51214.log.gz 2024-04-26T23:21:18.251 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.56420.log: /var/log/ceph/ceph-client.admin.53075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56168.log 2024-04-26T23:21:18.251 INFO:teuthology.orchestra.run.smithi047.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56420.log.gz 2024-04-26T23:21:18.251 INFO:teuthology.orchestra.run.smithi047.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53075.log.gz 2024-04-26T23:21:18.251 INFO:teuthology.orchestra.run.smithi047.stderr:gzip/var/log/ceph/ceph-osd.2.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.51932.log 2024-04-26T23:21:18.251 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.56168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56882.log 2024-04-26T23:21:18.252 INFO:teuthology.orchestra.run.smithi047.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56168.log.gz 2024-04-26T23:21:18.252 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48324.log 2024-04-26T23:21:18.252 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.51932.log: gzip 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51932.log.gz -5 2024-04-26T23:21:18.252 INFO:teuthology.orchestra.run.smithi047.stderr: --verbose -- /var/log/ceph/ceph-client.admin.56126.log 2024-04-26T23:21:18.252 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.56882.log: /var/log/ceph/ceph-client.admin.48324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49084.log 2024-04-26T23:21:18.252 INFO:teuthology.orchestra.run.smithi047.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56882.log.gz 2024-04-26T23:21:18.252 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.56126.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48324.log.gz 2024-04-26T23:21:18.252 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57036.log 2024-04-26T23:21:18.252 INFO:teuthology.orchestra.run.smithi047.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56126.log.gz 2024-04-26T23:21:18.252 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.49084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50423.log 2024-04-26T23:21:18.252 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.57036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47124.log 2024-04-26T23:21:18.252 INFO:teuthology.orchestra.run.smithi047.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49084.log.gz 2024-04-26T23:21:18.252 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.50423.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57036.log.gz 2024-04-26T23:21:18.252 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47952.log 2024-04-26T23:21:18.252 INFO:teuthology.orchestra.run.smithi047.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50423.log.gz 2024-04-26T23:21:18.253 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.47124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53117.log 2024-04-26T23:21:18.253 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.47952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56546.log 2024-04-26T23:21:18.253 INFO:teuthology.orchestra.run.smithi047.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47124.log.gz 2024-04-26T23:21:18.253 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.53117.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47952.log.gz 2024-04-26T23:21:18.253 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53327.log 2024-04-26T23:21:18.253 INFO:teuthology.orchestra.run.smithi047.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53117.log.gz 2024-04-26T23:21:18.253 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47448.log 2024-04-26T23:21:18.253 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.56546.log: /var/log/ceph/ceph-client.admin.53327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56756.log 2024-04-26T23:21:18.253 INFO:teuthology.orchestra.run.smithi047.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56546.log.gz 2024-04-26T23:21:18.253 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.47448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49992.log 2024-04-26T23:21:18.253 INFO:teuthology.orchestra.run.smithi047.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47448.log.gz 2024-04-26T23:21:18.253 INFO:teuthology.orchestra.run.smithi047.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.53327.log.gz 2024-04-26T23:21:18.253 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.56756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43689.log 2024-04-26T23:21:18.253 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.49992.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56756.log.gz 2024-04-26T23:21:18.253 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-26T23:21:18.254 INFO:teuthology.orchestra.run.smithi047.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49992.log.gz 2024-04-26T23:21:18.254 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.43689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50549.log 2024-04-26T23:21:18.254 INFO:teuthology.orchestra.run.smithi047.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43689.log.gz 2024-04-26T23:21:18.254 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43521.log 2024-04-26T23:21:18.254 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.50549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53603.log 2024-04-26T23:21:18.254 INFO:teuthology.orchestra.run.smithi047.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50549.log.gz 2024-04-26T23:21:18.254 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.43521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42231.log 2024-04-26T23:21:18.254 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51638.log 2024-04-26T23:21:18.254 INFO:teuthology.orchestra.run.smithi047.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43521.log.gz 2024-04-26T23:21:18.254 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.0.53603.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.53603.log.gz 2024-04-26T23:21:18.254 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52016.log 2024-04-26T23:21:18.254 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.51638.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42231.log.gz 2024-04-26T23:21:18.254 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55894.log 2024-04-26T23:21:18.254 INFO:teuthology.orchestra.run.smithi047.stderr: 89.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-26T23:21:18.255 INFO:teuthology.orchestra.run.smithi047.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51638.log.gz 2024-04-26T23:21:18.255 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35148.log 2024-04-26T23:21:18.255 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.52016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55087.log 2024-04-26T23:21:18.255 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.55894.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52016.log.gz 2024-04-26T23:21:18.255 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.35148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52447.log 2024-04-26T23:21:18.255 INFO:teuthology.orchestra.run.smithi047.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55894.log.gz 2024-04-26T23:21:18.255 INFO:teuthology.orchestra.run.smithi047.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35148.log.gz 2024-04-26T23:21:18.255 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.55087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48239.log 2024-04-26T23:21:18.255 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.52447.log: /var/log/ceph/ceph-client.admin.48576.log 2024-04-26T23:21:18.255 INFO:teuthology.orchestra.run.smithi047.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.55087.log.gz 2024-04-26T23:21:18.255 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.48239.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52447.log.gz 2024-04-26T23:21:18.255 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40089.log 2024-04-26T23:21:18.255 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.48576.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48239.log.gz 2024-04-26T23:21:18.255 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53940.log 2024-04-26T23:21:18.255 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.40089.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48576.log.gz 2024-04-26T23:21:18.256 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43224.log 2024-04-26T23:21:18.256 INFO:teuthology.orchestra.run.smithi047.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40089.log.gz 2024-04-26T23:21:18.256 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40533.log 2024-04-26T23:21:18.256 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.53940.log: /var/log/ceph/ceph-client.admin.43224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43647.log 2024-04-26T23:21:18.256 INFO:teuthology.orchestra.run.smithi047.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53940.log.gz 2024-04-26T23:21:18.256 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.40533.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43224.log.gz 2024-04-26T23:21:18.256 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50711.log 2024-04-26T23:21:18.256 INFO:teuthology.orchestra.run.smithi047.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40533.log.gz 2024-04-26T23:21:18.256 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.43647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40659.log 2024-04-26T23:21:18.256 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.50711.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43647.log.gz 2024-04-26T23:21:18.256 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42399.log 2024-04-26T23:21:18.256 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.40659.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50711.log.gz 2024-04-26T23:21:18.256 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49708.log 2024-04-26T23:21:18.256 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42399.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40659.log.gz 2024-04-26T23:21:18.256 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46878.log 2024-04-26T23:21:18.257 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.49708.log: gzip -5 89.5% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.42399.log.gz 2024-04-26T23:21:18.257 INFO:teuthology.orchestra.run.smithi047.stderr: /var/log/ceph/ceph-client.admin.48916.log 2024-04-26T23:21:18.257 INFO:teuthology.orchestra.run.smithi047.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49708.log.gz 2024-04-26T23:21:18.257 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.46878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47700.log 2024-04-26T23:21:18.257 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.48916.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46878.log.gz 2024-04-26T23:21:18.257 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40491.log 2024-04-26T23:21:18.257 INFO:teuthology.orchestra.run.smithi047.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48916.log.gz 2024-04-26T23:21:18.257 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-26T23:21:18.257 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.47700.log: /var/log/ceph/ceph-client.admin.40491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45591.log 2024-04-26T23:21:18.257 INFO:teuthology.orchestra.run.smithi047.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47700.log.gz 2024-04-26T23:21:18.257 INFO:teuthology.orchestra.run.smithi047.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40491.log.gz 2024-04-26T23:21:18.257 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46053.log 2024-04-26T23:21:18.257 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48662.log 2024-04-26T23:21:18.258 INFO:teuthology.orchestra.run.smithi047.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45591.log.gz 2024-04-26T23:21:18.258 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.46053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49426.log 2024-04-26T23:21:18.258 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42957.log 2024-04-26T23:21:18.258 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.48662.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46053.log.gz 2024-04-26T23:21:18.258 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.0.49426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41943.log 2024-04-26T23:21:18.258 INFO:teuthology.orchestra.run.smithi047.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48662.log.gz 2024-04-26T23:21:18.258 INFO:teuthology.orchestra.run.smithi047.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.49426.log.gz 2024-04-26T23:21:18.258 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45213.log 2024-04-26T23:21:18.258 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42957.log: /var/log/ceph/ceph-client.admin.41943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53698.log 2024-04-26T23:21:18.258 INFO:teuthology.orchestra.run.smithi047.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42957.log.gz 2024-04-26T23:21:18.258 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45213.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41943.log.gz 2024-04-26T23:21:18.258 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51806.log 2024-04-26T23:21:18.258 INFO:teuthology.orchestra.run.smithi047.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45213.log.gz 2024-04-26T23:21:18.259 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.53698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46620.log 2024-04-26T23:21:18.259 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55516.log 2024-04-26T23:21:18.259 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.51806.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53698.log.gz 2024-04-26T23:21:18.259 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.46620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55045.log 2024-04-26T23:21:18.259 INFO:teuthology.orchestra.run.smithi047.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51806.log.gz 2024-04-26T23:21:18.259 INFO:teuthology.orchestra.run.smithi047.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46620.log.gz 2024-04-26T23:21:18.259 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50634.log 2024-04-26T23:21:18.259 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.55516.log: /var/log/ceph/ceph-client.admin.55045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55390.log 2024-04-26T23:21:18.259 INFO:teuthology.orchestra.run.smithi047.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55516.log.gz 2024-04-26T23:21:18.259 INFO:teuthology.orchestra.run.smithi047.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55045.log.gz 2024-04-26T23:21:18.259 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.50634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54209.log 2024-04-26T23:21:18.259 INFO:teuthology.orchestra.run.smithi047.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50634.log.gz 2024-04-26T23:21:18.259 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42613.log 2024-04-26T23:21:18.259 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.55390.log: /var/log/ceph/ceph-client.0.54209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55558.log 2024-04-26T23:21:18.259 INFO:teuthology.orchestra.run.smithi047.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.54209.log.gz 2024-04-26T23:21:18.260 INFO:teuthology.orchestra.run.smithi047.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.55390.log.gz 2024-04-26T23:21:18.260 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42613.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-26T23:21:18.260 INFO:teuthology.orchestra.run.smithi047.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42613.log.gz 2024-04-26T23:21:18.260 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.55558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51974.log 2024-04-26T23:21:18.260 INFO:teuthology.orchestra.run.smithi047.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55558.log.gz 2024-04-26T23:21:18.260 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42616.log 2024-04-26T23:21:18.260 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-mds.c.log: /var/log/ceph/ceph-client.admin.51974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50262.log 2024-04-26T23:21:18.260 INFO:teuthology.orchestra.run.smithi047.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51974.log.gz 2024-04-26T23:21:18.260 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56924.log 2024-04-26T23:21:18.260 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55306.log 2024-04-26T23:21:18.260 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.0.50262.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.42616.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.0.50262.log.gz 2024-04-26T23:21:18.260 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-26T23:21:18.260 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.56924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52100.log 2024-04-26T23:21:18.260 INFO:teuthology.orchestra.run.smithi047.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-26T23:21:18.260 INFO:teuthology.orchestra.run.smithi047.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56924.log.gz 2024-04-26T23:21:18.261 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46920.log 2024-04-26T23:21:18.261 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.55306.log: /var/log/ceph/ceph-client.admin.52100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46458.log 2024-04-26T23:21:18.261 INFO:teuthology.orchestra.run.smithi047.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55306.log.gz 2024-04-26T23:21:18.261 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.46920.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52100.log.gz 2024-04-26T23:21:18.261 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56630.log 2024-04-26T23:21:18.261 INFO:teuthology.orchestra.run.smithi047.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46920.log.gz 2024-04-26T23:21:18.261 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41859.log 2024-04-26T23:21:18.261 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.46458.log: /var/log/ceph/ceph-client.admin.56630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56672.log 2024-04-26T23:21:18.261 INFO:teuthology.orchestra.run.smithi047.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46458.log.gz 2024-04-26T23:21:18.261 INFO:teuthology.orchestra.run.smithi047.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.56630.log.gz 2024-04-26T23:21:18.261 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.41859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42357.log 2024-04-26T23:21:18.261 INFO:teuthology.orchestra.run.smithi047.stderr:gzip 89.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.41859.log.gz --verbose 2024-04-26T23:21:18.261 INFO:teuthology.orchestra.run.smithi047.stderr: -- /var/log/ceph/ceph-client.admin.42105.log 2024-04-26T23:21:18.261 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.56672.log: /var/log/ceph/ceph-client.admin.42357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56504.log 2024-04-26T23:21:18.262 INFO:teuthology.orchestra.run.smithi047.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56672.log.gz 2024-04-26T23:21:18.262 INFO:teuthology.orchestra.run.smithi047.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42357.log.gz 2024-04-26T23:21:18.262 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42105.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-26T23:21:18.262 INFO:teuthology.orchestra.run.smithi047.stderr:gzip 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42105.log.gz -5 2024-04-26T23:21:18.262 INFO:teuthology.orchestra.run.smithi047.stderr: --verbose -- /var/log/ceph/ceph-client.admin.48874.log 2024-04-26T23:21:18.262 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.56504.log: /var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-26T23:21:18.262 INFO:teuthology.orchestra.run.smithi047.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56504.log.gz 2024-04-26T23:21:18.262 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.48874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56252.log 2024-04-26T23:21:18.262 INFO:teuthology.orchestra.run.smithi047.stderr: 88.0%/var/log/ceph/ceph-mgr.y.log: -- replaced with /var/log/ceph/ceph-client.admin.48874.log.gz 2024-04-26T23:21:18.262 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41465.log 2024-04-26T23:21:18.262 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.56252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48408.log 2024-04-26T23:21:18.262 INFO:teuthology.orchestra.run.smithi047.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56252.log.gz 2024-04-26T23:21:18.262 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.41465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56588.log 2024-04-26T23:21:18.262 INFO:teuthology.orchestra.run.smithi047.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41465.log.gz 2024-04-26T23:21:18.263 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.48408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42952.log 2024-04-26T23:21:18.263 INFO:teuthology.orchestra.run.smithi047.stderr: 88.8%/var/log/ceph/ceph-client.admin.56588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48790.log 2024-04-26T23:21:18.263 INFO:teuthology.orchestra.run.smithi047.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48408.log.gz 2024-04-26T23:21:18.263 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56714.log 2024-04-26T23:21:18.263 INFO:teuthology.orchestra.run.smithi047.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56588.log.gz 2024-04-26T23:21:18.263 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.48790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42273.log 2024-04-26T23:21:18.263 INFO:teuthology.orchestra.run.smithi047.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42952.log.gz 2024-04-26T23:21:18.263 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-26T23:21:18.263 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.56714.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48790.log.gz 2024-04-26T23:21:18.263 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.42273.log: -- /var/log/ceph/ceph-client.admin.53033.log 2024-04-26T23:21:18.263 INFO:teuthology.orchestra.run.smithi047.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56714.log.gz 2024-04-26T23:21:18.263 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50753.log 2024-04-26T23:21:18.263 INFO:teuthology.orchestra.run.smithi047.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42273.log.gz 2024-04-26T23:21:18.263 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5/var/log/ceph/ceph-client.admin.53033.log: --verbose -- /var/log/ceph/ceph-client.admin.55684.log 2024-04-26T23:21:18.264 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.50753.log: 89.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49866.log 2024-04-26T23:21:18.264 INFO:teuthology.orchestra.run.smithi047.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53033.log.gz 2024-04-26T23:21:18.264 INFO:teuthology.orchestra.run.smithi047.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50753.log.gz 2024-04-26T23:21:18.264 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.55684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55003.log 2024-04-26T23:21:18.264 INFO:teuthology.orchestra.run.smithi047.stderr: 92.9% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-26T23:21:18.264 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.49866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46332.log 2024-04-26T23:21:18.264 INFO:teuthology.orchestra.run.smithi047.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55684.log.gz 2024-04-26T23:21:18.264 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49866.log.gz 2024-04-26T23:21:18.264 INFO:teuthology.orchestra.run.smithi047.stderr: --verbose --/var/log/ceph/ceph-client.admin.55003.log: /var/log/ceph/ceph-client.admin.56798.log 2024-04-26T23:21:18.264 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.46332.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55003.log.gz 2024-04-26T23:21:18.264 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41290.log 2024-04-26T23:21:18.264 INFO:teuthology.orchestra.run.smithi047.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46332.log.gz 2024-04-26T23:21:18.264 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.56798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54622.log 2024-04-26T23:21:18.264 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.41290.log: gzip 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56798.log.gz 2024-04-26T23:21:18.264 INFO:teuthology.orchestra.run.smithi047.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43857.log 2024-04-26T23:21:18.265 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.54622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43310.log 2024-04-26T23:21:18.265 INFO:teuthology.orchestra.run.smithi047.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54622.log.gz/var/log/ceph/ceph-client.admin.43857.log: 2024-04-26T23:21:18.265 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52527.log 2024-04-26T23:21:18.265 INFO:teuthology.orchestra.run.smithi047.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43857.log.gz 2024-04-26T23:21:18.265 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54017.log 2024-04-26T23:21:18.265 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.43310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51130.log 2024-04-26T23:21:18.265 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.52527.log: 88.6% 92.6% -- replaced with /var/log/ceph/ceph-client.admin.41290.log.gz 2024-04-26T23:21:18.265 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.54017.log: -- replaced with /var/log/ceph/ceph-client.admin.43310.log.gz 2024-04-26T23:21:18.265 INFO:teuthology.orchestra.run.smithi047.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52527.log.gz 2024-04-26T23:21:18.265 INFO:teuthology.orchestra.run.smithi047.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54017.log.gz 2024-04-26T23:21:18.265 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39923.log 2024-04-26T23:21:18.265 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.51130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46248.log 2024-04-26T23:21:18.265 INFO:teuthology.orchestra.run.smithi047.stderr: 88.7%/var/log/ceph/ceph-client.admin.39923.log: -- replaced with /var/log/ceph/ceph-client.admin.51130.log.gz 2024-04-26T23:21:18.265 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44382.log 2024-04-26T23:21:18.266 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.46248.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39923.log.gz 2024-04-26T23:21:18.266 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56996.log 2024-04-26T23:21:18.266 INFO:teuthology.orchestra.run.smithi047.stderr: 87.9%/var/log/ceph/ceph-client.0.44382.log: -- replaced with /var/log/ceph/ceph-client.admin.46248.log.gz 2024-04-26T23:21:18.266 INFO:teuthology.orchestra.run.smithi047.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44382.log.gz 2024-04-26T23:21:18.266 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47490.log 2024-04-26T23:21:18.266 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55264.log 2024-04-26T23:21:18.266 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.56996.log: /var/log/ceph/ceph-client.admin.47490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54810.log 2024-04-26T23:21:18.266 INFO:teuthology.orchestra.run.smithi047.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56996.log.gz 2024-04-26T23:21:18.266 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.55264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.44824.log 2024-04-26T23:21:18.266 INFO:teuthology.orchestra.run.smithi047.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47490.log.gz 2024-04-26T23:21:18.266 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49126.log 2024-04-26T23:21:18.266 INFO:teuthology.orchestra.run.smithi047.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55264.log.gz 2024-04-26T23:21:18.266 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.0.54810.log: gzip/var/log/ceph/ceph-client.2.44824.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.54810.log.gz 2024-04-26T23:21:18.267 INFO:teuthology.orchestra.run.smithi047.stderr: -5 0.0% --verbose -- replaced with /var/log/ceph/ceph-client.2.44824.log.gz 2024-04-26T23:21:18.267 INFO:teuthology.orchestra.run.smithi047.stderr: -- /var/log/ceph/ceph-client.admin.52907.log 2024-04-26T23:21:18.267 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.49126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52693.log 2024-04-26T23:21:18.267 INFO:teuthology.orchestra.run.smithi047.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49126.log.gz 2024-04-26T23:21:18.267 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.52907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43605.log 2024-04-26T23:21:18.267 INFO:teuthology.orchestra.run.smithi047.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52907.log.gz/var/log/ceph/ceph-client.admin.52693.log: 2024-04-26T23:21:18.267 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45927.log 2024-04-26T23:21:18.267 INFO:teuthology.orchestra.run.smithi047.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52693.log.gz 2024-04-26T23:21:18.267 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.43605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44520.log 2024-04-26T23:21:18.267 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45927.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43605.log.gz 2024-04-26T23:21:18.267 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42955.log 2024-04-26T23:21:18.267 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.44520.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45927.log.gz 2024-04-26T23:21:18.267 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47209.log 2024-04-26T23:21:18.267 INFO:teuthology.orchestra.run.smithi047.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.44520.log.gz 2024-04-26T23:21:18.267 INFO:teuthology.orchestra.run.smithi047.stderr:gzip/var/log/ceph/ceph-client.admin.42955.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.52735.log 2024-04-26T23:21:18.268 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.47209.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42955.log.gz 2024-04-26T23:21:18.268 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56084.log 2024-04-26T23:21:18.268 INFO:teuthology.orchestra.run.smithi047.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47209.log.gz 2024-04-26T23:21:18.268 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.52735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43156.log 2024-04-26T23:21:18.268 INFO:teuthology.orchestra.run.smithi047.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52735.log.gz 2024-04-26T23:21:18.268 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.56084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57078.log 2024-04-26T23:21:18.268 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.43156.log: gzip -5 --verbose 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56084.log.gz 2024-04-26T23:21:18.268 INFO:teuthology.orchestra.run.smithi047.stderr: -- /var/log/ceph/ceph-client.admin.42441.log 2024-04-26T23:21:18.268 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.57078.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43156.log.gz 2024-04-26T23:21:18.268 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46704.log 2024-04-26T23:21:18.268 INFO:teuthology.orchestra.run.smithi047.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57078.log.gz 2024-04-26T23:21:18.268 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47574.log 2024-04-26T23:21:18.268 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.46704.log: gzip -5 --verbose 89.6% -- replaced with /var/log/ceph/ceph-client.admin.42441.log.gz 2024-04-26T23:21:18.268 INFO:teuthology.orchestra.run.smithi047.stderr: -- /var/log/ceph/ceph-client.admin.41633.log 2024-04-26T23:21:18.268 INFO:teuthology.orchestra.run.smithi047.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46704.log.gz 2024-04-26T23:21:18.269 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.47574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47279.log 2024-04-26T23:21:18.269 INFO:teuthology.orchestra.run.smithi047.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47574.log.gz 2024-04-26T23:21:18.269 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.41633.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-26T23:21:18.269 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.47279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47321.log 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41633.log.gz 2024-04-26T23:21:18.269 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-26T23:21:18.269 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-mon.a.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47279.log.gz 2024-04-26T23:21:18.269 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44961.log 2024-04-26T23:21:18.269 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.47321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40004.log 2024-04-26T23:21:18.269 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.44961.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47321.log.gz 2024-04-26T23:21:18.269 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40191.log 2024-04-26T23:21:18.269 INFO:teuthology.orchestra.run.smithi047.stderr: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53599.log 2024-04-26T23:21:18.269 INFO:teuthology.orchestra.run.smithi047.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44961.log.gz 2024-04-26T23:21:18.269 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.40004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50591.log 2024-04-26T23:21:18.269 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.40191.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40004.log.gz 2024-04-26T23:21:18.269 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.0.53599.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40191.log.gz 2024-04-26T23:21:18.270 INFO:teuthology.orchestra.run.smithi047.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.53599.log.gz 2024-04-26T23:21:18.270 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51256.log 2024-04-26T23:21:18.270 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.50591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54205.log 2024-04-26T23:21:18.270 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.51256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52777.log 2024-04-26T23:21:18.270 INFO:teuthology.orchestra.run.smithi047.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50591.log.gz 2024-04-26T23:21:18.270 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.0.54205.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51256.log.gz 2024-04-26T23:21:18.270 INFO:teuthology.orchestra.run.smithi047.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.54205.log.gz 2024-04-26T23:21:18.270 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40785.log 2024-04-26T23:21:18.270 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.52777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49632.log 2024-04-26T23:21:18.270 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.40785.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52777.log.gz 2024-04-26T23:21:18.270 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56294.log 2024-04-26T23:21:18.270 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52821.log 2024-04-26T23:21:18.270 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.49632.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.40785.log.gz 2024-04-26T23:21:18.270 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.56294.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49632.log.gz 2024-04-26T23:21:18.271 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51554.log 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56294.log.gz 2024-04-26T23:21:18.271 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-26T23:21:18.271 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.52821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50941.log 2024-04-26T23:21:18.271 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.51554.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52821.log.gz 2024-04-26T23:21:18.271 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47868.log 2024-04-26T23:21:18.271 INFO:teuthology.orchestra.run.smithi047.stderr: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47406.log 2024-04-26T23:21:18.271 INFO:teuthology.orchestra.run.smithi047.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51554.log.gz 2024-04-26T23:21:18.271 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.0.50941.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.50941.log.gz 2024-04-26T23:21:18.271 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.47868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41591.log 2024-04-26T23:21:18.271 INFO:teuthology.orchestra.run.smithi047.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47868.log.gz 2024-04-26T23:21:18.271 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.47406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52142.log 2024-04-26T23:21:18.271 INFO:teuthology.orchestra.run.smithi047.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47406.log.gz 2024-04-26T23:21:18.271 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.41591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41775.log 2024-04-26T23:21:18.272 INFO:teuthology.orchestra.run.smithi047.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41591.log.gz 2024-04-26T23:21:18.272 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.52142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40275.log 2024-04-26T23:21:18.272 INFO:teuthology.orchestra.run.smithi047.stderr: 89.1%/var/log/ceph/ceph-client.admin.41775.log: -- replaced with /var/log/ceph/ceph-client.admin.52142.log.gz 2024-04-26T23:21:18.272 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55600.log 2024-04-26T23:21:18.272 INFO:teuthology.orchestra.run.smithi047.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41775.log.gz/var/log/ceph/ceph-client.admin.40275.log: 2024-04-26T23:21:18.272 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48618.log 2024-04-26T23:21:18.272 INFO:teuthology.orchestra.run.smithi047.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40275.log.gz 2024-04-26T23:21:18.272 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41901.log 2024-04-26T23:21:18.272 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.55600.log: /var/log/ceph/ceph-client.admin.48618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56336.log 2024-04-26T23:21:18.272 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.41901.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.55600.log.gz 2024-04-26T23:21:18.272 INFO:teuthology.orchestra.run.smithi047.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48618.log.gz 2024-04-26T23:21:18.272 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49782.log 2024-04-26T23:21:18.272 INFO:teuthology.orchestra.run.smithi047.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41901.log.gz 2024-04-26T23:21:18.272 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52651.log 2024-04-26T23:21:18.272 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.56336.log: /var/log/ceph/ceph-client.admin.49782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54377.log 2024-04-26T23:21:18.273 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.52651.log: 88.2% 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56336.log.gz 2024-04-26T23:21:18.273 INFO:teuthology.orchestra.run.smithi047.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49782.log.gz 2024-04-26T23:21:18.273 INFO:teuthology.orchestra.run.smithi047.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.52651.log.gz 2024-04-26T23:21:18.273 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51722.log 2024-04-26T23:21:18.273 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40233.log 2024-04-26T23:21:18.273 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.54377.log: /var/log/ceph/ceph-client.admin.51722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47364.log 2024-04-26T23:21:18.273 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.40233.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54377.log.gz 2024-04-26T23:21:18.273 INFO:teuthology.orchestra.run.smithi047.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51722.log.gz 2024-04-26T23:21:18.273 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51890.log 2024-04-26T23:21:18.273 INFO:teuthology.orchestra.run.smithi047.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40233.log.gz 2024-04-26T23:21:18.273 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43353.log 2024-04-26T23:21:18.273 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.47364.log: /var/log/ceph/ceph-client.admin.51890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45129.log 2024-04-26T23:21:18.273 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.43353.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47364.log.gz 2024-04-26T23:21:18.273 INFO:teuthology.orchestra.run.smithi047.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51890.log.gz 2024-04-26T23:21:18.274 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55726.log 2024-04-26T23:21:18.274 INFO:teuthology.orchestra.run.smithi047.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43353.log.gz 2024-04-26T23:21:18.274 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51470.log 2024-04-26T23:21:18.274 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45129.log: /var/log/ceph/ceph-client.admin.55726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51848.log 2024-04-26T23:21:18.274 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.51470.log: 88.4% 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45129.log.gz 2024-04-26T23:21:18.274 INFO:teuthology.orchestra.run.smithi047.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55726.log.gz 2024-04-26T23:21:18.274 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56462.log 2024-04-26T23:21:18.274 INFO:teuthology.orchestra.run.smithi047.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51470.log.gz 2024-04-26T23:21:18.274 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48036.log 2024-04-26T23:21:18.274 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.51848.log: /var/log/ceph/ceph-client.admin.56462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49824.log 2024-04-26T23:21:18.274 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.48036.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56462.log.gz 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51848.log.gz 2024-04-26T23:21:18.274 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-26T23:21:18.274 INFO:teuthology.orchestra.run.smithi047.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48036.log.gz 2024-04-26T23:21:18.274 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40363.log 2024-04-26T23:21:18.275 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41675.log 2024-04-26T23:21:18.275 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.49824.log: /var/log/ceph/ceph-client.admin.40363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52991.log 2024-04-26T23:21:18.275 INFO:teuthology.orchestra.run.smithi047.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40363.log.gz 2024-04-26T23:21:18.275 INFO:teuthology.orchestra.run.smithi047.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49824.log.gz 2024-04-26T23:21:18.275 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.41675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45381.log 2024-04-26T23:21:18.275 INFO:teuthology.orchestra.run.smithi047.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41675.log.gz 2024-04-26T23:21:18.275 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.52991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45549.log 2024-04-26T23:21:18.275 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43437.log 2024-04-26T23:21:18.275 INFO:teuthology.orchestra.run.smithi047.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52991.log.gz 2024-04-26T23:21:18.275 INFO:teuthology.orchestra.run.smithi047.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45381.log.gz 2024-04-26T23:21:18.275 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45549.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45549.log.gz 2024-04-26T23:21:18.275 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49422.log 2024-04-26T23:21:18.275 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.43437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49908.log 2024-04-26T23:21:18.275 INFO:teuthology.orchestra.run.smithi047.stderr: 88.7%/var/log/ceph/ceph-client.0.49422.log: -- replaced with /var/log/ceph/ceph-client.admin.43437.log.gz 2024-04-26T23:21:18.276 INFO:teuthology.orchestra.run.smithi047.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.49422.log.gz 2024-04-26T23:21:18.276 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56840.log 2024-04-26T23:21:18.276 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54461.log 2024-04-26T23:21:18.276 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.49908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56210.log 2024-04-26T23:21:18.276 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.56840.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49908.log.gz 2024-04-26T23:21:18.276 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.54461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39748.log 2024-04-26T23:21:18.276 INFO:teuthology.orchestra.run.smithi047.stderr: 88.0% 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54461.log.gz 2024-04-26T23:21:18.276 INFO:teuthology.orchestra.run.smithi047.stderr: -- replaced with /var/log/ceph/ceph-client.admin.56840.log.gz 2024-04-26T23:21:18.276 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.56210.log: gzip 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56210.log.gz 2024-04-26T23:21:18.276 INFO:teuthology.orchestra.run.smithi047.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.54303.log 2024-04-26T23:21:18.276 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45087.log 2024-04-26T23:21:18.276 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.39748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45255.log 2024-04-26T23:21:18.276 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.54303.log: 88.3% 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39748.log.gz 2024-04-26T23:21:18.276 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45087.log: -- replaced with /var/log/ceph/ceph-client.admin.54303.log.gz 2024-04-26T23:21:18.276 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45087.log.gz 2024-04-26T23:21:18.277 INFO:teuthology.orchestra.run.smithi047.stderr: -- /var/log/ceph/ceph-client.admin.53898.log 2024-04-26T23:21:18.277 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42021.log 2024-04-26T23:21:18.277 INFO:teuthology.orchestra.run.smithi047.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45255.log.gz 2024-04-26T23:21:18.277 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.53898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51088.log 2024-04-26T23:21:18.277 INFO:teuthology.orchestra.run.smithi047.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53898.log.gz 2024-04-26T23:21:18.277 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48534.log 2024-04-26T23:21:18.277 INFO:teuthology.orchestra.run.smithi047.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42021.log.gz/var/log/ceph/ceph-client.admin.51088.log: 2024-04-26T23:21:18.277 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54545.log 2024-04-26T23:21:18.277 INFO:teuthology.orchestra.run.smithi047.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51088.log.gz 2024-04-26T23:21:18.277 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.48534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43563.log 2024-04-26T23:21:18.277 INFO:teuthology.orchestra.run.smithi047.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48534.log.gz 2024-04-26T23:21:18.277 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.54545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42948.log 2024-04-26T23:21:18.277 INFO:teuthology.orchestra.run.smithi047.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54545.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48958.log 2024-04-26T23:21:18.277 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-26T23:21:18.277 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.43563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49042.log 2024-04-26T23:21:18.278 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42948.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43563.log.gz 2024-04-26T23:21:18.278 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.48958.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.42948.log.gz 2024-04-26T23:21:18.278 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48958.log.gz 2024-04-26T23:21:18.278 INFO:teuthology.orchestra.run.smithi047.stderr: -- /var/log/ceph/ceph-client.admin.46662.log 2024-04-26T23:21:18.278 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.49042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53159.log 2024-04-26T23:21:18.278 INFO:teuthology.orchestra.run.smithi047.stderr: 88.2%/var/log/ceph/ceph-client.admin.46662.log: -- replaced with /var/log/ceph/ceph-client.admin.49042.log.gz 2024-04-26T23:21:18.278 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42189.log 2024-04-26T23:21:18.278 INFO:teuthology.orchestra.run.smithi047.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46662.log.gz 2024-04-26T23:21:18.278 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.44828.log 2024-04-26T23:21:18.278 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.53159.log: /var/log/ceph/ceph-client.admin.42189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43815.log 2024-04-26T23:21:18.278 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.2.44828.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53159.log.gz 2024-04-26T23:21:18.278 INFO:teuthology.orchestra.run.smithi047.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42189.log.gz 2024-04-26T23:21:18.278 INFO:teuthology.orchestra.run.smithi047.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.44828.log.gz 2024-04-26T23:21:18.279 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47167.log 2024-04-26T23:21:18.279 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.43815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42620.log 2024-04-26T23:21:18.279 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.47167.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43815.log.gz 2024-04-26T23:21:18.279 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45003.log 2024-04-26T23:21:18.279 INFO:teuthology.orchestra.run.smithi047.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47167.log.gz 2024-04-26T23:21:18.279 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49234.log 2024-04-26T23:21:18.279 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42620.log: /var/log/ceph/ceph-client.admin.45003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40405.log 2024-04-26T23:21:18.279 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.49234.log: 88.4% 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45003.log.gz 2024-04-26T23:21:18.279 INFO:teuthology.orchestra.run.smithi047.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42620.log.gz 2024-04-26T23:21:18.279 INFO:teuthology.orchestra.run.smithi047.stderr:gzip 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49234.log.gz 2024-04-26T23:21:18.279 INFO:teuthology.orchestra.run.smithi047.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.49950.log 2024-04-26T23:21:18.279 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.40405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48748.log 2024-04-26T23:21:18.279 INFO:teuthology.orchestra.run.smithi047.stderr: 88.8%/var/log/ceph/ceph-client.admin.49950.log: -- replaced with /var/log/ceph/ceph-client.admin.40405.log.gz 2024-04-26T23:21:18.279 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55129.log 2024-04-26T23:21:18.279 INFO:teuthology.orchestra.run.smithi047.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49950.log.gz 2024-04-26T23:21:18.280 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41817.log 2024-04-26T23:21:18.280 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.48748.log: /var/log/ceph/ceph-client.admin.55129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52865.log 2024-04-26T23:21:18.280 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.41817.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48748.log.gz 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55129.log.gz 2024-04-26T23:21:18.280 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-26T23:21:18.280 INFO:teuthology.orchestra.run.smithi047.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41817.log.gz 2024-04-26T23:21:18.280 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52184.log 2024-04-26T23:21:18.280 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.52865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42622.log 2024-04-26T23:21:18.280 INFO:teuthology.orchestra.run.smithi047.stderr: 89.3%/var/log/ceph/ceph-client.admin.52184.log: -- replaced with /var/log/ceph/ceph-client.admin.52865.log.gz 2024-04-26T23:21:18.280 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42623.log 2024-04-26T23:21:18.280 INFO:teuthology.orchestra.run.smithi047.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52184.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46578.log 2024-04-26T23:21:18.280 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-26T23:21:18.280 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42622.log: /var/log/ceph/ceph-client.admin.42623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49548.log 2024-04-26T23:21:18.280 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.46578.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42622.log.gz 2024-04-26T23:21:18.280 INFO:teuthology.orchestra.run.smithi047.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42623.log.gz 2024-04-26T23:21:18.280 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54814.log 2024-04-26T23:21:18.281 INFO:teuthology.orchestra.run.smithi047.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46578.log.gz 2024-04-26T23:21:18.281 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.49548.log: /var/log/ceph/ceph-client.admin.51375.log 2024-04-26T23:21:18.281 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.0.54814.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40851.log 2024-04-26T23:21:18.281 INFO:teuthology.orchestra.run.smithi047.stderr: -- replaced with /var/log/ceph/ceph-client.0.54814.log.gz 2024-04-26T23:21:18.281 INFO:teuthology.orchestra.run.smithi047.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49548.log.gz 2024-04-26T23:21:18.281 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56378.log 2024-04-26T23:21:18.281 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.51375.log: /var/log/ceph/ceph-client.admin.40851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47616.log 2024-04-26T23:21:18.281 INFO:teuthology.orchestra.run.smithi047.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51375.log.gz 2024-04-26T23:21:18.281 INFO:teuthology.orchestra.run.smithi047.stderr: 88.4%/var/log/ceph/ceph-client.admin.56378.log: -- replaced with /var/log/ceph/ceph-client.admin.40851.log.gz 2024-04-26T23:21:18.281 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48281.log 2024-04-26T23:21:18.281 INFO:teuthology.orchestra.run.smithi047.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56378.log.gz 2024-04-26T23:21:18.281 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.47616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55810.log 2024-04-26T23:21:18.281 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46746.log 2024-04-26T23:21:18.281 INFO:teuthology.orchestra.run.smithi047.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47616.log.gz/var/log/ceph/ceph-client.admin.48281.log: 2024-04-26T23:21:18.282 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.55810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45759.log 2024-04-26T23:21:18.282 INFO:teuthology.orchestra.run.smithi047.stderr: 89.0% 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55810.log.gz 2024-04-26T23:21:18.282 INFO:teuthology.orchestra.run.smithi047.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48281.log.gz 2024-04-26T23:21:18.282 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.46746.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46746.log.gz 2024-04-26T23:21:18.282 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47532.log 2024-04-26T23:21:18.282 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40149.log 2024-04-26T23:21:18.282 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.47532.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45759.log.gz 2024-04-26T23:21:18.282 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53243.log 2024-04-26T23:21:18.282 INFO:teuthology.orchestra.run.smithi047.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47532.log.gz 2024-04-26T23:21:18.282 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51512.log 2024-04-26T23:21:18.282 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.40149.log: /var/log/ceph/ceph-client.admin.53243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51172.log 2024-04-26T23:21:18.282 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.51512.log: 89.2% 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40149.log.gz 2024-04-26T23:21:18.282 INFO:teuthology.orchestra.run.smithi047.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53243.log.gz 2024-04-26T23:21:18.283 INFO:teuthology.orchestra.run.smithi047.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51512.log.gz 2024-04-26T23:21:18.283 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54908.log 2024-04-26T23:21:18.283 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.51172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43773.log 2024-04-26T23:21:18.283 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.54908.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51172.log.gz 2024-04-26T23:21:18.283 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53814.log 2024-04-26T23:21:18.283 INFO:teuthology.orchestra.run.smithi047.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.54908.log.gz 2024-04-26T23:21:18.283 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46137.log 2024-04-26T23:21:18.283 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.43773.log: /var/log/ceph/ceph-client.admin.53814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42621.log 2024-04-26T23:21:18.283 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.46137.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43773.log.gz 2024-04-26T23:21:18.283 INFO:teuthology.orchestra.run.smithi047.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53814.log.gz 2024-04-26T23:21:18.283 INFO:teuthology.orchestra.run.smithi047.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.46137.log.gz 2024-04-26T23:21:18.283 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46290.log 2024-04-26T23:21:18.283 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47004.log 2024-04-26T23:21:18.283 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.46290.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42621.log.gz 2024-04-26T23:21:18.283 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48197.log 2024-04-26T23:21:18.284 INFO:teuthology.orchestra.run.smithi047.stderr: 87.7%/var/log/ceph/ceph-client.admin.47004.log: -- replaced with /var/log/ceph/ceph-client.admin.46290.log.gz 2024-04-26T23:21:18.284 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42612.log 2024-04-26T23:21:18.284 INFO:teuthology.orchestra.run.smithi047.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47004.log.gz 2024-04-26T23:21:18.284 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44063.log 2024-04-26T23:21:18.284 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.48197.log: /var/log/ceph/ceph-client.admin.42612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40447.log 2024-04-26T23:21:18.284 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.44063.log: 88.2% 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42612.log.gz 2024-04-26T23:21:18.284 INFO:teuthology.orchestra.run.smithi047.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48197.log.gz 2024-04-26T23:21:18.284 INFO:teuthology.orchestra.run.smithi047.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44063.log.gz 2024-04-26T23:21:18.284 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43731.log 2024-04-26T23:21:18.284 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48366.log 2024-04-26T23:21:18.284 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.40447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52058.log 2024-04-26T23:21:18.284 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.43731.log: 87.7%/var/log/ceph/ceph-client.admin.48366.log: -- replaced with /var/log/ceph/ceph-client.admin.40447.log.gz 2024-04-26T23:21:18.284 INFO:teuthology.orchestra.run.smithi047.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43731.log.gz 2024-04-26T23:21:18.284 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48366.log.gz 2024-04-26T23:21:18.285 INFO:teuthology.orchestra.run.smithi047.stderr: --verbose -- /var/log/ceph/ceph-client.admin.51298.log 2024-04-26T23:21:18.285 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.52058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45969.log 2024-04-26T23:21:18.285 INFO:teuthology.orchestra.run.smithi047.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52058.log.gz 2024-04-26T23:21:18.285 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.51298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40701.log 2024-04-26T23:21:18.285 INFO:teuthology.orchestra.run.smithi047.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51298.log.gz 2024-04-26T23:21:18.285 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48706.log 2024-04-26T23:21:18.285 INFO:teuthology.orchestra.run.smithi047.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45969.log.gz 2024-04-26T23:21:18.285 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45171.log 2024-04-26T23:21:18.285 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.40701.log: /var/log/ceph/ceph-client.admin.48706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47784.log 2024-04-26T23:21:18.285 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45171.log: 88.6% 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48706.log.gz 2024-04-26T23:21:18.285 INFO:teuthology.orchestra.run.smithi047.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40701.log.gz 2024-04-26T23:21:18.285 INFO:teuthology.orchestra.run.smithi047.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45171.log.gz 2024-04-26T23:21:18.285 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42959.log 2024-04-26T23:21:18.285 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.47784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53201.log 2024-04-26T23:21:18.286 INFO:teuthology.orchestra.run.smithi047.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47784.log.gz 2024-04-26T23:21:18.286 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55474.log 2024-04-26T23:21:18.286 INFO:teuthology.orchestra.run.smithi047.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42959.log.gz 2024-04-26T23:21:18.286 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.53201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46536.log 2024-04-26T23:21:18.286 INFO:teuthology.orchestra.run.smithi047.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53201.log.gz 2024-04-26T23:21:18.286 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.55474.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-26T23:21:18.286 INFO:teuthology.orchestra.run.smithi047.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.55474.log.gz 2024-04-26T23:21:18.286 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.46536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50945.log 2024-04-26T23:21:18.286 INFO:teuthology.orchestra.run.smithi047.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46536.log.gz 2024-04-26T23:21:18.286 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42956.log 2024-04-26T23:21:18.286 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40743.log 2024-04-26T23:21:18.286 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.0.50945.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.50945.log.gz 2024-04-26T23:21:18.286 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53285.log 2024-04-26T23:21:18.286 INFO:teuthology.orchestra.run.smithi047.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42956.log.gz 2024-04-26T23:21:18.287 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-26T23:21:18.287 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.40743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52352.log 2024-04-26T23:21:18.287 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.53285.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.28765.log 2024-04-26T23:21:18.287 INFO:teuthology.orchestra.run.smithi047.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40743.log.gz 2024-04-26T23:21:18.287 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.52352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46374.log 2024-04-26T23:21:18.287 INFO:teuthology.orchestra.run.smithi047.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53285.log.gz 2024-04-26T23:21:18.287 INFO:teuthology.orchestra.run.smithi047.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52352.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40575.log 2024-04-26T23:21:18.287 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-26T23:21:18.287 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph.tmp-client.admin.28765.log: /var/log/ceph/ceph-client.admin.46374.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.28765.log.gz 2024-04-26T23:21:18.287 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50070.log 2024-04-26T23:21:18.287 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.40575.log: 88.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50507.log 2024-04-26T23:21:18.287 INFO:teuthology.orchestra.run.smithi047.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46374.log.gz 2024-04-26T23:21:18.287 INFO:teuthology.orchestra.run.smithi047.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40575.log.gz 2024-04-26T23:21:18.287 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45675.log 2024-04-26T23:21:18.287 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.50070.log: /var/log/ceph/ceph-client.admin.50507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48116.log 2024-04-26T23:21:18.288 INFO:teuthology.orchestra.run.smithi047.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50070.log.gz 2024-04-26T23:21:18.288 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45675.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50507.log.gz 2024-04-26T23:21:18.288 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 88.5% --verbose -- replaced with /var/log/ceph/ceph-client.admin.45675.log.gz -- 2024-04-26T23:21:18.288 INFO:teuthology.orchestra.run.smithi047.stderr: /var/log/ceph/ceph-client.admin.53393.log 2024-04-26T23:21:18.288 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.48116.log: /var/log/ceph/ceph-client.admin.47826.log 2024-04-26T23:21:18.288 INFO:teuthology.orchestra.run.smithi047.stderr: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45423.log 2024-04-26T23:21:18.288 INFO:teuthology.orchestra.run.smithi047.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48116.log.gz 2024-04-26T23:21:18.288 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.53393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46962.log 2024-04-26T23:21:18.288 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.47826.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53393.log.gz 2024-04-26T23:21:18.288 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55348.log 2024-04-26T23:21:18.288 INFO:teuthology.orchestra.run.smithi047.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47826.log.gz 2024-04-26T23:21:18.288 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.46962.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45423.log.gz 2024-04-26T23:21:18.288 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42147.log 2024-04-26T23:21:18.288 INFO:teuthology.orchestra.run.smithi047.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46962.log.gz 2024-04-26T23:21:18.288 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51596.log 2024-04-26T23:21:18.289 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.55348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45801.log 2024-04-26T23:21:18.289 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42147.log: 89.2%/var/log/ceph/ceph-client.admin.51596.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42147.log.gz 2024-04-26T23:21:18.289 INFO:teuthology.orchestra.run.smithi047.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55348.log.gz 2024-04-26T23:21:18.289 INFO:teuthology.orchestra.run.smithi047.stderr: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54503.log 2024-04-26T23:21:18.289 INFO:teuthology.orchestra.run.smithi047.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51596.log.gz 2024-04-26T23:21:18.289 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34781.log 2024-04-26T23:21:18.289 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.54503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45633.log 2024-04-26T23:21:18.289 INFO:teuthology.orchestra.run.smithi047.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45801.log.gz 2024-04-26T23:21:18.289 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.34781.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54503.log.gz 2024-04-26T23:21:18.289 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41549.log 2024-04-26T23:21:18.289 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42063.log 2024-04-26T23:21:18.289 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45633.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.34781.log.gz 2024-04-26T23:21:18.289 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.41549.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45633.log.gz 2024-04-26T23:21:18.290 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41733.log 2024-04-26T23:21:18.290 INFO:teuthology.orchestra.run.smithi047.stderr: 88.9%/var/log/ceph/ceph-client.admin.42063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47910.log 2024-04-26T23:21:18.290 INFO:teuthology.orchestra.run.smithi047.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41549.log.gz 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42063.log.gz 2024-04-26T23:21:18.290 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.41733.log: 2024-04-26T23:21:18.290 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41733.log.gz 2024-04-26T23:21:18.290 INFO:teuthology.orchestra.run.smithi047.stderr: --verbose -- /var/log/ceph/ceph-client.0.50258.log 2024-04-26T23:21:18.290 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.47910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47742.log 2024-04-26T23:21:18.290 INFO:teuthology.orchestra.run.smithi047.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47910.log.gz 2024-04-26T23:21:18.290 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.0.50258.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.50258.log.gz 2024-04-26T23:21:18.290 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42315.log 2024-04-26T23:21:18.290 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.47742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42958.log 2024-04-26T23:21:18.290 INFO:teuthology.orchestra.run.smithi047.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47742.log.gz 2024-04-26T23:21:18.290 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55171.log 2024-04-26T23:21:18.290 INFO:teuthology.orchestra.run.smithi047.stderr: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43268.log 2024-04-26T23:21:18.291 INFO:teuthology.orchestra.run.smithi047.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42315.log.gz 2024-04-26T23:21:18.291 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42958.log: /var/log/ceph/ceph-client.admin.55171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52226.log 2024-04-26T23:21:18.291 INFO:teuthology.orchestra.run.smithi047.stderr: 88.3%/var/log/ceph/ceph-client.admin.43268.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55171.log.gz 2024-04-26T23:21:18.291 INFO:teuthology.orchestra.run.smithi047.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42958.log.gz 2024-04-26T23:21:18.291 INFO:teuthology.orchestra.run.smithi047.stderr: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50465.log 2024-04-26T23:21:18.291 INFO:teuthology.orchestra.run.smithi047.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43268.log.gz 2024-04-26T23:21:18.291 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.52226.log: -- /var/log/ceph/ceph-client.admin.47658.log 2024-04-26T23:21:18.291 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.50465.log: 88.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43899.log 2024-04-26T23:21:18.291 INFO:teuthology.orchestra.run.smithi047.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52226.log.gz 2024-04-26T23:21:18.291 INFO:teuthology.orchestra.run.smithi047.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50465.log.gz 2024-04-26T23:21:18.291 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51764.log 2024-04-26T23:21:18.291 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.47658.log: /var/log/ceph/ceph-client.admin.43899.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47658.log.gz/var/log/ceph/ceph-client.admin.51764.log: 2024-04-26T23:21:18.291 INFO:teuthology.orchestra.run.smithi047.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51764.log.gz 2024-04-26T23:21:18.291 INFO:teuthology.orchestra.run.smithi047.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.43899.log.gz 2024-04-26T23:21:18.294 INFO:teuthology.orchestra.run.smithi203.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-26T23:21:18.393 INFO:teuthology.orchestra.run.smithi047.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-26T23:21:18.778 INFO:teuthology.orchestra.run.smithi047.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-26T23:21:18.805 INFO:teuthology.orchestra.run.smithi203.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-26T23:21:18.842 INFO:teuthology.orchestra.run.smithi203.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-26T23:21:18.849 INFO:teuthology.orchestra.run.smithi203.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-26T23:21:18.861 INFO:teuthology.orchestra.run.smithi047.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-26T23:21:18.885 INFO:teuthology.orchestra.run.smithi203.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-26T23:21:18.887 INFO:teuthology.orchestra.run.smithi203.stderr: 2024-04-26T23:21:18.887 INFO:teuthology.orchestra.run.smithi203.stderr:real 0m0.775s 2024-04-26T23:21:18.887 INFO:teuthology.orchestra.run.smithi203.stderr:user 0m3.230s 2024-04-26T23:21:18.887 INFO:teuthology.orchestra.run.smithi203.stderr:sys 0m0.120s 2024-04-26T23:21:18.953 INFO:teuthology.orchestra.run.smithi047.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-26T23:21:18.963 INFO:teuthology.orchestra.run.smithi047.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-26T23:21:18.963 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-26T23:21:18.963 INFO:teuthology.orchestra.run.smithi047.stderr:real 0m0.894s 2024-04-26T23:21:18.963 INFO:teuthology.orchestra.run.smithi047.stderr:user 0m3.432s 2024-04-26T23:21:18.963 INFO:teuthology.orchestra.run.smithi047.stderr:sys 0m0.318s 2024-04-26T23:21:18.963 INFO:tasks.ceph:Archiving logs... 2024-04-26T23:21:18.964 DEBUG:teuthology.misc:Transferring archived files from smithi047:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675294/remote/smithi047/log 2024-04-26T23:21:18.965 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T23:21:19.832 DEBUG:teuthology.misc:Transferring archived files from smithi203:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675294/remote/smithi203/log 2024-04-26T23:21:19.833 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T23:21:20.370 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-26T23:21:20.382 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-26T23:21:20.383 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T23:21:20.386 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T23:21:20.461 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-26T23:21:20.473 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-26T23:21:20.473 DEBUG:teuthology.orchestra.run.smithi047:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T23:21:20.476 DEBUG:teuthology.orchestra.run.smithi203:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T23:21:20.491 INFO:teuthology.orchestra.run.smithi047.stderr:bash: line 1: ntpq: command not found 2024-04-26T23:21:20.491 INFO:teuthology.orchestra.run.smithi203.stderr:bash: line 1: ntpq: command not found 2024-04-26T23:21:20.496 INFO:teuthology.orchestra.run.smithi047.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T23:21:20.496 INFO:teuthology.orchestra.run.smithi047.stdout:=============================================================================== 2024-04-26T23:21:20.497 INFO:teuthology.orchestra.run.smithi047.stdout:^* hv01.front.sepia.ceph.com 3 6 377 38 +1539us[+1542us] +/- 52ms 2024-04-26T23:21:20.497 INFO:teuthology.orchestra.run.smithi047.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 38 +420us[ +424us] +/- 54ms 2024-04-26T23:21:20.497 INFO:teuthology.orchestra.run.smithi047.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 39 -2543us[-2539us] +/- 65ms 2024-04-26T23:21:20.497 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T23:21:20.498 INFO:teuthology.orchestra.run.smithi203.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T23:21:20.499 INFO:teuthology.orchestra.run.smithi203.stdout:=============================================================================== 2024-04-26T23:21:20.499 INFO:teuthology.orchestra.run.smithi203.stdout:^* hv01.front.sepia.ceph.com 3 6 377 38 +1543us[+1547us] +/- 52ms 2024-04-26T23:21:20.499 INFO:teuthology.orchestra.run.smithi203.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 38 +432us[ +436us] +/- 54ms 2024-04-26T23:21:20.499 INFO:teuthology.orchestra.run.smithi203.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 39 -2589us[-2584us] +/- 65ms 2024-04-26T23:21:20.499 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T23:21:20.499 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-26T23:21:20.511 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-26T23:21:20.512 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-26T23:21:20.538 DEBUG:teuthology.orchestra.run.smithi047:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T23:21:20.616 DEBUG:teuthology.orchestra.run.smithi203:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T23:21:20.678 DEBUG:teuthology.orchestra.run.smithi047:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T23:21:20.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T23:21:20.708 DEBUG:teuthology.orchestra.run.smithi203:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T23:21:20.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T23:21:20.735 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-26T23:21:20.745 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-26T23:21:20.769 INFO:teuthology.task.internal:Duration was 579.996845 seconds 2024-04-26T23:21:20.770 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-26T23:21:20.809 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-26T23:21:20.809 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T23:21:20.812 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T23:21:20.849 INFO:teuthology.orchestra.run.smithi203.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T23:21:20.850 INFO:teuthology.orchestra.run.smithi047.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T23:21:21.026 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-26T23:21:21.026 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi047.front.sepia.ceph.com 2024-04-26T23:21:21.026 DEBUG:teuthology.orchestra.run.smithi047:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T23:21:21.087 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi203.front.sepia.ceph.com 2024-04-26T23:21:21.087 DEBUG:teuthology.orchestra.run.smithi203:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T23:21:21.113 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-26T23:21:21.113 DEBUG:teuthology.orchestra.run.smithi047:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T23:21:21.129 DEBUG:teuthology.orchestra.run.smithi203:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T23:21:21.178 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-26T23:21:21.178 DEBUG:teuthology.orchestra.run.smithi047:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T23:21:21.196 DEBUG:teuthology.orchestra.run.smithi203:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T23:21:21.316 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-26T23:21:21.326 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-26T23:21:21.326 DEBUG:teuthology.orchestra.run.smithi047:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T23:21:21.351 DEBUG:teuthology.orchestra.run.smithi203:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T23:21:21.375 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-26T23:21:21.388 DEBUG:teuthology.orchestra.run.smithi047:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T23:21:21.393 DEBUG:teuthology.orchestra.run.smithi203:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T23:21:21.414 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern = core 2024-04-26T23:21:21.438 INFO:teuthology.orchestra.run.smithi203.stdout:kernel.core_pattern = core 2024-04-26T23:21:21.450 DEBUG:teuthology.orchestra.run.smithi047:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T23:21:21.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T23:21:21.487 DEBUG:teuthology.orchestra.run.smithi203:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T23:21:21.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T23:21:21.504 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-26T23:21:21.517 INFO:teuthology.task.internal:Transferring archived files... 2024-04-26T23:21:21.518 DEBUG:teuthology.misc:Transferring archived files from smithi047:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675294/remote/smithi047 2024-04-26T23:21:21.518 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T23:21:21.568 DEBUG:teuthology.misc:Transferring archived files from smithi203:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675294/remote/smithi203 2024-04-26T23:21:21.568 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T23:21:21.609 INFO:teuthology.task.internal:Removing archive directory... 2024-04-26T23:21:21.609 DEBUG:teuthology.orchestra.run.smithi047:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T23:21:21.612 DEBUG:teuthology.orchestra.run.smithi203:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T23:21:21.654 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-26T23:21:21.665 INFO:teuthology.task.internal:Not uploading archives. 2024-04-26T23:21:21.666 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-26T23:21:21.688 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-26T23:21:21.689 DEBUG:teuthology.orchestra.run.smithi047:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T23:21:21.691 DEBUG:teuthology.orchestra.run.smithi203:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T23:21:21.704 INFO:teuthology.orchestra.run.smithi047.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 23:21 /home/ubuntu/cephtest 2024-04-26T23:21:21.708 INFO:teuthology.orchestra.run.smithi203.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 23:21 /home/ubuntu/cephtest 2024-04-26T23:21:21.709 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-26T23:21:21.725 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-26T23:21:21.794 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/pool-perm} duration: 579.9968450069427 flavor: default owner: scheduled_rishabh@teuthology success: true 2024-04-26T23:21:21.794 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T23:21:21.882 INFO:teuthology.run:pass